Data Collection Methods and Bias

Active learning works well for this topic because students need to confront messy data directly to understand how errors and biases affect analysis. Reading about data problems in a textbook doesn’t create the same urgency or lasting impression as fixing a real survey with missing responses or identifying skewed response patterns.

• 9th Grade
• Computer Science
• 3 activities
• 20 min–45 min

Learning Objectives

1. Compare potential biases in at least two different data collection methods, such as surveys versus observational studies.
2. Analyze how specific sampling techniques can introduce bias into a dataset, leading to skewed results.
3. Design a data collection plan for a given research question that actively mitigates at least two common sources of bias.
4. Explain the ethical implications of collecting biased data in real-world scenarios, citing potential harms.
5. Critique a provided dataset for potential biases and suggest methods for correction or further investigation.

Inquiry Circle

⏱ 45 min·Small Groups

Inquiry Circle: The Messy Survey

Give groups a raw dataset from a fictional school survey with intentional errors (typos, impossible ages like 200, missing names). Groups must decide on a set of 'cleaning rules' and produce a clean version of the data.

Prepare & details

Analyze how bias in data collection can lead to inaccurate or harmful conclusions.

Facilitation Tip: During Collaborative Investigation: The Messy Survey, circulate with a red pen to mark where students hesitate or make assumptions, turning those moments into mini-lessons about acceptable assumptions.

Setup: Groups at tables with access to source materials

Materials: Source material collection, Inquiry cycle worksheet, Question generation protocol, Findings presentation template

• Analyze
• Evaluate
• Create
• Self-Management
• Self-Awareness

Think-Pair-Share

⏱ 20 min·Pairs

Think-Pair-Share: Bias Detectives

Show students a headline based on a flawed data collection method (e.g., '90% of people love winter', but the survey was only taken at a ski resort). Pairs identify the bias and suggest a better collection method.

Prepare & details

Compare different data collection methods and their potential sources of bias.

Facilitation Tip: For Think-Pair-Share: Bias Detectives, provide sentence frames on the slide to keep discussions grounded in evidence rather than opinions.

Setup: Standard classroom seating; students turn to a neighbor

Materials: Discussion prompt (projected or printed), Optional: recording sheet for pairs

• Understand
• Apply
• Analyze
• Self-Awareness
• Relationship Skills

Gallery Walk

⏱ 30 min·Individual

Gallery Walk: Data Sources

Post different methods of data collection (online polls, sensors, government records, social media scraping). Students walk around and list one 'pro' and one 'con' for the reliability of each source.

Prepare & details

Design a data collection strategy that minimizes bias for a specific research question.

Facilitation Tip: During Gallery Walk: Data Sources, display examples at different heights so students must move and compare, reinforcing that source credibility isn’t just about content but also about presentation.

Setup: Wall space or tables arranged around room perimeter

Materials: Large paper/poster boards, Markers, Sticky notes for feedback

• Understand
• Apply
• Analyze
• Create
• Relationship Skills
• Social Awareness

Teaching This Topic

Teachers should emphasize that data cleaning is a creative process, not just a mechanical one. Avoid presenting data cleaning as a chore; instead, frame it as detective work where students set their own rules for what counts as valid. Research shows that students grasp bias better when they experience the tension between wanting easy answers and needing reliable ones, so plan moments where students have to choose between two imperfect data sets.

What to Expect

Successful learning looks like students confidently discussing why data isn’t ready for analysis right away and proposing concrete steps to clean and validate it. They should recognize bias not as an abstract concept but as a real issue they can spot and explain in a survey or data set.

Watch Out for These Misconceptions

Common Misconception

During Collaborative Investigation: The Messy Survey, watch for students assuming that the survey platform automatically fixes errors or that missing values are acceptable.

What to Teach Instead

In this activity, hand students a printed copy of the survey with visible red marks on missing values and format errors. Have them write a rule on a sticky note about how to handle each type of error, then post the rules on a class chart for reference during cleaning.

Common Misconception

During Think-Pair-Share: Bias Detectives, watch for students thinking that a larger survey size always reduces bias.

What to Teach Instead

In this activity, provide two survey scenarios with the same number of responses but different sampling methods. Have students calculate response rates and discuss why a larger biased sample can still produce misleading results.

Assessment Ideas

Quick Check

After Collaborative Investigation: The Messy Survey, present students with a cleaned and an uncleaned version of the same data set. Ask them to write one sentence explaining how the cleaned version will lead to more reliable conclusions than the original.

Discussion Prompt

During Think-Pair-Share: Bias Detectives, ask students to share their pair’s findings on one source of bias in the survey. Listen for explanations that reference sampling method, question wording, or response options.

Exit Ticket

After Gallery Walk: Data Sources, ask students to write one potential bias in the data source they found most convincing and one step the researcher could take to reduce that bias.

Extensions & Scaffolding

• Challenge: Ask students to design a follow-up survey question that would help clean or validate their messy data set.
• Scaffolding: Provide a checklist with three specific things to look for (missing values, outliers, formatting errors) and model one example before students work.
• Deeper exploration: Invite students to research how one historical data set was cleaned or corrected, and present the before-and-after changes to the class.

Key Vocabulary

Sampling Bias	Systematic error introduced into a sample when individuals or groups are not represented in the same proportion as they are in the population. This can lead to inaccurate generalizations.
Selection Bias	Bias introduced when the sample selected is not representative of the target population. This can occur if certain individuals are more likely to be included or excluded from the study.
Measurement Bias	Bias that occurs when the method of measurement or the instrument used consistently produces inaccurate results. This can happen with poorly worded survey questions or faulty equipment.
Confirmation Bias	The tendency to search for, interpret, favor, and recall information in a way that confirms one's pre-existing beliefs or hypotheses. In data collection, this can influence question design or data interpretation.
Convenience Sampling	A method of data collection where participants are selected based on their easy availability and proximity. This method often leads to biased samples because it does not represent the broader population.