Computer Science · 9th Grade · The Impact of Artificial Intelligence · Weeks 28-36

Sources of Algorithmic Bias

Students will analyze how human prejudices can be encoded into software and the resulting social impact.

• Common Core State Standards
• CSTA: 3A-IC-24
• CSTA: 3A-IC-25

About This Topic

Algorithmic bias occurs when automated systems produce outcomes that systematically favor or disadvantage certain groups. For 9th graders, understanding the sources of this bias is as important as knowing it exists. Human decisions embed bias at every stage: what data is collected, how it is labeled, which features are included, and what outcomes the system is optimized for. The result is that software can encode and scale human prejudice in ways that affect hiring, lending, healthcare, and criminal justice.

In the US K-12 context, this topic addresses CSTA 3A-IC-24 and 3A-IC-25 and connects to social studies, civics, and statistics. Students benefit from seeing concrete documented cases , the COMPAS recidivism tool, Amazon's hiring algorithm, and facial recognition accuracy disparities across demographic groups are all well-documented examples at an accessible reading level.

Active learning is particularly effective here because students bring strong prior beliefs about whether computers are objective. Direct examination of real biased outputs challenges that assumption in a way that lecture cannot, and structured analysis develops the critical framework students need to evaluate AI systems they will encounter throughout their lives.

Key Questions

1. Analyze how human prejudices can be encoded into software and the resulting social impact.
2. Differentiate between various sources of algorithmic bias (e.g., data bias, design bias).
3. Explain how algorithmic bias can perpetuate or exacerbate existing inequalities.

Learning Objectives

• Analyze specific examples to identify how human biases are encoded into algorithmic systems.
• Compare and contrast data bias and design bias, providing examples of each.
• Explain the social impact of algorithmic bias in at least two real-world scenarios, such as hiring or loan applications.
• Critique an algorithm's potential for bias by examining its data sources and intended function.

Before You Start

Introduction to Algorithms

Why: Students need a foundational understanding of what algorithms are and how they process information before analyzing their potential for bias.

Data Collection and Representation

Why: Understanding how data is gathered and structured is crucial for identifying how biases can be introduced during the data collection phase.

Key Vocabulary

Algorithmic Bias	Systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others.
Data Bias	Bias that occurs when the data used to train an algorithm is not representative of the real world or contains historical prejudices.
Design Bias	Bias introduced by the choices made by developers when designing an algorithm, including feature selection, objective functions, and evaluation metrics.
Proxy Variable	A variable that is correlated with a protected attribute (like race or gender) and can inadvertently lead to discrimination even if the protected attribute itself is not used.

Watch Out for These Misconceptions

Common Misconception

Algorithms are objective because they use math, not opinions.

What to Teach Instead

Every algorithm reflects human choices about what data to collect, what to optimize for, and what errors are acceptable. Math executes those choices at scale , it does not neutralize the bias in them. Case study analysis makes this concrete by showing the specific human decisions that produced biased real-world outcomes.

Common Misconception

Algorithmic bias only comes from biased training data.

What to Teach Instead

Bias can enter at every stage: data collection, feature selection, labeling, model design, threshold choices, and deployment context. A model trained on perfectly representative data can still produce biased outcomes if the optimization target is misaligned with fairness. Students who understand the full pipeline recognize more intervention points.

Common Misconception

Fixing bias is simply a matter of removing sensitive attributes like race or gender from the model.

What to Teach Instead

Removing a protected attribute does not eliminate bias if proxy variables (zip code, school name, browsing history) correlate with that attribute. This is called proxy discrimination, and it is one reason why bias remediation requires careful analysis of the full feature set and outcome distributions, not just attribute exclusion.

Active Learning Ideas

Case Study Analysis: Real Algorithmic Bias

Groups each receive one documented case of algorithmic bias (COMPAS, Amazon hiring tool, facial recognition accuracy, predictive policing). Each group identifies the source of bias, the affected group, and the real-world harm. Groups present their case using a shared analysis template, then the class maps patterns across all cases.

40 min·Small Groups

Think-Pair-Share: Where Did the Bias Come From?

Show students a short description of a biased AI outcome (e.g., a loan approval system that denies more applications from a particular zip code). Individually, students trace back through the development process to identify at least two points where bias could have entered. Pairs compare their traces, then share with the class.

20 min·Pairs

Bias Source Mapping: From Data to Decision

Using a simplified flowchart of how a hiring algorithm works (data collection, feature selection, model training, threshold setting, deployment), groups annotate each stage with the types of bias that could enter there. The class builds a composite map on the board showing how bias accumulates through a pipeline.

30 min·Small Groups

Perspective Role-Play: Who Is Harmed?

Students take on roles of people affected by a biased algorithm (loan applicant, job candidate, parolee, medical patient). Each writes a one-paragraph account from their perspective describing the decision they received and why it may be unfair. Class discusses whose perspective is typically absent from algorithmic development teams.

30 min·Individual

Real-World Connections

• Developers at companies like Amazon have faced scrutiny for AI hiring tools that showed bias against female candidates because the training data reflected historical male dominance in certain roles.
• The COMPAS recidivism prediction software, used in some US court systems, has been shown to disproportionately label Black defendants as high risk compared to white defendants with similar criminal histories.
• Facial recognition systems have demonstrated lower accuracy rates for individuals with darker skin tones and for women, raising concerns about their use in law enforcement and security.

Assessment Ideas

Exit Ticket

Provide students with a brief description of a hypothetical AI system (e.g., a loan application screener). Ask them to write one sentence identifying a potential source of bias (data or design) and one sentence explaining how it could lead to unfair outcomes.

Discussion Prompt

Pose the question: 'If an algorithm is trained on historical data, how can it ever be truly fair?' Facilitate a class discussion, encouraging students to reference specific types of bias and their real-world consequences.

Quick Check

Present students with two short case studies of algorithmic bias. Ask them to categorize the primary source of bias in each case (data bias or design bias) and briefly justify their choice.

Frequently Asked Questions

What are the main sources of algorithmic bias?

Bias can enter through historical data that reflects past discrimination, underrepresentation of certain groups in training data, labels that encode subjective human judgments, features that serve as proxies for protected attributes, and optimization targets that do not account for distributional fairness across groups. Each source requires different mitigation strategies.

Can you give a real example of algorithmic bias that affected people?

The COMPAS recidivism tool, used in US courts to assess reoffending risk, was found by ProPublica to incorrectly flag Black defendants as high risk at nearly twice the rate as white defendants. Amazon's internal hiring algorithm, trained on historical resumes, learned to downgrade resumes that included the word 'women's.' Both cases are well-documented and accessible for classroom discussion.

How does algorithmic bias relate to existing social inequalities?

Algorithms trained on historical data learn patterns from a world that already contained inequality. When deployed at scale, they can reproduce and amplify those patterns, making discriminatory outcomes faster and harder to contest because they appear objective. This is why algorithmic systems in high-stakes domains require ongoing audit and accountability structures.

How does active learning help students grasp the sources of algorithmic bias?

Students often arrive believing that computers are inherently objective. Direct case study analysis where they trace bias from a real documented outcome back through a development pipeline challenges that belief through evidence rather than argument. Annotating a real algorithm's decision pathway builds the analytical habit of asking 'where did this come from?' rather than accepting outputs at face value.