Computer Science · 11th Grade · Artificial Intelligence and Ethics · Weeks 19-27

AI and Decision Making: Transparency and Accountability

Discussing the importance of understanding how AI makes decisions and holding AI systems accountable.

• Common Core State Standards
• CSTA: 3B-IC-26

About This Topic

When an AI system denies a loan, flags a student for plagiarism, or recommends a medical treatment, the people affected by those decisions deserve to understand how they were made. This topic addresses the demand for transparency and accountability in AI decision-making, focusing on why opaque systems create risks and what technical and regulatory responses exist. In the US K-12 context, this connects to student experiences with content recommendation algorithms, school-based plagiarism detectors, and automated grading tools they may already encounter.

Transparency in AI ranges from full interpretability (understanding every step of the model's logic) to explainability (providing a coherent post-hoc justification for a specific decision). Neither is trivial with modern deep learning systems. Students learn to distinguish these concepts and to evaluate why transparency requirements vary by context. CSTA standard 3B-IC-26 specifically asks students to evaluate the tradeoffs in computational systems that affect communities.

Accountability questions are equally pressing: when an autonomous system makes a harmful error, who bears responsibility? The developer, the deploying organization, or the individual using it? Active learning formats like mock hearings and impact analysis workshops help students reason through these genuinely contested questions rather than arriving at easy answers.

Key Questions

1. Explain why it's important to understand how AI systems arrive at their decisions.
2. Analyze scenarios where AI decisions have significant real-world consequences.
3. Justify the need for accountability when AI systems make errors or biased decisions.

Learning Objectives

• Evaluate the tradeoffs between model accuracy and interpretability in AI systems for loan application processing.
• Analyze a case study of an AI-driven hiring tool to identify potential sources of bias and their impact.
• Justify the need for specific accountability mechanisms for autonomous vehicle software failures.
• Compare different approaches to AI transparency, such as LIME and SHAP, in explaining predictive model outcomes.
• Design a framework for auditing an AI content recommendation algorithm for fairness and unintended consequences.

Before You Start

Introduction to Machine Learning Concepts

Why: Students need a basic understanding of how machine learning models are trained and make predictions to grasp the challenges of transparency and accountability.

Ethical Considerations in Technology

Why: Prior exposure to ethical frameworks and discussions about technology's societal impact will help students engage with the fairness and accountability aspects of AI.

Key Vocabulary

Interpretability	The degree to which a human can understand the cause of a decision made by an AI model, examining its internal logic.
Explainability	The ability to provide a human-understandable justification for a specific AI decision after it has been made, even if the internal logic is complex.
Algorithmic Bias	Systematic and repeatable errors in an AI system that create unfair outcomes, such as privileging one arbitrary group of users over others.
Accountability	The obligation to accept responsibility for the outcomes of an AI system's decisions, including errors or harmful impacts.
Black Box Model	An AI system, often a deep neural network, where the internal workings are so complex that it is difficult or impossible to understand how it arrives at a specific output from a given input.

Watch Out for These Misconceptions

Common Misconception

If an AI gives a reason for its decision, that reason is a full and accurate explanation.

What to Teach Instead

Post-hoc explanations are often approximations, not complete accounts of the model's internal logic. Students who compare decision tree traces with neural network outputs firsthand see this gap clearly.

Common Misconception

Transparency and accuracy are always in tension: you have to sacrifice one for the other.

What to Teach Instead

The tradeoff exists but is context-dependent. Simpler, interpretable models often perform comparably to complex ones for structured tabular data. Exploring model comparisons in class prevents students from treating this as an absolute rule.

Common Misconception

Accountability means blaming one person or company when AI causes harm.

What to Teach Instead

AI systems involve distributed responsibility across data collectors, model developers, deployers, and regulators. Mock hearing exercises that assign distinct roles help students map this complexity rather than looking for a single blameworthy party.

Active Learning Ideas

Mock Congressional Hearing: AI Accountability

Assign roles: AI company representative, affected community member, regulator, and independent auditor. Groups prepare three-minute testimony and question rounds based on a provided AI decision-making scenario, then debrief on what accountability structures they would support.

45 min·Small Groups

Think-Pair-Share: Black Box Scenarios

Present three brief scenarios where an AI decision affected someone's life (credit denial, medical triage, college admissions). Students individually write who should be accountable and why, compare reasoning with a partner, then share the most contested case with the whole class.

20 min·Pairs

Explainability Comparison: Simple vs. Complex Models

Provide two classifiers for the same task: a decision tree students can trace by hand, and a neural network described as a black box. Groups compare the output of both on three test cases, then argue which model they would trust more for each scenario and why.

30 min·Small Groups

Real-World Connections

• In healthcare, AI diagnostic tools used by radiologists at Massachusetts General Hospital must be transparent enough for clinicians to trust their recommendations for patient treatment plans.
• Financial institutions like Capital One use AI for credit scoring; regulators require explanations for loan denials to ensure compliance with fair lending laws and prevent discrimination.
• The National Highway Traffic Safety Administration (NHTSA) investigates accidents involving self-driving cars to determine liability and establish safety standards for autonomous vehicle AI.

Assessment Ideas

Discussion Prompt

Present students with a scenario: An AI system used by a school district flags students for potential truancy based on their online activity. Ask: 'What information would you need to understand why a specific student was flagged? Who should be responsible if the system incorrectly flags a student who has done nothing wrong?'

Quick Check

Provide students with two brief descriptions of AI decision-making explanations: one focusing on technical model details and another on user-friendly justifications. Ask them to identify which is more interpretable and which is more explainable, and to write one sentence justifying their choice for each.

Exit Ticket

Ask students to name one profession where AI transparency is critical and explain in 1-2 sentences why. Then, ask them to identify one potential consequence of a lack of accountability in AI systems.

Frequently Asked Questions

What is an AI black box and why does it matter?

A black box AI system produces outputs without providing an understandable account of how inputs were transformed into decisions. This matters when decisions affect people's lives because individuals and oversight bodies cannot identify errors, challenge decisions, or assess fairness without understanding the reasoning process.

What is explainable AI (XAI)?

Explainable AI refers to techniques that make machine learning model decisions interpretable to humans. Methods include LIME, SHAP, and saliency maps, which highlight which input features most influenced a particular output. These explanations are useful but are approximations, not complete descriptions of the model's internal mechanics.

Who is legally responsible when an AI system makes a harmful mistake?

US law on AI liability is still developing. Currently, responsibility may fall on developers under product liability law, deploying organizations under anti-discrimination law, or individual operators. The EU AI Act and proposed US federal frameworks are beginning to assign liability based on risk level and use context.

How can active learning help students think through AI accountability?

Role-playing hearings and structured debates place students in the position of different stakeholders, making abstract accountability questions feel concrete. Students who argue multiple positions develop more nuanced frameworks than those who passively read case studies, better preparing them to reason about real-world AI governance.