Computer Science · 10th Grade · Advanced Data Structures and Management · Weeks 10-18

Ethical Considerations in Data Management

Students discuss the ethical implications of data collection, storage, and usage, including privacy and bias.

• Common Core State Standards
• CSTA: 3A-IC-24
• CSTA: 3A-IC-25

About This Topic

Ethical considerations in data management ask students to reason about the responsibilities that come with collecting, storing, and using personal data. As organizations gather increasing amounts of information about individuals -- through apps, sensors, purchases, and health records -- questions about consent, purpose limitation, data minimization, security, and bias become pressing social and technical concerns. This topic aligns with CSTA standards 3A-IC-24 and 3A-IC-25, which ask students to evaluate the social and economic implications of computing and analyze how technology affects civil liberties.

Students examine real cases: how facial recognition systems exhibit demographic bias, how app permissions collect data far beyond what is necessary, how data breaches expose millions of people, and how algorithmic decisions in hiring and lending can perpetuate historical inequities. These examples make abstract ethical principles concrete and show that technical decisions have human consequences.

This topic works especially well with discussion-based active learning. Ethical dilemmas rarely have clean answers, and the process of arguing, listening to opposing views, and refining positions builds the kind of ethical reasoning that prepares students for responsible participation in a data-driven society.

Key Questions

1. Evaluate the ethical responsibilities of organizations handling personal data.
2. Analyze how data collection practices can infringe on individual privacy.
3. Justify policies that protect user data while enabling beneficial data analysis.

Learning Objectives

• Analyze how specific data collection methods, such as app permissions or sensor data, can infringe on individual privacy.
• Evaluate the ethical responsibilities of organizations that collect, store, and use personal data, considering principles like consent and data minimization.
• Justify proposed policies or technical solutions that aim to protect user data while still allowing for beneficial data analysis.
• Critique real-world examples of algorithmic bias in systems like facial recognition or loan applications, identifying the data-related causes.
• Compare and contrast different approaches to data anonymization and their effectiveness in protecting privacy.

Before You Start

Introduction to Data Types and Structures

Why: Students need a foundational understanding of what data is and how it can be organized before discussing its management and ethical implications.

Basic Programming Concepts

Why: Understanding how data is processed and manipulated by programs is essential for grasping how ethical issues arise in computing systems.

Key Vocabulary

Data Privacy	The right of individuals to control how their personal information is collected, used, and shared by organizations.
Algorithmic Bias	Systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others.
Data Minimization	The practice of collecting and retaining only the data that is strictly necessary for a specific, defined purpose.
Informed Consent	A process where individuals voluntarily agree to share their personal data after being fully informed about how it will be used and protected.
Data Breach	An incident where sensitive, protected, or confidential data is accessed, stolen, or used by an unauthorized individual.

Watch Out for These Misconceptions

Common Misconception

If data is anonymized, it cannot be misused.

What to Teach Instead

Anonymized datasets can often be re-identified by combining them with other publicly available data. Researchers have shown that 87% of Americans can be uniquely identified using just ZIP code, birth date, and sex. Students who analyze real re-identification cases understand why 'anonymized' is not synonymous with 'safe.'

Common Misconception

Algorithmic decisions are objective because they are based on data and math.

What to Teach Instead

Algorithms reflect the biases embedded in their training data and the choices made by their designers about what to optimize. A hiring algorithm trained on historical hiring decisions will encode historical discrimination. Students who examine real disparities in algorithmic outputs -- and trace them to data collection or design choices -- develop a much more accurate model of how bias enters automated systems.

Active Learning Ideas

Formal Debate: Data Collection Policy

Present a scenario where a school wants to install AI attendance tracking using facial recognition. Groups are assigned stakeholder roles (students, parents, administrators, civil liberties advocates, technology vendors) and must argue their position in a structured town hall format, then negotiate a policy that addresses the core concerns of each group.

50 min·Small Groups

Inquiry Circle: Privacy Policy Audit

Small groups select a popular app (social media, gaming, educational) and analyze its actual privacy policy against a provided checklist covering data collected, stated purposes, third-party sharing, retention periods, and user rights. Groups report findings to the class and vote on which policy is most and least protective of user interests.

40 min·Small Groups

Think-Pair-Share: The Bias Audit

Present students with a dataset showing demographic disparities in loan approval rates from an algorithmic system. Pairs discuss whether the disparity constitutes bias, what data might have caused it, and whether the company bears responsibility. The class then hears all pairs and builds a shared framework for evaluating algorithmic fairness.

30 min·Pairs

Gallery Walk: Data Ethics Case Studies

Post six real-world data ethics case studies (Cambridge Analytica, health app data sales, predictive policing, credit scoring algorithms, clearview AI, student data brokers). Student groups rotate and annotate each case with the harm caused, who was responsible, and what policy or technical change would have prevented the harm.

45 min·Small Groups

Real-World Connections

• Tech companies like Google and Apple face scrutiny over how they collect user data through their operating systems and apps, impacting millions of smartphone users globally.
• Financial institutions use algorithms to assess loan applications; these systems can perpetuate historical biases if not carefully designed and monitored, affecting access to credit for certain communities.
• Healthcare providers must adhere to strict regulations like HIPAA when managing patient records, balancing the need for data access for treatment with the critical requirement of patient privacy.

Assessment Ideas

Discussion Prompt

Present students with a scenario: 'A social media company wants to use user posts to train a new AI model for content moderation. What ethical questions should they consider regarding user privacy and data ownership? What steps should they take to ensure informed consent?'

Exit Ticket

Ask students to write down one specific example of data collection they encountered today (e.g., app permission, website cookie). Then, have them explain one potential ethical concern related to that collection and suggest one way to mitigate it.

Quick Check

Provide students with a short case study about a data breach. Ask them to identify: 1) What type of data was compromised? 2) What were the potential consequences for individuals? 3) What preventative measures could the organization have implemented?

Frequently Asked Questions

What is data privacy and why does it matter?

Data privacy is the right of individuals to control how information about them is collected, used, and shared. It matters because personal data can be used to manipulate behavior, discriminate in hiring or lending, enable surveillance, or expose individuals to identity theft. Laws like FERPA (student records), HIPAA (health data), and COPPA (children's data) establish minimum privacy protections in the US.

How can data collection practices infringe on privacy?

Privacy is compromised when data is collected without meaningful consent, used for purposes beyond what users agreed to, retained longer than necessary, shared with third parties without disclosure, or insufficiently secured against breaches. Many apps collect location, contact, microphone, or browsing data that is not required for their stated function.

What is algorithmic bias and how does it arise?

Algorithmic bias occurs when an automated system produces systematically unfair outcomes for certain groups. It typically arises from biased training data (historical decisions that reflected discrimination), flawed feature selection (using proxies for protected characteristics), or optimization choices that do not account for fairness across groups. It is a technical problem with social causes and consequences.

How does active learning help students engage with data ethics?

Ethical questions do not have provable answers, which makes them ideal for discussion formats. Role-playing stakeholder debates forces students to understand perspectives they might not naturally hold. Case study analysis grounds abstract principles in specific harms. When students negotiate policies as a group, they practice the democratic reasoning that governance of data-driven systems actually requires.