Mathematics · 11th Grade · Statistical Inference and Data Analysis · Weeks 19-27

Significance Tests for Proportions

Students will perform significance tests for population proportions and interpret p-values.

• Common Core State Standards
• CCSS.Math.Content.HSS.IC.B.5

About This Topic

Once students understand the framework of hypothesis testing, they apply it to proportions , one of the most common scenarios in real statistical practice. A significance test for a proportion asks whether the observed sample proportion differs enough from a claimed population value to cast doubt on that claim. Students compute a z-test statistic using the formula z equals p-hat minus p-naught divided by the square root of p-naught times (1 minus p-naught) divided by n, then use the p-value to make a decision. CCSS.Math.Content.HSS.IC.B.5 requires that students understand how to interpret p-values and draw properly worded conclusions.

The p-value is the probability of getting a test statistic as extreme as the one observed, assuming H0 is true. A small p-value (typically below 0.05) suggests the sample result would be very unlikely under H0 , providing evidence against it. Students often confuse the p-value with the probability that the null hypothesis is true, a misconception that must be addressed directly and repeatedly.

Active learning is particularly valuable here because significance testing requires a chain of logical reasoning: state hypotheses, check conditions, compute the test statistic, find the p-value, and draw a conclusion in context. Small-group practice where students compare conclusions reinforces the sequential logic and surfaces common errors for class discussion.

Key Questions

1. Justify the use of a p-value to make a decision in hypothesis testing.
2. Analyze the meaning of 'statistical significance' in the context of a p-value.
3. Construct a conclusion for a hypothesis test based on the p-value and significance level.

Learning Objectives

• Calculate the z-test statistic for a population proportion using sample data.
• Determine the p-value associated with a given z-test statistic and alternative hypothesis.
• Interpret the meaning of a p-value in the context of a specific hypothesis test scenario.
• Formulate a statistically sound conclusion for a significance test based on the p-value and a chosen significance level.
• Critique the validity of a claim about a population proportion using evidence from a significance test.

Before You Start

Introduction to Probability

Why: Students need a foundational understanding of probability to grasp the concept of a p-value as a probability.

Sampling Distributions of Proportions

Why: Understanding the distribution of sample proportions is essential for calculating the z-test statistic and interpreting its meaning relative to the population proportion.

Confidence Intervals for Proportions

Why: Familiarity with confidence intervals helps students understand the concept of estimating population parameters from sample data, which is related to hypothesis testing.

Key Vocabulary

Null Hypothesis (H0)	A statement of no effect or no difference, representing the default assumption that the significance test aims to challenge.
Alternative Hypothesis (Ha)	A statement that contradicts the null hypothesis, proposing that there is an effect or difference to be detected.
p-value	The probability of observing a sample statistic as extreme as, or more extreme than, the one obtained, assuming the null hypothesis is true.
Significance Level (alpha)	A predetermined threshold (commonly 0.05) used to decide whether to reject the null hypothesis; if the p-value is less than alpha, the result is considered statistically significant.
z-test statistic	A standardized value calculated from sample data that measures how many standard errors a sample proportion is away from the hypothesized population proportion.

Watch Out for These Misconceptions

Common Misconception

A small p-value proves the alternative hypothesis is true.

What to Teach Instead

A small p-value indicates the observed data would be unlikely if H0 were true , it is evidence against H0, not proof for Ha. Statistical significance does not confirm the alternative is true; it only suggests the null is implausible given the data. Peer review of written conclusions helps students refine the precision of their language.

Common Misconception

The p-value is the probability that the null hypothesis is true.

What to Teach Instead

This is one of the most pervasive misconceptions in statistics. The p-value is computed assuming H0 is true; it is the probability of the observed data (or more extreme data), not the probability of the hypothesis itself. Using simulation where students compute p-values from known populations helps build the correct mental model.

Common Misconception

Failing to reject the null means the null hypothesis is true.

What to Teach Instead

Insufficient evidence against H0 is not proof of H0. The null might be false but the sample too small to detect it. Partner discussion of low-power scenarios (small sample size, small true effect) helps students understand why absence of evidence is not evidence of absence.

Active Learning Ideas

Real Data Test: Is Our Class Different?

Students test whether the proportion of the class who prefer a certain response on a survey question differs from a known national benchmark. Different pairs are responsible for each of the five steps, then they combine their work to reach a conclusion and discuss what the p-value actually tells them.

35 min·Whole Class

P-Value Interpretation Workshop

Groups receive five written conclusions from significance tests, some correct and some incorrectly equating the p-value with 'the probability H0 is true.' Groups annotate each statement, explain the error, and rewrite any flawed interpretations before sharing their corrections with the class.

25 min·Small Groups

Think-Pair-Share: Change the Alpha, Change the Conclusion?

Pairs examine a test with p-value 0.037. With alpha of 0.05 they reject H0; with alpha of 0.01 they do not. Partners discuss what changed and what did not, and what this implies about the role of the chosen significance level in a decision.

15 min·Pairs

Case Study Analysis: A Medical Claim

Groups receive a realistic scenario where a pharmaceutical company claims their treatment helps 80% of patients, but a study finds only 73 out of 100 patients improved. Groups perform the full five-step significance test and write a conclusion appropriate for a medical decision-maker audience.

30 min·Small Groups

Real-World Connections

• Political pollsters use significance tests to determine if a candidate's reported approval rating is statistically different from a previous survey, influencing campaign strategy and media coverage.
• Pharmaceutical companies conduct clinical trials and use significance tests to assess if a new drug's success rate is significantly higher than a placebo or existing treatment, impacting FDA approval and public health.
• Quality control managers in manufacturing plants perform significance tests on sample product defect rates to decide if a production process needs adjustment to meet quality standards.

Assessment Ideas

Exit Ticket

Provide students with a scenario: A company claims 80% of customers are satisfied. A sample of 200 customers shows 75% satisfaction. Ask them to: 1. State the null and alternative hypotheses. 2. Calculate the z-test statistic (provide the formula if needed). 3. Explain what a p-value of 0.03 would mean in this context.

Quick Check

Present students with a completed hypothesis test summary including the p-value and significance level. Ask them to write a two-sentence conclusion in context, stating whether the null hypothesis is rejected and what that implies about the population proportion.

Discussion Prompt

Pose the question: 'Imagine a p-value is 0.45. What does this tell us about the null hypothesis? Is this result statistically significant at the alpha = 0.05 level? Why or why not?' Facilitate a class discussion to clarify the relationship between p-value and significance level.

Frequently Asked Questions

What does a p-value actually mean?

A p-value is the probability of observing a test statistic as extreme as yours, or more extreme, assuming the null hypothesis is true. A small p-value means your data would be unlikely under H0 , suggesting H0 may not accurately describe reality. It does not tell you the probability that H0 is true or false.

What does it mean for results to be statistically significant?

Statistical significance means the p-value fell below the chosen significance level alpha (commonly 0.05). It indicates the observed result was unlikely to occur by random chance if H0 were true , providing enough evidence to reject the null. Statistical significance does not indicate the result is large, important, or practically meaningful.

What are the conditions for a significance test for a proportion?

The sample must be random, the observations must be independent, and both the expected number of successes and failures under H0 must be at least 10 (n times p-naught at least 10 and n times (1 minus p-naught) at least 10). These conditions ensure the normal approximation to the sampling distribution is reliable.

How does active learning make significance testing clearer?

Significance testing involves a chain of reasoning where students must get each link right. Small-group work through each step , with structured peer review , surfaces errors like miswriting hypotheses or misinterpreting p-values before they become habits. A class debate about whether a p-value of 0.047 is convincing reveals more about conceptual understanding than any individual quiz.