Bias in sampling happens when certain responses are systematically favored over others, which makes a sample fail to represent the population. The main types you need to recognize are voluntary response bias, undercoverage bias, nonresponse bias, and response bias (including question wording problems), plus the fact that nonrandom methods like convenience sampling open the door to bias.
Why This Matters for the AP Statistics Exam
Collecting data is about 12 to 15 percent of the AP Statistics exam, and this topic shows up when you have to judge whether a sample's results can be trusted. You will be asked to identify the type of bias in a described study and explain how it affects the results. A strong answer names the specific bias and explains the likely direction of the error, such as whether the sample result is probably too high or too low compared to the true population value. Being precise here also sets you up for later units, where you can only generalize sample results to a population when the sample was selected randomly or is otherwise representative.
Key Takeaways
- Bias occurs when certain responses are systematically favored over others, either intentionally or unintentionally.
- Voluntary response bias (also called self-selection bias) happens when the sample is made up of people who chose to participate, so people with strong opinions are overrepresented.
- Undercoverage bias happens when part of the population has a reduced chance of being included, often because the sampling frame leaves a group out.
- Nonresponse bias happens when people selected for the sample cannot be reached or refuse to respond, and the non-responders differ from responders.
- Response bias comes from problems in the data-gathering process, including confusing or leading question wording and unreliable self-reported answers.
- Non-random methods like convenience sampling and voluntary response do not use chance, so they carry built-in potential for bias.
Bias in Sampling
Bias occurs when certain responses are systematically favored over others, either intentionally or unintentionally. A biased study tends to underestimate or overestimate the value you are trying to measure. When that happens, your sample is not representative of the population, and any conclusions you draw can be misleading.
You already know how to gather data well. This topic is about recognizing the ways data collection goes wrong.
Voluntary Response
Voluntary response bias, also called self-selection bias, occurs when a sample is made up entirely of people who choose to participate. This often shows up when a study relies on self-reported data or recruits participants through online surveys, television, or radio.
Because these samples are not randomly selected, they are usually not representative of the population. People with strong opinions or a strong interest in the topic are more likely to volunteer, which skews the results.
Example
Suppose you want to study how young adults feel about social media, so you post an online survey link on your personal accounts and in social media discussion groups, then ask friends to share it.
You get a lot of responses fast, but the sample has problems. Many respondents are very active on social media with strong opinions, while lighter users are underrepresented. People from urban areas respond far more than people from rural areas. The sample is mostly volunteers, and those volunteers do not represent young adults overall.
Undercoverage
Undercoverage bias occurs when part of the population has a reduced chance of being included in the sample. This often happens when the sampling frame, the list you actually draw from, is incomplete or when some groups are harder to reach.
Example
If you study college students' attitudes toward climate change but use a sampling frame that only includes students at four-year colleges and universities, you introduce undercoverage bias. Students at two-year colleges and trade schools are underrepresented, even though they are part of the larger population of college students.
Nonresponse
Nonresponse bias occurs when people are chosen for the sample but data cannot be obtained from them, either because they cannot be reached or because they refuse to respond. The problem is that the people who do not respond may differ in important ways from the people who do.
Example
If you send a survey on climate change to a random sample of 1000 college students but only 500 complete and return it, you may have nonresponse bias. The students who responded could differ from the ones who did not, even though both groups were selected for the sample.
When you write about nonresponse on the exam, push your explanation further by stating whether the missing responses likely make the sample result too high or too low compared to the true population value.
Wording in Questions
Question wording bias is a type of response bias. It occurs when the way a question is asked influences the answer. This happens when questions are confusing, leading, or ambiguous.
Example
Compare these two questions:
- "Do you think that the government should provide free healthcare for all citizens?"
- "Would you support a government healthcare program that would provide free healthcare for all citizens?"
Both ask about the same topic, but the first uses a leading phrase that may nudge respondents toward agreeing. The second presents the idea more neutrally. The responses can differ even though the topic is the same. Self-reported answers can add response bias too, since people do not always answer truthfully.
Convenience Sampling
Convenience sampling occurs when the sample is selected based on ease of access or availability instead of chance. Because no random process is used, the sample is likely biased and not representative of the population.
Example
Back to the researcher studying college students' attitudes toward climate change. Instead of using a random method, the researcher picks a campus close to home and interviews whoever happens to be passing by or hanging out in a common area.
This sample only represents students who were available and willing at one campus. It leaves out students who were not on campus or not interested. The results may not reflect the attitudes of college students overall.
Why This Matters
If you sample without checking for these problems, your conclusions will not represent the population. Recognizing the source of bias and explaining its likely effect is what turns a vague answer into a complete one.
How to Use This on the AP Statistics Exam
MCQ
- Read the description of how the sample was gathered, then match it to a specific bias. Voluntary response means people opted in. Undercoverage means a group had a reduced chance of being selected. Nonresponse means selected people did not provide data. Response bias points to the question or measurement process.
- Watch for the difference between undercoverage and nonresponse. Undercoverage is about who could be selected in the first place. Nonresponse is about who actually responds after being selected.
Free Response
- When asked to identify the bias, name it directly and clearly.
- Back up your answer with evidence from the scenario. For nonresponse, say whether non-responders likely differ in a way that makes the result too high or too low.
- Tie your explanation to the context of the question instead of giving a generic definition.
- Keep your sampling vocabulary separate from experiment vocabulary. Bias and random selection apply to sampling. Save random assignment language for experiments.
Common Trap
Saying a sample is biased "because it is too small." Bias is about a systematic tilt in which responses are favored, not about sample size. A large convenience sample can still be badly biased.
Practice Problem
You are a researcher interested in the attitudes of small business owners toward the economy, so you run a survey.
You create a list of all small business owners in your city and send the survey to a random sample of 1000 owners. You receive responses from only 500, and many respondents are from larger businesses with more than 50 employees.
After analysis, you realize your sample is not representative of small business owners in your city.
a) Identify the type of bias present in this sample.
b) Explain how this bias may have affected the results of the survey.
Answer
a) This sample shows nonresponse bias because only 500 of the selected owners responded. The sample is not representative of all small business owners in the city, since it leaves out the half who did not respond.
b) The attitudes of owners who did not respond may differ from those who did, even though both groups were selected. On top of that, the overrepresentation of larger businesses may pull the results away from the views of small business owners overall. As a result, the survey may not accurately reflect how small business owners feel about the economy.
Common Misconceptions
- Bias and sample size are different issues. A small sample can be unbiased, and a huge sample can still be biased if the method systematically favors some responses.
- Undercoverage and nonresponse are not the same. Undercoverage means a group has a reduced chance of being chosen at all. Nonresponse means selected people did not provide data.
- A larger sample does not fix bias. Collecting more responses through a biased method just gives you a bigger biased sample.
- Voluntary response is not just "any survey." It specifically means people chose to opt in, which overrepresents strong opinions.
- Random selection is what lets you generalize to a population. Do not mix it up with random assignment, which belongs to experiments.
Related AP Statistics Guides
Vocabulary
The following words are mentioned explicitly in the College Board Course and Exam Description for this topic.Term | Definition |
|---|---|
bias | A systematic tendency for certain responses to be favored over others in a sample, resulting in a sample that does not accurately represent the population. |
convenience sampling | A non-random sampling method where individuals are selected based on their accessibility or ease of inclusion, introducing potential bias. |
non-random sampling | Sampling methods that do not use chance to select individuals from the population, introducing potential for bias. |
nonresponse bias | Bias that occurs when individuals chosen for the sample cannot provide data or refuse to respond, and these individuals differ from those who do respond. |
question wording bias | A type of response bias caused by confusing or leading questions in a survey or data collection instrument. |
response bias | Bias that results from problems in the data gathering instrument or process, such as confusing or leading questions. |
self-reported responses | Data collected directly from individuals about their own characteristics, behaviors, or opinions, which may introduce response bias. |
undercoverage bias | Bias that occurs when part of the population has a reduced chance of being included in the sample, resulting in an unrepresentative sample. |
voluntary response bias | Bias that occurs when a sample is comprised entirely of volunteers or people who choose to participate, making the sample unrepresentative of the population. |
Frequently Asked Questions
What are the main types of sampling bias in AP Statistics?
The main types are voluntary response bias, undercoverage bias, nonresponse bias, and response bias. Convenience sampling and other non-random methods also introduce potential for bias.
What is voluntary response bias?
Voluntary response bias happens when people choose to participate, so the sample overrepresents people with strong opinions or strong interest in the topic.
What is undercoverage bias?
Undercoverage bias happens when part of the population has a reduced chance of being included, often because the sampling frame leaves out a group.
What is nonresponse bias?
Nonresponse bias happens when individuals selected for the sample cannot be reached or refuse to respond, and those nonresponders differ in important ways from the people who do respond.
What is question wording bias?
Question wording bias is a type of response bias caused by confusing, leading, or loaded wording. The wording influences how people answer instead of measuring their true views neutrally.
How do you explain bias on the AP Statistics exam?
Name the specific bias, identify who is overrepresented or underrepresented, and explain the likely direction of the error in context, such as whether the estimate is probably too high or too low.