STATISTICAL BIAS

What Is Selection Bias? Why Samples Mislead

You want to know what voters think. You only poll people who answer their phones. The results are wrong. That is selection bias.

Editorial illustration of a person looking at a non-representative sample
Creator Classic statistical conceptOrigin StatisticsYear ClassicalCategory Statistics, Research

QUICK ANSWER

Here is the idea in plain English.

Selection bias is a bias introduced when the sample you study is not representative of the population you want to understand. It occurs when the selection process systematically excludes or includes certain groups. The bias is common in research, polling, and data analysis. It leads to misleading conclusions.

If you remember only a few things, remember these.

The basic move

Selection bias is simple: your sample is not representative. You study one group. You think you are studying everyone. You are wrong.

Why it matters

If you poll only people who answer their phones, you miss people who do not answer. The results are biased.

Use it deliberately

When reading a study, ask: how was the sample selected? Is it representative?

CORE IDEA

The concept in its simplest useful form.

What Does Selection Bias Mean in Simple Terms?

Selection bias is simple: your sample is not representative. You study one group. You think you are studying everyone. You are wrong.

If you poll only people who answer their phones, you miss people who do not answer. The results are biased.

The solution is to ensure your sample is representative. Random sampling is the best way.

The small mechanism underneath the big idea.

01

The Story Behind Selection Bias

Selection bias has been recognized for centuries. In the 1930s, pollsters learned the lesson the hard way. The Literary Digest polled its readers to predict the 1936 US presidential election. The poll predicted a landslide victory for Alf Landon. Franklin Roosevelt won in a landslide. The poll was wrong because it only polled people who subscribed to the magazine. Those people were not representative.

The lesson is simple: the sample must be representative. If it is not, the results are meaningless.

Today, selection bias is a foundational concept in statistics and research. It is taught in every statistics class.

02

Why Selection Bias Became Famous

Selection bias became famous because of the Literary Digest poll. The poll was a spectacular failure. It showed the importance of representative sampling.

The concept is central to statistics and research. It explains why many studies are flawed.

Today, selection bias is a foundational concept in statistics. It is taught in every statistics class.

Diagram showing the difference between a representative and non-representative sample
A diagram showing a non-representative sample and a representative sample, and the difference in results.

Where this idea shows up outside the textbook.

History

The Literary Digest poll of 1936 is the classic example. The poll was wrong because the sample was not representative.

Research

A study of health only includes people who visit doctors. The results are biased. Sick people visit doctors more often.

Polling

A poll only includes people who answer their phones. The results are biased. Some people do not answer.

Business

A company surveys its customers. It only surveys satisfied customers. The results are biased.

CONCEPT MAP

Every idea has neighbors. This is where the current concept sits in the TinyThat knowledge graph.

Current concept

Selection Bias

A sample misleads because the way it was selected is distorted.

What people often get wrong about this idea.

Selection bias is the same as survivorship bias.

No. Survivorship bias is a specific type of selection bias. Selection bias is the broader category.

Selection bias only applies to research.

No. It applies to polling, business, and everyday life. Any time you make a generalization, selection bias can occur.

You can eliminate selection bias.

You cannot eliminate it. You can only reduce it. Random sampling is the best defense.

Useful ideas become dangerous when they are stretched too far.

Criticisms and Limitations of Selection Bias

Selection bias is a powerful concept, but it has limitations. Not every sample is biased. Some samples are representative.

The concept can be overused. Not every study is flawed. Sometimes the sample is good enough.

The concept is a guide, not a rule. Use it to think about samples, not to dismiss every study.

Three simple ways to apply the idea without turning it into a slogan.

1

When reading a study, ask: how was the sample selected? Is it representative?

When reading a study, ask: how was the sample selected? Is it representative?

2

When conducting research, use random sampling

When conducting research, use random sampling. Random sampling reduces selection bias.

3

When making generalizations, ask: who am I missing? Who is not represented?

When making generalizations, ask: who am I missing? Who is not represented?

EXPLORE NEXT

The best next ideas to read after this one.

Quick answers to common questions.

What is selection bias in simple terms?

Your sample is not representative. You study one group and think you are studying everyone. You are wrong.

What is an example of selection bias?

The Literary Digest poll of 1936. The sample was not representative. The poll was wrong.

How do you avoid selection bias?

Use random sampling. Ensure your sample is representative. Ask: who am I missing?

Why is selection bias a problem?

It leads to misleading conclusions. The results are wrong because the sample is wrong.