Probability and Statistical Biases

Multiple comparisons problem

Testing many relationships makes false positives more likely.

What Is Multiple comparisons problem?

Multiple comparisons problem is a thinking trap where testing many relationships makes false positives more likely.

How It Tricks You

It can make noisy, incomplete, or poorly framed data feel more meaningful than it is.

Real-World Example

One surprising link appears after testing hundreds of possible links.

Seen Online As

• The fast interpretation is doing more work than the evidence.
• The claim feels obvious before the check question is asked.
• A shortcut is making the judgment feel more certain than it is.

What To Ask Instead

How many tests were run?

Related Thinking Traps

• Anchoring bias
• Confirmation bias
• Framing effect
• Loss aversion

Common Situations

• Thinking Traps in Relationships
• Thinking Traps at Work
• Thinking Traps Online

Quick FAQ

What is Multiple comparisons problem?

Testing many relationships makes false positives more likely.

What is an example of Multiple comparisons problem?

One surprising link appears after testing hundreds of possible links.

How do I spot Multiple comparisons problem?

How many tests were run?