Back to library

📊Understand P-Hacking and Why It Wrecks Studies

See p-hacking with your own eyes — slice random data twenty ways, watch a 'significant' finding fall out of pure noise, then draft the pre-registration outline that would have stopped it.

Applied14 drops~2-week path · 5–8 min/daymath

Phase 1How Innocent Choices Manufacture Significance

Watch noise become significance through innocent slicing.

4 drops

  1. A coin flip looks significant if you flip it enough ways

    7 min

    A coin flip looks significant if you flip it enough ways

  2. Every analysis decision is a fork, and forks multiply

    7 min

    Every analysis decision is a fork, and forks multiply

  3. Slice your noise twenty ways and watch significance appear

    8 min

    Slice your noise twenty ways and watch significance appear

  4. Most p-hacking is exploration that forgot to label itself

    7 min

    Most p-hacking is exploration that forgot to label itself

Phase 2Spotting P-Hacking in the Wild

Spot the red flags in real-looking study descriptions.

5 drops

  1. If the outcome changed between protocol and paper, the result is exploratory

    7 min

    If the outcome changed between protocol and paper, the result is exploratory

  2. Stopping the moment p < 0.05 is its own kind of cheating

    7 min

    Stopping the moment p < 0.05 is its own kind of cheating

  3. Hypothesizing after the results are known

    7 min

    Hypothesizing after the results are known

  4. If the effect only shows up in left-handed Tuesday-born subjects, it isn't real

    8 min

    If the effect only shows up in left-handed Tuesday-born subjects, it isn't real

  5. Real p-values cluster smoothly. P-hacked ones pile up just under 0.05

    7 min

    Real p-values cluster smoothly. P-hacked ones pile up just under 0.05

Phase 3From P-Hacking to the Replication Crisis

Trace p-hacking to the replication crisis and its fixes.

4 drops

  1. A scenario — only 39 of 100 famous psychology findings replicated

    7 min

    A scenario — only 39 of 100 famous psychology findings replicated

  2. A scenario — your colleague's exciting finding hits your desk

    8 min

    A scenario — your colleague's exciting finding hits your desk

  3. A scenario — you're peer-reviewing for a journal that does registered reports

    8 min

    A scenario — you're peer-reviewing for a journal that does registered reports

  4. A scenario — your team picks one statistical reform to adopt this year

    8 min

    A scenario — your team picks one statistical reform to adopt this year

Phase 4Draft a Pre-Registration That Holds Up

Draft a pre-registration that closes every garden path.

1 drop

  1. Draft a pre-registration outline for a fake study

    8 min

    Draft a pre-registration outline for a fake study

Frequently asked questions

What is p-hacking and why is it considered a problem?
This is covered in the “Understand P-Hacking and Why It Wrecks Studies” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
How is p-hacking different from outright fraud?
This is covered in the “Understand P-Hacking and Why It Wrecks Studies” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
Why does the 0.05 significance threshold encourage p-hacking?
This is covered in the “Understand P-Hacking and Why It Wrecks Studies” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
What is pre-registration and how does it prevent p-hacking?
This is covered in the “Understand P-Hacking and Why It Wrecks Studies” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.
Can a study be p-hacked without the researcher realizing it?
This is covered in the “Understand P-Hacking and Why It Wrecks Studies” learning path. Start with daily 5-minute micro-lessons that build from fundamentals to hands-on application.

Related paths

∫Understand the Fundamental Theorem of Calculus

Build the Fundamental Theorem of Calculus from accumulation first — feel why differentiation undoes integration before you ever see the formal statement. You'll finish by pulling a real dataset and verifying the theorem numerically with your own code.

Applied~2-week path · 5-8 min/day

🧮Understand Vector Spaces: The Rulebook Behind Linear Algebra

See why polynomials, pixel images, and 3D arrows all live in the same mathematical world — then design your own vector space of polynomials up to degree two and compute its dimension from scratch.

Foundations~2-week path · 5-8 min/day

🔗Understand Correlation vs Causation With Real Examples

Go beyond 'correlation is not causation' — learn to spot confounders, recognize when randomization or natural experiments are doing the real work, and leave with a 30-second rule of thumb you'll actually use on tomorrow's headline.

Foundations~2-week path · 5-8 min/day

🔵Learn Set Theory Basics: The Language Every Math Class Assumes

Stop squinting at ∪, ∩, and ∁ — shade Venn diagrams first, then translate them into clean notation, until you can model anything from music genres to probability events as sets you can actually picture.

Foundations~2-week path · 5-8 min/day