Statistical Power, Explained with Worked Numbers

Last updated October 30, 2026·4 min read

Power is the probability that a study detects an effect of a stated size, if the effect is really there. It is a property of a design aimed at a specific effect, not a property of a result. A study with power of 0.80 aimed at a medium effect runs the experiment ten times in imagination and detects the effect about eight times. The other two times it reports nothing, not because the effect was absent, but because the design could not see it reliably.

The definition that matters

Four quantities lock together: effect size, significance level, sample size, and power. Fix three and the fourth follows. Everything in this guide is a consequence of that lock.

One comparison, three sample sizes

Take a two-group comparison of means, alpha of 0.05 two-sided, aimed at a medium standardized effect, d of 0.5:

• At 50 per group, power is 0.6969. The study misses a real medium effect about three times in ten.
• At 64 per group, power is 0.8015, the conventional target.
• At 100 per group, power is about 0.94, and the study misses a real medium effect about one time in seventeen.

Read the middle row as the price of the convention. Eighty percent power is not a law of nature; it is a settled trade between the cost of cases and the cost of missing effects. A pilot screening many candidate effects might accept 0.70 and a fast answer. A confirmatory study that a program decision rides on might buy 0.90 or 0.95. The error is not choosing a different trade. The error is never making the trade on purpose.

What underpowered studies actually cost

The obvious cost is the miss: a real effect, an inconclusive study, a field budget spent. The subtler cost is what underpowered studies do when they succeed. At low power, the only samples that reach significance are the ones where noise happened to inflate the observed effect. Published effects from underpowered designs therefore run systematically large, a pattern named the winner curse, and replications aimed at those inflated sizes then appear to fail. Low power does not just lose effects; it distorts the ones it finds.

A third cost lands on the reader. A nonsignificant result from a high-power study is informative: the effect, if present, is smaller than the size the study was built to see. The same sentence from a low-power study says almost nothing, because the study could not have seen the effect either way. Power is what gives a null result its meaning.

The four levers, and what each one buys

• Sample size buys power at a square-root rate on precision: the standard error shrinks with the square root of n, so doubling precision costs four times the cases.
• Effect size is partly nature and partly design. Sharper measurement, a stronger contrast between conditions, and a homogeneous population all raise the standardized effect without touching n.
• Alpha trades one error for another. Raising alpha from 0.05 to 0.10 buys power by accepting more false alarms. Sometimes that trade is defensible in screening work; it is never free, and it must be chosen before the data arrive.
• Design is the quiet lever. Pairing, blocking, covariate adjustment, and CUPED-style variance reduction in experiments all remove noise variance from the comparison, which raises power at fixed n. A paired design that halves the effective variance does the work of doubling the sample.

Minimum detectable effect: the planning number executives understand

When the sample is fixed by budget or population, turn the calculation around and report the minimum detectable effect: the smallest effect the fixed design detects with the stated power. It converts an abstract power figure into a decision sentence. This design, with 400 completed responses split evenly, detects a difference of about 0.28 standard deviations at power of 0.80. If an effect of 0.28 would not change the decision anyway, the study as funded cannot inform the decision, and that is better known before fieldwork than after.

Post hoc power: the number to decline

Computing power after the fact, at the observed effect size, adds no information: it is a restatement of the p value in another unit, low exactly when p is high. When a study ends nonsignificant, the useful statements are the confidence interval, which shows the effects the data remain compatible with, and the minimum detectable effect the design carried. Both are honest. Observed power is neither, and no result section improves by including it.

Run the numbers

The Sample Size and Power Planner in Analysis Studio solves for any one of the four quantities, and draws the power curve so the trade is visible rather than tabular:

Module 7: Sample Size and Power PlannerOpen in Analysis Studio →

Sources and method notes

Power for t tests is computed from the noncentral t distribution; the figures in this guide were recomputed at drafting (50 per group 0.6969; 64 per group 0.8015, both at d of 0.5) and are re-verified at build.