Effect Sizes: Reporting How Much, Not Just Whether

Last updated October 30, 2026·4 min read

A p value answers whether a result is distinguishable from noise at the sample size in hand. It scales with n, which means it confounds the size of the effect with the size of the study. Ten thousand cases make a trivial difference significant; forty cases make a substantial one ambiguous. Effect sizes answer the question the reader actually brought: how much. They are stated on scales that do not grow with the sample, and they carry intervals that show the precision honestly. Every module in the Studio reports them for this reason.

The standardized mean difference: Cohen d and Hedges g

The difference between two means, divided by a standard deviation. In the t test worked example used across this program, group means of 25.0 and 20.0 on a pooled within-group standard deviation of 1.94 give d of 2.58, a very large standardized difference, which is why five cases per group sufficed there. Hedges g applies a small-sample correction to d and is the safer report at small n.

The conventional bands, 0.2 small, 0.5 medium, 0.8 large, are Cohen own rough calibration for behavioral research, offered by him with reluctance and since hardened into law by citation. A d of 0.2 on a cheap intervention applied to a million people can matter more than a d of 0.8 on an expensive one applied to fifty. Report the band if your field expects it, and argue the subject in the same paragraph.

The odds ratio, and its honest companions

For binary outcomes, the odds ratio multiplies odds. The logistic worked example: treated odds 1.5 against control odds 0.43 give an odds ratio of 3.5, while the rates move from 30 to 60 percent, a relative risk of 2.0 and a risk difference of thirty points. All three describe the same table. The odds ratio is the largest of the three numbers and the one most often misread as a probability ratio, so a careful report carries the risk difference or the rates beside it. Average marginal effects translate the whole model into probability points, which is the scale decisions are made in.

Association in tables: Cramér V and phi

Cramér V scales chi-square association onto zero to one. The crosstab worked example, a two by two table with chi-square of 3.636 on forty cases, gives V of 0.302, where phi is the same number for a two by two table. Bands depend on the table dimensions; for two by two, 0.1, 0.3, and 0.5 mark small, medium, and large by convention. V earns its keep in comparisons across tables of different sizes, where raw chi-square values cannot be compared at all.

Variance explained: eta squared, omega squared, R squared

ANOVA effect sizes report the share of outcome variance associated with the grouping. Eta squared is the direct sample share and runs optimistic; omega squared corrects the bias and is the better report, at the price of occasionally landing at zero or below in weak designs, where it is truncated and noted. R squared in regression is the same idea at model level, and key driver analysis decomposes it into Shapley shares when the question turns from how much the model explains to which predictors do the explaining.

Ratios over time: the hazard ratio

Survival models report hazard ratios: the ratio of instantaneous event rates between groups. A hazard ratio of two means the exposed group events occur at twice the rate at every point in follow-up, under the proportional hazards assumption the model checks. It is not a ratio of survival probabilities at any single time, and translating it into one requires the baseline curve, which a complete report shows.

Intervals, always

An effect size without an interval is a point pretending to be a fact. The Studio attaches intervals by the standard routes: noncentral distributions for d and eta-type measures, profile or delta methods in regression families, bootstrap for importance shares. The interval is also the bridge back to significance for readers who want it: an interval that excludes the null value and a significant test are the same statement in two dialects, with the interval carrying the extra information about magnitude.

A minimal reporting standard

For every headline result: the effect size on its natural scale first (points, percent, dollars), the standardized size where the field uses one, its interval, and the sample it came from. That set lets a reader judge size, precision, and relevance without trusting the author summary, which is the entire point of reporting numbers at all.

Compute them

Every Analysis Studio module reports its effect sizes with intervals beside the tests:

Analysis StudioOpen Analysis Studio →

Sources and method notes

Worked figures are the shared examples from the calculator pages of this program (t test, logistic, crosstab), recomputed for this guide and re-verified at build. Band conventions follow Cohen (1988), cited as conventions throughout.