Fisher’s Exact Test Calculator

When a 2×2 table is small, the chi-square test’s approximation runs out of road — and Fisher’s exact test takes over, computing the p-value not approximately but exactly, by enumerating every table with the same margins and weighing each with the hypergeometric distribution. Enter your four counts, and this calculator returns the probability of your table, the one-sided p-values in both directions, and the two-sided p-value by the standard probability-ordering rule — for the lady tasting tea, the clinical pilot study, and every small table in between. Where the counts are small enough that approximations apologise, this is the instrument that does not have to.

The Lady, the Tea, and the Exact Distribution

The test was born as a parlour challenge. In the 1920s Muriel Bristol claimed she could taste whether milk had been poured before or after the tea; R. A. Fisher designed the experiment — eight cups, four each way, order randomised — and computed, exactly, the probability of her getting them all right by luck: 1 in C(8,4) = 1/70 ≈ 0.014. That computation is the test. Fix both margins of the table (there were four milk-first cups; she named four), and the single free cell’s count follows the hypergeometric distribution — the mathematics of drawing without replacement. The p-value for an observed table is then the total probability of all tables, margins fixed, at least as extreme: enumerable one by one, with no appeal to large-sample approximations. Fisher’s framework (and the famous dispute with Neyman and Pearson that it fuelled) still shapes how exact inference is taught, and the tea party remains the correct first example because its sample size is one at which approximation would be a farce.

Worked Example: A Pilot Study With a Punch

The defaults are a small trial: treatment row 8 successes against 2 failures; control row 1 against 9 — margins 10 and 10, column totals 9 and 11. Enumerating every table with those margins (the free cell can run from 0 to 9), the observed table’s own probability is 0.00268. The one-sided p-value in the direction of the effect — this table plus the one more extreme (9, 1, 0, 10) — is 0.00274. The two-sided p-value, summing every table whose probability is at most the observed one’s (the standard rule this calculator implements), is 0.00548. The odds ratio, (8×9)/(2×1) = 36, is enormous; the p-values say the pattern would arise by chance about five times in a thousand under the null. Small samples, strong signal, exact verdict — the precise corner of statistics where Fisher’s construction earns its enduring fame, and where a chi-square approximation would both warn about its own validity (expected cells of 4.5) and miss the exactitude the question deserved.

Reading the Outputs Honestly

Four reporting disciplines keep exact p-values from being over-read. Two-sided is the default confession: a one-sided p is half the story chosen after seeing the direction, legitimate only when the direction was pre-registered by theory or protocol. The p-value is not the effect: the odds ratio carries the size; p carries the surprise under the null — the worked example has both large, and plenty of real tables pair a tiny p with a trivial effect (huge samples) or a large effect with a modest p (small ones, this test’s home turf). Exact is not omnipotent: conditioning on both margins is Fisher’s design choice, conservative by construction; unconditional exact tests (Barnard’s) buy power at computational and philosophical price, and the mid-p adjustment softens the discreteness — report which you used, as this page does. And zero cells are information, not errors: a zero makes the odds ratio infinite or zero (the note flags it) while the p-value machinery runs undisturbed — one of the test’s quiet advantages over odds-ratio-based approximations, which break on the same zero.

When Fisher, When Chi-Square

The working rule taught everywhere: if any expected cell count (row total × column total ÷ N) falls below about 5, chi-square’s approximation is suspect and Fisher’s exact computation is the standard recourse; above that, the two agree closely enough that chi-square’s speed historically won — an argument that evaporates when the exact computation costs a webpage a few milliseconds. Fisher’s test extends beyond 2×2 (the Fisher–Freeman–Halton generalisation enumerates larger tables), at combinatorial cost this page’s 2×2 focus avoids. For paired binary data the correct instrument is McNemar’s test — a different design (the pairs are the unit), which the hub carries as its own item; for ordered categories, trend tests spend the information Fisher’s test, blind to ordering, politely ignores. Choosing among them is study design, and the enumeration above is the exact core the alternatives all approximate or extend.

The Original Experiment, Priced at Every Score

Run the lady’s own experiment through the machine to see Fisher’s design logic in its purest form. The table for a perfect performance — she names all four milk-first cups correctly — is a=4, b=0, c=0, d=4 with margins 4 and 4: its probability is 1/C(8,4) = 1/70 ≈ 0.0143, and it is the only table that extreme, so the one-sided p-value is 0.0143 itself. Now price imperfection: three correct of four gives the table a=3, b=1, c=1, d=3, with probability C(4,3)·C(4,1)/C(8,4) = 16/70, and the one-sided p for “3 or better” is 17/70 ≈ 0.243 — nowhere near significance. The lesson Fisher built into the design is the calculator’s whole philosophy in miniature: at n = 8, only perfection convicts, and a test whose possible verdicts are enumerable can be audited completely before a single cup is poured. (Bristol, legend holds, scored perfectly.)

Exactness Has a Price: A Non-Significant Giant

Small tables cut both ways, and one more run keeps the lesson honest. Split the same total of 20 as 7–3 against 3–7 — an odds ratio of (7×7)/(3×3) ≈ 5.4, a fivefold effect by any clinical standard. Its table probability is C(10,7)·C(10,3)/C(20,10) = 14,400/184,756 ≈ 0.078, and the two-sided p-value comes out well above 0.05: not significant. Fisher’s test will not pretend otherwise. The interpretation is the power lesson of all exact small-sample work: at this size, only enormous effects are certifiable, and a non-significant result means “unproven at n = 20”, not “disproven”. The remedy is a larger table, not a looser test — the enumeration will sharpen exactly as fast as the evidence does.

Related Tools

More from the Statistics Calculators hub: the Mann-Whitney U Test Calculator for the non-parametric comparison of two groups, the live False Positive Paradox Calculator for what a positive result really means, and the Descriptive Statistics Calculator for the table’s raw material.

Frequently Asked Questions

What exactly is the p-value Fisher’s test returns?

The probability, if row and column were truly independent, of obtaining a table at least as extreme as yours among all tables sharing your margins. “Extreme” is defined by table probability for the two-sided version here (the observed table and every less-likely one are summed) and by direction for the one-sided versions. It is a statement about the data under the null hypothesis — never the probability that the null is true, a distinction the next answer guards.

My p is 0.005. Does that mean a 0.5% chance the effect is not real?

No — the persistent misreading. It means: if there were no association, tables this lopsided would appear about 0.5% of the time. The probability that the effect is real depends additionally on the effect’s prior plausibility and the study’s design — the territory of the hub’s false-positive tools. Fisher himself insisted on the modest reading; honour it, and report the odds ratio beside the p so readers see size and surprise separately.

Why does the test fix both margins? My study fixed only the rows.

Conditioning on both margins is what makes the distribution exact and distribution-free — the column totals, though not fixed by your design, are treated as given because they carry no information about association under the null (they are, in the jargon, ancillary). The price is a little conservatism; the purchase is an exact p for any sample size, which is the trade Fisher chose and this calculator implements.

Can Fisher’s test handle bigger tables?

The mathematics generalises (Fisher–Freeman–Halton) but the enumeration explodes combinatorially; practical implementations switch to network algorithms or simulation beyond modest sizes. This page is deliberately the classical 2×2 instrument. For a 2×3 or 3×3 with small counts, dedicated software implements the generalisation; for large counts at any size, chi-square’s approximation is again entirely respectable.

What if my table has a zero cell?

Run it — the hypergeometric enumeration is untroubled. The odds ratio becomes infinite (zero in a denominator cell) or zero, which the note reports plainly; many analysts add a 0.5 continuity correction for the ratio’s confidence interval, but the p-values need no such patch. A zero cell in a small table is often the most informative entry in it, as the lady’s perfect score demonstrated.

One-sided or two-sided — which should I report?

Two-sided, unless the direction was genuinely fixed before the data existed (a registered protocol, a one-directional scientific claim). The one-sided figures are printed for completeness and for those pre-committed uses. Choosing the favourable tail after inspection quietly halves the reported p and doubles the field’s false positives; the calculator shows both so the choice is at least made in daylight.