This two-proportion z-test calculator compares two independent sample proportions (the A/B test classic) with the pooled z statistic, p-value and a confidence interval for the difference.

–

The Pooled z: A/B Testing's Workhorse

One sample against a target is the One-Proportion Z-Test; this page is two samples against each other. Part of our Statistics Calculators collection.

Worked Example (Verified)

  • 54/100 vs 40/100 (defaults): pooled p = 94/200 = 0.47, SE = 0.0706, z = 1.983, two-tailed p = 0.0473 — just significant; the 95% CI for the difference runs 0.3 to 27.7 points, a humbling reminder that “significant” and “precise” are different adjectives
  • Scale effect: the same proportions at n = 1,000 per group give z = 6.27, p < 0.0001 — significance is effect × √n, which is why giant platforms “prove” 0.1-point lifts
  • χ² identity check: z² = 3.93 = the chi-square statistic for this 2×2 table at df = 1 — two doors, one room

A/B Testing Discipline

  • Fix n (or the horizon) before launch; optional stopping manufactures significance from noise.
  • Guard the family: 20 variants at α = 0.05 expect one false winner — see the Bonferroni Correction Calculator.
  • A significant 0.2-point lift may still lose money after implementation cost — report the CI, not just the star.
  • Independent groups only: paired before/after data needs McNemar's test, not this one.

Related Tools

Frequently Asked Questions

How do you compare two proportions?

Pool the successes for the test SE: p̄ = (x₁+x₂)/(n₁+n₂), SE = √(p̄(1−p̄)(1/n₁+1/n₂)), z = (p̂₁−p̂₂)/SE. With 54/100 vs 40/100: z = 1.983, p = 0.0473. This calculator adds the (unpooled) confidence interval for the difference.

Why pool for the test but not the interval?

Under H₀ both groups share one true proportion, so the test's yardstick pools them. The interval estimates the difference without assuming H₀, so each group keeps its own p̂ in the SE. The asymmetry is deliberate and standard.

How large a sample does an A/B test need?

It depends on baseline rate and the smallest lift worth detecting — detecting 45% vs 40% at 80% power needs ≈ 1,530 per group; 41% vs 40% needs ≈ 37,000. Run those numbers before launch (Sample Size Calculator) instead of peeking until significance wanders in.

Is this the same as a chi-square test?

For a 2×2 table, yes mathematically: χ²(df 1) = z² and the p-values match. The z version adds direction and a natural confidence interval for the difference; chi-square generalizes to bigger tables (see the Chi-Square Calculator).

What if the counts are small?

When any expected cell drops below ~5–10, the normal approximation wobbles — Fisher's exact test is the small-sample referee. This page's defaults (n = 100 per group) sit comfortably in z territory.

Is this two-proportion z-test calculator free?

Yes — free, instant, no sign-up.