---
title: "Two-proportion z-test — statistical significance calculator"
description: "Free calculator: is the difference between two percentages statistically significant? Two-proportion z-test, p-value, examples and common mistakes."
lang: en
url: https://fidestat.com/en/tools/two-proportion-z-test/
---

# Two-proportion z-test

Check whether the difference between two groups is real or just chance — e.g. satisfaction in two stores, or two versions of a message. Enter the “yes” answers and respondents in each group.

- [Calculate now](https://fidestat.com/en/tools/two-proportion-z-test/#alat)

When two groups give different percentages, the first question is whether the difference is real or simply came about by chance, because you only asked some of the people. The calculator above applies the two-proportion z-test: from the number of “yes” answers and respondents in each group, it tells you whether the difference is statistically significant at your chosen confidence level.

## When to use it

- **A/B tests of a message or offer**: two email versions, two texts on a QR poster, two offers.
- **Comparing locations**: satisfaction in two stores, pharmacies or gyms.
- **Comparing segments**: new and returning customers, two departments in a company.
- **Comparing periods** when the respondents are different people: this quarter and last.

The groups must be independent — one person cannot be in both.

## How it is calculated

- Share in each group: p₁ = x₁ / n₁ and p₂ = x₂ / n₂, where x is the number of “yes” answers and n the number of respondents.
- Pooled share: p̂ = (x₁ + x₂) / (n₁ + n₂).
- Test statistic: z = (p₁ − p₂) / √(p̂(1 − p̂)(1/n₁ + 1/n₂)).
- The difference is significant when |z| exceeds the critical value: 1.645 for 90%, 1.960 for 95% and 2.576 for 99%.

The test starts from the assumption that there is no difference in the population and checks how unusual it would be for a difference this large to appear by chance. The larger |z|, the less plausible that assumption becomes.

## Worked example

Email A got 120 clicks from 400 recipients (30%), email B 150 from 400 (37.5%).

1. Pooled share: (120 + 150) / 800 = 0.3375.
2. Standard error of the difference: √(0.3375 · 0.6625 · (1/400 + 1/400)) ≈ 0.0334.
3. z = (0.375 − 0.30) / 0.0334 ≈ 2.24.
4. Since 2.24 is greater than 1.96, the difference is significant at 95% confidence (p ≈ 0.025), but not at 99%.

## How large a difference needs to be

| Respondents per group | Smallest significant difference (around 50%, 95% confidence) |
| --- | --- |
| 100 | about 14 points |
| 200 | about 10 points |
| 400 | about 7 points |
| 1,000 | about 4.4 points |

Values are approximate, for shares around 50%. If you expect a five-point difference, a hundred respondents per group won't detect it — not because it isn't there, but because the data isn't enough.

## Statistical vs business significance

Statistical significance says the difference is probably not chance. It does not say the difference is large enough to justify a decision. Before the survey, decide which difference matters (for example “we switch messages if the new one brings at least five points more clicks”) and plan the number of respondents for it. After the survey, look at the size of the difference, not only whether the test passed.

## When this test is not the right choice

- The same respondents before and after a change — the groups are not independent.
- Average scores (for example 1–5 or 0–10) — averages call for a t-test.
- More than two groups at once — use a chi-square test or a stricter level for each comparison.

## Examples

- Two email versions: 120 of 400 recipients clicked version A (30%), 150 of 400 clicked version B (37.5%). z ≈ 2.24, p ≈ 0.025 — the difference is significant at 95% confidence.
- Two gyms: 30 of 100 (30%) and 38 of 100 (38%) members are satisfied. An 8-point gap looks large, but z ≈ 1.19, p ≈ 0.23 — not significant. More responses are needed.
- A supermarket compares the share of customers using a loyalty card in two stores: 210 of 500 and 255 of 500. The 9-point difference is significant even at 99% confidence (z ≈ 2.85).

## Common mistakes

- Concluding “there is no difference” when the test isn't significant but the sample is small.
- Comparing many groups pair by pair without a stricter level — at 95%, one in twenty chance differences looks significant.
- Using the test for the same respondents before and after a change.
- Treating statistical significance as business significance: with 10,000 responses per group even a one-point difference is significant, but may not matter for the decision.
- Checking results repeatedly while collecting and stopping as soon as the test turns significant. Decide the number of responses in advance.

## Frequently asked questions

### What does “statistically significant” mean?

That the difference is large enough that, if there were no difference in the population, it would rarely appear by chance — less often than your chosen level allows (for example in under 5% of cases at 95%). It does not mean the difference is large or important for the business.

### When is the two-proportion z-test used?

When you compare the share of the same answer (“yes”, “satisfied”, “bought”) in two independent groups — two stores, two versions of a message, two periods with different respondents.

### How many respondents do I need in each group?

The test is reliable when each group has at least about ten “yes” and ten “no” answers. To detect a small difference (a few points) you need several hundred respondents per group.

### Can I use it to compare the same people before and after?

No. If the same people answered twice, the groups are not independent and you need a different test (for example McNemar's test). This calculator is for two different groups of respondents.

### What if the difference is not significant?

That doesn't prove the groups are the same — only that this data isn't enough to confirm a difference. A difference may exist, but the sample is too small to see it reliably.

### Can I compare more than two groups at once?

If you compare many pairs, some differences will look significant by chance. For several groups, use a test for multiple proportions (chi-square) or a stricter level for each individual comparison.

- [Start for free](https://app.fidestat.com/registracija)
