A/B Test Sample Size Calculator

Statistical significance (typically 95%) is the probability that your results aren't due to random chance. A 95% significance level means there's only a 5% chance of a false positive. Without proper sample sizes, you might implement changes that don't actually improve conversions.

MDE is the smallest improvement you want to be able to detect. Smaller MDEs require larger sample sizes. If you want to detect a 5% improvement, you need more traffic than detecting a 20% improvement. Set MDE based on what change would be worth implementing.

No—this is called "peeking" and dramatically increases false positive rates. Statistical significance can fluctuate during a test. Commit to your sample size before starting and only analyze final results. Some tools offer sequential testing methods that account for multiple looks.

Power (typically 80%) is the probability of detecting a real effect. Higher power requires larger samples but reduces false negatives. At 80% power, you have a 20% chance of missing a real improvement. For important tests, consider 90% power despite longer test duration.

More variants require proportionally more traffic. Testing 4 variants instead of 2 roughly doubles your sample size needs. For low-traffic sites, stick to A/B tests. Only run multi-variant tests if you have sufficient traffic to reach significance within 2-4 weeks.