Plan your A/B test, then
find out if it won.
Step 1 tells you how many weeks your test needs before you start. Step 2 reads the results when it’s done: whether the difference was real, and what it’s worth in revenue.
Step 1: Before the test
Plan your test
Find out how many weeks your test needs to run before you can trust what it tells you.
Your page
From your analytics, for the page you want to test.
Sessions or users in an average week.
Orders, sign-ups or whatever the test is for.
2 to 11, control included. 2 means the control and one challenger.
Test settings
The defaults suit most tests.
Worked out from your traffic and conversions. You can change it.
80% is the usual choice.
95% is the usual choice.
One-sided asks only “did it win?” Use two-sided when a drop matters as much as a lift.
Step 2: After the test
Read your results
Enter what your test recorded to see whether the new version really won, and what it’s worth. Already ran a test? You can start here.
Your test results
Visitors and conversions for each variation, from your testing tool.
Test settings
Use the settings you planned the test with.
95% is the usual choice.
One-sided asks only “did it win?” Use two-sided when a drop matters as much as a lift.
