A/B test your offer
This article shows you how to split viewers between two to four versions of a path, read which one sells more, and keep the winner.
Available on: Pro, Business and Agency plans.
How it works
An A/B split step sends each viewer to one version, by the shares you choose (for example 50% to A and 50% to B). Each version can lead to a different video, question or offer. The statistics then compare how many viewers of each version bought.
- Each viewer always sees the same version, also when they come back on the same device.
- Nothing identifies the viewer. The version is drawn on their device and remembered there.
Add a split
- In the funnel editor, click A/B split in the Add bar, or choose New A/B split in a step's Goes to menu.
- Give each version a name if you like, for example "Offer A: $197". Only you see it.
- Set each version's share in percent. The shares must add up to 100% ("The shares add up to 100%."). Share evenly does it for you.
- Choose where each version goes in Version A goes to, Version B goes to and so on.
- Click Add version for a third or fourth version.
To compare sales, add the line to your thank-you page. Without it, versions are compared by clicks on the offer button. See Count sales on your thank-you page.
Read the result
Click See which version wins in the split's panel, or open Statistics. Each test has its own section, A/B test: and the split's name, with a table of Version, Viewers, Clicked to buy and Bought, and one of these verdicts:
| Verdict | What it means |
|---|---|
| B is the winner | At a weekly check (from day 14 of the test), every version had at least 100 viewers and 10 sales, and PlayerMagick was more than 99% sure B sells best. |
| No clear winner yet | Enough viewers and sales, and the leader with its chance, for example "B leads, 92% likely best", but no weekly check has named a winner yet. The section says when the next check is. |
| About the same | From day 28, a weekly check found the versions so close that keeping either costs almost nothing. |
| No clear winner | 12 weeks without a winner: the versions are too close to call with this much traffic. |
| Too early to tell | A version has fewer than 100 viewers or 10 sales. "Each version needs at least 100 viewers and 10 sales before we name a winner." |
Under the table, a sentence puts the difference in plain words, for example "B sells to about 50% more people than A."
What "92% likely best" means, and why we check once a week
It is the chance, given the viewers and sales so far, that the leading version really sells better than the others, and is not just lucky. PlayerMagick works it out with a Bayesian method, for two to four versions. The figure is rounded down and never shows more than 99%.
A running test's figure moves every day. If the section named a winner whenever the leader looked 95% sure, a test of two identical versions would name one most of the time, just because it is looked at again and again. So PlayerMagick names a winner only at a weekly check, from day 14 (days 14, 21, 28 and so on), when it is more than 99% sure. Between checks you see the leader and its chance, never a winner. Each check counts the whole days before it.
- Small tests say "Too early to tell" on purpose. With a handful of sales, chance can easily make the weaker version look better.
- Tests you already ended (a version kept) keep the result they had.
- The verdict covers the whole current test, from the moment it started, whatever period you choose at the top of the page.
- Sales are as reported by your thank-you page. Under the table, "Compared by: bought, as reported by your thank-you page." says so. If your thank-you page can be reached without paying, the comparison can be off. See What counts and what doesn't.
- When no sale was counted during the test, the versions are compared by clicks: "No sales were counted during this test yet, so the versions are compared by clicks on the offer button."
Keep the winner
- When a weekly check names a winner, click Keep B for everyone in the test's section. When the versions do about the same, or after 12 weeks without a winner, there is a button for each version.
- Confirm with Keep B: "Everyone who reaches this split gets that version from now on, also viewers who saw another one before. The test results stay here."
The split stays in your funnel with 100% for that version. The panel says, for example, "Everyone sees version B. Change the shares to start a new test." Click Keep testing instead to leave the test running.
Before there is a winner you can still decide for yourself: click Keep a version anyway, choose the version, read the warning and click Keep B anyway. Early leads are often luck, so do this only when you have another reason to choose.
Change a running test
Changing the shares or the versions of a live test restarts it. The editor asks first ("This restarts the A/B test"):
- Restart the test: viewers from now on are split by the new shares, and the statistics start a new test. The results so far stay in the statistics.
- Keep the test running: the split goes back to its shares and versions as they were, and the test goes on.
Renaming a version does not restart the test.
Troubleshooting
- "The shares must add up to 100%." Change the shares, or click Share evenly.
- The test says "Too early to tell" for weeks. Each version needs at least 100 viewers and 10 sales. Send more traffic, or test a bigger difference.
- B has led at 97% for days, but there is no winner. A winner is named only at a weekly check, when B is more than 99% likely to be best. Leads at 95–97% often fade; wait for the next check, shown under the table.
- I see the same version every time I test. That is expected: each device keeps its version. Use a private window, or another device, to see another one.
- The numbers of my test reset. You changed the shares or versions and chose Restart the test. The earlier test is still listed in the statistics.