Split Testing... How many conversions till you decide?

7 replies
• |
Hey guys,
When it comes to split testing, how many conversions do you wait till you have received until you decide to stop the test and go with the higher converting page?
• Short answer: depends on the niche.

I tend to have a fair idea of an optimum EPC for any given product or service promoted on any given traffic source. I know, for instance, traffic source A (given how I'm securing that traffic) should be earning me \$0.xx per click. I'll split test until I'm close, on the mark, or surpass it. The same EPCs don't last forever, though, so it won't be long before I'm repeating the process. It's a business of constant testing and experimentation.

Tom
Signature

Learn one way I earn money: I give away free stuff.

{{ DiscussionBoard.errors[9728802].message }}
• Banned
 Originally Posted by charliemwallace When it comes to split testing, how many conversions do you wait till you have received until you decide to stop the test and go with the higher converting page?
It depends on the relative success-rates.

Achieving "statistical significance" of the kind where you can confidently say "the chances of this having happened by chance alone are less than 5%" (or "less than 1%") is a statistical function, dependant on the variability/standard deviation.

It's not trivially simple, but in general, the wider the disparity in "strike-rate" is, between the two things being measured, the smaller the sample size needed to assume that.

Examples:
• If one page succeeds 50% of the time and the other 10%, a sample-size of 100 is very big
• If one page succeeds 50% of the time and the other 48%, even a sample size of 1,000 is too small
• If one sales page converts at 2% and the other at 1% (only half as good), a sample size of 100 is very big
• If one sales page converts at 2% and the other at 1.9%, even a sample size of 1,000 is too small

.
{{ DiscussionBoard.errors[9728806].message }}
• this tool might help
:::SplitTester.com:::
{{ DiscussionBoard.errors[9728828].message }}
•  Originally Posted by Alexa Smith If one sales page converts at 2% and the other at 1% (only half as good), a sample size of 100 is very big.
Wow. I find it hard to understand it, but the split-testing calculator provided in the post above confirms your assertion. I just can't understand how such a small sample size and so few occurrences can give a statistical confidence of 99%.

Maybe that's why they won't let me design space rockets.
{{ DiscussionBoard.errors[9728974].message }}
• Banned
 Originally Posted by Lucian Lada I just can't understand how such a small sample size and so few occurrences can give a statistical confidence of 99%.
It's basically because the difference between 2% conversion and 1% conversion is absolutely huge: it's 100% (whereas the difference between 50% and say 48% is tiny): you have to start from "what you're comparing it with" and take one of the percentages as the figure with which you're comparing the other as "a percentage of a percentage". Funny subject, statistics.

 Originally Posted by Lucian Lada Maybe that's why they won't let me design space rockets.
Me also. They're probably right really - would be no good for the European space program having you and me designing the rockets: we'd probably find a way to screw up the whole thing for them. We'll have to stick to brain surgery, instead.

.
{{ DiscussionBoard.errors[9728997].message }}
•  Originally Posted by Alexa Smith We'll have to stick to brain surgery, instead.
I don't think brain surgery fits us, either. For brain surgery, precision counts and you can easily lose balance on 17-cm heels, and I will most certainly fart and infect the patient's brain.

We're useless.
{{ DiscussionBoard.errors[9729009].message }}
• Completely depends on the niche. If I'm getting leads for a dentist, 2 a day might be an astronomical success. If I'm getting opt ins for a free video in a target niche, 50 a day might not be enough. Use the split tester tool and your own goals/judgement.
Signature
FREE 500 word articles, PM me for yours!
(4 days only!)
{{ DiscussionBoard.errors[9728998].message }}

• I'm done..

ATTEK

Hi, In a total of like 7 years, i made like 100\$ online. I check daily for methods on forums. The methods i try just doesn't feel like they work. ... [read more]

• In-House vs Marketing Agency - which one is better?

WF Will

Hey Warriors! I've often come across this question/debate "Should I hire an in-house marketer or work with a marketing agency?" within my network of business owners and entrepreneurs. Curious as ... [read more]

• Why Aren't There As Many Women In IM ...

Jonathan 2.0

Just curious. : )

• What email platform can I upload and old but large email list to?

Robert X

Hello everyone, I have a slightly old email list that I kept in a file. I'm looking to upload it and revive it but it is several thousand strong. Will ... [read more]

• Same Content - Multiple Sites I Own

rmnnet in SEO

I currently run a job board website, with a few blog articles that rank well. I want to launch another job board in a similar market, and I have articles ... [read more]