How long do I split test
 

How long should I keep a split test running before the results become somewhat reliable?

100 clicks to each page? more? less?

Re: How long do I split test
 

As I see it... the math behind A/B testing is Bells Theorem. Mathematically speaking the theorem falls into its "best" stride right at 1000 views per side of your test. All of the numbers just fall into that sweet spot where a definitive yes or no A is better than B answer can be given.

That's my answer and Im sticking to it! LOL

Re: How long do I split test
 

Wow. Never heard of using Bell's theorem before. That's cool! Usually I just go 100 clicks for each side.

Re: How long do I split test
 

Oh jeeze really? If you look at A/B Significance Test you can insert your numbers. and it is using bells theorem to figure out the overall significance of you test. If the numbers are to close, on the far right column last 2 lines it will tell you basically yes trust the results, or no, not enough data.

Notice when you go to the page it is set for? 1000 LOL

Here is another link that has many different calculators: http://www.evanmiller.org/ab-testing/
including the top left one that helps you determine the sample size needed to run a decently accurate test.

Re: How long do I split test
 

Thanks! Those are some great resources :D

Re: How long do I split test
 

Ideally, you will need at least 1000 unique visitors per week to the page that you want to test - otherwise it will take too long to get a test result (many weeks).

And its not just about traffic too - you need to get enough 'conversions' (sales, leads, signups, subscribers etc) to get a valid test result. Ideally you will need at least 25 conversions per test variation (50 conversions per week in total).

You should also take a full week to do the test - as this will help take into account variances in traffic and differences in traffic by day (weekend versus weekday etc), and this also allows enough time for fluctuations in test results to settle down.

And if you don't have enough traffic to do testing, don't give up - there are still some good ways to test with lower traffic - I created a guide on A/B testing with low traffic to help with this.

Hope that helps! Good luck with your testing...

Re: How long do I split test
 

Split test until you have enough data and then just scale the best campaign.

Best regards,


Affiliate Manager at CPABeyond

Re: How long do I split test
 

I think this may be my favourite reply ever.

How long do I split test for?

"until you have enough data"

ahhhh thanks, how did I miss that! :P

Re: How long do I split test
 

Test when you see clear winner. Its depend on your existing traffic. 100 visitors/clicks maybe not enough if you have 10k-20k per days. :) Just be practical on it.

Re: How long do I split test
 

Im going to be THAT guy. there is nothing practical about A / B testing. it is what it is, a scientific method of comparison. I can not even begin to think how many times there was a "clear winner" early in testing and then once the full testing ran its course that "winner" ended up being the loser.

So I keep pretty good notation on all of the tests I run. I just went and looked over the past 300 test that I have run. 68% of these tests had an early leader. out of those 204 tests with early leaders, 172 of those ended up being losers.

that means to bail on the process early would have had me place the least converting option 84% of the time. Jeeze... that's bad! LOL

Re: How long do I split test
 

After I have at least 300 unique views daily, I do split testing for a month.

This has generally worked out for me for my own experiences

Re: How long do I split test
 

Right now you are split testing at 4500 impressions per side. Not a bad thing don't get me wrong. But if you mathematically only really need say 1000 per test variable, you could effectively be running 4.5 Different tests per month. Or one heck of a A B C D E F G H test LOL

Re: How long do I split test
 

I think that this is an endless game You always will compete and weed out the bad ones. You stop when you are in the range of 30% conversion rate.

Re: How long do I split test
 

The more the better! .. and as some recommended, at least 1000 clicks. Then track every single thing and look at the numbers to see what's working best!

I hope this helps!

Keep up the great work!

I wish you the best!

Cheers! :D