10 {{ upvoteCount | shortNum }}
10 {{ upvoteCount | shortNum }}

Split Testing... How many conversions till you decide?

by charliemwallace
7 replies
Hey guys,
When it comes to split testing, how many conversions do you wait till you have received until you decide to stop the test and go with the higher converting page?
#conversions #decide #split #till
  • Profile picture of the author Tom Addams
    Short answer: depends on the niche.

    I tend to have a fair idea of an optimum EPC for any given product or service promoted on any given traffic source. I know, for instance, traffic source A (given how I'm securing that traffic) should be earning me $0.xx per click. I'll split test until I'm close, on the mark, or surpass it. The same EPCs don't last forever, though, so it won't be long before I'm repeating the process. It's a business of constant testing and experimentation.

    Tom
    Signature

    I Coach: Learn More | My Latest WF Thread: Dead Domains/ Passive Traffic

    {{ DiscussionBoard.errors[9728802].message }}
  • Profile picture of the author Alexa Smith
    Banned
    Originally Posted by charliemwallace View Post

    When it comes to split testing, how many conversions do you wait till you have received until you decide to stop the test and go with the higher converting page?
    It depends on the relative success-rates.

    Achieving "statistical significance" of the kind where you can confidently say "the chances of this having happened by chance alone are less than 5%" (or "less than 1%") is a statistical function, dependant on the variability/standard deviation.

    It's not trivially simple, but in general, the wider the disparity in "strike-rate" is, between the two things being measured, the smaller the sample size needed to assume that.

    Examples:
    • If one page succeeds 50% of the time and the other 10%, a sample-size of 100 is very big
    • If one page succeeds 50% of the time and the other 48%, even a sample size of 1,000 is too small
    • If one sales page converts at 2% and the other at 1% (only half as good), a sample size of 100 is very big
    • If one sales page converts at 2% and the other at 1.9%, even a sample size of 1,000 is too small

    .
    {{ DiscussionBoard.errors[9728806].message }}
    • Profile picture of the author lexenz
      this tool might help
      :::SplitTester.com:::
      {{ DiscussionBoard.errors[9728828].message }}
    • Profile picture of the author Lucian Lada
      Originally Posted by Alexa Smith View Post

      If one sales page converts at 2% and the other at 1% (only half as good), a sample size of 100 is very big.
      Wow. I find it hard to understand it, but the split-testing calculator provided in the post above confirms your assertion. I just can't understand how such a small sample size and so few occurrences can give a statistical confidence of 99%.

      Maybe that's why they won't let me design space rockets.
      Signature
      Professional autoresponder email writer.
      I also write premium quality articles, e-books, etc.
      {{ DiscussionBoard.errors[9728974].message }}
      • Profile picture of the author Alexa Smith
        Banned
        Originally Posted by Lucian Lada View Post

        I just can't understand how such a small sample size and so few occurrences can give a statistical confidence of 99%.
        It's basically because the difference between 2% conversion and 1% conversion is absolutely huge: it's 100% (whereas the difference between 50% and say 48% is tiny): you have to start from "what you're comparing it with" and take one of the percentages as the figure with which you're comparing the other as "a percentage of a percentage". Funny subject, statistics.

        Originally Posted by Lucian Lada View Post

        Maybe that's why they won't let me design space rockets.
        Me also. They're probably right really - would be no good for the European space program having you and me designing the rockets: we'd probably find a way to screw up the whole thing for them. We'll have to stick to brain surgery, instead.


        .
        {{ DiscussionBoard.errors[9728997].message }}
  • Profile picture of the author ArticlePrince
    Completely depends on the niche. If I'm getting leads for a dentist, 2 a day might be an astronomical success. If I'm getting opt ins for a free video in a target niche, 50 a day might not be enough. Use the split tester tool and your own goals/judgement.
    Signature
    FREE 500 word articles, PM me for yours!
    (4 days only!)
    {{ DiscussionBoard.errors[9728998].message }}

Trending Topics