16 {{ upvoteCount | shortNum }}
16 {{ upvoteCount | shortNum }}

When should I end my split test?

by TheWebGuy
12 replies
Hey Warriors,

I'm doing a split test on my sales page right now and was wondering at what point should I conclude and determine the results?

Do you all do it based on how many conversions you've gotten?

for example...once you hit 50, 100, 200, 300, etc conversions do you end it and decide the better option?

As it stands right now, I currently have 20 sales for 1 sales page and 17 sales for the other sales page...is this significant? Or do I need more?

Thanks.
#end #split
  • Profile picture of the author Alexa Smith
    Banned
    Originally Posted by TheWebGuy View Post

    As it stands right now, I currently have 20 sales for 1 sales page and 17 sales for the other sales page...is this significant?
    It depends on the numbers of visitors, and the statistical standard deviation in the results, and can't be answered with certainty from the information you've offered. But it's very highly likely that you need more, and that what you have so far isn't yet statistically significant, let alone conclusive.
    {{ DiscussionBoard.errors[4677939].message }}
    • Profile picture of the author TheWebGuy
      Ok great. Thanks.

      Here's the exact stats now (as reported by google optimizer):

      Page 1: 23 sales - 844 visitors (Est. conv. rate: 2.73% ± 0.8%)

      Page 2: 26 sales - 815 visitors (Est. conv. rate: 3.19% ± 0.9%)

      At what point should I make a conclusion in your opinion?

      Thanks a lot for the help! I really appreciate this!
      {{ DiscussionBoard.errors[4678026].message }}
      • Profile picture of the author Chri5123
        Originally Posted by TheWebGuy View Post

        Ok great. Thanks.

        Here's the exact stats now (as reported by google optimizer):

        Page 1: 23 sales - 844 visitors (Est. conv. rate: 2.73% ± 0.8%)

        Page 2: 26 sales - 815 visitors (Est. conv. rate: 3.19% ± 0.9%)

        At what point should I make a conclusion in your opinion?

        Thanks a lot for the help! I really appreciate this!

        Well it looks good so far!

        The main thing you are looking for is conversions - keep it running to at least 50 sales and see if the pages start to run away from each other.

        I would keep it going till 100 sales at least for each page but like I say there is no set method - hence the reason it is called testing.

        What you are looking for is as much proof that you have the best converting page you can possible have.

        You should ALWAYS keep testing but keep it going longer IMO.

        Chris
        {{ DiscussionBoard.errors[4678060].message }}
  • Profile picture of the author Chri5123
    Kudos to you FOR split testing!

    A lot of people skip this part of the process.

    It really does depend on the niche, traffic and what you are selling however I normally look for patterns.

    You need quite a few sales to notice a pattern and you will start to see which page is performing better after 100 sales.

    If you notice one of the pages sky rockets before then and is bringing in a HUGE amount of sales then change the other page to match the aspect that you believe is making the sales.

    So you are playing "catch up" with the other page.

    Then if they BOTH get sales fast you know you are onto a winner.

    No REAL timescale just use your own judgement but as a rule of thumb 100 sales is a good place to start look for patterns.

    Chris Jones
    {{ DiscussionBoard.errors[4678009].message }}
  • Profile picture of the author WebPen
    I definitely think you should wait until you get at least 1000 visitors

    They seem pretty close right now.

    But you should keep in mind that you can start a completely new test once this one is done.
    Signature
    {{ DiscussionBoard.errors[4678032].message }}
  • Profile picture of the author Warrior X
    Congrats, those sales numbers already top what most folks can get. It would add that the top things to test are the headline, the offer, and the guarantee...but you probably already knew that
    ________
    Jeremy
    Signature
    #1 In WHITEBOARD VIDEOS - Great Way To Tell Your Story!
    Available Here
    {{ DiscussionBoard.errors[4679159].message }}
    • Profile picture of the author Daniel Evans
      Agree with the 1000. This will surface significant differences you can draw a conclusion from.
      {{ DiscussionBoard.errors[4679218].message }}
      • Profile picture of the author DaveWildash
        Take a look at splittester.com.

        Although it's set up for Adwords you can still plug in your stats and it will tell you there is approximately 99% chance your current best conversion will be the eventual winner.

        In other words you can go with the results you have now.
        {{ DiscussionBoard.errors[4680144].message }}
      • Profile picture of the author TheWebGuy
        You guys are very helpful and I appreciate you taking the time to help me out.

        Ok here are the results now:

        Sales Page 1: 50 sales / 1766 visitors , est conv rate: 2.83% ± 0.6%

        Sales Page 2: 55 Sales / 1708 visitors , est conv rate: 3.22% ± 0.6%


        Is it time to say page 2 wins and rap this up and move on to a new split test?

        Thanks!
        {{ DiscussionBoard.errors[4686696].message }}
  • Profile picture of the author Rus Sells
    Actually I can answer your question with one word,

    When should I end my split testing?

    Answer: Never
    Signature
    {{ DiscussionBoard.errors[4686916].message }}
    • Profile picture of the author Digital Info Diva
      Good answer Rus!

      Maybe I missed this in the thread but, you should always use the same number of visitors when split testing.

      I start with 200; 500, and 1000. When I first started in IM, I had a hard time getting enough traffic to hit 1000 but, with more sites I found that I could rely on 200 to initially test a site, but never stopped testing!

      The same is true of keywords - I used to get away with checking keyword stats every 6 - 12 months. Now there's so many new people coming to the Internet than ever before that I have found dramatic changes in keyword use in just 6 weeks!
      {{ DiscussionBoard.errors[4687283].message }}
  • Profile picture of the author TheWebGuy
    Thanks for the replies!

    Now it's

    Page 1) 51 / 1823 visitors

    Page 2) 56 / 1759 visitors

    So I think I'm going to drop page 1 and create a new split test vs. page 2! If you think it's too soon let me know. tkx
    {{ DiscussionBoard.errors[4687302].message }}

Trending Topics