| | ||||||||
| |||||||||
 |
11-06-2009, 07:35 PM
| #1 |
| ArticleToolChest.com War Room Member  Join Date: Apr 2005 Location: Boynton Beach, Florida
Posts: 565
Thanks: 3
Thanked 111 Times in 17 Posts
|  Significant Split Test Results?
I have a question for those of you that regularly run split tests. I was hoping for something that would jump out and slap me in the face, but the results seem almost insignificant. After about 200 subscribers, running an A/B split test, one page resulted in 94 subscribers, the other is at 104. Is that enough to consider it a significant run and start testing the better page against another? Thanks |
| | |
|  |
|
11-06-2009, 10:16 PM
| #2 |
| Enlightened Warrior War Room Member  Join Date: Nov 2007 Location: New York, USA.
Posts: 536
Thanks: 12
Thanked 95 Times in 70 Posts
| Re: Significant Split Test Results?
No - those numbers aren't enough to be statistically significant. Are the pages very similar? If feasible, you'll want to continue testing until you get statistically significant results. If pressed for time, just make a decision and go with the better producing one.
|
| | |
|
| |
|
11-06-2009, 10:20 PM
| #3 | |
| Copywriter and Marketer War Room Member  Join Date: Apr 2005 Location: Philly Suburbs, USA
Posts: 2,783
Thanks: 784
Thanked 694 Times in 371 Posts
| Re: Significant Split Test Results? Quote:
The more important question is, why isn't the testing software you are using telling you if it's statistically significant or not? That's a pretty common feature in many commercial testing scripts. | |
| | ||
|
| |
|
11-06-2009, 10:29 PM
| #4 |
| Senior Warrior Member War Room Member  Join Date: Jun 2003 Location: USA
Posts: 2,442
Blog Entries: 1 Thanks: 7
Thanked 97 Times in 75 Posts
| Re: Significant Split Test Results?
I would use the better version and use it as the "control" and split test another version against the current winning version. Frank Bruno |
| | |
|
| |
|
11-06-2009, 10:39 PM
| #5 |
| Dr. Michael Quadlander War Room Member  Join Date: Aug 2007 Location: Texas...And the place I call home, Is just south of Austin, in the Hill Country north of San Antone!
Posts: 33
Thanks: 9
Thanked 7 Times in 5 Posts
| Re: Significant Split Test Results?
Yes JaggyJay... TimS should use much different copy angles to test, not just something slightly different. Add video, or short vs. long copy... not just changing a word here or there. Although simple headline changes can often make a huge difference. I think the approximate 100 ACTIONS from each test IS significant enough to pick the winner, although it is only about 10% better in this case. If it is 100 exposures (not actions), then no... But 100 ACTIONS/CONVERSIONS is significant, whether it is opt-ins, or sales. Some statisticians say 50 ACTIONS/CONVERSIONS by both controls is enough to be 90% certain of the winner. From my learning and experience... |
| | |
|
| |
|
11-07-2009, 05:30 AM
| #6 |
| ArticleToolChest.com War Room Member Join Date: Apr 2005 Location: Boynton Beach, Florida
Posts: 565
Thanks: 3
Thanked 111 Times in 17 Posts
| Re: Significant Split Test Results?
Thanks for the replies. I'm not using an A/B script, I'm using a PHP script that pulls one of the two pages at random. It loads them evenly over the course of time and gives me a good split. The pages are very different. One is with a headline and copy, the other is with a link that asks them to fill out the form w/ a video included on the page. The text is well below the fold. Thanks, Tim |
| | |
|
| |
 |
|
| Tags |
| results, significant, split |
| Thread Tools | |
| |
 |
