Went From 2k To 5k From This Simple Change

Hey Stephen -- How are you?

I also use a 95% confidence level.

My point was the blog author is making the decision after 15 total conversions and 600 (or so) page views, not page clicks. Based on his screen shot, I'm not sure SplitTester is the right calculator to use since it's asking for Page Clicks, not views.

Both versions also didn't get an equal amount of views which is a no-no for straight split-testing.

Either his testing software didn't rotate the versions equally or he ran two separate tests and cut one of them off early b/c he thought he had the winner. If it didn't rotate them equally, then it showed the lower converting one which is contrary to what some of the "auto-optimize" testing scripts will do.

Maybe it was outside traffic to that one version which bumped up its specific page views.

The blog author doesn't explain the 70+ page view difference in his article so your guess is as good as mine.

In terms of the 100 action or 1000 uniques rule of thumb...

I've seen a variation start out hot on a split test (about 11.3%) and cool off over a 1000 unique visitors to a 1.2% conversion rate. It still wound up being the winner for that particular case but it was being much closer to the other variations. If I had cut the test early, I could have picked the wrong variation as the winner.

Okay... I just spotted another problem with his stats.

The Belcher button didn't just have a different order button shape but had the credit card logos added to it as well. The Kajabi style order button had no credit card logos. You could also argue the Belcher had a different design style to it as well.

So was it the shape of the Belcher button or the credit card logos that bumped response rates? Or maybe it was the design style?

Would the Kajabi button convert better if we had added the credit card logos beneath it?

Bottom line... He didn't test one variation (order button shape) of another element against another. He actually tested three variations on the same test (button shape, button style, and credit card logos) and lumped them all into one "answer".

Yes, Belcher button may convert better but I'd still caution people to TEST it on their own product & niche instead of just assuming it will convert better.

My 3 cents,

Mike

P.S. Just to be clear, I'm not picking on the Belcher button. I've used it on certain salesletters I've written. I'm picking on the inaccuracies of the test example that was shared.