Tiny Little Split Tests - Share Your Stories

[JamesPenn]

Hey

I decided to use the Aweber split test function yesterday for the first time ever. It was only a very simple split test to a very small number of subscribers and it can't draw any definite results, but it may prove useful to some of you.

I was emailing about a video I'd just uploaded to Youtube. The only change I made between the two emails was the title of the email. Here's the titles for each version.

Version one:

'Something different... A hilarious home video of me as a kid'

Version two:

'Something Different... A Hilarious Home Video of Me as a Kid'

The only changes were that version two was in title case.

The results:

Both versions were sent to 302 subscribers. 18 hours later, version one has had 39 click-thru's and version two has had 22 click-thru's.

I now know to send version one to the rest of my list when I mail to the rest of my 3k subscribers.

This is only a tiny split test, so don't take it as gospel that for greater open rates and click-thru's you shouldn't use title case. It all depends on the content of the email and so many other factors.

What small split tests have you done recently and what results have you found?

Your tiny split tests could be worth thousands of dollars to other people.

James

[J. David Rogers]

I did a small split test with Google Adsense blocks on one of my blogs. It was clear after a few thousand impressions that while both blocks had roughly equivalent click through rates, one of them earned me a much higher pay per click. You can guess which ad block I kept and which one I dumped!!

[Rachel Rofe]

Ahhh - just be careful - the results could vary wildly with bigger test numbers. I like to use 2000 minimum for my tests to get a decent indication - though even then, I get surprised often.

That being said, I see your point and awesome job. Split testing can shock you sometimes! =)

[JamesPenn]

Hi Rachel

I agree. 600 is definitely not a big enough number to draw definitive conclusions.

James