Register Advertise with usHelp Desk Today's Posts Search

Closed Thread
Thread Tools Search this Thread
Unread 27th Oct 2014, 06:41 PM   #1
HyperActive Warrior
 
Join Date: 2012
Posts: 242
Thanks: 8
Thanked 5 Times in 5 Posts
Default
How do you split-test?
Share on: 
fb share twitter share gplus share more share

I'm testing response rates to my Craigslist ad.
Here are some major elements I can test:
a) headline
b) location
c) picture/no picture

I have two different accounts to post ads. So my strategy is posting the same ad (hence same headlines and picture/no picture) in city A and city B one week, then change the headline for weeks 2 and 3. With the headline winner, repeat with testing c. Then post the same ad in different cities.

If you were in my shoes, what would you change about the testing process? Does it sound like the testing methods are on the right track?

As an aside, when do you tell whether the mental costs of split-testing outweighs the resulting benefits?

Thank you in advance for your feedback!
stanigator is offline  
Unread 27th Oct 2014, 07:39 PM   #2
VIP Warrior
 
dburk's Avatar
 
Join Date: 2005
Location: Tampa, Florida
Posts: 6,367
Thanks: 333
Thanked 1,622 Times in 1,288 Posts
Default
Re: How do you split-test?
Share on: 
fb share twitter share gplus share more share

Hi stanigator,

What you are proposing to do is called a serial test, not a split test. A serial test is not quite as accurate as a split test and you generally need to let your test run for a longer period of time to get enough data for a valid conclusion.

For best results make sure that you are changing just one element at a time and that you are making a significant change. For example, when testing headlines, do not make a minor insignificant change to the headline, make it radically different so that you are more likely to see at least a 20% difference in performance, otherwise the test results will hold no significance.

Be sure to setup your tests with a specific purpose in mind for what you are to learn as a result of the test. This is commonly referred to as a hypothesis. Analyse the results, of testing your hypothesis, to form a theory as to why it worked the way it did. Next, devise tests that will disprove your the theory if it isn't completely valid. If your theory holds true during your validation tests, you have a winning insight that can be applied to your entire marketing strategy.

I suggest you start by identify various value propositions and test those against each other first, so that when you find a winner it will be clear that it is the value proposition rather than clever wording that people are responding to. Insights, into value propositions that work, have the broadest potential application for conversion rate optimisation, IMHO.

As long as your testing is showing significant benefits you should continue. In marketing you should never stop testing until you have no more competitors and are ready to retire.

dburk is offline  
Unread 27th Oct 2014, 08:48 PM   #3
HyperActive Warrior
 
Join Date: 2012
Posts: 242
Thanks: 8
Thanked 5 Times in 5 Posts
Default
Re: How do you split-test?
Share on: 
fb share twitter share gplus share more share

I was suggesting a serial test b/c split-testing isn't practical in my environment, as I wasn't allowed to post multiple ads with the same account. This leaves me to one more question: when do you use split-test vs. serial-test? Is it an accuracy vs. practicality tradeoff?
stanigator is offline  
Unread 27th Oct 2014, 09:06 PM   #4
VIP Warrior
 
dburk's Avatar
 
Join Date: 2005
Location: Tampa, Florida
Posts: 6,367
Thanks: 333
Thanked 1,622 Times in 1,288 Posts
Default
Re: How do you split-test?
Share on: 
fb share twitter share gplus share more share

Hi stanigator,

Generally, I would recommend split testing over serial testing. The only time you should use serial testing is when it is more practical than split testing, as in the case you described.

dburk is offline  
Unread 27th Oct 2014, 10:19 PM   #5
HyperActive Warrior
 
Join Date: 2012
Posts: 242
Thanks: 8
Thanked 5 Times in 5 Posts
Default
Re: How do you split-test?
Share on: 
fb share twitter share gplus share more share

What factors determine the practicality? I may have another situation that says so.
stanigator is offline  
Unread 28th Oct 2014, 06:08 AM   #6
VIP Warrior
 
dburk's Avatar
 
Join Date: 2005
Location: Tampa, Florida
Posts: 6,367
Thanks: 333
Thanked 1,622 Times in 1,288 Posts
Default
Re: How do you split-test?
Share on: 
fb share twitter share gplus share more share

Originally Posted by stanigator View Post

What factors determine the practicality? I may have another situation that says so.
Anything that prohibits you from using 2 or more alternative page variants at the same time would make split testing impractical.

If you can use javascript code snippets or run simultaneous variants of a page then you can, and should, split test.

dburk is offline  
Closed Thread


Bookmarks

Tags
splittest


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




All times are GMT -6. The time now is 05:05 PM.