What to measure and what indication to look, when doing A/B split testing ?
 

Hello!

I want to A/B split test several sales copy pages elements. Like for instance several Headlines, content pieces, images, call to action offers etc.

What data I should look in my Google Analytics to determine the efficiency when A/B split testing between them.

I know I'll test them one vs another at a time. However if I don't have any sales during the test what is best to measure? Time spent on page, bounce rate ...?

Should there be anything else that I'm missing ?

What is your best approach when doing A/B split tests ? Also how much time should the test runs and how much visitors per page is enough to determine if page performs or not ?

ty

Re: What to measure and what indication to look, when doing A/B split testing ?
 

I would say that in testing there is ultimately one thing you are testing for, and that's conversion. With that in mind you are then left with basically 2 parameters worth measuring. Conversion itself, and bounce. ( how many people get to the page, how many buy, and how many don't. ) It really is that simple.

You can use elements such as time on page etc... to see where there may be a hang up in the ultimate goal of conversion, but these would not be what I would consider matrix to determine if one page is better than a another.

So the question at hand.. how do you tell if a page is better than the next if there is no conversion? the answer is you really cant. if you test away and produce what think is the best page you can present and it has not produced a conversion in the testing process... well that probably is not going to change. In the testing process you simply have to have conversions as the ultimate line of success or failure.

Re: What to measure and what indication to look, when doing A/B split testing ?
 

OK thank you for your reply. But now a new question arises. How many conversions are required to determine if page A is better than page B ?

In the first series of test I want to test between 7 different headlines (rest of sales page is same). What is the best and time efficient way to do it. If I have to wait for several conversions between each page that would take a looong time. I plan to do it with Google Analytics Experiments option where one experiment can last between a week and a month.

How do you think about that ?

Re: What to measure and what indication to look, when doing A/B split testing ?
 

to do it "Correctly" you run no less than 1000 impressions and or 1 week across each change. and you place your totals in a tool such as this: A/B Significance Test .

Proper testing flat out takes time - UNLESS you simply have unruly amounts of traffic coming from a single source.

Re: What to measure and what indication to look, when doing A/B split testing ?
 

Is there any software packages or link tracking that can test these automatically and keep a live tally? That would be something that could come in handy.

Re: What to measure and what indication to look, when doing A/B split testing ?
 

Most if not all click tracking services or software will have split testing
built into the software. You can even use Google Analytics which is free!

A Guide to A/B Testing with Google Analytics

-Ray Edwards

Re: What to measure and what indication to look, when doing A/B split testing ?
 

Great post I just read it few more times. However I have a question I couldn't find the answer to:

If one have several pages (variations) to test is it better to test pages 1 vs 1 at a time (A vs B) for 3 weeks lets say, and then the next 3 weeks to test another 1 vs 1 (B vs C) and so on...

OR

is better to run the test all at once testing 1 page vs. all the other variations (A vs B,C,D,E...) at once.

In other words which is more accurate: adding multiple variation pages or testing one variation page at a time?

ty

Re: What to measure and what indication to look, when doing A/B split testing ?
 

Setup funnels and goals and try to see the exit point of each and make changes accordingly, though I use another tool to check my goals.. see signature ;)

Re: What to measure and what indication to look, when doing A/B split testing ?
 

Yes that is one thing and thanks, but I was asking should I test all pages at same time or cycle them 1 by 1 ?