AdWords A/B Testing Tips?

Hi maddystruve,

As a general rule you want to control the variables in a test so you can isolate the cause of any difference in the results.

Having said that, you should consider exactly what "it" is that you are testing when testing matched pairs of landing pages and ad messages. You want the value proposition in your ad text to match the value proposition found on your landing page. It's import to realize it is the "message" that you are testing in this case, not just the ad, or just the landing page.

I would setup a split test to compare the new messaging with the old messaging, first by altering just the landing page message, not the design. You want to isolate design issues from corrupting the results of your test. You could also test the old message against the new message on the new landing page design. That will tell you if the design itself is an improvement or not as well as the messaging. This is called a multi-variant test. It just takes more data to reach statistical significance, but gives you more usable results.

The bottom line it to test one thing at a time, if it is the "message" then make sure the message is consitent on both the landing page as well as the ad text, else you are not really testing the message, are you? I hope that makes sense.

HTH,

Don Burk

Hi maddystruve,

It depends upon what "it" is that you are testing.

You mentioned a new landing page, but what is the one thing you changed on the new landing page? Or, are you testing a radical redesign of the landing page?

If you are testing a radical redesign then you want to split traffic evenly between the 2 landing pages using the same set of control ads. The point of a radical redesign is to find the best version of a the the 2 radically different designs to discover which is better to use as a baseline for testing going forward.

As with all tests, you need to control the variables. You want to keep everything the same for both the control and the experiment, except for the one thing you are testing. Try to test only one variable at a time, that way you can confidently attribute the performance difference to the one thing your were testing. If you try to test more then one thing at a time you will not know with certainty which elements contributed to performance changes and which did not. You won't learn much by testing too many things at once.

So if you are testing a landing page "design", using a totally new layout, then you need to use the exact same ad message, landing page message, and traffic source for comparison in your marketing experiment.

If however, you are testing a new value proposition, not a new page layout design, then you would use the same landing page design and traffic source with a different messaging on the ads and the landing page. The idea is to always keep the messaging consistent for users that read the ad and then read the landing page message, do not change the message for users within the same session. Keep the message consistent, otherwise your test results will be corrupted by mixed messages.

As far as keeping the history, that should not be an issue as your experiments have a specific start and ending date. History is never lost as long as you are not deleting data, and keep your activities documented to help you understand what you did, when, and why.

HTH,

Don Burk