Care to explain split testing?

Hey Guys and Gals,

I'm reading through Clayton's course at the moment and it has touched on something I've wondered about before. As the title shows, this is split testing.

For marketing online:

When testing a piece for something online, how is this done?

Do you run one version of a salesletter for a period of time and then switch? Or do you simply test between two websites?

It doesn't seem logical to me to split between two sites because in some cases that would imply having/needing two suitable domains, double the traffic generation, etc. Plus if you're trying to build a brand in the process, it could create confusion.

So...

In retrospect - do you just simply direct to two directories on your page?
Ex. - mysqueezepage.com/a vs. mysqueezepage.com/b

For marketing offline:

I don't really know how to ask this, so I will just cite an example from the course.

It was said that in the process of boosting response, sometimes changes for a current control were made and tested by running "two headlines and four offer variations". I'm assuming that's one headline and two offer variations per mailing piece. Also, I understand that those specific numbers and changes aren't an only option, just what they did.

What I'm not getting, in regard to how to split test them, is this:

1.) Is this done by splitting a current list in half and running your changes against eachother simultaneously, or...

2.) Run one version, wait for your responses and then push the other out?

Given the term ("split testing"), the first option is the only one that really makes any sense. Not to mention the amount of time you would have to wait for the postal service to mail, deliver and return for the second to move.

However, I'm unsure, so I just want to know in case I am wrong.

I haven't reached the point of mailing anything out or anything online, but I'm the kind of person that won't be able to concentrate until I properly comprehend whatever is bothering me.


Thanks for your time,
Mike