When Does Split-Testing Stop?

[Joshua Uebergang]

Most people say to keep testing, but I've been thinking, surely there comes a point where you must stop testing?

You test and test and test. The end result is a nice 6% conversion. You test a few other things, but they fail. Because the conversion hasn't improved for sometime, the failed tests are hurting your sales. There must be a point where you become satisfied with what you have?

[maximus242]

No it doesnt. You just test on a small scale while taking your control and rolling it out on the large scale.

Say you get 10,000 visitors, you might have 8,000 of them view the control and 2,000 of them view test versions.

[Joshua Uebergang]

What you said makes sense.

What's the go then with experts, such as Eben Pagan, who have supposedly kept their salesletters the same? (No guarantee about that, but the headlines and other obvious things haven't seem to change for several years.)

[Fender85]

Well, generally people just get lazy. But the "real" answer is to never stop. I mean the guys who write world-class copy talk about how the copy they wrote for ______ was "the control for 5 years." That means that there were other sales letters actively trying to beat theirs for YEARS at a time.

Ads generally get tired eventually, it's just a matter of when, and also how proactive you want to be. You can continue to always try to improve, or you can split test for a while to get something really sticky, and then when the numbers start to slip, try testing some new aspects. The latter is my preferred method.

[Kyle Tully]

Most people (gurus included) don't test.

They just release new products

The answer is you never really stop testing.

Even if you get 6% today, the market will have changed 6 months from now, or you'll have saturated your market with that message, and results will slide.

[Party Plan Pat]

Originally Posted by Joshua Uebergang

Most people say to keep testing, but I've been thinking, surely there comes a point where you must stop testing?

You test and test and test. The end result is a nice 6% conversion. You test a few other things, but they fail. Because the conversion hasn't improved for sometime, the failed tests are hurting your sales. There must be a point where you become satisfied with what you have?

When the world is gone, then you stop. As long as people are buying online you keep testing. Times change, interests change, what was hot yesterday in to soo hot next week. The Conversion Doc is constantly testing. Amazon with thier 13% conversion ratio are always testing. I was on Amazon last night and there was a new feature that made it virtually impossible for me to read, well went back today and it was gone, looks like i wasn't the only one digging it! My friend you never stop testing. Unless you are looking to end up like the lehman brothers, then stop testing now!

[Loren Woirhaye]

Your product or offer will generally run out of steam
before exhausting all areas for testing. Sometimes it's
best to take theproduct off the market.

In the old direct mail days if you stopped mailing you
would stop making sales. Now with the web you can
keep an offer running long after it's no longer timely
for the marketplace.

There is a point of diminishing returns:
The product runs it's course or the competition gets
better with their offers. It's an ongoing process -
in direct mail it's often a matter of working specific
lists for a year or two until there is no longer
sufficient response to justify further mailing.

On the internet the closest analogy is a product
launch... but those often play out in a matter of
days or weeks instead of months or years as
in direct mail.

Are internet marketing people making products
looking for a short-term quick hit? I think so, yes,
and it's a mistake. The market is already correcting
itself and if trends continue there may be not much
business to go around. Sylie Fortin has an eye-opening
3-part report about these issues you might want to
check out.

[Stephen Dean]

As was mentioned, you don't have to test 1 page against another and show them both 50% of the time. Muvar, for example, shows winners way more often than new variables.

Cheers,
Stephen Dean

[Raydal]

My simple answer would be when the cost of testing exceeds the
improved profits you are making from these test.

In other words, when you have diminishing returns from the testing
campaigns.

-Ray Edwards

[Kenneth L]

Originally Posted by Raydal

My simple answer would be when the cost of testing exceeds the
improved profits you are making from these test.

In other words, when you have diminishing returns from the testing
campaigns.

-Ray Edwards

That is the correct answer. Game, set and match. Thanks for playing!

[budgawnings]

What would be the reason to stop? How is that helping you? Unless somehow, you are making much more money doing what you are doing with your time instead of refining your campaign. Think of it this way, the internet and the economy are dynamic -- if you want to keep up with them, you have to be dynamic too.