What sample size for a split test?

3 replies
what's the formula to figure out what is a statistically significant sample size for a split test?

this is for a direct mail letter sent snail mail.
#sample #size #split
  • I don't think there is an exact formula.

    Bob Stones book "Successful Direct Marketing Methods" does delve into the math with some interesting number - crunching.

    My view is...

    A low priced product may have more prospects - so the numbers will be higher.

    A high end product may have less prospects - so the numbers will be lower.

    As a guideline...

    Everything I've read over the years suggests about 2,000 pieces.

    But I've done "measurable" tests with much lower numbers. They key is of course to reach your target audience (I know it's obvious but so important).

    Again it all depends what you're selling.

    Another critical point - when you have a winner on a "split test"

    Don't roll out huge numbers - because there are so many variables - just about anything can happen - so limit them to 5,000 at a time. You can save yourself a fortune if unforeseen events do take place.

    Hope this helps a bit,

    Steve

    P.S. I remember reading Brian Keith Voiles who wrote "Advertising Magic" one of the first major copywriting courses - in his early days he did a "test" for his magic show - the number of letters he sent - a massive 27 - and he got a fabulous response.

    As always - the better the "pitch" - the better the results.
    {{ DiscussionBoard.errors[5166998].message }}

Trending Topics