22 {{ upvoteCount | shortNum }}
22 {{ upvoteCount | shortNum }}

How Would You Explain These Split Testing Results?

by sethczerepak
17 replies
I'm testing a hypothesis in the copywriting niche. This is one of 20 emails I'm using to do it.

(Sent 36 days from opt in date)

Control Version:

How to Hijack Your Reader's "Inner Dialogue..."
36.4% Clicks - 34.3% Opens

Tested Version:

How to Become the Voice Inside Your Reader's Head
71.4% Clicks - 57.1% Opens

I have my own theory on this, but I'm interested in what you guys think caused it...

HINT: I think the timing of the email has a part in it.
#explain #results #split
  • Profile picture of the author NickN
    The tested version is clearer and the benefit is more apparent. It makes me feel more powerful or something -- like I'm John Cusack controlling Malkovich's brain in Being John Malkovich.

    Hijacking someone's "Inner Dialogue" is kind of vague... but then again, I'm not sure the kinds of emails you sent to your list before this. Maybe you've explained "Inner Dialogue" to your audience in previous emails, so they already know what it is. I don't know.

    As for the timing -- no idea. Would it help to see the email sequence prior to this email?

    -Nick
    Signature

    {{ DiscussionBoard.errors[9051829].message }}
    • Profile picture of the author Chriswrighto
      People would rather "become" than "hijack" the readers inner voice... then they have complete persuasive power.

      And...

      When it comes to "hijacking," I think of a physical act... rather than mental. This leaves the subject feeling incongruent.
      Signature

      Wealthcopywriter.com :)

      {{ DiscussionBoard.errors[9051896].message }}
      • Profile picture of the author sethczerepak
        Originally Posted by Chriswrighto View Post

        People would rather "become" than "hijack" the readers inner voice... then they have complete persuasive power.

        And...

        When it comes to "hijacking," I think of a physical act... rather than mental. This leaves the subject feeling incongruent.
        Okay, you guys got me thinking in a different direction lol.

        Interesting though, the validation of "becoming" something to someone.
        Signature
        {{ DiscussionBoard.errors[9052184].message }}
    • Profile picture of the author sethczerepak
      Originally Posted by NickN View Post

      The tested version is clearer and the benefit is more apparent. It makes me feel more powerful or something -- like I'm John Cusack controlling Malkovich's brain in Being John Malkovich.

      Hijacking someone's "Inner Dialogue" is kind of vague... but then again, I'm not sure the kinds of emails you sent to your list before this. Maybe you've explained "Inner Dialogue" to your audience in previous emails, so they already know what it is. I don't know.

      As for the timing -- no idea. Would it help to see the email sequence prior to this email?

      -Nick
      That's interesting perspective about the "becoming..." I'm still waiting for the test results, but I'll post my hypothesis when they come in.
      Signature
      {{ DiscussionBoard.errors[9052151].message }}
  • Profile picture of the author RickDuris
    I have no idea. All I can say is I like them both!

    Good job, Seth!

    - Rick Duris
    Signature
    {{ DiscussionBoard.errors[9052147].message }}
  • Profile picture of the author Mark Pescetti
    I agree with Rick. Both are pretty kick ass.

    I think the first one might be too much of a thinker for some people.

    The second one has clearer benefits.

    My take.

    Mark
    Signature

    Do you want a 9 figure copywriter and biz owner to Write With You? I'll work with you, on zoom, to help write your copy or client copy... while you learn from one of the few copywriters to legit hit 9 figures in gross sales! Discover More

    {{ DiscussionBoard.errors[9052204].message }}
    • Profile picture of the author Alex Cohen
      The tested version stats could be an anomaly. As you said, the timing of the email was different.

      How about the volume of the test email... was it statistically significant?

      Alex
      {{ DiscussionBoard.errors[9052585].message }}
      • Profile picture of the author sethczerepak
        Originally Posted by Alex Cohen View Post

        The tested version stats could be an anomaly. As you said, the timing of the email was different.

        How about the volume of the test email... was it statistically significant?

        Alex
        If you mean how many subscribers it was sent to, the volume was the same. Same time of day too. It's an autoresponder message so the results span a near 90 day period so far.

        So if our definition of "anomaly," is the same, it's not an anomaly.

        The only difference was the headline, and one other variable, which I'll explain once the testing cycle completes. It's done the end of this month.

        If my hypothesis is right, which I think it is, this could be a new variable to consider when creating follow up messages.
        Signature
        {{ DiscussionBoard.errors[9052610].message }}
        • Profile picture of the author Chriswrighto
          Originally Posted by sethczerepak View Post

          If you mean how many subscribers it was sent to, the volume was the same. Same time of day too. It's an autoresponder message so the results span a near 90 day period so far.

          So if our definition of "anomaly," is the same, it's not an anomaly.

          The only difference was the headline, and one other variable, which I'll explain once the testing cycle completes. It's done the end of this month.

          If my hypothesis is right, which I think it is, this could be a new variable to consider when creating follow up messages.
          Ooh, ooh, ooh...

          Pick me...

          Is it something to do with the time of year?
          Signature

          Wealthcopywriter.com :)

          {{ DiscussionBoard.errors[9052625].message }}
          • Profile picture of the author sethczerepak
            Originally Posted by Chriswrighto View Post

            Ooh, ooh, ooh...

            Pick me...

            Is it something to do with the time of year?
            No, has to do with how familiar the reader is with the sender and one other variable...more to come.
            Signature
            {{ DiscussionBoard.errors[9054557].message }}
        • Profile picture of the author Alex Cohen
          Originally Posted by sethczerepak View Post

          If you mean how many subscribers it was sent to, the volume was the same. Same time of day too. It's an autoresponder message so the results span a near 90 day period so far.
          My question was, is the test statistically significant? In other words, are you getting enough volume to have confidence in the numbers from each test?

          If you're not getting at least 1,000 emails delivered, and basing your Opens and CTRs on that number, the numbers are not statistically significant.

          Alex
          {{ DiscussionBoard.errors[9053320].message }}
          • Profile picture of the author sethczerepak
            Originally Posted by Alex Cohen View Post

            My question was, is the test statistically significant? In other words, are you getting enough volume to have confidence in the numbers from each test?

            If you're not getting at least 1,000 emails delivered, and basing your Opens and CTRs on that number, the numbers are not statistically significant.

            Alex
            Dammit Alex, no scientific discussions allowed on the Warriorforum!

            Seriously, I'll be publishing the confidence rates on my blog when I finish the experiment. The test is much bigger than just this headline though. I'm testing a theory which I got from Eugene Schwartz's 5 levels of sophistication and using multiple lists across five niches.

            I'll post a link here when the results are in.
            Signature
            {{ DiscussionBoard.errors[9054550].message }}
  • Profile picture of the author GregBrooks
    I also think the word hijack has a negative connotation
    {{ DiscussionBoard.errors[9052571].message }}
  • Profile picture of the author BlingFiles
    For me the tested version is more simple and clear than the first one.
    {{ DiscussionBoard.errors[9052590].message }}
  • Profile picture of the author Arzak
    The second one is more clear. The first sounds slightly hypey (and reminds me of WSOs or the typical email I'd get from opt-ins).
    {{ DiscussionBoard.errors[9053723].message }}
  • Profile picture of the author angiecolee
    I do like both, but there's definitely a connotation thing for me.

    Hijack seems more violent, like I'm forcing them to do something against their will. Becoming their inner voice makes it seem like I'm hypnotizing them or otherwise becoming their purchasing conscience. They're doing what I want them to do without me having to hold a gun to their head. To me, that's more appealing.

    To your list, I don't know honestly.
    Signature

    Aspiring copywriters: if you need 1:1 advice from an experienced copy chief, head over to my Phone a Friend page.

    {{ DiscussionBoard.errors[9055734].message }}
    • Profile picture of the author OutOfThisWord
      Both the words 'hijack' and 'dialogue' are not words people think or feel when they want something.

      #2 can reach an 8th grader with the words 'voice' 'head' and you can feel those words.

      Thanks for posting!
      {{ DiscussionBoard.errors[9056320].message }}

Trending Topics