Case Study: Two Opt-in Processes Tested

I'm not here a lot and many of you don't know me, but in terms of my overall forum participation, the Warrior Forum is one of my most frequented forums and I always appreciate that it is here.

So, what I'd like to share with you are two very simple tests that I ran, and the results, including the "mindset" lessons learned.

It's a small case study which I hope will be of some use to you.

+++++ Case Study +++++

General Topic: Sales System Management
Project Stage: Prototype

== Summary ==

In this case study I'll share with you two tests that I ran on two different opt-in processes that are part of the same sales system. Both tests revealed a clear winner and the overall process helped me make a decision about what to do next with the proejct.

== Objectives ==

Objective 1) Maximize Current Opt-in Conversion (to get the most out of the traffic I'm getting now)

I have to say "maximize conversion" rather than something more specific because this was my first round of testing. I am just now establishing a baseline conversion rate so I will soon be able to be much more specific such as "Increase Current 26% Opt-in Conversion Rate to 30%".

With every new system, I start at the front of the funnel with my optimization efforts.

I want to make sure I'm getting as high of an opt-in conversion rate as I can before I start working on maximizing profit. It may not be the *best* approach, but it seems to be the one I always take.

Lots of people would suggest focusing first on increasing profit per visitor, but opt-in rate and profit per visitor are tied hand in hand, so I start with the simpler one first.

I can always monetize a list, but I can't re-gain lost visitors who should have been prospects.

Right now, I'm using two opt-in processes (as explained below) so the complexity of testing both processes is double what it normally is, though manageable.

Objective 2) Define/Validate Future System Model (to determine the next step in the development of this project)

This is a critical objective at this stage of the project. When I started this test, I was still in the process of deciding between two very different system models. I won't go into the various options I was considering because that will take too long and it will cloud the main point. The main point is this: I needed to decide between two very different approaches for a major project, and I needed *objective* information in order to make an informed decision.

One of the key differences between the two models is the opt-in process. It isn't the *only* key difference, but it is probably the most important one. So, part of the decision of defining the system model will be informed by which opt-in process is better. This makes both of my objectives, nicely complimentary, which always makes testing easier.

== System Description ==

This test was run on a single sales system that includes two opt-in processes. It is a straightforward opt-in process.

Traffic -> Opt-in Offer -> Thank You Page with Product Offer

After the initial product offer, subscribers continue to get frequent content and offers by email.

== Description of Tests ==

Test 1

Background: When I first started tracking the number of opt-ins I was getting from one of my blogs, it took about 3 days for the suckiness of my conversion rates to become crystal clear. So I decided to test another opt-in process, this time adding a lightbox pop-up to the picture. The lightbox contains a headline, sub-head, left column of bullet points and right column opt-in form and it is shown only once to each visitor.

Page Type: WP Blog

Control: Subscribe By Email' Opt-in Box in Upper Right Sidebar
Test: Current opt-in process with a lightbox pop-up opt-in offer

Test 2

Background: I have a 'sales letter' type opt-in process that I use for the same sales system. Most of the traffic for this page comes from my direct promotions (free eBook links, articles, etc.) and referrals from a handful of JV partners and affiliates who have *integrated* the offer into their thank you pages, autoresponders, etc.

Because I believe in split testing these types of pages, I wanted to kick off a split test as soon as I became able to track the results easily. I didn't spend much time on it, I just deleted the header from the test page and started alternating the traffic and tracking the results.

Page Type: Sales Letter

Control: Sales Letter with Graphic Header
Test: Sales Letter without Graphic Header

== Results ==

Test 1

Test one didn't produce many surprises. The test beat the control by a significant margin.

Conversion for Control: 1.2%
Conversion for Test: 7.1%

This means the lightbox + blog box (test) beat the blog box by itself (control) by a spread of 5.9% Clearly the test improved results, so the lightbox will stay. And I won't be running another test on this system because the whole thing is likely to go away in February in favor of a different model.

The conversion rate on the blog box by itself was 1.2% before the test and .9% after the test, so the conversion rate of the blog box itself went down but the combination of blog box plus lightbox pop-up is a huge improvement.

13% of the total optins during the test period came from the blog box. This means that they first said "no" to the lightbox and then saw the content and said "yes" to the blog box optin offer.

I suspect that the .3% decline in blog box opt-ins was due more to poor content than it was to the effect of using the lightbox. We have just improved the content process, so I expect that in a week or so, blog box opt-in rates will be back to normal or may even increase. I really think the lightbox pop-up delivers incremental subscribers without negatively affecting the blog box conversion rates. I'll need more testing to know for sure.

Test 2

Test two surprised me. I have always heard that graphic headers reduce conversion rates, but I've not really tested it on my own sales letters. In this case, I have to call the test in favor of the version of the sales letter that included the graphic header (control) because conversion was worse on the one without header (test).

Conversion for Control: 35.5%
Conversion for Test: 26.3%

This means that for every 100 visitors the control (with header) would bring in 35 opt-ins (on average) and the test would bring in 26. If I had gone with my intuition and just assumed the without header was best, I would lost 9 out of every 100 visitors just based on that one element.

But, as I mentioned, I'm also evaluating one model against another (objective #2). By comparing my conversion rate from Test 1 against Test 2, I can clearly see that the sales letter greatly outperforms the blog.

I knew that would be the case when I set up the blog, but I also expected to get some good search engine juice, links from other blogs, comments, etc. None of those things are happening right now, and I know why, but the effort to make them happen just isn't worth it.

I can get higher quality traffic from JV and affiliate referrals with much less effort, and comments (community interaction) aren't really that important at this point.

== What I Learned ==

These are, of course, results that are based on my system so they are only relevant to my system. If I tried the exact same tests on a different system and offer, I may get much different results.

Also, I know there are LOTS of variables here. I can't make definitive empirical judgments based on my results here, but I can make more money. :-)

I don't really need to be too scientific about it. As long as I'm seeing a general improvement from the optimization steps I take, then things are moving in the right direction.

Major take-aways for me:

* A Lightbox pop-up plus a blog opt-in Box performs better than the blog box by itself.

* My sales letter with graphic header works better than the same letter without graphic header.

* An 'opt-in first' model works better than a 'content first' model (all things considered).

But there is a more important lesson here for me... having ready access to the conversion data I need has changed 'optimization' from a concept to an action for me. Because I have the right information, I am able to quickly make good decisions about how to improve my results.

As soon as I started measuring my results, they started to improve.

I can't wait to see what this same system looks like a year from now. I can say this, it will either be wildly profitable or gone. With the numbers staring me in the face, I've become pretty ruthless about what I spend my time on.

Sorry for the long post, but I wanted to share this information with you.