17 replies
Hello everyone,

I run a website which is essentially SAAS.

I want to do A\B testing for 2 of my pages:

1. The billing page (where the user submits his details for payment) .

2. The plans page (where a user can choose from verious paying plans (pro, enterprise etc.).

Would you please suggest 2 or maybe even 3 of the most important test I should conduct on each one of the pages?

ALSO - how long should I run the test? when will it be complete and I can decide the "winner"?

Thanks!
  • Profile picture of the author Yvon Boulianne
    it`s not 2-3 test you have to do but a hundreds or so...
    Signature
    Stop Struggling With Your Website and Marketing
    30$ / task, pay after done!
    {{ DiscussionBoard.errors[10992342].message }}
  • Profile picture of the author rritz
    When I do split testing I usually start out with two or three completely different pages. From there I go into the details. Usually I start on testing different backgrounds holding the same content.

    On each and every page there are a hundred details that can affect your conversion rate. Start testing the ones that have most impact. If you are unsure, test the bigger things first, or the most important ones. On the plans page this might be the way you represent the different plans for instance.

    It takes a bit of experience to find out what parts of your pages are most important.
    Maybe you could use a heatmap to see the way people interact with your page.

    Things to split test are fonts, colors, images, font size, wording, backgrounds, videos, graphics etc.

    When split testing you need statistically relevant data. A rule of thumb is 500 views on each split test page.

    But, in your industry you may experience busier and laxer times during the day and during the week - giving you different qualities of traffic.
    If so, and you gather all the views during the weekend, this might not be representative of business customers, who visit from their office desk during the week.
    So it was suggested that a split test should continue during one full circle at least: a full week, or maybe even a full month, irreverent of the number of views.

    I once read an interesting blog post about this, I think it was called "9 ways you screw up your split tests" or something like that. Maybe you can find it
    {{ DiscussionBoard.errors[10992363].message }}
  • Profile picture of the author JohnMcCabe
    Originally Posted by Mishel Oneders Baron View Post

    Hello everyone,

    I run a website which is essentially SAAS.

    I want to do AB testing for 2 of my pages:

    1. The billing page (where the user submits his details for payment) .

    2. The plans page (where a user can choose from verious paying plans (pro, enterprise etc.).

    Would you please suggest 2 or maybe even 3 of the most important test I should conduct on each one of the pages?

    ALSO - how long should I run the test? when will it be complete and I can decide the "winner"?

    Thanks!
    Start with whichever page comes first in your process. If they don't get past that, the other won't matter.

    rritz had it right when they said to start testing with big things, but blew it when they said

    Things to split test are fonts, colors, images, font size, wording, backgrounds, videos, graphics etc.
    To be fair, the big things could be contained in that list, but it's not specific enough.

    I'd start with the offer itself. Things like

    > free trial vs. cheap trial
    > length of trial
    > descriptions of the plans
    > Pricing, including various prices for prepaid blocks (like a discount for paying a year upfront vs. month to month).

    The advice about the heatmap is spot on for pages like this.

    As far as how long to run the test, you want to run it until you have enough responses (not just page views) to be statistically significant. Lots of calculators available to see what your "magic number" is.

    You also have to decide how precise you want to be with your test results. You can test until you reach 99% confidence level in the results, but that will take a lot more responses than going for 95%, 90% or 85% confidence that the result is actually due to the difference in the test pages rather than random influences.

    For most businesses, 85-90% confidence is fine, because it allows you to spend time testing additional factors (like colors, etc.) that may have a bigger effect on your bottom line than wringing an extra 0.1% out of a single variable.
    {{ DiscussionBoard.errors[10992390].message }}
  • Hey guys,

    Attached are the 2 pages needed for the A\B testing (the plans page and the "more followers" page.

    Please have a look and tell me what you think is the 2 most important tests i need to conduct?

    Thanks!

    Attachment 24558

    Attachment 24559
    {{ DiscussionBoard.errors[10992525].message }}
  • Profile picture of the author johndetlefs
    Headline every time.

    Before buttons, before colours (Assuming everything is readable), before fonts, before tag lines, before image sizes, before anything you must...

    Test the headline.
    Signature

    You only get one shot at life - make it awesome.

    Everyone else also gets just one crack at it - help make theirs awesome too... or, politely step out of their way.

    {{ DiscussionBoard.errors[10992536].message }}
  • Profile picture of the author yukon
    Banned
    I'll tell you now, you're only guessing about things like headlines.

    You need to be tracking actions, things that traffic does on a webpage, button clicks, heatmaps, analytics, etc...

    Here's an example of a heatmap (below), it shows where on the webpage traffic is clicking.

    Once you have this data then you go back and make edits to your A/B test pages and fine tune things like headlines, button location, images, etc...

    ...then you keep tracking with a heatmap and compare the new data with the old data. This way you have real data to work with.

    Always save your old data even If it sucks, this way you have something to compare and progress over time. This worked, this didn't work, etc...





    {{ DiscussionBoard.errors[10992846].message }}
    • Profile picture of the author johndetlefs
      Originally Posted by yukon View Post

      I'll tell you now, you're only guessing about things like headlines.
      I'm a huge fan of heat maps, and in general agree with your points, but the example you've given in no way matches the OP's pages.

      I'm using the pages (images) give by the OP. In the context of the pages given I 100% stand by the headline being the number one element on the page to split test. It's a no brainer.
      Signature

      You only get one shot at life - make it awesome.

      Everyone else also gets just one crack at it - help make theirs awesome too... or, politely step out of their way.

      {{ DiscussionBoard.errors[10992865].message }}
      • john thanks,
        I understand that the headlines is the number 1 thing to check, but could you please tell me another important test for this?
        thanks!
        {{ DiscussionBoard.errors[10992872].message }}
      • Profile picture of the author yukon
        Banned
        Originally Posted by johndetlefs View Post

        I'm a huge fan of heat maps, and in general agree with your points, but the example you've given in no way matches the OP's pages.

        I'm using the pages (images) give by the OP. In the context of the pages given I 100% stand by the headline being the number one element on the page to split test. It's a no brainer.


        Changing a headline tells you nothing.

        You need a known reaction, an event done by traffic.

        For all you know traffic didn't even read the headline, maybe an image on the same page triggered traffic into performing an action.

        Now If you made that headline a clickable hyperlink and tracked the click event then you know for a fact traffic based their click/action on the headline or whatever, it could be an image with text (ex: banner).

        Just changing a headline isn't good enough.
        {{ DiscussionBoard.errors[10992879].message }}
        • Profile picture of the author johndetlefs
          Originally Posted by yukon View Post

          For all you know traffic didn't even read the headline, maybe an image on the same page triggered traffic into performing an action.
          Unless I'm mistaken you seem to be talking about the mechanics of how you would test/track the headlines, or any other element on the page. I'm not talking about how, but what.

          Going back to the example give by the OP, we have an initial sales page with a dark background, a headline and a button below.

          In this instance I say test the headline.

          @Mishel:

          You don't currently have all that many elements on the page. So after the headlines I'd go for CTA text (button text) followed by button colour.

          I'd personally also play around with your background image/overlay. Yukon's right though, the only way to test these things properly is to do click tracking / heat mapping, and I'm also a big fan of Eyequant predictive eye tracking, and I use Hotjar to record screen sessions.

          I have, by far, learnt the most about my sales pages by watching over recorded screen sessions.. you'd be amazed how easy it is to confuse users without meaning to.
          Signature

          You only get one shot at life - make it awesome.

          Everyone else also gets just one crack at it - help make theirs awesome too... or, politely step out of their way.

          {{ DiscussionBoard.errors[10992888].message }}
  • Guys another question please:

    Lets assume I want to add tweets scheduling to my product (online product, SAAS) since people are paying for it.

    Could you please advise me on how to add the schedule functionality to my product, in a way that will be competitive and people will use it?

    Of course i'm not looking for you to explain it thoroughly, I just need it in pointers (e.g: 1. do this. 2. do that and so on...)

    Thank you!!
    {{ DiscussionBoard.errors[10992896].message }}
    • Profile picture of the author johndetlefs
      Originally Posted by Mishel Oneders Baron View Post

      Lets assume I want to add tweets scheduling to my product (online product, SAAS) since people are paying for it.
      No idea, my friend, depends on your setup.. you might be better off hitting the dev section of the forum, or even Stack Overflow for this sort of stuff. Be warned though, SO can be tough on you if you haven't thoroughly written out the question, including things you've already tried etc.
      Signature

      You only get one shot at life - make it awesome.

      Everyone else also gets just one crack at it - help make theirs awesome too... or, politely step out of their way.

      {{ DiscussionBoard.errors[10992901].message }}
  • thanks john, already posted in that forum, waiting for reply.

    May I ask yet another question?

    my head of performance told me that he would like to create an algorithm to predict the probability that a user will purchase our premium product. What type of data should I provide him?

    Thanks!
    {{ DiscussionBoard.errors[10992936].message }}
    • Profile picture of the author johndetlefs
      Originally Posted by Mishel Oneders Baron View Post

      my head of performance told me that he would like to create an algorithm to predict the probability that a user will purchase our premium product. What type of data should I provide him?
      Unless he has a direct line to God or a seer who can tell the future, the answer is spend some money to send traffic to your sales page and see. Then you won't have to worry about probabilities.. you'll have an exact answer!
      Signature

      You only get one shot at life - make it awesome.

      Everyone else also gets just one crack at it - help make theirs awesome too... or, politely step out of their way.

      {{ DiscussionBoard.errors[10992940].message }}
      • Ok, but I should provide him SOME stats no?

        Like (for example only) amount of clicks, CTR rate and such...

        Thanks!
        {{ DiscussionBoard.errors[10992941].message }}
  • Profile picture of the author johndetlefs
    Gonna have to can this one Mishel, it's 9pm here, and I'm on holidays tomorrow!

    For your last question though, sure, if you have some stats on how people are navigating through your site, then I guess you can make some predictions.. really though, the only way to know is to give it a whirl. Anything else is a guess.

    I'm sure you already know this, but advertising is always a game that starts off with high risk and educated(?) guesses, and over time ends up being low risk with decisions being made on actual data.

    I'm sure there are some services out there that purport to be able to give you an accurate prediction of how your sales are going to go on an untested page, but I don't know of them, and personally wouldn't trust them anyway!

    Best of luck with this Mishel, and have a great weekend.
    Signature

    You only get one shot at life - make it awesome.

    Everyone else also gets just one crack at it - help make theirs awesome too... or, politely step out of their way.

    {{ DiscussionBoard.errors[10993029].message }}
  • Profile picture of the author ankitkumarkhanna
    Hello Mishel,

    If you have started doing AB testing of your website then following things must remenber :-
    1. Your Call to Action Button (CTA)
    2. Your page content which should be related to your product.
    3. 2-3 eye catchy areas like punch lines or images which attract more visitors.

    Do changes and then test on different platforms so that you get to know which page is performing better and getting more traffic to your site.
    {{ DiscussionBoard.errors[10993057].message }}

Trending Topics