13 {{ upvoteCount | shortNum }}
13 {{ upvoteCount | shortNum }}

Use of multi-armed bandit model in A/B split testing

by eternalwarrior
10 replies
I heard from many marketers that traditional way of A/B split testing to find a high converting landing page is no longer effective and suggested to use multi-armed bandit model in A/B split test.

I have absolutely no idea of multi-armed bandit. How do I use that model in split testing?
#bandit #model #multiarmed #split
  • Profile picture of the author vask
    The multi-armed bandit model (which is a stupid ass name) is the same thing as A/B testing, except that you start putting more weight on the variable that does better as time goes on.

    Here's an example:

    Lets say you have Landing Page A (LP-A) and Landing Page B (LP-B). If you send 1000 clicks to your campaign, then with traditional A/B testing you would send 50% of the clicks to both landing pages. Then you would analyze it after the 1000 clicks and make your decision there.

    But with the multi-armed bandit model, you would analyze your data multiple times during the test and you would change the click weight according to which one does better. So instead of sending 50% to both landing pages - you might take a look at the data after 200 clicks and you see the LP-A did better, so instead of doing a 50/50 split between clicks, you might do a 60/40 or 70/30 leaning towards the landing page with better stats.

    This is done so you lose less money while testing. You can easily do this in CPVLab as they have a weight option.

    This article (which is against the multi-armed bandit model) is also an interesting read.
    {{ DiscussionBoard.errors[9080746].message }}
  • Profile picture of the author biggenius
    This method have its own drawbacks.
    {{ DiscussionBoard.errors[9081614].message }}
    • Profile picture of the author JustinDupre
      Originally Posted by biggenius View Post

      This method have its own drawbacks.
      Lol. And you aren't going to say what those drawbacks are? Way to be vague.

      I've never knew this had such a goofy name, but I use it for all my campaigns. Whether it be offer, landing page, or ad. It's similar to the 80/20 rule. Put 80% of your effort/money into things that are working, 20% into new tests. Save money, keep making money. There really isn't any drawback to this method, and if you aren't already naturally doing it, you're losing money.
      Signature
      I offer CPA coaching and investment opportunities for those SERIOUSLY interested in making money directly or indirectly with affiliate marketing. PM me for details.


      Read More about CPA/Affiliate Marketing on my Blog
      {{ DiscussionBoard.errors[9081934].message }}
  • Profile picture of the author Greedy
    There really isn't any drawback to this method, and if you aren't already naturally doing it, you're losing money.
    Yes, I always felt it was the natural progression of testing of landing page. A new nickname for it = easier to sell to people.
    Signature
    {{ DiscussionBoard.errors[9082940].message }}
    • Profile picture of the author pcalvert
      Actually, the term "multi-armed bandit" isn't new, nor is it a name invented by some marketer. It comes from probability theory.

      See: Multi-armed bandit

      Phil
      Signature

      "If a cat sits on a hot stove, that cat won't sit on a hot stove again.
      That cat won't sit on a cold stove either." - Mark Twain

      {{ DiscussionBoard.errors[9086862].message }}
      • Profile picture of the author eternalwarrior
        Yeah, the name is not something new. I was not aware of the term before; and when I saw it for the first time I was like "WTF is this method?!"

        Now it's clear to me that it's basically the same as A/B testing except that more traffic is send to the winning landing page variation in order to minimize the loss.

        Originally Posted by pcalvert View Post

        Actually, the term "multi-armed bandit" isn't new, nor is it a name invented by some marketer. It comes from probability theory.

        See: Multi-armed bandit

        Phil
        {{ DiscussionBoard.errors[9087822].message }}
  • Profile picture of the author Jaxi
    been doing this since i started ppv 2 weeks ago. Thought this was pretty obvious?
    {{ DiscussionBoard.errors[9097632].message }}
    • Profile picture of the author eternalwarrior
      Originally Posted by Jaxi View Post

      been doing this since i started ppv 2 weeks ago. Thought this was pretty obvious?
      How's your PPV campaign going?
      {{ DiscussionBoard.errors[9098347].message }}

Trending Topics