5 {{ upvoteCount | shortNum }}
5 {{ upvoteCount | shortNum }}

Not seeing 50/50 traffic allocation in AB Test!

by SheilaC
4 replies
Hi all!

Any recommendations which service to use for classical, traditional AB testing with an even split of traffic between variations?

I used Experimentor app in Shopify and though I selected an even split of traffic, 50% traffic to old variation and 50% traffic to new variation, the results did not show even traffic allocation between the two variations.

Apparently, Optimizely and Convertize do not ensure even traffic allocation as well. Both Optimizely and Convertize support team cannot guarantee that.

If anyone knows of one service that can ensure that, I would like to know!

Much thanks!
#50 or 50 #allocation #traffic
  • Profile picture of the author timokeefe
    Originally Posted by SheilaC View Post

    I used Experimentor app in Shopify and though I selected an even split of traffic, 50% traffic to old variation and 50% traffic to new variation, the results did not show even traffic allocation between the two variations.

    ... snip

    Apparently, Optimizely and Convertize do not ensure even traffic allocation as well. Both Optimizely and Convertize support team cannot guarantee that.
    When you say it wasn't 50/50, how far off was it? Not guaranteeing an even split isn't necessarily an issue, as long as the split is reasonably likely for a 50/50 coin toss, which is fairly straightforward to test.
    Signature

    I'm part of the Warrior Forum team, hit me up with any suggestions that could help improve the forum!

    {{ DiscussionBoard.errors[11146816].message }}
  • Profile picture of the author techugo
    Hello,
    Thanks,For posting to the community can you post a screenshots of the traffic allocation that is setup? also can you post your screenshots for your result page....then we can see your page...
    Signature

    Ankit Singh
    Techugo - Top Mobile App Development Company

    {{ DiscussionBoard.errors[11146922].message }}
  • Profile picture of the author SheilaC
    Hi, this is my content experiments results page. At any point in time, there was about 20-30 sessions difference between the variations. Is this like a normal thing?
    {{ DiscussionBoard.errors[11150792].message }}
  • Profile picture of the author timokeefe
    Yep, that's normal. To avoid any kind of bias (and having to remember state) all the system is doing is flipping a coin for every new visitor and showing them one variant for heads and the other for tails.

    So for your example you've essentially flipped a coin 1,122 times and you've seen 550 heads and 572 tails. I haven't put that into a calculator, but I'm pretty confident that can occur by chance

    That slight difference in the number of people who've seen each variant also has no impact on the results, because you care about the conversion rate, not the count. So the fact that one variant had more people doesn't matter, because you only care about the proportion that converted, not the raw amount.
    Signature

    I'm part of the Warrior Forum team, hit me up with any suggestions that could help improve the forum!

    {{ DiscussionBoard.errors[11150817].message }}

Trending Topics