What do you mean test?

by 25 replies
29
I keep hearing people in the forum say that the key is to test, test, test but I never actually understood that. For instance, I've been trying to study POF as a traffic source for couple of months and when it comes to test more, I find myself pretty lost.

When I start campaigns, I usually make a batch of creatives to test, something like 80 creatives and then test them for a whole week. After a week, I get rid of the losers and keep all the ads with 0.1xx CTR or higher.

At this point I feel kind of lost because everybody keep telling me to test new creatives but I don't know how and in which order. Also, sometimes, the ads that had 0.1xx CTR or higher now has very poor CTR and I don't know how to manage it.

What I'm trying to say is that I know how to start testing but I get stuck everytime when it comes to test another creatives or to move on to the next level.

If someone could point me to the right direction or to give me some tips I would really appreaciate it!

Sincerely,
David.
#ad networks (cpm/cpl, display)
  • Hey David,

    Testing 80 ads over a week may not be the best use of your money. Try testing something like 10 at a time.

    I like to test an ad for a minimum 2x the offer payout ($5 offer = $10 spent per creative). So testing 80 ads would cost you $800 to get any real indication of what is or isn't working.

    Don't get too caught up on CTR. High CTR ads aren't always the most profitable nor are they always the best quality leads for the advertiser/merchant. Conversions/profit is what's important. After running an ad for 2x offer payout, did you make money or loose money? Keep the goodies and replace the losers.

    Personally I find it more important to test landers and offers first.

    Offers I judge by EPC and Landers I judge by CVR. Before I make any descisions on these I ideally want multiple conversions and a few hundred clicks.

    Hope it helps as a new starting point. Great to see you persevering
    • [1] reply
    • I understand what you're saying but how can I test 10 ads at a time when the time could be a factor here? What I mean is that if I test 10 creatives and next week I would test another 10, maybe the time would interfere my testing and result in bad data.

      Also, I'm looking for tips about testing in small budget($200).
      • [1] reply
  • [DELETED]
  • 99.9999% of what you read here on WF about testing is sheer nonsense. As someone who has taught the stuff in graduate school I tell you any real, genuine test comparing one A to another B or C, D.E...etc costs $$$thousands and is very difficult. Maybe you can use one of the AB testing programs which are sold here and get results which indicate that this A is better than that B. But I'll guarantee you that you can repeat the AB testing next week or next month and get the exact opposite results.

    Your intuition is probably better than any testing you can afford. Just make sure you get up on the right side of the bed.
    • [1] reply
    • A lot of people here don't have the luxury of testing that extensively. Companies can pour in thousands of dollars into A/B testing and can come up with a solid 90%+ confidence level.

      Although testing on a smaller scale isn't accurate, it is better than not testing at all.

      I 100% agree with this.
      • [1] reply
  • Yeah you might be over testing.

    Create 2 ads, split test those for a 3 day period. Or longer if you want more accurate results. Then take the winner and get rid of the loser. Run the winning ad 80% of the time and create another test ad to run 20% of the time. Compare again.

    Don't focus on CTR though. Focus on ROI. That's what matters!

    • [2] replies
    • So basically you're saying I should always keep testing ads and find winners. Is that what you're saying? If that's so then at what poing will I scale up this business?

      Thanks for your help!
    • its amazing how often people LITERALLY split test at 50/50. why would you put a winner on as often as an unknown.
  • Can someone please explain to me what is A/B testing?
    Also if you can help me out with the process of testing I would really appreaciate it.
    I mean how to test, generally.
    • [2] replies
    • A/B testing is when you test 2 different variations or versions of something.

      For example version A and version B.

      Typically you will be testing both versions and the same time to see which preforms better.
      • [1] reply
    • You really have testing down... different creatives, different demos in different campaigns, etc

      Anything past that, try changing one variable at a time and record the results, see if that variable worked for your or not.
  • If you have a small budget, you're going to be relying on luck more then anything.

    • [1] reply
    • Luck or good instinct and experience? Experience is often gained by throwing crap at the wall and seeing what sticks.

      from your teespring thread:
      This is nice theory but we are here to make money.

      Sometimes, perfection and even statistical significance are not the wisest of investments. With the limited life span of most AM campaigns you are often better served making hay while the sun shines. You can eek out 1% here and 3% there or you can take what's working and scale.

      If I have a choice of spending $500 at 100% ROI or 2k at 50% I know what I will choose.

      As you can see from this thread, there is not one clear cut answer. You need to weed through all the ideas and opinions and find what makes sense to you and your budget. Then you need to test it
      • [ 2 ] Thanks
  • Don't overwhelm yourself too much at the beginning. It'll only confuse you. Try start slow and increase your campaigns along the way. Also putting yourself in the shoes of your audience and trying to think like them will help a lot. "follow" is a good place to start.
    • [1] reply
    • I know that but what I'm trying to figure out is the main phases of testing. I'm looking for some guidelines that I can follow generally. I'm not looking for a secret formula that will make me rich overnight. I'm just searching general concept that I can follow because on the advanced stages of testing I can find myself pretty lost.

      Do you know what I'm saying?
  • Superbowl coming up. It's a simple A/B test. If the two teams who meet in the Superbowl also met at some time during the season we know how the Superbowl will turn out! If team A beat team B when they met earlier in the season, we know for sure that team A will beat team B in the Superbowl also. My handy-dandy A/B testing program told me so. A/B testing never lies.

    /sarcasm
    • [1] reply
    • Dude stop being such a douche. Okay okay, I know. You've "taught this in graduate school" and you're mister big shot and we are all in the "little leagues".

      Let me throw some numbers at you then.

      I netted low 6 figures last year with my affiliate campaigns and many of my campaigns are based off of less than $1000 in testing before I decide to scale it up. And guess what? It's working.

      Low 6 figures may not be a lot to you, but it sure is not just a "coincidence".

      When all you have is a pen and a knife, you just gotta make due.

      Unless you got something to contribute, don't contribute at all. /sarcasm
      • [1] reply
  • you can live very comfortably on a low 6figure salary.

    hell, you can live pretty well on a mid 5 figures salary if youre frugal.
Join the discussion

Next Topics on Trending Feed

  • 29

    What do you mean test?

    I keep hearing people in the forum say that the key is to test, test, test but I never actually understood that. For instance, I've been trying to study POF as a traffic source for couple of months and when it comes to test more, I find myself pretty lost. When I start campaigns, I usually make a batch of creatives to test, something like 80 creatives and then test them for a whole week. After a week, I get rid of the losers and keep all the ads with 0.1xx CTR or higher.