Which numbers are statistically significant for keywords?

9 replies
  • |
I know that for Ads and Landing pages there should be at least 30 actions, clicks, conversions etc., if there is a big difference in the results, if there is not much difference, one should wait for 50 or even 100 clicks. I can also use some splittest tools for this.

But what is the rule of thumb for keywords? I cannot wait for 30 clicks for each keyword, if I just want to try 100 or more keywords, just to see which ones have the best CTR.
And if there are 200 impressions and no clicks, I pause the KW and it is not statistically significant, maybe later the same KW would perform much better.

#keywords #numbers #significant #statistically
  • Profile picture of the author jamescanz
    What numbers are significant for keywords?

    There's a few I look at.

    1. Cost per click
    2. Cost per action (optin / sale)

    And most importantly

    3. Profit / ROI

    I don't mind paying a little more if it's getting me sales...

    So every keyword in every campaign will differ.
    {{ DiscussionBoard.errors[10606823].message }}
  • Profile picture of the author techppc
    Originally Posted by Michael9 View Post


    And if there are 200 impressions and no clicks, I pause the KW and it is not statistically significant, maybe later the same KW would perform much better.

    no clicks does not mean that keyword is not important because it can be due to many reasons including your ad position .so keyword may be important or significant but your ad rank is low and people are unable to see your ad and hence 0 clicks
    {{ DiscussionBoard.errors[10608780].message }}
    • Profile picture of the author Michael9
      Thanks, but the question is: When can I be 95% sure, that the numbers will be the same in the future. How many clicks and impressions do I need to be 95% sure?

      If I get 20 impressions and 1 click, it doesnt mean this KW will get 5% CTR forever.
      If I get 200 impressions and 4 clicks, I can be little bit more confident this KW will get 2% CTR in the long run.

      But how many impressions and clicks do I need to be 95% sure?
      {{ DiscussionBoard.errors[10609131].message }}
  • Profile picture of the author Evan887
    CTR's vary over time and are not always consistent. I run over $5 million in paid search and we look at CTR's at the account level. For my vertical, I know that 3.5% on the account is good. For an ad group (not keyword), I like to see at least a 1% CTR. If I don't have a 1% CTR for an ad group, then I adjust the ad copy until I do.

    I find that CTR's are a good indicator, but as mentioned in a previous post you should really focus on the cost-per-conversion. How much does that keyword or ad group (preferably ad group) cost to gain a conversion on your landing page? That should be your primary focus. That is something you have more control over than CPC's.
    {{ DiscussionBoard.errors[10629443].message }}
  • Profile picture of the author Analytical
    In general, you need more like 300 clicks to gauge conversion rate. Rather than pausing keywords after only 200 impressions, you might want to look at the search terms actually generating those impressions because you might need to add some negative keywords.
    Best tools to improve Google Analytics
    - Free apps, custom reports & dashboards
    {{ DiscussionBoard.errors[10634182].message }}
    • Profile picture of the author Michael9
      Thanks, I am looking for the top 20% of keywords bringing 80% of conversions and I want to be sure to have statisticaly significant numbers to choose the right keywords, not only the ones being lucky and getting more conversions during the first 100 or 200 impressions.

      For the rest I would try negative keywords or creating new adgroups for them.
      {{ DiscussionBoard.errors[10635191].message }}
  • Profile picture of the author dburk
    Hi Michael9,

    The answer depends upon the data collected.

    There is no standard minimum number for finding statically significance. It is a calculation based on the data collected and how much it varies. Since your question is about an unknown set of variables there is no standard correct answer.

    You will need to gather data for an initial pilot study to determine standard deviation and use that in your calculation. You need data to calculate standard deviation, and you need the standard deviation to calculate statistical significance.

    One thing is for sure, the number of impressions and clicks you need are likely to be far more than your numbers you cited in your examples.

    You don't always have to wait for statistical significance to see actionable data. For new campaigns I will usually wait until an ad has had at least 1000 impressions before looking at CTR data, and then only make adjustments where I see relatively high CTRs, or less than 1 or 2 clicks. I will then reassess after 3000, 5000, and 10000 impressions.
    {{ DiscussionBoard.errors[10635150].message }}
  • Profile picture of the author Marty Foley
    Hi Michael,

    Hopefully you're working with a higher volume account that generates enough actionable data in a reasonable amount of time. But reaching a high level of statistical confidence is at best difficult and impractical for lower budget, lower volume PPC accounts. It would simply take too much money and time to reach statistical validity.

    Obviously you asked this because you're testing different variables. That's great because most overlook its value and only give it lip service.

    A smart goal when testing is to come up with a test variation(s) that outperforms the other(s) so much that statistical validity is less of an issue.

    Assuming you're accurately tracking PPC conversions to the keyword level and have set a maximum amount you want to spend for each conversion, you can use such numbers to determine how much you should spend to test a given keyword before determining if it's worth keeping or not.

    If you're thoroughly testing ads, as CTR improves and avg CPC drops over time, you can afford to buy more clicks to test a given keyword, before it exceeds the "maximum cost per conversion" threshold you've set and deserves being paused.

    Another tip: Consider also comparing other metrics in determining winning / losing test variables, especially if you don't have much traffic volume. For example, if you have Google Analytics linked to an Adwords account, you can import and display bounce rate and time on page metrics and use that additional data to compare performance of ads, keywords, etc. Similar with attribution data like impression- and click-assisted conversions.

    Marty Foley ~ PPC Traffic & Conversion Mad Scientist
    {{ DiscussionBoard.errors[10654525].message }}
  • Profile picture of the author Michael9
    Thanks, there are interesting ideas in your post, like not waiting for conversions and measuring smaller actions. But is it not the same problem like with clicks and conversions? If some KW gets more clicks it doesn´t mean that it gets also more conversions.

    On the other hand I think for the most campaigns waiting for statistically significant numbers would be too expensive, considering how expensive the clicks are and how many hundreds or thousands of them you need just for one test.

    It´s easy to separate the top 20% of KWs from the bottom 20%, but what to do with the rest? Probably it would be OK just to wait for probability of 51% or more, so that one could be sure to get the right keywords in more than the half of the time, so one could be sure to go up more often than go down. Getting 51% probability could be maybe 10x cheaper than 95% probability, so it could be a solution.
    {{ DiscussionBoard.errors[10656120].message }}

Trending Topics