Which numbers are statistically significant for keywords?

What numbers are significant for keywords?

There's a few I look at.

1. Cost per click
2. Cost per action (optin / sale)

And most importantly

3. Profit / ROI

I don't mind paying a little more if it's getting me sales...

So every keyword in every campaign will differ.

no clicks does not mean that keyword is not important because it can be due to many reasons including your ad position .so keyword may be important or significant but your ad rank is low and people are unable to see your ad and hence 0 clicks

CTR's vary over time and are not always consistent. I run over $5 million in paid search and we look at CTR's at the account level. For my vertical, I know that 3.5% on the account is good. For an ad group (not keyword), I like to see at least a 1% CTR. If I don't have a 1% CTR for an ad group, then I adjust the ad copy until I do.

I find that CTR's are a good indicator, but as mentioned in a previous post you should really focus on the cost-per-conversion. How much does that keyword or ad group (preferably ad group) cost to gain a conversion on your landing page? That should be your primary focus. That is something you have more control over than CPC's.

In general, you need more like 300 clicks to gauge conversion rate. Rather than pausing keywords after only 200 impressions, you might want to look at the search terms actually generating those impressions because you might need to add some negative keywords.

Hi Michael9,

The answer depends upon the data collected.

There is no standard minimum number for finding statically significance. It is a calculation based on the data collected and how much it varies. Since your question is about an unknown set of variables there is no standard correct answer.

You will need to gather data for an initial pilot study to determine standard deviation and use that in your calculation. You need data to calculate standard deviation, and you need the standard deviation to calculate statistical significance.

One thing is for sure, the number of impressions and clicks you need are likely to be far more than your numbers you cited in your examples.

You don't always have to wait for statistical significance to see actionable data. For new campaigns I will usually wait until an ad has had at least 1000 impressions before looking at CTR data, and then only make adjustments where I see relatively high CTRs, or less than 1 or 2 clicks. I will then reassess after 3000, 5000, and 10000 impressions.

Hi Michael,

Hopefully you're working with a higher volume account that generates enough actionable data in a reasonable amount of time. But reaching a high level of statistical confidence is at best difficult and impractical for lower budget, lower volume PPC accounts. It would simply take too much money and time to reach statistical validity.

Obviously you asked this because you're testing different variables. That's great because most overlook its value and only give it lip service.

A smart goal when testing is to come up with a test variation(s) that outperforms the other(s) so much that statistical validity is less of an issue.

Assuming you're accurately tracking PPC conversions to the keyword level and have set a maximum amount you want to spend for each conversion, you can use such numbers to determine how much you should spend to test a given keyword before determining if it's worth keeping or not.

If you're thoroughly testing ads, as CTR improves and avg CPC drops over time, you can afford to buy more clicks to test a given keyword, before it exceeds the "maximum cost per conversion" threshold you've set and deserves being paused.

Another tip: Consider also comparing other metrics in determining winning / losing test variables, especially if you don't have much traffic volume. For example, if you have Google Analytics linked to an Adwords account, you can import and display bounce rate and time on page metrics and use that additional data to compare performance of ads, keywords, etc. Similar with attribution data like impression- and click-assisted conversions.

Marty Foley ~ PPC Traffic & Conversion Mad Scientist

Thanks, there are interesting ideas in your post, like not waiting for conversions and measuring smaller actions. But is it not the same problem like with clicks and conversions? If some KW gets more clicks it doesn´t mean that it gets also more conversions.

On the other hand I think for the most campaigns waiting for statistically significant numbers would be too expensive, considering how expensive the clicks are and how many hundreds or thousands of them you need just for one test.

It´s easy to separate the top 20% of KWs from the bottom 20%, but what to do with the rest? Probably it would be OK just to wait for probability of 51% or more, so that one could be sure to get the right keywords in more than the half of the time, so one could be sure to go up more often than go down. Getting 51% probability could be maybe 10x cheaper than 95% probability, so it could be a solution.