Market Samurai uses BROAD match by default; do you use EXACT? Which are numbers you look for?

Wanted to jump in here because this post is leaving a lot of FUD, and causing problems for us

We've spent a lot of time and effort analyzing the accuracy of keyword data (because we have 197,000 users relying on us to give them accurate data via Market Samurai).

There is an issue around Search Network data being included - but it's not as bad as is suggested in this post.

Instead, I think what you're seeing here is a sampling issue.
If I review the data for "Fish Pond Supplies" (exact match, UK, English),

I'm getting this data:
- Search-Based Keyword Tool: 66
- Beta Google Keyword Tool: 210
- Legacy Google Keyword Tool: 14,800 (Note: July data)

Obviously not everyone who searches for a keyword will go to 1 site.
So if you use the AOL CTR data to work out what traffic you should get from this...

...Assume 54.5% of people click ANY result...
...Assume 8.5% of them will click #3...
8.5% * 54.5% * 210 = 10 clicks.
10 clicks is very close to 12 clicks.

Based on this, the Google Keyword tool comes up trumps. (Compared to an estimate of 3 clicks from the SKTool.)
(Run this on a few more keywords and you'll see that the Google Keyword Tool is actually quite accurate. This is why we use it inside Market Samurai.)

We often get support tickets where people have seen one set of figures one month, and a very different set of figures the next month... the "gold nugget" keyword they've found turns into something worthless, and they feel cheated.

So we've had to analyze literally hundreds of keywords, like this.
I believe what people are seeing here are sampling issues.
Obviously it wouldn't make commercial sense for Google to count every search through a turnstyle every month. With 88,000,000,000 searches a month, you're looking at a HUGE amount of computing power.

What they are more likely doing is taking a sample (a few percent) of searches, analyzing this, and coming up with "Estimated" searches.
But even with 99.9% statistical certainty, 1 in every 1,000 keywords will show "bad" data.

(The percentage of "bad" keywords is probably higher - maybe as high as 1%, although I've never gathered a few hundred thousand keywords and watched them over the next few months to know for certain.)
This is why I believe it's a sampling issue, rather than the data coming from the search network.

If the problem was a sampling issue, we'd see a spike one month, and then the next month (when the data was re-sampled) the figure would be back within reasonable bounds ("fish pond supplies" is now back to 210 per month - and I can point to a dozen other examples from our users.)
If the problem was just the search network, we'd see inflated searches on product-specific keywords (Amazon is part of the search network) - but we wouldn't see temporary spikes like this - the traffic would be more consistent.

If the problem was sites like Mahalo redirecting traffic to different search queries, we'd expect that these redirects would probably be applied manually (otherwise, why not just use Google's own ad matching algorithms). And if the redirects were applied manually, we'd expect that the redirects would be permanent (i.e. no spikes), and would affect only a small percentage of high traffic keywords (i.e. would be big, and rare, to maximise ROI).

Big and rare variances I can agree with - but in this case, the next month, the figures were back to 210 searches. Permanent? No.
On balance, I don't see how this can be caused by the search network.
There's every chance that the issues above MAY be affecting the data. And you're right - there are some issues with using the Google Keyword Tool data for keyword research.

But I don't think they're the cause of your specific issue.
In this case, a sampling issue is just the simplest explanation.
I stand by the Google Keyword Tool - it's still the most accurate source of data around (as I demonstrated with the 10 clicks estimated vs 12 clicks actual traffic - pretty close, I say.)

I'd love it if they gave us 100% accuracy on keyword data - but I understand it's not feasible. 98, 99, 99.5% confidence on the largest sampling made available and the most accurate data available - I can live with this
Brent