Keyword Research - It *Can't* Be This Easy...

I know there are "advanced" (for lack of a better term) ways to do keyword research, and was wondering what exactly that entailed?

I ask b/c I'm planning my next product, and was doing so basic keyword research.

Thus far, my keyword research has consisted of:

-using Google keyword tool to get an idea of search volume
-also using said tool get an idea of competition for advertising (to see just how many people are interested in this keyword)
-typing keyword into Google to see how many results there are
-scanning the results on the first page, to see "who" my competition is (if a 'big'/widely popular site - i.e. somebody too big to compete with - comes up in the first few results, I'm not gonna try and compete with that - I'll either long-tail it some more, or go after a different base keyword)

There has to be more / better ways.

Doing some keyword research in the above fashion, I came up with a keyword that I thought would be a good idea. It gets a lot of hits, yet has relatively few results in Google (we're talking results are only maybe 3.5x the number of searches). Yet, it's got crazy competition as far as PPC goes (which I was expecting given the keyword). However, when I scan the first page, the results are pretty..."iffy". For example, a simple ezine article ranked something like #5. There were no actual products of any kind on the entire 1st page, and I was kinda surprised to see that none (that I remember) of at least the top 5 results even had the keyword in the title of the webpages themselves.

And the keyword is a very basic keyword - not long-tail or anything like that.

I'm pretty certain I didn't just stumble onto what would be an apparent "gold mine", given that there seems to be a ton of searches with few results (and what may even appear to not be very good results at that).

I'm bound to be missing the boat here somewhere - can one of y'all tell me what I'm goofing up, not doing, or possibly misinterpreting?

Thanks in advance.