7 {{ upvoteCount | shortNum }}
7 {{ upvoteCount | shortNum }}

Where can I get massive lists of keywords like SEMRush?

by Dan B Rusu
5 replies
Tools like SEMRush - service for competitors research, shows Google organic and AdWords keywords for any site or domain, SpyFu, and http://www.keywordspy.com/ have all collected tons of data using millions of keywords. SEMRush, if you look to the bottom right of their site, claim to have 80M+ keywords.

My question is, where do they get (or how do they develop) this list of keywords? I've been racking my brains and have got nothing. It would have to be a very extensive list that would include tons of industries and topics.

Pointing me in the right direction would be greatly appreciated!

Dan
#keywords #lists #massive #semrush
  • Profile picture of the author Mr Steve
    Try nichebotclassic. It always works for me.
    {{ DiscussionBoard.errors[4306317].message }}
  • Profile picture of the author razorico
    Hey, why do you want to know this? And where're you going to use this info? Or maybe you want to become a competitor? How can you be a competitor, if you can't find out solution yourself.
    {{ DiscussionBoard.errors[4316236].message }}
    • Profile picture of the author Dan B Rusu
      Originally Posted by Mr Steve View Post

      Try nichebotclassic. It always works for me.
      Thanks. I took a look and not exactly what I'm looking for. I'm looking for just a huge list of keywords. Kindof like the free dictionaries on here: Kevin's Word List Page, but instead of words, keywords.

      Originally Posted by razorico View Post

      Hey, why do you want to know this? And where're you going to use this info? Or maybe you want to become a competitor? How can you be a competitor, if you can't find out solution yourself.
      Uhh i want to know this so I can develop a program that uses massive lists of keywords... Why should I dig for hours to find a solution when someone on here probably has already. Isn't that the point of a forum?
      Signature
      {{ DiscussionBoard.errors[4324686].message }}
      • Profile picture of the author razorico
        Uhh i want to know this so I can develop a program that uses massive lists of keywords... Why should I dig for hours to find a solution when someone on here probably has already. Isn't that the point of a forum?
        About purpose of the forum you're absolutely right, but I think that the information you're looking for is a commercial secret, and only company's employees possess it . I think you should ask them, maybe you are lucky and they will give you something interesting.
        {{ DiscussionBoard.errors[4345888].message }}
  • Profile picture of the author Subsonic
    It's possible to build that kind of list (of course it will take heck lot of time to build a list of 10s of millions of keywords..) by scraping keyword suggestions from sites like Google, Yahoo, Ebay and Amazon. The problem with that sort of activity is that it's against the TOS of those sites. That's why I doubt SEMRush would be doing it that way, however it's possible!

    I've developed a program for my own use which scrapes keyword suggestions from Google using a seed keyword. The keyword suggestions are actually downloaded from the same server as where the Google engine downloads them. Here's the query for getting keyword suggestions using a seed keyword:

    clients1.google.com/complete/search?hl=en&gl=&q=[KEYWORD HERE]

    If I test this using a keyword "Daniel" I will get this list as a result (the original list looks different, this is a parsed version):

    daniel tosh
    daniel radcliffe
    danielle staub
    daniel craig
    daniel day lewis
    daniel defense
    daniel boone
    danielle fishel
    daniel pearl
    daniel fast

    I guess that this is a good starting point for you. There are certain techniques for scraping huge amounts of keywords but I guess you'll figure them out by yourself if you decide to take a closer look at this stuff

    I hope that this helps!

    Edit: Okay I wasn't thinking clearly. I was only talking about keywords but forgot about all that traffic, CPC and other stuff! Some people use scripts or applications to scrape that data from Google's Keyword Suggestion Tool (that Adwords tool) but it has been increasingly hard lately because G has changed the tool a lot and made it harder for developers to scrape content.

    That's of course because Google wants people to use the official API and pay for that information. SEMRush might use Google Adwords API to get all that data but if it does, it's extremely expensive. However I think that it's probably what they do and it's also legal!

    Oops, my post looks horrible now but I hope that you get something out of it, haha!
    {{ DiscussionBoard.errors[4346124].message }}

Trending Topics