4 {{ upvoteCount | shortNum }}
4 {{ upvoteCount | shortNum }}

Email Scraper with DB Connectivity

by ben341
3 replies
Hi there,

Not sure if this is the right place. There are a lot of email scrapers out there. I personally extract data from google places and yell.

On one program that extracts the data from GP it does extract email addresses from the website associated with the profile, but it has limitations.

Extracting data from yell, or other places, does not usually extract this data.

It is possible to take the URLs from the list that do not have an email address in the extraction results and run it through a program like scrapebox to obtain email addresses, but you just get a list of addresses and no idea which company it links to.

Basically, it would be great if I could find a program that could read my csv or mysql database of extracted data, lookup the url for each record, scrape an email address from the website and place it in an 'email address' column within my db or csv. This would improve the yeild in responses, and allow me to remove inappropriate email addresses and so on.

In addition, it would be good if the program could be configured to ignore certain words, such as webmaster, spam, oops, admin and so on.

I have looked for a little while now and found nothing that matches this what I would have considered to be simple functionality. If anyone could point me in the right direction of something that could do this I would most appreicate it.

Thank you,
Ben
#connectivity #email #scraper
  • Profile picture of the author MaxReferrals
    Sounds pretty simple, and probably best to build it. Might want to post over in the WFH or Programming Talk section.
    Signature
    {{ DiscussionBoard.errors[5791931].message }}
  • Profile picture of the author xsonz
    Hey Ben,

    I have been looking for the same thing today and that's how I ended up here.

    Well, I can tell you one thing for sure. I am looking for exact same thing as you do and the best program I found so far is Web Data Extractor Pro which unfortunately costs $199 :/

    However, you can download a trial version to see how it works (I am pretty impressed btw) but that is the far it goes because you can't extract more than 10 columns.

    Take a look for yourself, maybe you find it useful.

    Take care,
    Xsonz
    {{ DiscussionBoard.errors[6990712].message }}
    • Profile picture of the author lovol
      Hi

      I've just built exactly what you describe pretty much!

      My App:

      1. Takes a search term and location, enters into Yell - the UK's local business search engine (UK site)
      2. Get's all the details from yell, including keywords etc.
      3. Visits each website in the database (from step 2)
      4. Finds email address on home page if exists - stores this to database
      5. Finds the 'contact us' page or similar - stores this to database
      6. Finds email address(s) on contact us page or similar - stores this to database

      It's fully working, but doesn't 'look nice', it's a console app, you just run it and off it goes. (if you interrupt it, it will continue from where it left off)

      System Requirements
      1. Windows Vista or above with MS .Net 4.5 install (free)
      2. MS SQL Server Express installed (free)

      If you're interested, let me know.
      {{ DiscussionBoard.errors[8643121].message }}

Trending Topics