Scraping/crawling data

5 replies
Not sure if this is the correct board to post this on, but I would like to discuss algorithmic data gathering here.

Specifically, I have been more and more involved in scraping data for various clients and I've gotten quite good at it.

Recently I acquired about 1.5GB of UK property sales data for a client, and it got me thinking:

- is there an organized market where one could sell such data?
- what is the legal status of scraping? some resources seem to say it's legal because you're just gathering information that's already publicly posted, some say it's in some sort of grey area or dependent on the specific website's terms of service
- has anyone made good money through scraping and would like to share their experience?
#data #scraping or crawling
Avatar of Unregistered
  • Profile picture of the author TripLoop
    What did you use to scrap data?
    {{ DiscussionBoard.errors[11469575].message }}
    • Profile picture of the author Monsignor
      Python's "scrapy" library.
      Signature

      The intelligent investor is a realist who sells to optimists and buys from pessimists. - B. Graham

      {{ DiscussionBoard.errors[11469938].message }}
  • Profile picture of the author bartology
    What you're talking about would be lead generation. The value of the data would be in if you could analyze it to be a set of data that could convert to leads for a business that could use the data. If you can do that, you can make money.
    {{ DiscussionBoard.errors[11469856].message }}
    • Profile picture of the author Monsignor
      Thanks. But how would I go about selling the leads? Is there an organized marketplace for this sort of thing, or do I just find and contact businesses directly?
      Signature

      The intelligent investor is a realist who sells to optimists and buys from pessimists. - B. Graham

      {{ DiscussionBoard.errors[11469939].message }}
  • Profile picture of the author ryanbiddulph
    The only glaring issue with this approach; it lacks a warm, personal, genuine, human touch. Stick with building bonds to build a thriving business, the right way. I Junked-Trashed a few more emails today from bloggers scraping my data and email, sending me unsolicited emails, Silly approach because you chase out of fear, while I attract out of love. Peep my Seen At Page to see how it has worked for me hehehehe
    Signature
    Ryan Biddulph, Blogger, Author, World Traveling Digital Nomad
    If you want to become a full time blogger you can buy my eBook here
    {{ DiscussionBoard.errors[11469974].message }}
Avatar of Unregistered

Trending Topics