Web Scraping .. for cheap?

sOliver · 05-16-2009, 10:00 AM

Well, I need to extract a lot of data from some websites. I know there are some tools to do it and I also tried doing it via Xpath and Curl, but it's still too much work.

Does anyone have experience with it and can recommend me a data mining/web scraping service? Or would it be cheaper to hire someone to do "data entry"? I actually don't know how many entries there are but I guess around +-100,000.

I'm on a tight budget, so it should be pretty cheap.

Any tips are appreciated.

dvduval · 05-16-2009, 10:02 AM

The problem is more related to server loads and storage and then searchability. We are actually doing this for a customer right now, and as they grow, the hardware gets really important.

05-16-2009, 10:00 AM	#1
sOliver Active Warrior Join Date: Mar 2009 Posts: 85 Thanks: 3 Thanked 7 Times in 6 Posts	Web Scraping .. for cheap? Well, I need to extract a lot of data from some websites. I know there are some tools to do it and I also tried doing it via Xpath and Curl, but it's still too much work. Does anyone have experience with it and can recommend me a data mining/web scraping service? Or would it be cheaper to hire someone to do "data entry"? I actually don't know how many entries there are but I guess around +-100,000. I'm on a tight budget, so it should be pretty cheap. Any tips are appreciated.

05-16-2009, 10:02 AM	#2
dvduval phpLD master War Room Member Join Date: Dec 2006 Location: Silicon Valley Posts: 3,120 Blog Entries: 3 Thanks: 49 Thanked 334 Times in 273 Posts Social Networking	Re: Web Scraping .. for cheap? The problem is more related to server loads and storage and then searchability. We are actually doing this for a customer right now, and as they grow, the hardware gets really important.