How to crawl a website & build a product feed?

willyboy104 · 02-22-2013, 03:37 AM

I've generally outsourced this activity for most of my clients however I am researching in to building product feeds for use with Google Shopping myself however I am wondering how do I go about crawling a site to get all the appropriate data, such as:

Product URL
Image URL
Product Description
Price
Brand

e.t.c

I could obviously do this manually, but when a client has in excess of 5,000 products it is not feasible.

Anyone got any advice?

Thank you!

willyboy104 · 02-22-2013, 05:32 AM

Thanks, but no thanks - for the pointless, blatant promotion of your own services/lead generation in your site links.

-_-

Sillysoft · 02-22-2013, 07:17 AM

I built something like this a long time ago. It was a php spider that would spider a site, get the product information and put it into an xml file then auto submit to places like froogle.

What site are you trying to obtain these products? I wouldnt mind seeing if I could get my old script to try and grab those products.

02-22-2013, 03:37 AM	#1
willyboy104 Advanced Warrior War Room Member Join Date: Apr 2009 Location: London, UK Posts: 865 Thanks: 311 Thanked 120 Times in 57 Posts Social Networking Contact Info	How to crawl a website & build a product feed? I've generally outsourced this activity for most of my clients however I am researching in to building product feeds for use with Google Shopping myself however I am wondering how do I go about crawling a site to get all the appropriate data, such as: Product URL Image URL Product Description Price Brand e.t.c I could obviously do this manually, but when a client has in excess of 5,000 products it is not feasible. Anyone got any advice? Thank you!
	Looking for a PPC Professional - I can help. Paid Media Expert *Manage International Brands - American Express, Nespresso, Game Group, Boots & More* PM or Add me to Skype, and let's discuss! skype: willyboy104

02-22-2013, 05:32 AM	#2
willyboy104 Advanced Warrior War Room Member Join Date: Apr 2009 Location: London, UK Posts: 865 Thanks: 311 Thanked 120 Times in 57 Posts Social Networking Contact Info	Re: How to crawl a website & build a product feed? Thanks, but no thanks - for the pointless, blatant promotion of your own services/lead generation in your site links. -_-
	Looking for a PPC Professional - I can help. Paid Media Expert *Manage International Brands - American Express, Nespresso, Game Group, Boots & More* PM or Add me to Skype, and let's discuss! skype: willyboy104

02-22-2013, 07:17 AM	#3
Sillysoft SumoCoder Join Date: Jan 2012 Posts: 596 Thanks: 15 Thanked 70 Times in 66 Posts	Re: How to crawl a website & build a product feed? I built something like this a long time ago. It was a php spider that would spider a site, get the product information and put it into an xml file then auto submit to places like froogle. What site are you trying to obtain these products? I wouldnt mind seeing if I could get my old script to try and grab those products.
	No more...