GoogleBot History Question

Sebulba · 08-19-2009, 08:24 AM

I know that I can look at my Cached information in Google for my site and see the last Time Google crawled my site, but does anyone know if there is a way to tell when it was crawled before that? Like a list of times it was crawled?

Thanks

Seb

Ben Roy · 08-19-2009, 08:35 AM

In the webmaster tools they show some information about frequency of crawling, pages pre day, etc. There's a graph. It's not terribly detailed. If you want lots of detail about what's happening, you'd need to go to your web server logs. That will show you every single page request that the crawler (all crawlers, really) make to your site.

Peter Adamson · 08-19-2009, 09:57 AM

Your own web logs, if you keep them. Unfortunately most hosting providers bin them after just a few days so if you don't download them regularly you lose them. If you keep your logs, use Sawmill or something similar and you can get all the information you want.

Sebulba · 08-19-2009, 01:06 PM

Thanks for the help

Seb

08-19-2009, 08:24 AM	#1
Sebulba Wyoming Warrior War Room Member Join Date: Mar 2009 Posts: 191 Thanks: 60 Thanked 13 Times in 9 Posts	GoogleBot History Question I know that I can look at my Cached information in Google for my site and see the last Time Google crawled my site, but does anyone know if there is a way to tell when it was crawled before that? Like a list of times it was crawled? Thanks Seb
	Learning about offline and online marketing

08-19-2009, 08:35 AM	#2
Ben Roy Advanced Warrior War Room Member Join Date: Jan 2007 Location: Minnesota Posts: 612 Thanks: 13 Thanked 47 Times in 41 Posts Social Networking Contact Info	Re: GoogleBot History Question In the webmaster tools they show some information about frequency of crawling, pages pre day, etc. There's a graph. It's not terribly detailed. If you want lots of detail about what's happening, you'd need to go to your web server logs. That will show you every single page request that the crawler (all crawlers, really) make to your site.
	Stop social bookmarking the old fashioned way - the future is here

08-19-2009, 09:57 AM	#3
Peter Adamson Active Warrior War Room Member Join Date: Jun 2009 Location: United Kingdom Posts: 341 Thanks: 15 Thanked 144 Times in 55 Posts	Re: GoogleBot History Question Your own web logs, if you keep them. Unfortunately most hosting providers bin them after just a few days so if you don't download them regularly you lose them. If you keep your logs, use Sawmill or something similar and you can get all the information you want.
	Get backlinks! Up to 1,500 social media backlinks per month for $47! Earn passive recurring revenues as an affiliate, too!!! One month FREE trial! Get The Link Juicer now!

08-19-2009, 01:06 PM	#4
Sebulba Wyoming Warrior War Room Member Join Date: Mar 2009 Posts: 191 Thanks: 60 Thanked 13 Times in 9 Posts	Re: GoogleBot History Question Thanks for the help Seb
	Learning about offline and online marketing