Fetch as GOOGLEbot FAIL

[lovboa]

Is anyone else having problems with their Google Webmaster Tools "Fetch as Googlebot"? Could somebody please check for me?

I just changed my theme a minute ago and then logged into Webmaster to Fetch as Googlebot and it is failing all 5 times I tried.
The page keeps refreshing, but it shows Pending status and then Failed.


Thanks, everybody

[lovboa]

Well....im using Google Chrome...... heh

[vasionit]

well recently there are on going changes in google webmaster, analytics and adsense interface.. i guess they are testing all at once and which i also notice when i add new site in webmaster and still shows pending status..

I guess you check back after 24-48 hours time period.. it will be fine.. i notice my site pending status gone after almost next day..

[SuzanneH]

There are a number of people having this problem over at the Google Webmaster Tools forum (Crawling, indexing & ranking - Webmaster Central Help ) People there are trying to find what they have in common. Are you, by any chance, using CloudFlare? (I'm not saying that's the problem, necessarily. But CloudFlare *appears* to be something they have in common.) Check out the Google forum link I posted to search for more info.

Suzanne

[damoncloudflare]

We've done some testing at CloudFlare with the domains that have reported an issue and we haven't had any indication that it is limited to CloudFlare. Some CloudFlare customers have turned us off as a test & the same issue is happening with CloudFlare off or nameservers switched away in some cases. We've also been able to grab the robots.txt files using our IPs, so it doesn't appear that we're being blocked either.

It looks like a Google employee is responding in this thread & Google did make some tweaks that should help the crawl rate, apparently.

[Josh555]

If you are using a wordpress cms/core - and using the google analyticator plugin - this fix should work for "fetch as googlebot failed".

www(dot)joshclosser(dot)com/2011/11/23/fetch-googlebot-failed-fix-2031180.jsp

[damoncloudflare]

Originally Posted by damoncloudflare

We've done some testing at CloudFlare with the domains that have reported an issue and we haven't had any indication that it is limited to CloudFlare. Some CloudFlare customers have turned us off as a test & the same issue is happening with CloudFlare off or nameservers switched away in some cases. We've also been able to grab the robots.txt files using our IPs, so it doesn't appear that we're being blocked either.

It looks like a Google employee is responding in this thread & Google did make some tweaks that should help the crawl rate, apparently.

This issue with Google should now be resolved & we've confirmed that they have fixed any issues with crawling relative to CloudFlare. Please do let us know if you see any issues & we can reach out to the Google crawl team.