Does Hostgator block Googlebot?

[HannahLowell]

I checked my Google webmaster tools couple of days ago and saw that there were some errors on my sitemap. I didn't worry about it much at first but this morning I checked the tools again with the hope of these errrors were fixed. However they were not fixed, so I decided to check these problems deeply.

I had 8 'network unreachable' error in my google webmaster tool. All these were from my sitemap. I checked my robots.txt and .htaccess files. But there was not any change since December 2009 as it is the date when I just set up my website.

And then I started to make some more research just to find out similar issues other people had like mine. It seems that most of these people figure out that the problem is caused by their hosting company. Here are some examples (similar stories):

forums (dot) hostgator (dot) com/my-1-year-hostgator-review-sigh-t33383 (dot) html

cravingtech (dot) com/google-sitemap-error-network-unreachable (dot) html

homewithandrew (dot) com/index (dot) php/debugging-the-network-unreachable-robots-txt-unreachable-error/


So I contacted my hosting company (Hostgator) and explained them the problem. At first they told me that they do NOT block Google bots, then they said that this was happening because google bot was hitting my server too often and their firewall thought it's a spam (they think googlebot got suddenly more aggressive in crawling my server). So then they said they put some delay on my robot.txt for googlebot and should be OK from now on.

After I got this answer I went to my google webmaster tools and wanted to fetch my website as a googlebot. However all the tries returned as still with 'network unreachable' error. So actually this delay did not work and my website is still not accessible by googlebot.

As a result of all the information I got from my research, I really suspect that Hostgator is blocking googlebot for some reason. Maybe accidentally... they may not be aware of the situation. The latest answer I got from Hostgator is that I have to wait for 6-12 hours so that googlebot can crawl my website again.

This sounds like a disaster to me as my website already got dropped in SERPs for almost all of my keywords and this cost already too much for me. Now I do not want to wait 6-12 more hours because waiting will hurt my website further.

Is there anything I can do on my own without hosting's help? Anyone experienced the same problem and somehow solved it with the help of their hosting company? So at least I can show my hosting company these experiences as a reference... And they may solve the issue faster.

Thanks for the help in advance...

[Groovystar]

Hostgator hosts a LOT of sites. I don't think they would try to then block Google from seeing them. That seems kind of like it'd turn off a lot of their customers for no good reason. And if they can't handle traffic from just the google bot, what about the millions of human visitors to the sites?

Maybe I'm just missing something here. Sorry if I am.

[yukon]

I run HostGator sites, trust me they don't block anything...