Google Says If You Are Blocking a Country's Traffic, You Should Be Careful Not to Block Googlebot, T
- SEO |
A new article on Search Engine Journal reports that website publishers need to be careful not to block Googlebot when they're blocking entire countries from accessing their site.
This was Google's John Mueller speaking on the company's Search Central hangout recently, in which he advised site owners to be careful not to block Googlebot if they choose to block traffic from an entire country. Mueller lives in Switzerland, so he isn't that fond of blocking entire countries, but he understands why people do it:
For SEO professionals and web publishers, the key point here is to not block Googlebot from crawling a website:
This was Google's John Mueller speaking on the company's Search Central hangout recently, in which he advised site owners to be careful not to block Googlebot if they choose to block traffic from an entire country. Mueller lives in Switzerland, so he isn't that fond of blocking entire countries, but he understands why people do it:
|
"I've heard some people block individual countries because they see a lot of problematic traffic from individual countries. Personally, I don't really like that approach because Switzerland is one of those countries as well that people tend to say, "oh, this is another European country, I will block it. So I don't like it from a personal point of view. But if you're seeing significant issues with traffic from an individual country, then maybe that makes sense." |
|
"From a search point of view, we primarily crawl from the US. We don't always crawl from the U.S., but we primarily do so. And essentially, that means, if you need to block individual users and individual countries, then you need to make sure that you're not blocking Googlebot, so you're not blocking users in the U.S., for example. So that's kind of the search-specific aspect I would watch out for there. If you need to block individual countries, just make sure that's not a country where Googlebot is crawling your site from. You can confirm Googlebot traffic with a reverse DNS lookup if you want to do that. But, essentially, for most websites we would crawl from the U.S. I think it's very rare that we would crawl sites from other locations, and we generally don't crawl sites from multiple locations. So it's not going to be the case that we would crawl from the U.S. and from, I don't know, Singapore or some random other country. We would generally pick one country to crawl this website from." |
AMRAY List of Top 50 Web Directories <> Best 5 Web Directories Reviews