fake googlebot?

by 10 replies
12
I have some serious bandwidth problem on my website/hosting account, but I cannot tie all the details together. Maybe someone has experience with this.

1. According to the host's webalytics a few of my sites are getting hundreds of visitors. According to Google Analytics, these sames sites get between 2 and 20 visitors per day (instead of hundreds).

2. According to the host's webalytics, 95% to 98% of the hits are caused by agents/bots, of which the main one has the name Googlebot. According to Google Webmastertools, the spider only crawls about 25K per dag and just a few thousand hits per day (which is only a few hits per minute).

Could it be fake bots that are attaching my site under the Googlebot name? How can I make sure (e.g. via robots.txt) that only real Bots are visiting my sites? Any other ideas or experience with this is welcome!
#programming #fake #googlebot
  • Which host are you using? Their analytics may not be accurate.

    You could be suffering a DDOS attack. Well, I am not an expet at this but I think that's the way those attacks work by simulating visitors to your site

    • [1] reply
    • Its a Dutch provider (coloprovider) and I have a VPS with them. It is not a one time thing, but continues workload on the server. I'm sure it is a bot, but I just want to make sure its Google (despite webmastertools says its not).
      • [1] reply
  • Even if it's genuine Google Bot, you can change crawling rate for your website from Google Webmaster Tools.
  • [DELETED]
  • Hi
    Our company recently released a study of fake Googlebot behavior that shows an average of 21% for fake Googlebot visits (75% of these were directly harmful)

    Read more here: Fake Googlebot study

    To prevent Googlebot impersonation attempts, you can use this Validation technique suggested by Google.

    Also, if these are not fake visits, you can control the visit frequency via WMT.
  • You must kill the fake Googlebots, before they kill your business.

    Also, if you can trace the IP address to a person in the U.S.(doubtful)< they owe you for lost revenue.
  • This is my answer based my experience :

    1. Check bot's IP address (if their IP address is not match with google/other SE, then it's bot).

    2. Maybe someone steal your contents, yeah unbelievable right?
    They harvesting your content, yes I have done that, and creates PHP script to crawling other website is very easy. Be careful. Back to step #1
  • If you have total control over it then you can do that.
    Or you can give some complains to the google team.
Join the discussion

Next Topics on Trending Feed

  • 12

    fake googlebot?

    I have some serious bandwidth problem on my website/hosting account, but I cannot tie all the details together. Maybe someone has experience with this. 1. According to the host's webalytics a few of my sites are getting hundreds of visitors. According to Google Analytics, these sames sites get between 2 and 20 visitors per day (instead of hundreds).