Website Spiders Gone Wild

3 replies
What's going on with all the spiders?

I launched a site on Monday and so far 1786 spiders have visited
my site. Granted, I built up the site first and launched it with 30
or so pages on it - but it seems nearly 2000 spiders in a week is
a bit much, right? 298 spiders visited the site yesterday.

I'm not sure if I should enjoyed being crawled all over by spiders
or take out a can of Raid and somehow blast these suckers away.

I only mention the above because my other websites are not hit
by spiders this much - not even close. I don't know if I should
be happy or concerned by all this activity.

Thoughts???
#spiders #website #wild
  • Profile picture of the author 2WDHost
    Hi.

    Generally, as long as it doesn't affect the speed of your site etc., there should be nothing to worry about. But, just in case:
    Where exactly do you check the spider stats? Can you see host names of those spiders and what exactly they are trying to access?
    {{ DiscussionBoard.errors[5295849].message }}
  • Profile picture of the author waqasikd
    May B you loss you bandwith
    {{ DiscussionBoard.errors[5320605].message }}
  • Profile picture of the author ashleysmith12
    hi
    its good that you crawled, better to stick what your doing.
    {{ DiscussionBoard.errors[5320768].message }}

Trending Topics