6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

Web crawlers messing up my stats - what to do?

by saxguru
5 replies
Hi all,

I am an amazon affiliate and I promote products on my website as well as social media. Each new post on my website is posted to facebook and twitter and then it seems to be crawled by facebook bots (according to the Wordpress Statcounter Plugin)

My problem is, these bots seem to be messing up my conversion rate states etc on amazon. Currently I have 48k clicks and 10 conversions which is absolutely ridiculous. I started the website 2 weeks ago and I know that it doesn't get many unique visitors yet. A more suitable figure would be about 300 click and 10 conversions.

Is there anything I can do about this at all?

thanks
#crawlers #messing #stats #web
  • Profile picture of the author radhika
    Ak your plugin author to filter the bot IP numbers or give you an option to filter bots in plugin interface.

    This way you can see only (mostly) human visitor IPs. Still few IP numbers you can not resolve to bots.

    .
    Signature
    Follow up Autoresponder PRO :: 33% Discount!!
    FREE Upgrades! IMPROVED Email Deliverability!!
    {{ DiscussionBoard.errors[8962063].message }}
  • Profile picture of the author onSubie
    You can block them with your robots.txt file.

    Block all bots:
    Code:
    User-agent: * 
Disallow: /
    Block one bot:
    Code:
    User-agent: name_of_bot 
Disallow: /
    You can have multiple entries for multiple bots.
    {{ DiscussionBoard.errors[8962078].message }}
    • Profile picture of the author radhika
      Originally Posted by onSubie View Post

      You can block them with your robots.txt file.

      Block all bots:
      Code:
      User-agent: * 
Disallow: /
      Block one bot:
      Code:
      User-agent: name_of_bot 
Disallow: /
      You can have multiple entries for multiple bots.
      This completely stops the site crawled and listed in search engines.


      .
      Signature
      Follow up Autoresponder PRO :: 33% Discount!!
      FREE Upgrades! IMPROVED Email Deliverability!!
      {{ DiscussionBoard.errors[8962091].message }}
      • Profile picture of the author saxguru
        Originally Posted by radhika View Post

        This completely stops the site crawled and listed in search engines.
        I definitely won't be doing that then. 90% of my traffic is organic from the search engines
        {{ DiscussionBoard.errors[8962106].message }}
      • Profile picture of the author onSubie
        Originally Posted by radhika View Post

        This completely stops the site crawled and listed in search engines.
        Only if you block all bots with *.

        You can block individual bots by name.

        You can also block all bots but whitelist search engine bots like Google.

        If you don't know your way around robots.txt there are security tools and plugins you can use. They usually just give an interface for options you want and then update files like .htaccess and robots.txt for you.

        Edit: I found this with Google. It gives more detail and examples
        http://www.robotstxt.org/robotstxt.html
        {{ DiscussionBoard.errors[8962131].message }}

Trending Topics