Show me a site with blocked Ahrefs bots

by Murkr
9 replies
  • SEO
  • |
I'm not sure if one of my competitors are blocking Ahrefs and other SEO bots.

What does it look like in ahrefs when they are blocking Ahrefs bots? Anyone know of a website I can check out?

Does everything say 0, or does everything just stick with however it was before it was blocked?
#ahrefs #blocked #bots #show #site
Avatar of Unregistered
  • Profile picture of the author Ba C
    if they are blocking ahrefs they are most likely doing it through their robots.txt file. Check it out theirdomain.com/robots.txt should tell you if they are or not.
    {{ DiscussionBoard.errors[11372408].message }}
  • Profile picture of the author dilipcybex
    Does Ahref respect robots.txt ? I was told that most of them ignore it
    {{ DiscussionBoard.errors[11372539].message }}
    • Profile picture of the author Ba C
      most reputable seo tools like ahrefs will respect robots.txt. You can see here that ahrefs respects robots.txt and how to block it. https://ahrefs.com/robot
      {{ DiscussionBoard.errors[11372661].message }}
      • Profile picture of the author Murkr
        Originally Posted by Ba C View Post

        most reputable seo tools like ahrefs will respect robots.txt. You can see here that ahrefs respects robots.txt and how to block it. https://ahrefs.com/robot
        Do you know unreputable ones who don't haha. Then we'd be able to find a PBN right?


        When a site has blocked bots and you try to search that site in ahrefs does everything come back 0??
        {{ DiscussionBoard.errors[11372810].message }}
        • Profile picture of the author MikeFriedman
          Originally Posted by Murkr View Post

          Do you know unreputable ones who don't haha. Then we'd be able to find a PBN right?


          When a site has blocked bots and you try to search that site in ahrefs does everything come back 0??
          That's not how it works.

          You can only block a bot from crawling a site you own.

          If you want to block all your links in Ahrefs, you cannot do it unless you own every site where a link appears to your site.
          Signature

          For SEO news, discussions, tactics, and more.
          {{ DiscussionBoard.errors[11372812].message }}
          • Profile picture of the author Mike Anthony
            Originally Posted by MikeFriedman View Post


            If you want to block all your links in Ahrefs, you cannot do it unless you own every site where a link appears to your site.
            and you still can get deindexed by Google because if you block Google from your PBNs you get no juice.
            Signature

            {{ DiscussionBoard.errors[11372906].message }}
          • Profile picture of the author Murkr
            Originally Posted by MikeFriedman View Post

            That's not how it works.

            You can only block a bot from crawling a site you own.

            If you want to block all your links in Ahrefs, you cannot do it unless you own every site where a link appears to your site.
            ahh makes total sense now, thank you.
            {{ DiscussionBoard.errors[11372971].message }}
  • Profile picture of the author MikeFriedman
    If they are doing it right, they are blocking it in the .htaccess file.
    Signature

    For SEO news, discussions, tactics, and more.
    {{ DiscussionBoard.errors[11372679].message }}
  • Profile picture of the author Mike Anthony
    Originally Posted by Murkr View Post

    What does it look like in ahrefs when they are blocking Ahrefs bots? Anyone know of a website I can check out?
    The only way you are going to know is if the link profile does not reflect the rank. However be aware of your own bias. Most webmasters tend to always believe their site is better than the ones outranking them
    Signature

    {{ DiscussionBoard.errors[11372907].message }}
Avatar of Unregistered

Trending Topics