Is your host blocking SEMrush?
- SEO |
Most of the hosts are owned by EIG aka Endurance International Group - about 84 at time of this writing. To be certain it is necessary to contact your host support in the event of changes.
Two popular hosts and hosting packages recommended on this forum and others owned by EIG are BlueHost and HostGator - Shared Hosting. To see if your host is owned by EIG you can reference this article EIG list (not affiliated).
You can read more about it on Reddit (not affiliated):
FYI SEMrush bot is blacklisted by Bluehost server
A SEMrush representative confirmed it and basically says the SEMrush bot is blocked by the web host on shared hosting packages and they have tried to contact EIG, but EIG will not respond:
FeelTheRush11, Sam from SEMrush Hi, Sam from SEMrush here! Thanks for your question! First of all we feel sorry for this negative experience you've faced! However, currently, we are experiencing a block of our crawlers from Bluehost hosting provider. We believe that such actions from Bluehost are not justified. We are constantly trying to open a conversation with their leadership but so far all our attempts to establish productive conversation with them have been ignored. I am afraid the common whitelisting of our bot doesn't work on their side for shared servers. The only solution at the moment is to switch to VPS / Dedicated Hosting and whitelist our bot. I would like to assure you that this is a top priority for us but we can't give you any specific timeframes as we solely depend on Bluehost in this matter. We hope for your understanding! Referring to your question, if our crawler slows down your website and you do not have a crawl delay directive in your robots.txt file, you can tell SEMrush to crawl 1 URL per 2 seconds. This may force your audit to take longer to complete, but it will cause less potential speed issues for actual users on your website during the audit. |
There are many a thread on this forum wherein some SEO Agencies recommend SEMrush to their clients. There are also many SEO Agencies that do "not" recommend SEMrush. There are also many a member that recommend SEMrush. There are also many a member that do "not" recommend SEMrush.
In my experience the main reason some (not all) SEO Agencies recommend SEMrush is because it gives the SEO Agency a reference to a metric that the customer is lead to believe as proof of the SEO Agency's competence.
Now that it is verified by some web host support staff (according to some reddit members) the SEMrush bot is blocked/blacklisted will SEO Agencies stop recommending SEMrush?
If SEO Agencies stop using SEMrush what are the alternitives and will the alternatives eventually be blocked by web hosts?
-
freelancernational Banned{{ DiscussionBoard.errors[11619132].message }}-
Jeffery -
Thanks - Reply
SignatureIn the minute it took me to write this post.. someone died of Covid 19. RIP.The Best Source of Free Traffic is in This Very Forum.Sticky: < ! S U C C E S S ! > See how I get my "first 10,000 Youtube Subscribers"{{ DiscussionBoard.errors[11619147].message }} -
-
-
dsimms -
Thanks - Reply
- 1 reply
{{ DiscussionBoard.errors[11619140].message }}-
Jeffery -
Thanks - Reply
SignatureIn the minute it took me to write this post.. someone died of Covid 19. RIP.The Best Source of Free Traffic is in This Very Forum.Sticky: < ! S U C C E S S ! > See how I get my "first 10,000 Youtube Subscribers"{{ DiscussionBoard.errors[11619142].message }} -
-