Website Design

  • 8 {{ upvoteCount | shortNum }}

    Blocking Bots is this right?

    AffiliatingAlan in Web Design

    I want to block these bots from seeing my crawling my site.Which of these is the best and will not brick my site? PHP Code: RewriteEngine onRewriteCond %{HTTP_USER_AGENT} rogerbot [OR]RewriteCond %{HTTP_USER_AGENT} exabot [OR]RewriteCond %{HTTP_USER_AGENT} mj12bot [OR]RewriteCond %{HTTP_USER_AGENT} dotbot [OR]RewriteCond %{HTTP_USER_AGENT} gigabot [OR]RewriteCond %{HTTP_USER_AGENT} ahrefsbot [OR]RewriteCond %{HTTP_USER_AGENT} sitebotRewriteRule .* - [F]  PHP Code: RewriteEngine OnRewriteBase /RewriteCond %{HTTP_USER_AGENT} ^rogerbot [OR]RewriteCond %{HTTP_USER_AGENT} ^exabot [OR]RewriteCond %{HTTP_USER_AGENT} ^MJ12bot [OR]RewriteCond %{HTTP_USER_AGENT} ^dotbot [OR]RewriteCond %{HTTP_USER_AGENT} ^gigabot [OR]RewriteCond %{HTTP_USER_AGENT} ^AhrefsBotRewriteRule ^.* - [F,L]  PHP ... [read more]