robots.txt for Wordpress sample / advice needed

by Gec
2 replies
  • SEO
  • |
Hey folks!

In your experience, what are the best practices for robots.txt file on a Wordpress blog?

I'm looking for general advice, but I also have specific questions regarding the categories, archives, and tags - should these be disallowed?

Thanks!
#advice #needed #robotstxt #sample #wordpress
  • Profile picture of the author PROmotions LLC
    [DELETED]
    {{ DiscussionBoard.errors[8151359].message }}
    • Profile picture of the author Gec
      Thanks! I already have a robots.txt file and plugin, I was asking about "best practices", things that I can control by settings... for example, specific questions:

      1. Should category pages be allowed or disallowed?
      2. Should tag pages be allowed or disallowed?
      3. Should archive pages be allowed or disallowed?

      In your experience, for SEO purposes, what works better? Thanks again!
      Signature

      -

      {{ DiscussionBoard.errors[8151383].message }}
  • Profile picture of the author Aniket
    This is my Robot.txt

    User-agent: *
    Disallow: /cgi-bin/
    Disallow: /wp-admin/
    Disallow: /wp-content/
    Disallow: /wp-includes/
    Disallow: /recommended/
    Disallow: /comments/feed/
    Disallow: /trackback/
    Disallow: /index.php
    Disallow: /xmlrpc.php
    Disallow: /search?
    Disallow: /wp-content/plugins/
    {{ DiscussionBoard.errors[8151990].message }}

Trending Topics