Need Robots.txt help
I never had robots files on my sites. They are all wordpress blogs. I came up with a file, and want an opinion before I put it on my sites.
Also is it best to put it in the public_html of the domain, or is the wordpress plugin for a robots.txt file just as good. I also noticed that in the sitemap plugin, you can click an option to have it built in the root directory, and then edit it.
Which way is best?
Here is the robots.txt file I am thinking of using. Thanks Glenn
User-agent: *
Disallow: /feeds/
Disallow: /feeds
Disallow: /auction.php
Disallow: /auctions.php
Disallow: /get.php
Disallow: /wp-content/
Disallow: /wp-content
Disallow: /wp-content/plugins/
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /trackback
Disallow: /comments
Disallow: */trackback
Disallow: /category/*/*
Disallow: */comments
Disallow: /*?*
Disallow: /*?
-
Karen Blundell -
Thanks
Signature---------------{{ DiscussionBoard.errors[1549262].message }} -
-
customertools -
Thanks
{{ DiscussionBoard.errors[1549892].message }} -