Robot txt template for wordpress

by 3 replies
4
Hi all,
I have been advise that my previous robot text was rubbish, yet most of the information was taken from the websites regarding the best seo practises for wordpress. I would like to know on your thoughts of the best template robot txt disallows / allows / no index for wordpress websites?
Thank you
#search engine optimization #robot #template #txt #wordpress
  • Well obviously your site needs to allow the search spiders to access it if you want to rank. The only time I think you would really need to worry about setting up a robots would be if there were particular directories/images/files that you didn't want indexed. If you want everything to be found and indexed by Google then I don't think a robots.txt is really necessary. Is there any particular file or folder that you do not want indexed?
  • Banned
    I have no idea why anyone would even attempt to block a category page from Google SERPs.

    It's like throwing away traffic just because you can.

    Good luck with that.
  • What is all the dfd.name stuff?

    You could just disallow it all by doing:

    Disallow: /*dfd.name
  • [DELETED]
Join the discussion

Next Topics on Trending Feed

  • 4

    Robot txt template for wordpress

    Hi all, I have been advise that my previous robot text was rubbish, yet most of the information was taken from the websites regarding the best seo practises for wordpress. I would like to know on your thoughts of the best template robot txt disallows / allows / no index for wordpress websites? Thank you