Should I Block Some Forum Pages with robots.txt?
- SEO |
A few months ago, I added forum and I've noticed that there were also some irrelevant pages to the content of my website such as FAQ, help, etc. I use URL rewriting for all the content pages on the forum so I can recognize pages that do not attribute content by ? in the URL.
I figured that inclusion of all those pages could negatively affect my SEO efforts and Google was marking "forum" keyword as most significant keyword on my site.
Couple of weeks ago I decided to block those pages in robots.txt by adding this line for all user agents:
Disallow: /forum/*?
Ever since, when I log into my Webmaster Tools in Google, I get warning stating that there are severe health issues with important pages being blocked by robots.txt. The pages I get forwarded to are the ones I don't want showing in search results. I was just ignoring this for the last few weeks but it' s not going away.
Should I worry about this?
Thanks,
Robert
I figured that inclusion of all those pages could negatively affect my SEO efforts and Google was marking "forum" keyword as most significant keyword on my site.
Couple of weeks ago I decided to block those pages in robots.txt by adding this line for all user agents:
Disallow: /forum/*?
Ever since, when I log into my Webmaster Tools in Google, I get warning stating that there are severe health issues with important pages being blocked by robots.txt. The pages I get forwarded to are the ones I don't want showing in search results. I was just ignoring this for the last few weeks but it' s not going away.
Should I worry about this?
Thanks,
Robert