Wordpress + Robots.txt fails

MaxZimmer · 07-29-2011, 04:35 AM

Good morning WF.

I have several wordpress blogs and never had problems. Lately I read that there could be a problem with duplicate content(e.g. single post + archives).

I decided to upload a robots.txt to avoid this problem. But instead, Google has indexed all the sites that I do not want to be indexed? What am I doing wrong?!

Here's my robots.txt:

Code:

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /feed
Disallow: /*/feed
Disallow: /comments
Disallow: /author
Disallow: /tag
Disallow: /archives
Disallow: /2010/*
Disallow: /2011/*
Disallow: /2012/*
Disallow: /iframes
Disallow: /category/*/*
Disallow: */trackback
Disallow: /*.php$
Allow: /sitemap.xml
Allow: /images

For example, the categories are indexed :/

Thanks in Advance!

MaxZimmer · 07-31-2011, 09:52 AM

No ideas anyone?
Need the help urgently :/

Thanks in Advance!
Max

07-29-2011, 04:35 AM	#1
MaxZimmer Active Warrior Join Date: Apr 2011 Location: Berlin, Germany Posts: 49 Thanks: 10 Thanked 2 Times in 2 Posts	Wordpress + Robots.txt fails Good morning WF. I have several wordpress blogs and never had problems. Lately I read that there could be a problem with duplicate content(e.g. single post + archives). I decided to upload a robots.txt to avoid this problem. But instead, Google has indexed all the sites that I do not want to be indexed? What am I doing wrong?! Here's my robots.txt: Code: User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /feed Disallow: //feed Disallow: /comments Disallow: /author Disallow: /tag Disallow: /archives Disallow: /2010/ Disallow: /2011/* Disallow: /2012/* Disallow: /iframes Disallow: /category// Disallow: /trackback Disallow: /.php$ Allow: /sitemap.xml Allow: /images For example, the categories are indexed :/ Thanks in Advance!

07-31-2011, 09:52 AM	#2
MaxZimmer Active Warrior Join Date: Apr 2011 Location: Berlin, Germany Posts: 49 Thanks: 10 Thanked 2 Times in 2 Posts	Re: Wordpress + Robots.txt fails No ideas anyone? Need the help urgently :/ Thanks in Advance! Max