Is This the Correct Code for Robot.txt??

wm42150 · 11-11-2011, 09:43 AM

Hey all,

Is the following the correct code for to put in my robot.text file?

User-agent: *
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /cgi-bin
Disallow: /search
Disallow: /feed
Disallow: /rss
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$

User-agent: *
Disallow: /blog/wp-admin
Disallow: /blog/wp-includes
Disallow: /blog/wp-content/plugins
Disallow: /blog/wp-content/cache
Disallow: /blog/wp-content/themes
Disallow: /blog/trackback
Disallow: /cgi-bin
Disallow: /search
Disallow: /blog/feed
Disallow: /blog/rss
Disallow: /blog/comments/feed
Disallow: /blog/feed/$
Disallow: /blog/*/feed/$
Disallow: /blog/*/feed/rss/$
Disallow: /blog/*/trackback/$

I have a blog attached to the main domain called www.123.com/blog.... do I put my blog robot text in the main domain robot.txt file as shown above? eg www.123.com/robot.txt

Thanks!

11-11-2011, 09:43 AM	#1
wm42150 Active Warrior Join Date: Jan 2008 Location: , , . Posts: 31 Thanks: 2 Thanked 0 Times in 0 Posts	Is This the Correct Code for Robot.txt?? Hey all, Is the following the correct code for to put in my robot.text file? User-agent: * Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /trackback Disallow: /cgi-bin Disallow: /search Disallow: /feed Disallow: /rss Disallow: /comments/feed Disallow: /feed/$ Disallow: //feed/$ Disallow: //feed/rss/$ Disallow: //trackback/$ User-agent: Disallow: /blog/wp-admin Disallow: /blog/wp-includes Disallow: /blog/wp-content/plugins Disallow: /blog/wp-content/cache Disallow: /blog/wp-content/themes Disallow: /blog/trackback Disallow: /cgi-bin Disallow: /search Disallow: /blog/feed Disallow: /blog/rss Disallow: /blog/comments/feed Disallow: /blog/feed/$ Disallow: /blog//feed/$ Disallow: /blog//feed/rss/$ Disallow: /blog/*/trackback/$ I have a blog attached to the main domain called www.123.com/blog.... do I put my blog robot text in the main domain robot.txt file as shown above? eg www.123.com/robot.txt Thanks!