How to block subdomain in Robots.txt

PromoDirect · 11-10-2011, 07:43 AM

User-agent: *
Disallow: /

=================

This is how my current robots.txt file looks like. I want to block a subdomain (i.e http://subdomain.mywebsite.com) using robots.txt, I don't want even a single page of my subdomain to be indexed in Google.

I added the above text to prevent my Subdomain from indexing... but it still got indexed into Google. What should I do to entirely prevent my Subdomain from Indexing.

webdevpro · 11-10-2011, 12:55 PM

You need to create a separate robots.txt file for each subdomain you want to control and put the above code in that. Be sure it should not be accessed by the main domain or your main site will be blocked. Its not that easy but you need some programing skill to get this effectively.

PromoDirect · 11-16-2011, 07:28 AM

I have the robots.txt separate under the sub domain. Someone suggested me the following:
Instead of:
User-agent: *
Disallow: /

I must have in robots.txt
User-agent: *
Disallow: /http://subdomain.mywebsite.com

I can see that after 3 weeks I have 22000 pages still showing indexed out of 24000 that was showing earlier. Does Google usually takes so long to remove pages from its index? Is there a quick way of doing it?

11-10-2011, 07:43 AM	#1
PromoDirect HyperActive Warrior Join Date: Jan 2011 Posts: 179 Thanks: 0 Thanked 7 Times in 7 Posts	How to block subdomain in Robots.txt User-agent: * Disallow: / ================= This is how my current robots.txt file looks like. I want to block a subdomain (i.e http://subdomain.mywebsite.com) using robots.txt, I don't want even a single page of my subdomain to be indexed in Google. I added the above text to prevent my Subdomain from indexing... but it still got indexed into Google. What should I do to entirely prevent my Subdomain from Indexing.
	Promo Direct -- We bring more than 20 years of experience in the Promotional Products industry. Imprinted Promotional Calendars 2012 \| Custom Imprinted Promotional Bags \| Imprinted Personalized Promotional Pens

11-10-2011, 12:55 PM	#2
webdevpro HyperActive Warrior Join Date: Nov 2009 Posts: 294 Thanks: 0 Thanked 23 Times in 22 Posts	Re: How to block subdomain in Robots.txt You need to create a separate robots.txt file for each subdomain you want to control and put the above code in that. Be sure it should not be accessed by the main domain or your main site will be blocked. Its not that easy but you need some programing skill to get this effectively.
	Dofollow Backlink Checker \| 60+ Free Webmaster Tools \| Fake PR Checker \| SEO Forums 22,000+ List of Directories including Blogs, Forums, Press Release, Social Media... (sort by PR & Alexa)

11-16-2011, 07:28 AM	#3
PromoDirect HyperActive Warrior Join Date: Jan 2011 Posts: 179 Thanks: 0 Thanked 7 Times in 7 Posts	Re: How to block subdomain in Robots.txt I have the robots.txt separate under the sub domain. Someone suggested me the following: Instead of: User-agent: * Disallow: / I must have in robots.txt User-agent: * Disallow: /http://subdomain.mywebsite.com I can see that after 3 weeks I have 22000 pages still showing indexed out of 24000 that was showing earlier. Does Google usually takes so long to remove pages from its index? Is there a quick way of doing it?
	Promo Direct -- We bring more than 20 years of experience in the Promotional Products industry. Imprinted Promotional Calendars 2012 \| Custom Imprinted Promotional Bags \| Imprinted Personalized Promotional Pens