3 {{ upvoteCount | shortNum }}
3 {{ upvoteCount | shortNum }}

How to block subdomain in Robots.txt

by PromoDirect
2 replies
  • SEO
    • |
User-agent: *
Disallow: /

=================

This is how my current robots.txt file looks like. I want to block a subdomain (i.e http://subdomain.mywebsite.com) using robots.txt, I don't want even a single page of my subdomain to be indexed in Google.

I added the above text to prevent my Subdomain from indexing... but it still got indexed into Google. What should I do to entirely prevent my Subdomain from Indexing.
#block #robotstxt #subdomain
  • Profile picture of the author webdevpro
    You need to create a separate robots.txt file for each subdomain you want to control and put the above code in that. Be sure it should not be accessed by the main domain or your main site will be blocked. Its not that easy but you need some programing skill to get this effectively.
    Signature
    {{ DiscussionBoard.errors[5028231].message }}
  • Profile picture of the author PromoDirect
    I have the robots.txt separate under the sub domain. Someone suggested me the following:
    Instead of:
    User-agent: *
    Disallow: /

    I must have in robots.txt
    User-agent: *
    Disallow: /http://subdomain.mywebsite.com

    I can see that after 3 weeks I have 22000 pages still showing indexed out of 24000 that was showing earlier. Does Google usually takes so long to remove pages from its index? Is there a quick way of doing it?
    {{ DiscussionBoard.errors[5062587].message }}

Trending Topics