4 {{ upvoteCount | shortNum }}
4 {{ upvoteCount | shortNum }}

Blocking Search Engines from indexing Pages

by TeamJ
3 replies
  • SEO
    • |
Hello,

What happens if you were to block a page in robots.txt and add a no index tag?

Would the no index tag be redundant since Google wouldn't be able to crawl the page on robots.txt?

Thanks
#blocking #engines #indexing #pages #search
  • Profile picture of the author yukon
    Banned
    Originally Posted by TeamJ View Post

    Hello,

    What happens if you were to block a page in robots.txt and add a no index tag?

    Would the no index tag be redundant since Google wouldn't be able to crawl the page on robots.txt?

    Thanks


    Google will still crawl a page with a noindex tag. The noindex tag tells Google you don't want the page listed in the SERPs.

    Put the noindex tag in the page HTML source code <head> because Google could easily find an alternative route to the page from any other link on the web or your host. This way once Google lands on your page the noindex tag is one of the first things they'll find.
    {{ DiscussionBoard.errors[10915210].message }}
  • Profile picture of the author TeamJ
    My concern is that because the page is listed in robots.txt, the no index tag won't be crawled. Google will still be able to find the no index tag? Also, would it be more efficient to remove the page from robots.txt in order for no index to be put into practice quicker.
    {{ DiscussionBoard.errors[10915550].message }}
    • Profile picture of the author yukon
      Banned
      Originally Posted by TeamJ View Post

      My concern is that because the page is listed in robots.txt, the no index tag won't be crawled. Google will still be able to find the no index tag? Also, would it be more efficient to remove the page from robots.txt in order for no index to be put into practice quicker.


      That doesn't matter (URL in robots.txt).

      Google still has to prove the page exist, so, they'll find the noindex tag in the <head> of the page source code.

      You can create a bazillion URLs, doesn't mean the pages actually exist.

      Just put the noindex tag in the <head> of the page you want removed from Google SERPs and be done with it.
      {{ DiscussionBoard.errors[10915732].message }}

Trending Topics