6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

How to apply robots.txt to unwanted url's?

by kavyajain
5 replies
  • SEO
    • |
Hey guys I just wanted to ask for robots.txt file after applying which I wont be able to see the unwanted url's in my website.I want to remove some url's from my website using robots.txt
If you have any idea please let me know
#apply #bespoke #bespoke digital media #digital #media #robots.txt #robotstxt #search engine help #seo #unwanted #url
  • Profile picture of the author jessikaload
    If you want to remove urls from search engines results use that "Remove Url" option in Web master tools.

    If you want to block the urls from the search engine bots use robots.txt


    User-agent: *
    Disallow: /directory/file.html
    {{ DiscussionBoard.errors[10281987].message }}
  • Profile picture of the author nmwf
    A robots.txt won't remove a URL from your website. It'll only hide it from search engines as jessikaload said. If you want to remove a URL from your website, delete its file at the server level. (That would be, "Delete Post" for blog users.)
    Signature
    Write comprehensible articles on *any* topic in seconds with First Draft...
    First Draft's: Download | Add-Ons | Templates | Purchase | Support | Affiliates
    {{ DiscussionBoard.errors[10282045].message }}
    • Profile picture of the author nettiapina
      Originally Posted by nmwf View Post

      A robots.txt won't remove a URL from your website. It'll only hide it from search engines as jessikaload said.
      This is correct. Also note that it doesn't even hide anything from search engines. It's just a suggestion that the well-behaving bots will probably follow.
      Signature
      Links in signature will not help your SEO. Not on this site, and not on any other forum.
      Who told me this? An ex Google web spam engineer.

      What's your excuse?
      {{ DiscussionBoard.errors[10282075].message }}
  • Profile picture of the author NetgainsTech
    User-agent: *
    Disallow: /url-path
    {{ DiscussionBoard.errors[10282067].message }}
  • Profile picture of the author Sherylpeter
    robots.txt file is just a request from webmasters to search engines that not to crawl specific sections of my website these are private.
    Signature
    Android apps development is now becomes easy and fast with Dotstudioapps.com
    {{ DiscussionBoard.errors[10293924].message }}

Trending Topics