9 {{ upvoteCount | shortNum }}
9 {{ upvoteCount | shortNum }}

How can I create a robot.txt file using google?

by suprity
8 replies
  • SEO
    • |
I have a new website. I used wordpress platform. Now I want to use robot.txt file for avoid some page google crawling. Is it possible? Can anyone help me?
#create #file #google #google crawler #robot.txt #robottxt
  • Profile picture of the author movielinks
    offcourse it's possible. Open Notepad++ or any html editor, Create a new file and name it robots.txt , Add requisite code and upload it in your root folder (on server)

    This may help you to learn more about robots.txt
    The Web Robots Pages
    {{ DiscussionBoard.errors[7419961].message }}
  • Profile picture of the author Jose Princely
    In WordPress there won't be ronot.txt file as default in back end to access it easily? Sorry, I am not familiar with CMS
    Signature
    Magento development Services | Prestashop modules
    {{ DiscussionBoard.errors[7420572].message }}
  • Profile picture of the author Avian Media
    Banned
    you can create a robots.txt file by going in to the webmaster official site and after that u will get the create robots.txt file

    use
    user-agent: *
    Allow: /
    {{ DiscussionBoard.errors[7420628].message }}
  • Profile picture of the author Sanket Patel
    Originally Posted by suprity View Post

    I have a new website. I used wordpress platform. Now I want to use robot.txt file for avoid some page google crawling. Is it possible? Can anyone help me?
    Yes it is possible to create robots.txt file for wordpress site. Open Edit+ or any other html editor and save that file name with robots.txt. Mention Disallow to that page root directory which you do not want to crawl search engine and upload it in server. For example

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    {{ DiscussionBoard.errors[7420782].message }}
  • Profile picture of the author anexplore
    Originally Posted by suprity View Post

    I have a new website. I used wordpress platform. Now I want to use robot.txt file for avoid some page google crawling. Is it possible? Can anyone help me?
    You may read this article.
    Thanks
    Signature
    Electrical Engineering | Online Electrical | Engineering Forum | The Explore | Aliexpress Product | Hot Deal
    {{ DiscussionBoard.errors[7420864].message }}
  • Profile picture of the author dwayne123
    type this in
    User-agent: * Disallow:

    notepad and save it as robots.txt
    {{ DiscussionBoard.errors[7421467].message }}
  • Profile picture of the author danb12
    User-agent: *
    Disallow: /images/

    this is an example... open notepad, and just save a robots, then upload to your root.
    Signature
    UK Coupon Website PR1 making £300+ per month - QUICK SALE - CHEAP SALE - CONTACT ME
    {{ DiscussionBoard.errors[7421477].message }}
  • Profile picture of the author Elfen Lied
    You can search for plugins to noindex author, category, tag etc pages.

    Just make sure you don't accidently noindex pages that you do want indexed.. like your homepage.
    {{ DiscussionBoard.errors[7428421].message }}

Trending Topics