robots.txt

by vithobaaseo 8 replies
Hi friends I have small doubt regarding robots.txt. How to add robots.txt to our website? While analysing the website in woorank SEO tool its showing the website is missing robots.txt. Most of them would have done it previously please share me you ideas.. thanks
#search engine optimization #robotstxt
Avatar of Unregistered
  • Profile picture of the author sidharthsharma
    you can take help of google create robots txt coding and use this your site on page you can set robots txt coding on which pages that you don't want crawl by google.
    {{ DiscussionBoard.errors[10075515].message }}
  • Profile picture of the author uditsh
    Originally Posted by vithobaaseo View Post

    Hi friends I have small doubt regarding robots.txt. How to add robots.txt to our website? While analysing the website in woorank SEO tool its showing the website is missing robots.txt. Most of them would have done it previously please share me you ideas.. thanks
    Robots.txt is a simple text file (.txt extension) that you need to upload at root of your website.
    Search engines will look for a robots.txt file at the root of your website whenever they crawl your website.

    Note: Please consult with a SEO specialist before uploading or making changes in it. Any improper or wrong robots.txt file may harm seo of your website.
    {{ DiscussionBoard.errors[10075590].message }}
  • Profile picture of the author MikeFriedman
    Originally Posted by vithobaaseo View Post

    Hi friends I have small doubt regarding robots.txt. How to add robots.txt to our website? While analysing the website in woorank SEO tool its showing the website is missing robots.txt. Most of them would have done it previously please share me you ideas.. thanks
    How do you offer SEO services (in your signature) and not know how to add a robots.txt file to a site?

    WTF?
    {{ DiscussionBoard.errors[10075889].message }}
    • Profile picture of the author jamesfreddyc
      Originally Posted by MikeFriedman View Post

      How do you offer SEO services (in your signature) and not know how to add a robots.txt file to a site?

      WTF?
      Sometimes I just peruse the SEO section of this place and it gives me comfort knowing I am competing with "SEO Experts".
      {{ DiscussionBoard.errors[10088249].message }}
  • Profile picture of the author kavyasharma561
    Robots.txt is a file in the root directory of your web site that instructs web crawlers what parts, or all, or none of your site they are allowed examine.
    {{ DiscussionBoard.errors[10087810].message }}
  • Profile picture of the author addbizglobal
    Robots.txt is a text file in the root director. Before you put in the root directory you should to know disallow command.
    {{ DiscussionBoard.errors[10087886].message }}
  • Profile picture of the author RevConnect
    Use Googles' Webmaster Tools for everything related to seo and page improvements. There you'll find the Robots.txt generator, too.
    {{ DiscussionBoard.errors[10088520].message }}
  • Profile picture of the author MichaelMcKinney
    Yes, Webmaster Tools is a tool through which you can know about webpage structure and you can easily find here robot.txt.
    {{ DiscussionBoard.errors[10090885].message }}
Avatar of Unregistered

Trending Topics

  • 54 {{ upvoteCount | shortNum }}

    Better/cheaper solution than Mailchimp?

    Hi all, I have an e-commerce on Squarespace and I make my own product (jewelry). I want to start building an email list in order to promote 1 great offer ... [read more]

  • 5 {{ upvoteCount | shortNum }}

    Whats the Best Way To Sell Domain Names?

    As Above. I have a portfolio of premium domain names that I would like to sell. I've used Flippa in the past, with little success; the fees tend to prevent ... [read more]

  • 55 {{ upvoteCount | shortNum }}

    Are you the Warrior of the Month?

    so.....we are launching the Warrior of the Month mini competition... Yes... every Month we will announce who was the Warrior with the most valuable post. This is our own small ... [read more]

  • 9 {{ upvoteCount | shortNum }}
  • 6 {{ upvoteCount | shortNum }}

    Any Alternative Of Hootsuite?

    Pr1nc3 in Social Networks

    Suggest me some website or tool like Hootsuite. Free tool or website would be appreciate.. Thanks in Advance.