robots.txt

by vithobaaseo 8 replies
Hi friends I have small doubt regarding robots.txt. How to add robots.txt to our website? While analysing the website in woorank SEO tool its showing the website is missing robots.txt. Most of them would have done it previously please share me you ideas.. thanks
#search engine optimization #robotstxt
Avatar of Unregistered
  • Profile picture of the author sidharthsharma
    you can take help of google create robots txt coding and use this your site on page you can set robots txt coding on which pages that you don't want crawl by google.
    {{ DiscussionBoard.errors[10075515].message }}
  • Profile picture of the author uditsh
    Originally Posted by vithobaaseo View Post

    Hi friends I have small doubt regarding robots.txt. How to add robots.txt to our website? While analysing the website in woorank SEO tool its showing the website is missing robots.txt. Most of them would have done it previously please share me you ideas.. thanks
    Robots.txt is a simple text file (.txt extension) that you need to upload at root of your website.
    Search engines will look for a robots.txt file at the root of your website whenever they crawl your website.

    Note: Please consult with a SEO specialist before uploading or making changes in it. Any improper or wrong robots.txt file may harm seo of your website.
    {{ DiscussionBoard.errors[10075590].message }}
  • Profile picture of the author MikeFriedman
    Originally Posted by vithobaaseo View Post

    Hi friends I have small doubt regarding robots.txt. How to add robots.txt to our website? While analysing the website in woorank SEO tool its showing the website is missing robots.txt. Most of them would have done it previously please share me you ideas.. thanks
    How do you offer SEO services (in your signature) and not know how to add a robots.txt file to a site?

    WTF?
    {{ DiscussionBoard.errors[10075889].message }}
    • Profile picture of the author jamesfreddyc
      Originally Posted by MikeFriedman View Post

      How do you offer SEO services (in your signature) and not know how to add a robots.txt file to a site?

      WTF?
      Sometimes I just peruse the SEO section of this place and it gives me comfort knowing I am competing with "SEO Experts".
      {{ DiscussionBoard.errors[10088249].message }}
  • Profile picture of the author kavyasharma561
    Robots.txt is a file in the root directory of your web site that instructs web crawlers what parts, or all, or none of your site they are allowed examine.
    {{ DiscussionBoard.errors[10087810].message }}
  • Profile picture of the author addbizglobal
    Robots.txt is a text file in the root director. Before you put in the root directory you should to know disallow command.
    {{ DiscussionBoard.errors[10087886].message }}
  • Profile picture of the author RevConnect
    Use Googles' Webmaster Tools for everything related to seo and page improvements. There you'll find the Robots.txt generator, too.
    {{ DiscussionBoard.errors[10088520].message }}
  • Profile picture of the author MichaelMcKinney
    Yes, Webmaster Tools is a tool through which you can know about webpage structure and you can easily find here robot.txt.
    {{ DiscussionBoard.errors[10090885].message }}
Avatar of Unregistered

Trending Topics

  • 48 {{ upvoteCount | shortNum }}

    Warriors .... Warriors.... what would you like to LEARN?

    Good Morning awesome Warriors... I'm happy to announce that The Warrior Forum is planning to create our first COURSE. Yes... Powered by Warrior Wisdom... yeah!! Made at home. But before ... [read more]

  • 3 {{ upvoteCount | shortNum }}

    Drop shipping/print on demand - Niche or micro-niche ?

    This question is for those of you who are building a drop shipping or print on demand business and are monetizing through your facebook fan pages. When you create the ... [read more]

  • 8 {{ upvoteCount | shortNum }}

    I don't want to get another cease and desist letter about my domain name...

    I was once threatened by a Lawsuit concerning a domain name that I owned - and I don't want to make the same mistake. So therefore, I want to make ... [read more]

  • 43 {{ upvoteCount | shortNum }}

    How Do Highly Successful People Stay Positive?

    Hi Gang, I just stumbled upon this great article and wanted to share! 6 Ways Highly Successful People Stay Positive So the question begs, how do YOU stay positive? What ... [read more]

  • 3 {{ upvoteCount | shortNum }}

    The website is frozen after 301 redirect

    AdamTear12 in SEO

    Hello mates, I made a 301 redirect on one of my websites. Purchased a new domain, installed WordPress on it, and then did a 301 redirect in order to inject ... [read more]