robots.txt

by 6 replies
8
Hi there I recently done a quick analysis of my website and it said i did not have a robots.txt file or an author tag. Can someone tell me what these are and how I go about getting them?
thanks
#search engine optimization #robotstxt
  • What is it that you are trying to do?
  • well for the next month until March I am trying to get my site from page 2 on google to page 1. This analysis tool said i was missing these and I needed them?
  • Here you can find some interesting tips
    hxxp://www.free-seo-news.com/all-about-robots-txt.htm
  • The robots.txt file is super easy. If you don't have any files on your server you want to hide form search engine robots, just create a blank .txt file in Notepad and save it as robots.txt. Then upload that to your root directory, which is the public_html directory for most web hosts. Same place you put your index.html page, etc. Then the search engines will see the robots.txt file, scan it, see that it's not disallowing them anywhere, and scour your whole site (eventually).

    John
    • [ 1 ] Thanks
    • [2] replies
    • Here is a link to a robots.txt generator

      Robots.txt Generator - McAnerin International Inc.

      Just fill out the form and it will create a text file you can upload to your server.
      • [ 2 ] Thanks
    • John this is the perfect answer to this, thank you, I'm sure others will find it useful too!

      If you are hiding certain pages from the Search Engine, what do you susggest?

      Thanks!
  • Banned
    [DELETED]
Join the discussion

Next Topics on Trending Feed

  • 8

    robots.txt

    Hi there I recently done a quick analysis of my website and it said i did not have a robots.txt file or an author tag. Can someone tell me what these are and how I go about getting them? thanks