5 {{ upvoteCount | shortNum }}
5 {{ upvoteCount | shortNum }}

Help with robot.html pages

by MikeHayes
4 replies
  • WEB DESIGN
    • |
After all these years I suddenly hear about robot.html protection pages but can't find what I need to know.
Question...what does the robot.html page contain?
Question...where do you place it on your site?

Thanks.

Quicksilver
The Mike Hayes Total Info Site

PS What's the function of the "Trackback" option below?
#pages #robot #robothtml #robotstxt
  • Profile picture of the author Thomas
    Originally Posted by quicksilver View Post

    After all these years I suddenly hear about robot.html protection pages but can't find what I need to know.
    Question...what does the robot.html page contain?
    Question...where do you place it on your site?

    Thanks.

    Quicksilver
    The Mike Hayes Total Info Site

    PS What's the function of the "Trackback" option below?
    Here's ya go, Mike:

    The Web Robots Pages
    {{ DiscussionBoard.errors[85216].message }}
  • Profile picture of the author bendiggs
    Originally Posted by quicksilver View Post

    After all these years I suddenly hear about robot.html protection pages but can't find what I need to know.
    Question...what does the robot.html page contain?
    Question...where do you place it on your site?

    Thanks.
    robots.txt is a simple text file placed at the root of your domain that tells obedient robots/spiders/crawlers what they are and are not allowed to access. If you have certain pages you don't want Google to index (test.html in the tmp folder for example) you'd put in a robots.txt file with this:

    Code:
    User-agent: Google
Disallow: /tmp/test.html
    {{ DiscussionBoard.errors[86311].message }}

Trending Topics