I have a question about robot.txt files:
Do you use the robot.txt to keep a directory with a .pdf file from being indexed...so it doesn't even show up in search engines?
For example, if I have a .pdf file in a host directory...like:
and I want to keep that link from ever showing up in the search engines, do you do this with the robots.txt file?
And this would be the robots.txt file right in:
Is this right?