Understanding robots.txt files Please Help
Hello,
I have a site that I want a specific URL to be blocked from search engines. I also have a PDF file that shows up on Google that I would like blocked.
So here is what I did (not sure if it was right or not):
I downloaded "PC-Robots.txt Plugin"
Opened the settings and added these following texts in the robots.txt file:
User-agent: *
Disallow: /(this is where I added the last piece of the URL of the page that I wanted blocked)
Disallow: /*.pdf$
Then I saved the file. I tried refreshing my Google search but it all was still there and still available. I'm not sure if I need to manually do something else to make these changes take effect or if what I did is even the right thing to do for what I'm looking for.
Any help is highly appreciated. Thank you so much!
I have a site that I want a specific URL to be blocked from search engines. I also have a PDF file that shows up on Google that I would like blocked.
So here is what I did (not sure if it was right or not):
I downloaded "PC-Robots.txt Plugin"
Opened the settings and added these following texts in the robots.txt file:
User-agent: *
Disallow: /(this is where I added the last piece of the URL of the page that I wanted blocked)
Disallow: /*.pdf$
Then I saved the file. I tried refreshing my Google search but it all was still there and still available. I'm not sure if I need to manually do something else to make these changes take effect or if what I did is even the right thing to do for what I'm looking for.
Any help is highly appreciated. Thank you so much!
-
1stranked -
[ 1 ] Thanks - 1 reply
SignatureSSD VPS Servers starting from $5 - Click to apply!{{ DiscussionBoard.errors[8144625].message }}-
chase8705 -
Thanks - 1 reply
{{ DiscussionBoard.errors[8144645].message }}-
1stranked -
[ 1 ] Thanks - 1 reply
SignatureSSD VPS Servers starting from $5 - Click to apply!{{ DiscussionBoard.errors[8144649].message }}-
chase8705 -
Thanks
{{ DiscussionBoard.errors[8144652].message }} -
-
-
-