![]() | | ||||||||
| | #1 |
| HyperActive Warrior War Room Member Join Date: Oct 2009
Posts: 275
Thanks: 10
Thanked 4 Times in 4 Posts
|
Hi wordpress 3 I've got some trouble with PC Robots.txt plugin for wordpress. The problem is that when I click view the robots.txt file (link from plugin admin) then I see the robots.txt. but it contain different info This is what I see: User-agent: * Disallow: / Sitemap: http://www.mydomain/sitemap.xml.gz This info I thought the robots.txt should also contain: User-agent: Alexibot Disallow: / User-agent: Aqua_Products Disallow: / User-agent: asterias Disallow: / What am I doing wrong here |
| | |
| | #2 |
| Penguins ate my PC War Room Member Join Date: Dec 2010 Location: Missouri
Posts: 84
Blog Entries: 5 Thanks: 0
Thanked 11 Times in 8 Posts
|
Sometimes plugins that change file structures don't not work right. The best thing to do is: Log into your cpanel or hosting panel. Open the file directory for that domain name. locate the robots.txt file. (if there is not one go to the add file button and create one) Right click the robots.txt file and click edit. User-agent: * Disallow: /cgi-bin/ Disallow: /privatedir/ Disallow: /tutorials/blank.htm This is an example of course but you can use it to block whatever directories you want. |
| | |
| | #3 |
| You're Welcome War Room Member Join Date: Sep 2010 Location: On Top Of Your Head!
Posts: 322
Blog Entries: 1 Thanks: 0
Thanked 25 Times in 23 Posts
|
If you want to specifically disallow those user agents then you can but unless you have a specific reason to you don't really need to block those.
|
| They All Told Me Making Money Online Was Just A Scam For The Gullible... Even Though I Was Already Pulling In $1k/Day. Now You Say My Resellers Can Not Make $1k/day But... | |
| | |
![]() |
|
| Tags |
| <wordpress>, <wordpress>, robotstxt, trouble |
| Thread Tools | |
| |
![]() |