Go Back   WarriorForum - Internet Marketing Forums > The Warrior Forum > Adsense / PPC / SEO Discussion Forum
Register Blogs FAQ Social Groups CalendarHelp Desk

Reply
 
LinkBack Thread Tools
Old 12-16-2010, 02:32 PM   #1
HyperActive Warrior
War Room Member
 
Join Date: Oct 2009
Posts: 275
Thanks: 10
Thanked 4 Times in 4 Posts
Default <Wordpress> PC Robots.txt trouble

Hi

wordpress 3

I've got some trouble with PC Robots.txt plugin for wordpress. The problem is that when I click view the robots.txt file (link from plugin admin) then I see the robots.txt. but it contain different info

This is what I see:
User-agent: *
Disallow: /

Sitemap: http://www.mydomain/sitemap.xml.gz

This info I thought the robots.txt should also contain:
User-agent: Alexibot
Disallow: /

User-agent: Aqua_Products
Disallow: /

User-agent: asterias
Disallow: /

What am I doing wrong here
loginname is offline   Reply With Quote
Old 12-16-2010, 02:43 PM   #2
Penguins ate my PC
War Room Member
 
cbrauer's Avatar
 
Join Date: Dec 2010
Location: Missouri
Posts: 84
Blog Entries: 5
Thanks: 0
Thanked 11 Times in 8 Posts
Social Networking View Member's Myspace Profile 
Default Re: <Wordpress> PC Robots.txt trouble

Sometimes plugins that change file structures don't not work right. The best thing to do is:

Log into your cpanel or hosting panel.
Open the file directory for that domain name.
locate the robots.txt file. (if there is not one go to the add file button and create one)
Right click the robots.txt file and click edit.
User-agent: *
Disallow: /cgi-bin/
Disallow: /privatedir/
Disallow: /tutorials/blank.htm
This is an example of course but you can use it to block whatever directories you want.
cbrauer is offline   Reply With Quote
Old 12-17-2010, 03:30 PM   #3
You're Welcome
War Room Member
 
Join Date: Sep 2010
Location: On Top Of Your Head!
Posts: 322
Blog Entries: 1
Thanks: 0
Thanked 25 Times in 23 Posts
Contact Info
Send a message via AIM to mad.hat Send a message via Yahoo to mad.hat Send a message via Skype™ to mad.hat
Default Re: <Wordpress> PC Robots.txt trouble

If you want to specifically disallow those user agents then you can but unless you have a specific reason to you don't really need to block those.

They All Told Me Making Money Online Was Just A Scam For The Gullible... Even Though I Was Already Pulling In $1k/Day. Now You Say My Resellers Can Not Make $1k/day But...
mad.hat is offline   Reply With Quote
Reply

  WarriorForum - Internet Marketing Forums > The Warrior Forum > Adsense / PPC / SEO Discussion Forum

Tags
&ltwordpress&gt, <wordpress>, robotstxt, trouble

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off



All times are GMT -6. The time now is 11:43 PM.