![]() |
| ||||||||
|
|||||||
![]() |
|
|
LinkBack | Thread Tools |
|
|
#1 |
|
Lovin' Life
War Room Member
Join Date: Sep 2008
Location: Cape Town
Posts: 397
Thanks: 67
Thanked 35 Times in 23 Posts
|
Hi all,
I was wondering if this has happened to someone before... I am trying to submit my sitemap to Webmastertools it says it cant crawl my site because it is restricted by my robots.txt. My websites robots is: User-agent: * Disallow: And google is telling me it's User-agent: * Disallow: / Pretty frustrating. Its a wordpress site, I have checked the privacy setting under Settings and its fine Does anyone have any ideas? Cheers, Pete |
|
|
|
|
|
|
|
|
#2 |
|
Senior Warrior Member
War Room Member
Join Date: Apr 2006
Location: , , USA.
Posts: 1,711
Thanks: 17
Thanked 41 Times in 40 Posts
|
If that is the whole thing, just delete the file, otherwise recreate it.
|
|
|
|
|
|
#3 |
|
Captain
War Room Member
Join Date: Dec 2008
Location: South Africa
Posts: 784
Blog Entries: 1
Thanks: 48
Thanked 30 Times in 29 Posts
|
I also had a problem like this once, dam it was fustrating
![]() Try what has been suggested, creating a new site map might be the answer. Also check in the domains root file on your hosting account and see if there is a robots.txt file there - if there is then delete it. Hope that helps you
|
|
XSitePro - Start Designing Professional Websites Yourself
|
|
|
|
|
![]() |
|
| Tags |
| sitemap, tools, webmaster |
| Thread Tools | |
|
|
![]() |