| | ||||||||
| |||||||||
 |
06-28-2011, 09:36 PM
| #1 |
| Active Warrior War Room Member  Join Date: Jan 2011
Posts: 87
Thanks: 17
Thanked 3 Times in 3 Posts
|  robots.txt
Hi just a quick question.... this is what google says my robots.txt file is User-agent: * Disallow: I thought this meant to allow googlebot right? because gwt is telling me 16 of my pages are restricted... does this robot.txt indeed allow or not google to crawl??? Ive done a thorough forum search but info is confusing thanks |
| | |
|  |
|
06-28-2011, 09:39 PM
| #2 |
| Jim Marq  Join Date: May 2011 Location: S.E. Minnesota
Posts: 38
Thanks: 12
Thanked 4 Times in 4 Posts
| Re: robots.txt
That doesn't block anything the way it's written, another words, nothing is disallowed. It's the same as not having a robots.txt file at all. This would tell all the robots not to visit any pages on the site. Bad for SEO lol User-agent: * Disallow: / |
| | |
|
| |
|
06-28-2011, 09:56 PM
| #3 |
| Active Warrior War Room Member Join Date: Jan 2011
Posts: 87
Thanks: 17
Thanked 3 Times in 3 Posts
| Re: robots.txt
thanks 2stroke guess ill jsut wait for google to recrawl site...
|
| | |
|
| |
 |
|
| Tags |
| robotstxt |
| Thread Tools | |
| |
 |
