| | ||||||||
| |||||||||
 |
04-16-2011, 06:49 PM
| #1 |
| Warrior Member Join Date: Mar 2011 Location: Earth
Posts: 20
Thanks: 0
Thanked 0 Times in 0 Posts
|  help with robots.txt file
i want to make sure that google and other serch engines can crawl my site, i wanted to know if this txt is allowing them to crawl or not? User-agent: Mediapartners-Google Disallow: User-agent: * Disallow: /search |
|  |
|
04-16-2011, 07:41 PM
| #2 | |
| HyperActive Warrior  Join Date: Jan 2011 Location: Norfolk U.K.
Posts: 227
Thanks: 11
Thanked 39 Times in 34 Posts
| Re: help with robots.txt file Quote:
Yes, they are crawling its the forward slash ( / ) that blocks the crawl, thus: User-agent: Mediapartners-Google Disallow: / now blocked  Edit: Do not remove this: User-agent: * Disallow: /search | |
| Last edited by Tim3; 04-16-2011 at 07:45 PM. Reason: missed a bit off | ||
|
| |
|
09-19-2011, 11:28 AM
| #3 |
| Warrior Member Join Date: Jun 2011
Posts: 9
Thanks: 0
Thanked 0 Times in 0 Posts
| Re: help with robots.txt file
my site went from page one to 44 and no meta disc. and goes to a page called stuff gate dont know why
|
|
| |
|
09-19-2011, 11:47 AM
| #4 |
| Rakesh Jain Join Date: Jul 2011 Location: Chandigarh
Posts: 40
Thanks: 0
Thanked 0 Times in 0 Posts
| Re: help with robots.txt file
yes you have the right code. you can check it in google webmasters tools also. |
| SEO Company India with Google top 10 ranking guarantee
| |
|
| |
 |
|
| Tags |
| file, robotstxt |
| Thread Tools | |
| |
 |
