| | ||||||||
| |||||||||
 |
08-14-2010, 04:26 PM
| #1 |
| Active Warrior Join Date: Jul 2010
Posts: 65
Thanks: 1
Thanked 0 Times in 0 Posts
|  Need help in customizing robots.txt File
My robots.txt file shows like this User-agent: * Disallow: Sitemap: http://www.mysitegoeshere.com/sitemap.xml So does that mean that the robots.txt is disabling my sites Sitemap to be not displayed in the Search Engine.Should I do something with this to avoid this? |
|  |
|
08-14-2010, 04:29 PM
| #2 |
| HyperActive Warrior  Join Date: Jun 2010
Posts: 184
Thanks: 16
Thanked 46 Times in 33 Posts
| Re: Need help in customizing robots.txt File
Here's an example of a proper robots.txt http://www.hovercoupon.com/robots.txt Hope that helps. You only Disallow stuff you want disallowed... IF you Disallow / - Google ignores the whole site. |
| Hover Coupon | Affordable Domain Names and Personalized, Surname Email for Less Site5 Rebate Claim Form | Save Big on Reliable Shared and Reseller Web Hosting | |
|
| |
|
08-15-2010, 03:05 PM
| #3 |
| Active Warrior Join Date: Jul 2010
Posts: 65
Thanks: 1
Thanked 0 Times in 0 Posts
| Re: Need help in customizing robots.txt File
So should I disallow Sitemap MY site shows like above where my Sitemap is shown along with the Disallow tab |
|
| |
|
08-15-2010, 03:22 PM
| #4 |
| HyperActive Warrior Join Date: Jun 2010
Posts: 184
Thanks: 16
Thanked 46 Times in 33 Posts
| Re: Need help in customizing robots.txt File
No... you don't want to disallow your sitemap. That kind of defeats the purpose of creating a sitemap.
|
| Hover Coupon | Affordable Domain Names and Personalized, Surname Email for Less Site5 Rebate Claim Form | Save Big on Reliable Shared and Reseller Web Hosting | |
|
| |
|
08-16-2010, 08:26 PM
| #5 | |
| Active Warrior Join Date: Jul 2010
Posts: 65
Thanks: 1
Thanked 0 Times in 0 Posts
| Re: Need help in customizing robots.txt File Quote:
User-agent: * Disallow: Sitemap: http://www.urladdress.net/sitemap.xml Now does that mean that my robots.txt is set to Disallow my Sitemap,so should I remove the Sitemap: http://www.urladdress.net/sitemap.xml | |
|
| |
|
08-17-2010, 01:28 AM
| #6 |
| HyperActive Warrior War Room Member  Join Date: Mar 2010
Posts: 351
Thanks: 33
Thanked 61 Times in 46 Posts
| Re: Need help in customizing robots.txt File
No, This is telling the robots where your sitemap is located so they can find it faster. Read this post: Must See SEO Guide for All WordPress Bloggers It can help you better understand the robots.txt file among other things. It also give you an exact example of how to structure your robots.txt file. You can also simply install the PC Robots.txt Wordpress plugin - get it here: WordPress › PC Robots.txt WordPress Plugins |
|
Find out What's Working For me Now on my Blog and if you are looking for some Honest IM Reviews than you should definitely read some of mine. Product Launch Domination Blueprint - My Latest WSO! PINGBACK POWER - Free WordPress Plugin | |
|
| |
|
08-17-2010, 07:04 PM
| #7 | |
| Active Warrior Join Date: Jul 2010
Posts: 65
Thanks: 1
Thanked 0 Times in 0 Posts
| Re: Need help in customizing robots.txt File Quote:
Disallow: means its not Disallowing any of my Content right.So I dont need to do anything with my robots.txt right and Sitemap Content is showed in a different line? | |
|
| |
|
08-18-2010, 07:59 AM
| #8 |
| Wordpress SEO Expert  Join Date: Sep 2008 Location: France
Posts: 18
Thanks: 0
Thanked 2 Times in 2 Posts
| Re: Need help in customizing robots.txt File
I recommend the iRobots.txt SEO Wordpress plugin.
|
| | |
|
| |
|
08-19-2010, 07:45 PM
| #9 |
| Active Warrior Join Date: Jul 2010
Posts: 65
Thanks: 1
Thanked 0 Times in 0 Posts
| Re: Need help in customizing robots.txt File
No,I just want to know what exactly is my robots.txt saying.Do I need to edit any text or should I just leave it like that.And I believe the Site map line is in the next line after the Disallow line right?
|
|
| |
 |
|
| Tags |
| customizing, file, robotstxt |
| Thread Tools | |
| |
 |
