![]() | | ||||||||
| | #1 |
| HyperActive Warrior War Room Member Join Date: Oct 2009
Posts: 275
Thanks: 10
Thanked 4 Times in 4 Posts
|
Hi This site is already indexed by google using both the .xml and the .xml.gz sitemaps. I thought I should update the robots.txt file correctly so that it also contain links to the sitemap. I just wonder if I should link the .xml version or the .xml.gz version so basically which one of these should I use: Sitemap: http://www.mydomain.com/sitemap.xml Sitemap: http://www.mydomain.com/sitemap.xml.gz I'm a bit worried as this site is 8 months old and I've trusted PC Robots.txt plugin in wordpress to do it's job, when I checked it displays an empty robots.txt so I uploaded a real robots.txt with the content I wanted. I've seen the error before but didn't pay much attention to it, but now as google don't displays ads on my site. I think it is somehow related to robots.txt issues... any suggestions? |
| | |
| | #2 |
| Plundering the Web War Room Member Join Date: Feb 2007 Location: , , .
Posts: 4,849
Thanks: 804
Thanked 1,199 Times in 886 Posts
|
"How to link sitemap to the robots.txt?" What for? Completely useless. Every "major" (okay, like 2) automatically looks for a robots.txt. If it's there, fine. If it's not there, fine. You don't really need a robots.txt if you are like the 99.999% of the rest of us. You certainly don't need to link to it. You don't really even need a sitemap, but that's sacrilegious to a lot of SEO gurus. There would only be an issue if your robots text told google to ignore something. As stated, not really needed. Make sure it's named robots.txt and in the root directory. I see no reason to disallow anything. Google is a smart cookie. If you put some garbage error stuff in it, my advice is to delete it, or just leave it blank. I can't even see how an error in it would matter. Search engine would ignore anything it can't figure out. If you put: User-agent: * Disallow: / That means it is not looking at squat. Get rid of the slash mark. User-agent: * Disallow: That means look at everything. But what's the point? Best advice: Don't have a robots.txt Done and done. If that does not fix the google ad thing, you have a problem somewhere else. But I can't fathom robots.txt causing google to not show ads anyway. That may be another matter. Paul |
| How to Make Money off Facebook: Login to your account. Deactivate your account. Get your butt to work.
| |
| | |
| | #3 |
| HyperActive Warrior Join Date: Nov 2010
Posts: 343
Thanks: 20
Thanked 20 Times in 20 Posts
|
In your robots.txt file, add this line : Sitemap: http://www.mydomain.com/sitemap.xml The gunzipped version is smaller to download, so you can add that too. But I have never tried GZ version. Why dont you add the sitemap in Google Webmaster Tools and check if it can retrieve and extract the URLs. |
| | |
| | |
| | #4 |
| SEO Strategist War Room Member Join Date: Jun 2010
Posts: 6,532
Thanks: 355
Thanked 1,992 Times in 1,273 Posts
|
I run: Sitemap: http://domain.com/sitemap.xml.gz Works just fine. |
| | |
| | #5 |
| Warrior Member Join Date: Dec 2010
Posts: 4
Thanks: 0
Thanked 0 Times in 0 Posts
|
Site link is not in itself a symbol of the REP. Which is only there to help our crawlers find your map. usual rules apply EPR, then you can block robots from indexing your site.
|
| | |
| | |
![]() |
|
| Tags |
| link, robotstxt, sitemap |
| Thread Tools | |
| |
![]() |