How important is robots.txt for SEO

diff122 · 16th Sep 2010, 03:17 AM

Hi Warriors,

Just need some suggestions regarding the importance of robots.txt for good SEO. How important is that?

The reason why I am asking is I have been ranking on page 1 and 2 of google for certain long tail keywords and my alexa ranking has improved quite a lot since my website launch, though I do not have any robots.txt for my site.

Hence, I am just wondering how exactly is that file important.
Waiting for some feedback.

Thanks,
Aviroop

Allen Lundy · 16th Sep 2010, 11:32 AM

Good Question!

The robots.txt file is what you would use to tell the search engines what files to disallow - NOT to index...

Here is a good article explaining what a robots.txt file is and on how to set up your own robots.txt file for your websites.

Robots.txt: All you need to know

Allen Lundy

cooljazz · 16th Sep 2010, 06:31 PM

Article explained it all.

diff122 · 16th Sep 2010, 11:33 PM

Thanks a lot Allen,

That was a very helpful article.

Regards,
Aviroop

16th Sep 2010, 03:17 AM	#1
diff122 Warrior Member Join Date: 2010 Location: Calcutta, India Posts: 84 Thanks: 8 Thanked 16 Times in 12 Posts	How important is robots.txt for SEO Share on: Hi Warriors, Just need some suggestions regarding the importance of robots.txt for good SEO. How important is that? The reason why I am asking is I have been ranking on page 1 and 2 of google for certain long tail keywords and my alexa ranking has improved quite a lot since my website launch, though I do not have any robots.txt for my site. Hence, I am just wondering how exactly is that file important. Waiting for some feedback. Thanks, Aviroop

16th Sep 2010, 11:32 AM	#2
Allen Lundy Article Marketer War Room Member Join Date: 2005 Location: Orange, Ca. , USA. Posts: 558 Thanks: 151 Thanked 270 Times in 184 Posts	Re: How important is robots.txt for SEO Share on: Good Question! The robots.txt file is what you would use to tell the search engines what files to disallow - NOT to index... Here is a good article explaining what a robots.txt file is and on how to set up your own robots.txt file for your websites. Robots.txt: All you need to know Allen Lundy
	Amazing Deals On Mens and Womens Jewelry - - Gies Jewelry Shop Watches for all occasions -Gies Watch Shop The latest in Cameras and Photography - Good Tyme Photography Need a new phone or upgrade? - Gies Mobile Phones Number 1 Source For Kids Toys & Games - Shop For Toys & Games

16th Sep 2010, 06:31 PM	#3
cooljazz Active Warrior Join Date: 2010 Posts: 67 Thanks: 0 Thanked 1 Time in 1 Post	Re: How important is robots.txt for SEO Share on: Article explained it all.
	Everything You Wanted To Know About How To Create a Your Own WebSite . Need Help Finding Profitable Niche ? Free Website Templetes

16th Sep 2010, 11:33 PM	#4
diff122 Warrior Member Join Date: 2010 Location: Calcutta, India Posts: 84 Thanks: 8 Thanked 16 Times in 12 Posts	Re: How important is robots.txt for SEO Share on: Thanks a lot Allen, That was a very helpful article. Regards, Aviroop