How important is robots.txt for SEO
 

Hi Warriors,

Just need some suggestions regarding the importance of robots.txt for good SEO. How important is that?

The reason why I am asking is I have been ranking on page 1 and 2 of google for certain long tail keywords and my alexa ranking has improved quite a lot since my website launch, though I do not have any robots.txt for my site.

Hence, I am just wondering how exactly is that file important.
Waiting for some feedback.

Thanks,
Aviroop

Re: How important is robots.txt for SEO
 

Good Question!

The robots.txt file is what you would use to tell the search engines what files to disallow - NOT to index...

Here is a good article explaining what a robots.txt file is and on how to set up your own robots.txt file for your websites.

Robots.txt: All you need to know



Allen Lundy

Re: How important is robots.txt for SEO
 

Article explained it all.

Re: How important is robots.txt for SEO
 

Thanks a lot Allen,

That was a very helpful article.


Regards,
Aviroop