Blocking bots with htaccss and robots.txt files
- SEO |
Does anyone know why you should use both the htaccess file and robots.txt to block certain bots from accessing your site instead of just using the robots.txt?
I thought the htaccess was just for internal access.
Thanks
Forum Management & Promotion, SEO Tips, Money Making tips etc.
Forum Management & Promotion, SEO Tips, Money Making tips etc.