4 {{ upvoteCount | shortNum }}
4 {{ upvoteCount | shortNum }}

robots.txt - where to put it?

by osekdomains
3 replies
  • SEO
    • |
OK. I have set up redirect for my domain so it should always go to http://www.mydomain.com , whether user types in www or not.

Now my question is, where should i put robots.txt? So google doesn't index the pages twice. Into the public_html directory?
#put #robotstxt
  • Profile picture of the author bobcarlsjr
    yes.. root directory.. which is public_html in 99% of the case..

    some hosting use a different root folder but if you have a public_html.. and your index file is in there, that's the folder to use..
    {{ DiscussionBoard.errors[5649712].message }}
    • Profile picture of the author osekdomains
      ok that's what i thought i just wanted to make sure before i go ahead and do it.

      as for all you SEO-experienced people: is robots.txt really that important? is the penalty that big?

      and how do you choose what domain is "the right one"? i mean i chose www.mydomain.com because people are used to this kind of format. is there any difference which domain will be included in robots.txt?
      Signature

      Film Screenshot Quiz - win cash and prizes
      http://www.screenshotquiz.com

      SuperBMX - czech website for freestyle BMX beginners
      http://www.superbmx.cz

      {{ DiscussionBoard.errors[5649732].message }}
      • Profile picture of the author bobcarlsjr
        Originally Posted by osekdomains View Post

        ok that's what i thought i just wanted to make sure before i go ahead and do it.

        as for all you SEO-experienced people: is robots.txt really that important? is the penalty that big?

        and how do you choose what domain is "the right one"? i mean i chose www.mydomain.com because people are used to this kind of format. is there any difference which domain will be included in robots.txt?
        some will say yes and some will say no.. personally, i've never bothered with robots.txt and rank just fine.. most of my websites are #1 for many keywords

        www and no www is just preference.......... however.. be sure to perm redirect (301) one to the other.. otherwise the big G will see them as 2 seperate pages and you will water down your pr ranking, backlinks, etc..

        you can do this via .htaccess or via your hosting control panel (cpanel, plesk, etc) if you have one..
        {{ DiscussionBoard.errors[5655453].message }}

Trending Topics