6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

Robot Text?

by Trixxie12
5 replies
Hey guys, im trying to stop a "secret" page on my website being able to be crawled by search engines, this page contains all the downloads that should only be available after a customer purchases.

Right now i have a big problem, and that is that my main page isnt appearing in the search engines, but the secret page is.

I phoned GoDaddy and asked them what to do, they explained i needed to submit my url to google, via google.com/addurl

And then use robot text to block search engines picking up the secret page.

Aparently i have to add "<meta name="robots" content="noindex">" into my HTML code, but where do i put it?


thanks
#robot #text
  • Profile picture of the author jacktackett
    place an index.htm in that folder with only the header there. You can also create a robots.txt file in your root folder and place any folders you do not want indexed in that.

    examples:

    robots.txt:
    User-agent: *
    Disallow: /alphavax
    Disallow: /brochures
    Disallow: /pentair
    Disallow: /ppdiftp
    Disallow: /ultimus

    index.htm:


    <html>
    <head>
    <title>...</title>
    <meta name="robots" content="noindex,nofollow">
    </head>
    <body>
    </body>
    </html>

    best,
    --Jack

    Thanks ehicks - edits made.
    Signature
    Let's get Tim the kidney he needs!HELP Tim
    Mega Monster WSO for KimW http://ow.ly/4JdHm


    {{ DiscussionBoard.errors[701750].message }}
    • Profile picture of the author ehicks727
      Originally Posted by jacktackett View Post

      <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
      Sorry, I'm going to be an HTML nazi for a second...

      Please don't use all caps in HTML. It's not standard.

      Also, make sure you close your meta tags...

      <meta name="robots" content="noindex,nofollow" />


      Google likes you better if your HTML validates... the code in the previous post does not validate.
      {{ DiscussionBoard.errors[701759].message }}
      • Profile picture of the author Trixxie12
        Originally Posted by ehicks727 View Post


        Google likes you better if your HTML validates... the code in the previous post does not validate.
        That's the code google gave me
        {{ DiscussionBoard.errors[701810].message }}
        • Profile picture of the author jacktackett
          Originally Posted by Trixxie12 View Post

          That's the code google gave me
          Trixie - I edited the post to correct the errors ehicks727 rightly pointed out - instead of posting a reply (not really looking to inflate my post count ;-) - opps.

          Apologies for the scare!

          best,
          --Jack
          Signature
          Let's get Tim the kidney he needs!HELP Tim
          Mega Monster WSO for KimW http://ow.ly/4JdHm


          {{ DiscussionBoard.errors[702200].message }}
          • Profile picture of the author Trixxie12
            Originally Posted by jacktackett View Post

            Trixie - I edited the post to correct the errors ehicks727 rightly pointed out - instead of posting a reply (not really looking to inflate my post count ;-) - opps.

            Apologies for the scare!

            best,
            --Jack
            Thanks, ill try it out
            {{ DiscussionBoard.errors[702257].message }}

Trending Topics