Help Me Understand Blocking Search Engines From Viewing Pages Please

7 replies
Hi I'm getting to the point now where I will be launching all of my videos on a page instead of people having to download a zipped file..

Ok here's what I need laid out for me.

I need for certain pages to be only accessed by certain people after they opt in. So for instance if anyone never opted in, that particular page is not even existent in the search engines.

How do I do that?

Thanks in advance
#blocking #engines #pages #search #understand #viewing
  • Profile picture of the author bazzais
    You will have to have a login system that will protect the pages unless a user is logged in (this will stop robots from visiting the pages because they are protected) or have a password protected directory at server level.

    Its the only fullproof way.

    Ta

    Baz
    {{ DiscussionBoard.errors[1416377].message }}
    • Profile picture of the author Jeff Baxter
      Originally Posted by bazzais View Post

      You will have to have a login system that will protect the pages unless a user is logged in (this will stop robots from visiting the pages because they are protected) or have a password protected directory at server level.

      Its the only fullproof way.

      Ta

      Baz
      Thanks but i believe that having a password protection might not be the only way. I'm putting my videos on sort of like a regular page and I seen others do this. That's how I know that having a login might not be mandatory...??
      {{ DiscussionBoard.errors[1416392].message }}
  • Profile picture of the author Jason Stepp
    I've never ran a site with different access levels, so I can't really help you out with that part of your question.

    However, if you don't want search engines to index a specific page, you can add a no-index meta tag to the top of the pages you don't want indexed.

    Code:
    <meta name="robots" content="noindex" />
    You insert this code in the same location as all of your other meta tags.

    It's important to note that this meta tag is not honored by all search engines. However, Google does honor the meta tag.

    Read more about the no-index meta tag from Google here:
    Using meta tags to block access to your site - Webmasters/Site owners Help
    {{ DiscussionBoard.errors[1416380].message }}
    • Profile picture of the author Jeff Baxter
      Originally Posted by Jay284 View Post

      I've never ran a site with different access levels, so I can't really help you out with that part of your question.

      However, if you don't want search engines to index a specific page, you can add a no-index meta tag to the top of the pages you don't want indexed.

      Code:
      <meta name="robots" content="noindex" />
      You insert this code in the same location as all of your other meta tags.

      It's important to note that this meta tag is not honored by all search engines. However, Google does honor the meta tag.

      Read more about the no-index meta tag from Google here:
      Using meta tags to block access to your site - Webmasters/Site owners Help

      Thats what I needed ..Thanks bro
      {{ DiscussionBoard.errors[1416403].message }}
      • Profile picture of the author bazzais
        Thanks jay284 your correct that will stop spiders.

        Jeffery you can also use this in your robots.txt file

        User-agent: *
        Disallow: /thedirectorythatyourvidsarein

        Ta

        Baz
        {{ DiscussionBoard.errors[1416408].message }}
      • Profile picture of the author Dan C. Rinnert
        Originally Posted by Jeffery Baxter View Post

        I need for certain pages to be only accessed by certain people after they opt in. So for instance if anyone never opted in, that particular page is not even existent in the search engines.
        FYI, if you list the page in your robots.txt file, it will keep it from being indexed by Google, but it won't keep people out. For example, if I want to find your video page without opting in to your list, all I need do is take a look at your robots.txt file to find the location.
        Signature

        Dan's content is irregularly read by handfuls of people. Join the elite few by reading his blog: dcrBlogs.com, following him on Twitter: dcrTweets.com or reading his fiction: dcrWrites.com but NOT by Clicking Here!

        Dan also writes content for hire, but you can't afford him anyway.
        {{ DiscussionBoard.errors[1416442].message }}
  • Profile picture of the author Jason Stepp
    No problem Jeffery. Glad I could help!
    {{ DiscussionBoard.errors[1416422].message }}

Trending Topics