Help Me Understand Blocking Search Engines From Viewing Pages Please

[Jeffery Baxter]

Hi I'm getting to the point now where I will be launching all of my videos on a page instead of people having to download a zipped file..

Ok here's what I need laid out for me.

I need for certain pages to be only accessed by certain people after they opt in. So for instance if anyone never opted in, that particular page is not even existent in the search engines.

How do I do that?

Thanks in advance

[bazzais]

You will have to have a login system that will protect the pages unless a user is logged in (this will stop robots from visiting the pages because they are protected) or have a password protected directory at server level.

Its the only fullproof way.

Ta

Baz

[Jason Stepp]

I've never ran a site with different access levels, so I can't really help you out with that part of your question.

However, if you don't want search engines to index a specific page, you can add a no-index meta tag to the top of the pages you don't want indexed.

<meta name="robots" content="noindex" />

You insert this code in the same location as all of your other meta tags.

It's important to note that this meta tag is not honored by all search engines. However, Google does honor the meta tag.

Read more about the no-index meta tag from Google here:
Using meta tags to block access to your site - Webmasters/Site owners Help

[Jeffery Baxter]

Originally Posted by bazzais

You will have to have a login system that will protect the pages unless a user is logged in (this will stop robots from visiting the pages because they are protected) or have a password protected directory at server level.

Its the only fullproof way.

Ta

Baz

Thanks but i believe that having a password protection might not be the only way. I'm putting my videos on sort of like a regular page and I seen others do this. That's how I know that having a login might not be mandatory...??

[Jeffery Baxter]

Originally Posted by Jay284

I've never ran a site with different access levels, so I can't really help you out with that part of your question.

However, if you don't want search engines to index a specific page, you can add a no-index meta tag to the top of the pages you don't want indexed.

<meta name="robots" content="noindex" />

You insert this code in the same location as all of your other meta tags.

It's important to note that this meta tag is not honored by all search engines. However, Google does honor the meta tag.

Read more about the no-index meta tag from Google here:
Using meta tags to block access to your site - Webmasters/Site owners Help

Thats what I needed ..Thanks bro

[bazzais]

Thanks jay284 your correct that will stop spiders.

Jeffery you can also use this in your robots.txt file

User-agent: *
Disallow: /thedirectorythatyourvidsarein

Ta

Baz

[Jason Stepp]

No problem Jeffery. Glad I could help!

[Dan C. Rinnert]

Originally Posted by Jeffery Baxter

I need for certain pages to be only accessed by certain people after they opt in. So for instance if anyone never opted in, that particular page is not even existent in the search engines.

FYI, if you list the page in your robots.txt file, it will keep it from being indexed by Google, but it won't keep people out. For example, if I want to find your video page without opting in to your list, all I need do is take a look at your robots.txt file to find the location.