How to avoid an entire site being Google Indexed?

Anonymous Affiliate · 06-26-2009, 02:47 AM

Hey warriors,

I have a domain where I upload personal photos and documents which are only meant to be viewed by family members. Obviously, I'd like the entire site to NOT be indexed by Google.

I figured that if I moved the entire site into a password-protected folder like www.MyDomain.com/family/ , being "family" the password-protected folder, Google bots would not be able to index any content within.

Is that correct? I just don't want anyone to be able to access (or even to be aware of) the contents unless he/she has the family password.

Thanks!

bryanclark · 06-26-2009, 02:52 AM

Even if they could spider it (which they shouldn't be able to if you set the permissions correctly), they couldn't see anything because you'd have to log in.

ken_p · 06-26-2009, 02:55 AM

I think the password will take care of it. They would be aware of your domain, and the family folder, but beyond that only those who has the password can access it.

Adrian Cooper · 06-26-2009, 03:00 AM

Quote:

Originally Posted by Anonymous Affiliate

Hey warriors,

I have a domain where I upload personal photos and documents which are only meant to be viewed by family members. Obviously, I'd like the entire site to NOT be indexed by Google.

I figured that if I moved the entire site into a password-protected folder like www.MyDomain.com/family/ , being "family" the password-protected folder, Google bots would not be able to index any content within.

Is that correct? I just don't want anyone to be able to access (or even to be aware of) the contents unless he/she has the family password.

Thanks!

You can use the "nocache" and "noindex" tags as well as "nofollow" in your headers to instruct Google to stay away.

Alternatively you can instruct the 'bots through "robots.txt" or .htaccess.

You can of course create a password access as has already been metioned though cPanel or .htaccess.

WendellC · 06-26-2009, 03:05 AM

In addition to password protecting the directory, I believe you can also create a robots.txt file with the following two lines in it.

User-agent: *
Disallow: /

Upload that to the root directory and that should also tell the spiders not to index any part of the site.

At least in theory...

Wendell

Amenda Jessera · 06-26-2009, 05:01 AM

You have to disallow indexing via robots.txt of that page...... then goolge will not idex it.....

KirkMcD · 06-26-2009, 08:39 AM

The only way to not be index by any search engine is to password protect the folder.

seo-authority · 06-26-2009, 08:53 AM

quick check is to do a search on site:yourdomain.com and check what search engines have picked up already. If your havnt already applied a password file which will resolve this then the spiders may have already indexed your site and therefore stored a cache of this. worth checking

06-26-2009, 02:47 AM	#1
Anonymous Affiliate Senior Warrior Member War Room Member Join Date: May 2009 Posts: 1,886 Thanks: 256 Thanked 332 Times in 244 Posts	How to avoid an entire site being Google Indexed? Hey warriors, I have a domain where I upload personal photos and documents which are only meant to be viewed by family members. Obviously, I'd like the entire site to NOT be indexed by Google. I figured that if I moved the entire site into a password-protected folder like www.MyDomain.com/family/ , being "family" the password-protected folder, Google bots would not be able to index any content within. Is that correct? I just don't want anyone to be able to access (or even to be aware of) the contents unless he/she has the family password. Thanks!

06-26-2009, 02:52 AM	#2
bryanclark Marketing Mogul Join Date: Jun 2009 Location: St. Louis Posts: 8 Thanks: 0 Thanked 3 Times in 2 Posts Social Networking	Re: How to avoid an entire site being Google Indexed? Even if they could spider it (which they shouldn't be able to if you set the permissions correctly), they couldn't see anything because you'd have to log in.

06-26-2009, 02:55 AM	#3
ken_p Newbie Warrior Join Date: Apr 2009 Location: The Tropics Posts: 243 Thanks: 31 Thanked 9 Times in 9 Posts Social Networking	Re: How to avoid an entire site being Google Indexed? I think the password will take care of it. They would be aware of your domain, and the family folder, but beyond that only those who has the password can access it.
	hates the rain

06-26-2009, 03:05 AM	#5
WendellC HyperActive Warrior War Room Member Join Date: Jan 2008 Location: Southern California Posts: 190 Thanks: 9 Thanked 52 Times in 34 Posts Social Networking	Re: How to avoid an entire site being Google Indexed? In addition to password protecting the directory, I believe you can also create a robots.txt file with the following two lines in it. User-agent: * Disallow: / Upload that to the root directory and that should also tell the spiders not to index any part of the site. At least in theory... Wendell
	Get a free one-way link from No Opt In Required for your no opt-in product. PM for details. Twitt..er...Me!

06-26-2009, 05:01 AM	#6
Amenda Jessera Theeban-Advanced Warrior Join Date: May 2009 Location: at the edge of the world Posts: 525 Blog Entries: 1 Thanks: 58 Thanked 21 Times in 21 Posts Contact Info	Re: How to avoid an entire site being Google Indexed? You have to disallow indexing via robots.txt of that page...... then goolge will not idex it.....

06-26-2009, 08:39 AM	#7
KirkMcD Senior Warrior Member War Room Member Join Date: Apr 2006 Location: , , USA. Posts: 2,433 Thanks: 40 Thanked 155 Times in 148 Posts	Re: How to avoid an entire site being Google Indexed? The only way to not be index by any search engine is to password protect the folder.

06-26-2009, 08:53 AM	#8
seo-authority Warrior Member Join Date: Jun 2009 Posts: 14 Thanks: 0 Thanked 0 Times in 0 Posts	Re: How to avoid an entire site being Google Indexed? quick check is to do a search on site:yourdomain.com and check what search engines have picked up already. If your havnt already applied a password file which will resolve this then the spiders may have already indexed your site and therefore stored a cache of this. worth checking
	Chris Smith SEO Consultant & Internet Stratagist Affordable SEO \| Yahoo SEO \| Search Engine Optimisation