Do search engines index secure webpages?

redbrad0 · 12-14-2008, 08:36 PM

I have done some research and it appears that they do, but I wanted to get some other opinions. On one of my websites I have secure pages for the ordering, but switching users between secure and unsure pages gets to be a pain sometimes so I was thinking about just making all pages on the website secure.

Steven Carl Kelly · 12-14-2008, 08:38 PM

Yes, they do. I am familiar with several sites that can ONLY be accessed through port 443 and all of them are indexed.

Sylvan · 12-14-2008, 09:14 PM

I think it depends on your cookies settings. how secure is your page? I always disallow access to my download directories withot the proper link and disallow spides to index those pages.

Mike Bogowski · 12-14-2008, 09:27 PM

They will index secure pages just fine.

You'll need to consider the issues of running your entire site through https though. This can be a real CPU killer if you are getting a lot of traffic.

onera · 12-14-2008, 09:32 PM

Google will index anything that it can reach unless your tell the google crawler not to index a directory using robots.txt file.

Curt7 · 12-14-2008, 09:51 PM

I'm aware of the robots.txt file being used to stop indexing. Can anyone tell me what to put in it to stop search engines from indexing an entire site? I have one site that I use just for my thank you pages and downloading my products.

Lavinco · 12-14-2008, 09:58 PM

Quote:

Originally Posted by Curt Dillion

I'm aware of the robots.txt file being used to stop indexing. Can anyone tell me what to put in it to stop search engines from indexing an entire site? I have one site that I use just for my thank you pages and downloading my products.

Make sure you want to do this for sure first.

Simply create a file named robots.txt and add
User-agent: *
Disallow: /

Curt7 · 12-15-2008, 08:19 AM

Quote:

Originally Posted by Lavinco

Make sure you want to do this for sure first.

Simply create a file named robots.txt and add
User-agent: *
Disallow: /

Thank you.

Lavinco · 12-16-2008, 08:12 AM

Quote:

Originally Posted by Curt Dillion

Thank you.

You're welcome.
To better explain what this is doing, whould be where the / is represents the entire root directory.

User-agent: *
Disallow: /

If you specified a subfolder like

User-agent: *
Disallow: /images/

then you sre telling the bots to stay out of the /images/ folder. When you simply add the / alone, this instructs the bot to stay out of the entire site.

12-14-2008, 08:36 PM	#1
redbrad0 Advanced Warrior War Room Member Join Date: Aug 2003 Location: Oklahoma City, OK Posts: 514 Thanks: 0 Thanked 0 Times in 0 Posts Social Networking	Do search engines index secure webpages? I have done some research and it appears that they do, but I wanted to get some other opinions. On one of my websites I have secure pages for the ordering, but switching users between secure and unsure pages gets to be a pain sometimes so I was thinking about just making all pages on the website secure.
	StubWire - Full Service Ticketing company with low cost ticket printing

12-14-2008, 08:38 PM	#2
Steven Carl Kelly Trust Christ Alone War Room Member Join Date: Sep 2008 Location: Central Florida Posts: 3,000 Thanks: 63 Thanked 810 Times in 391 Posts	Re: Do search engines index secure webpages? Yes, they do. I am familiar with several sites that can ONLY be accessed through port 443 and all of them are indexed.
	Read this SURPRISING REPORT Before You Buy ANY WSO! Click Here FREE REPORT: Split Test Your Landing Pages the Easy Way

12-14-2008, 09:14 PM	#3
Sylvan Banned War Room Member Join Date: Jul 2008 Location: , , USA. Posts: 256 Thanks: 12 Thanked 4 Times in 4 Posts	Re: Do search engines index secure webpages? I think it depends on your cookies settings. how secure is your page? I always disallow access to my download directories withot the proper link and disallow spides to index those pages.

12-14-2008, 09:27 PM	#4
Mike Bogowski Puppet Master Join Date: Nov 2007 Location: Sydney, Australia. Posts: 627 Thanks: 7 Thanked 63 Times in 18 Posts	Re: Do search engines index secure webpages? They will index secure pages just fine. You'll need to consider the issues of running your entire site through https though. This can be a real CPU killer if you are getting a lot of traffic.
	Puppets are people too

12-14-2008, 09:32 PM	#5
onera Read Product Reviews Join Date: Dec 2007 Location: USA. Posts: 176 Thanks: 0 Thanked 9 Times in 8 Posts Contact Info	Re: Do search engines index secure webpages? Google will index anything that it can reach unless your tell the google crawler not to index a directory using robots.txt file.
	Post Unlimited Free Classified Ads in the Oldest Online Classified Site. Build Links, Increase Traffic, and Make Money by Creating MySpace type Free Pages For Linking to Digg Style Submissions

12-14-2008, 09:51 PM	#6
Curt7 HyperActive Warrior War Room Member Join Date: Aug 2005 Location: Near Seattle, Washington, USA. Posts: 387 Thanks: 26 Thanked 41 Times in 27 Posts Contact Info	Re: Do search engines index secure webpages? I'm aware of the robots.txt file being used to stop indexing. Can anyone tell me what to put in it to stop search engines from indexing an entire site? I have one site that I use just for my thank you pages and downloading my products.