How to stop Google from crawling my site until COMPLETION

Seofarmer · 09-01-2011, 11:47 PM

It usually takes me around 1-2 weeks to fully complete my WP site with contents etc. How can I stop Google from crawling/indexing my pages before my site is 100% complete and live?

Any suggestions? Thanks in advance!

Jim Hudson · 09-02-2011, 12:05 AM

You have an option in wp dashboard...under settings> Privacy> tick the box that says..

"I would like to block search engines, but allow normal visitors"

This will keep it from being hit by the search engines until you are ready.

Jim

lalitbansal004 · 09-02-2011, 01:08 AM

Yes This is The Option:-
In wp dashboard...under settings> Privacy> tick the box that says..

Block Search Engine and enable Visitor..but when your site completed Then enable the Google The Search engine ON.

Thanks.

techbabyy · 09-02-2011, 01:16 AM

Create a Robots.txt file and upload it to the root directory with following content:

Quote:

User-agent: *
Disallow: /

When your work is complete, remove the "/" and customize the robots.txt file as per your requirements

Seofarmer · 09-02-2011, 01:25 AM

Thanks alot guys!

Oh, and one more thing,
i was playing with permalink under settings, and I now renamed my pages nicely for SEO. (ex. domain.com/keyword)

However, i noticed that Google has already picked up and indexed some of my old pages (domain.com/?p=123 etc.). I am aware that it will show 404 error as those pages no longer exist (since they have been renamed). What should I do at this point?

Is there a way to clear those old pages form my database etc.?

Thanks for your time again

winsoar · 09-02-2011, 01:31 AM

I wouldn't publish the site until its ready. But I don't think it matters with Wordpress sites.

Srinagarhouseboats · 09-02-2011, 03:34 AM

Use robots.txt . upload this to site root directory

techbabyy · 09-02-2011, 07:32 AM

Quote:

Originally Posted by Seofarmer

Thanks alot guys!

Oh, and one more thing,
i was playing with permalink under settings, and I now renamed my pages nicely for SEO. (ex. domain.com/keyword)

However, i noticed that Google has already picked up and indexed some of my old pages (domain.com/?p=123 etc.). I am aware that it will show 404 error as those pages no longer exist (since they have been renamed). What should I do at this point?

Is there a way to clear those old pages form my database etc.?

Thanks for your time again

WordPress will automatically redirect Google Bots to the New Permalink and Google will change the URLs after sometime

For better results, make sure you are using Google XML Sitemap Generator WordPress Plugin

m1985nehal · 09-02-2011, 07:36 AM

robot.txt optimization is best solution for this !

09-01-2011, 11:47 PM	#1
Seofarmer HyperActive Warrior Join Date: Jun 2011 Posts: 191 Thanks: 14 Thanked 2 Times in 2 Posts	How to stop Google from crawling my site until COMPLETION It usually takes me around 1-2 weeks to fully complete my WP site with contents etc. How can I stop Google from crawling/indexing my pages before my site is 100% complete and live? Any suggestions? Thanks in advance!

09-02-2011, 12:05 AM	#2
Jim Hudson sixfigureadsensecode.com War Room Member Join Date: Aug 2007 Location: Montana, USA. Posts: 329 Blog Entries: 2 Thanks: 23 Thanked 131 Times in 70 Posts Social Networking Contact Info	Re: How to stop Google from crawling my site until COMPLETION You have an option in wp dashboard...under settings> Privacy> tick the box that says.. "I would like to block search engines, but allow normal visitors" This will keep it from being hit by the search engines until you are ready. Jim
	"In order to get a mental picture..You must first put film in the camera.." Jim Hudson Learn how I made good money with the Adsense program. I have built my first course to help you do just that. Six Figure Adsense Code Launches June 6th! Six Figure Adsense Code JV Invite Page

09-02-2011, 01:08 AM	#3
lalitbansal004 HyperActive Warrior War Room Member Join Date: May 2011 Location: Internet Posts: 399 Thanks: 15 Thanked 37 Times in 31 Posts Social Networking Contact Info	Re: How to stop Google from crawling my site until COMPLETION Yes This is The Option:- In wp dashboard...under settings> Privacy> tick the box that says.. Block Search Engine and enable Visitor..but when your site completed Then enable the Google The Search engine ON. Thanks.

09-02-2011, 01:25 AM	#5
Seofarmer HyperActive Warrior Join Date: Jun 2011 Posts: 191 Thanks: 14 Thanked 2 Times in 2 Posts	Re: How to stop Google from crawling my site until COMPLETION Thanks alot guys! Oh, and one more thing, i was playing with permalink under settings, and I now renamed my pages nicely for SEO. (ex. domain.com/keyword) However, i noticed that Google has already picked up and indexed some of my old pages (domain.com/?p=123 etc.). I am aware that it will show 404 error as those pages no longer exist (since they have been renamed). What should I do at this point? Is there a way to clear those old pages form my database etc.? Thanks for your time again

09-02-2011, 01:31 AM	#6
winsoar HyperActive Warrior War Room Member Join Date: Sep 2007 Location: Nottingham, United Kingdom. Posts: 277 Thanks: 30 Thanked 28 Times in 23 Posts Social Networking	Re: How to stop Google from crawling my site until COMPLETION I wouldn't publish the site until its ready. But I don't think it matters with Wordpress sites.

09-02-2011, 03:34 AM	#7
Srinagarhouseboats HyperActive Warrior Join Date: Jan 2011 Posts: 112 Thanks: 0 Thanked 4 Times in 4 Posts	Re: How to stop Google from crawling my site until COMPLETION Use robots.txt . upload this to site root directory
	Houseboats in Kashmir \| Kashmir Tourism \| Jammu and Kashmir Tourism

09-02-2011, 07:36 AM	#9
m1985nehal Pixel Advertising Company Join Date: Jul 2011 Posts: 42 Thanks: 0 Thanked 1 Time in 1 Post Social Networking	Re: How to stop Google from crawling my site until COMPLETION robot.txt optimization is best solution for this !
	Pixel Advertising Amazing Facts \| Strange Facts and Information