How to stop Google from crawling my site until COMPLETION

15 years ago

You have an option in wp dashboard...under settings> Privacy> tick the box that says..

"I would like to block search engines, but allow normal visitors"

This will keep it from being hit by the search engines until you are ready.

Jim

[ 1 ] Thanks
1 reply

Signature

"In order to get a mental picture..You must first put film in the camera.." Jim Hudson
Learn how to cloak affiliate links for Facebook for FREE Plus More!

{{ DiscussionBoard.errors[4593517].message }}

lalitbansal004 15 years ago

Yes This is The Option:-
In wp dashboard...under settings> Privacy> tick the box that says..

Block Search Engine and enable Visitor..but when your site completed Then enable the Google The Search engine ON.

Thanks.
- [ 1 ] Thanks
Signature
Boost Website SERP and Traffic With Article Submission and Social Bookmarking start with $8.

Cheap Social Bookmarking

Email - lalitbansal004@gmail.com | Yahoo - lalitbansal004 | Skype - lalitbansal004
{{ DiscussionBoard.errors[4593811].message }}

techbabyy

15 years ago

Create a Robots.txt file and upload it to the root directory with following content:

User-agent: *
Disallow: /

When your work is complete, remove the "/" and customize the robots.txt file as per your requirements

[ 1 ] Thanks
1 reply

{{ DiscussionBoard.errors[4593848].message }}

Seofarmer

15 years ago

Thanks alot guys!

Oh, and one more thing,
i was playing with permalink under settings, and I now renamed my pages nicely for SEO. (ex. domain.com/keyword)

However, i noticed that Google has already picked up and indexed some of my old pages (domain.com/?p=123 etc.). I am aware that it will show 404 error as those pages no longer exist (since they have been renamed). What should I do at this point?

Is there a way to clear those old pages form my database etc.?

Thanks for your time again

Thanks
1 reply

{{ DiscussionBoard.errors[4593890].message }}

techbabyy

15 years ago

Originally Posted by Seofarmer

Thanks alot guys!

Oh, and one more thing,
i was playing with permalink under settings, and I now renamed my pages nicely for SEO. (ex. domain.com/keyword)

However, i noticed that Google has already picked up and indexed some of my old pages (domain.com/?p=123 etc.). I am aware that it will show 404 error as those pages no longer exist (since they have been renamed). What should I do at this point?

Is there a way to clear those old pages form my database etc.?

Thanks for your time again

WordPress will automatically redirect Google Bots to the New Permalink and Google will change the URLs after sometime

For better results, make sure you are using Google XML Sitemap Generator WordPress Plugin