Is There a Way to Unrank a Site from the SERPs?

[MaxReferrals]

Say I have a site that is ranked. The site is aged 4-5 years, etc.


Is there a way in robots.txt (or otherwise) to get the site unranked/de-indexed, so essentially it drops off the SERPs?

The site still needs to be operationally functional... just not anywhere in the serps.


Any ideas folks?

[tonis]

Robots exclusion standard - Wikipedia, the free encyclopedia

Google it

[John Williamson]

You could block robots.txt, change URL structure, delete your backlinks and SEO efforts, send G a URL removal request. Any of those should work.

[MaxReferrals]

I forgot to also mention, what happens if I do *not* currently have a robots.txt file?

Just create one and add the disallow code?

[John Williamson]

Yes you can create one and put in the disallow code. I think there are some robots.txt file generators out there that will create one for you.

[Chasing Pace]

Would just changing your WordPress privacy settings to not allow search engines to index the site work?

[Maraun]

Yes it's essentially the same thing. But both methods will take a while to show effect.

[Feydakin]

All this works, but it could be months before the site is totally de-indexed. Google keeps data forever and I'm always running across ranked listings that point to long dead sites.

[paulgl]

Unfortunately, for things like this, there is never
any guarantee. You could password protect the page, or
at best, put a script in front so nobody gets in without
an action by a human. That would effectively kill any
searchbot at the door. That is, provided they crawl the
site frequently. Any other method is not fool proof.

The reason is rather simple. Not all searchbots follow
robots.txt instructions. Google is not the only place
people find sites. Also, your links are probably indexed,
meaning people can still "find" your site in unexpected
SERPS in the form of a link. Google can find your site
via other links, which is actually an end around on
the robots.txt file. I think people would be surprised
that the googlebot can go from an outside link
directly to your page and disregard the robots.txt.

I believe you need to ask google to remove it
regardless, or else the chances of it appearing are
still there. Do the robots.txt thing, request a removal.
That works in a perfect world.

The internet does not follow earthly laws of gravity.
If it exists online, chances are, someone finds it
listed somewhere.

Paul

[vlada111]

This is from google support team:





"To ensure your directory or site is permanently removed, you should use robots.txt to block crawler access to the directory (or, if you’re removing a site, to your whole site). We recommend doing this before or soon after requesting removal of the directory. Otherwise, your content may later reappear in search results. (More information about blocking Google.) Returning a 404 HTTP status code isn't enough, because a directory can return a 404 status code, but still serve out the files it contains.
Once you've blocked crawler access to the directory, follow these steps to remove a site or directory from Google:

1. On the Webmaster Tools home page, click the site you want.
2. On the Dashboard, click Site configuration on the left-hand menu.
3. Click Crawler access, and then click Remove URL.
4. Click New removal request.
5. Type the URL of the directory you want removed from search results and then click Continue. (How to find the right URL.)
   • The URL is case-sensitive—use exactly the same characters and capitalization that the site uses.
   • If you want to remove the whole site, you can leave this blank.
6. Click Remove directory.
7. Click Submit Request.

Important! The URL removal tool is intended for pages that urgently need to be removed—for example, because they contain confidential data that was accidentally exposed. Using the tool for other purposes may cause problems for your site. When not to use the URL removal tool."

[cardine]

I'd check out this page on Google; they describe pretty well what to do if you don't want them to spider your results (and what to do if you don't want your results anywhere).

What they say is do a robots.txt, and if you don't want it to appear anywhere on the web, password protect the page first. That way only people who know the password can visit the page and it'd be impossible for anyone who didn't know the page to be able to access it.

[ghazia]

Create a notepad file and name it as robots.txt. Add the below coding in that file and upload it into your root folder.

User-agent: *
Disallow:/

This will remove your site from SERP.

[AndyBlackSEO]

Just the noindex, nofollow wil suffice. I did this accidentally for one of my main sites and it completely dissappeared out of the index for a few weeks. Just the homepage that was... It was only when I checked the meta info and seen that I'd somehow swapped templates and added the noindex,nofollow meta.

After removing it the site re-appeared in exactly the same search engine places as before... just a few days later after being re-crawled.

[Witty]

If you just want it removed from Google, then you'll need to specifiy the Googlebot in your robots.txt, otherwise no need to specify any bot if you want it removed from all search engines.