PLEASE COMMENT below with opinions or suggestions or improvements or new information!
I'll be happy to help analyze your site if you post it in this thread. but do not send me PMs asking me to analyze your de-indexed sites! My mailbox has been flooded.
Every day there seems to be 5-10 posts about how someones site was dropped from Google, was banned from Google, was deindexed from Google, received a Google slap, etc. Call it what you will, but to see your traffic suddenly drop is scary to say the least.
Usually they are scared and confused and make a post looking for answers where the same questions get asked over and over and over.
I'm starting this thread to collect all the common wisdom and advice for those poor folks who have seen their site disappear and what they can expect.
First, let's clarify some terms here:
- Deindexed - When your domain is completely removed from Google. Also known as Banned.
- Penalized - When your domain still exists but none of your pages can be found through very direct search queries. Also known as Sent To Hell and Sandboxed.
- Sunk - Your domain wasn't Deindexed or Penalized, but the traffic you were getting from Google suddenly drops dramatically. Keywords you used to rank well for (#1-5) are now ranking poorly (#5-50+).
Check With Google's WebMaster Tools
Go to webmaster.google.com. If you haven't setup an account and claimed your site, do it. Google will tell you exactly if there are any technical issues with your site here that may be causing your issues.
How To Tell If You Were Deindexed
Do a "site:yourdomain.com" search in Google, where "yourdomain.com" is your domain. For example, I would type the following exactly into a search box, without the quotes: "site:electronplumber.com".
The pages that come back are all the pages Google has in their database for your site. People call this your INDEXED PAGES. If your site doesn't show and you get this message "Your search - site:electronplumber.com - did not match any documents." you've been deindexed. Note that ANY typo or mistake will cause you to lose a lot of sleep over nothing. If all my pages are at blog.electronplumber.com or a sub-folder like electronplumber.com/blog, you need to do a site: search there. Check them all before you panic.
Don't forget that Google's main aim is to provide quality content and the best results for searchers. Typically a complete deindexing occurs from one of these things:
- Major Navigation Problems - This happens far more often than people realize in the IM world. Your pop-up opt-in or script that keeps people from using their back button to leave your site correctly WILL get your site deindexed by Google.
- Duplicate Content - Eventually a site that features ONLY duplicate content from shopping sources such as Amazon or Shopzilla will be deindexed.
- Offline For Too Long - Your site was offline multiple times for long enough for Google to deem it 100% no longer active. Google does NOT want sites in it's results that don't exist anymore. You may not have noticed your site ever being down, but if your server reboots at the same time Google tries to crawl it and a line goes down to your hosting provider the next time Google tries to crawl it, your site could get temporarily declared dead. This happens far more often than people think. It may take days or weeks to become reindexed. Only time will tell here.
- Technical Difficulties - Your site is blocked for indexing by a robots.txt file. Most people don't realize that some automatic Wordpress installs have a setting to not allow search engines to find your site.
- Manual Action - Your site received a manual review by a Google Search Quality Engineer and was found in serious violation of the Google Webmaster guidelines and includes a great many things you could have done wrong.
- Malware Links - If you use autoblogging content on your site, you could be hosting links to malware or virus sites without knowing it. Go through each link on your site and make sure none of them link to sites that Google reports as hosting malware.
- Too Fast A Rise - Your giant link building scheme worked and your site shot to the top of the search results for a new site for a monitored keyword For example, too many links to a new site with too little content. Note that I have never seen a site that was deindexed for backlinks alone without a Google Engineer looking at your site. Not one. It's far more likely that your link building worked too well, which caused your site to rise in the SERPS, which caused your site to appear on a Google Quality Engineers manual review report, or caused your longer term competition to report your site to Google as spam.
How To Tell If You Were Penalized
If your site passes the index test above but is not showing up for searches, you might have been Penalized. Do a "yourdomain.com" and a "www.yourdomain.com" search in Google, not including the quotes. Does your site appear at the top for at least one of those? Any site that hasn't been hit with a Google penalty should appear as the first results for an exact match search.
If your site doesn't appear, you may have been Penalized by a Google Search Quality Engineer. If your site has violated the Google Webmaster guidelines but still has some redeeming value, your site may be marked in the Google Search system with a penalty to all it's rankings for some period of time. If this happens to you, take a long hard look at your site. The main reasons for being Penalized are:
- Linking To Bad Neighborhoods - Linking OUT to bad sites such as gambling, porn, hacks, torrents, and the like are seen as bad for your site.
- Buying links - Tons of debate on if buying links can hurt your site. Most people say the worst that can happen here is you don't get credit for the bought links. If not, then I could buy links to my competition report them to Google and have them penalized. If your site obviously only has paid backlinks though, that might get you into trouble.
- Selling links - Being caught selling links will definitely hurt your site.
- Engaging in linking schemes - Basically trying to hide buying or selling links by using three way trades and such.
- Sneaky Redirects - Attempting to redirect visitors but not search engine robots.
- Hiding Content - Serving different links or content to search robots than human visitors.
- Unnatural Linking Pattern - Your backlinks triggered some algorithmic flag known only to Google. Typically due to keyword stuffing. This is by far the most common cause of algorithmic Google penalties. If more than 90% of your links are for a few highly targeted keywords, you'll hit this.
How To Tell If You Were Sunk
If you weren't Deindexed and weren't Penalized but your site's traffic from Google keyword searches has dropped dramatically, your site might have Sunk.
There is no manual action by Google, no removal from the index. But keywords you were previously ranking well for are no longer ranking well. There are MANY reasons why your site may have sunk in the Google rankings, but some of the most frequent are:
- Site Redesign - Internal links count and a site redesign may have removed what Google considers to be "aged" links, to the determent of your rankings
- Link Shaping - So you read an article that says using "nofollow" on internal links is a good idea? You know what? It's NOT. Using "nofollow" on links doesn't cause them to save their link juice, it pours that juice down the sink instead. Don't use "nofollow" except as a way to discourage comment spam.
- Technical Errors - Setting your robots.txt by accident to prevent indexing on some inner pages can have a VERY negative effect on your rankings
- Loss of Backlinks - If you rely on a few high value backlinks that suddenly disappear, your rank will drop.
- Loss of Backlink Value - I once had a site of mine linked in an article on the NPR All Tech Considered blog, a PR 7 page. They only update once every few weeks, so for about three months while the article stayed on the front page I had a VERY powerful backlink. My search traffic to that site increased about 25%. Then three months later after the article dropped off the front page, my search traffic dropped back down again. And on the flip side, one of my bigger sites started as a Wordpress.com free site. After a year I wised up and transferred it to my own hosted domain, paying Wordpress.com $12 to redirect everything from my old site to my new site. The next year I decided not to pay the $12 again since I was no longer getting traffic from the redirects. A few weeks later, my search traffic dropped close to 50%! It was no longer getting credit for those nice aged backlinks that were previously redirected. I coughed up the $12 and a few days later my traffic went back to normal.
- Lack of Backlink Growth - If you have been doing lots of backlink building and then suddenly drop off.
- Lack of Fresh Content - Google loves updates and fresh content gives your entire site a boost. Some people call this QDF or "Query Deserves Freshness"
- Competition - Someone is now outranking you.
- Google Update - Like the famed "Google Farmer Update", sometimes Google just changes their algorithm and you can get hit.
- Too Much Duplicate Content, Not Enough Links - I've seen this happen to many autoblog sites. After a few hundred articles without enough backlinks, you'll trigger some math formula where Google gets wise and you'll see search traffic plummet across the board.
- Outbound Link Junk - Too many outbound links can dilute any internal links you have. As a general rule, never have more than 5-10% outbound links on one page unless they're to one of your own sites. Don't increase your internal links to hit this percentage though!
- Penalized - Anything from the Penalized or Deindexed list above can cause your site to Sink instead of a harsher penalty if you are lucky.
- Loss of Honeymoon Effect - Google actually gives every new site a chance to be discovered. For the first few months your new site with new content will get a boost in the SERPS (Search Engine Ranking Positions) to give it a chance of being discovered and becoming popular, even when it has no links. When this effect goes away, your rankings will drop.
- Slow Site - Google takes into account site speed and loading times for their algorithms. If you are on a shared server, make sure your sites is optimized for speed as much as possible.
- Site Dilution - Too Much Content, Not Enough Backlink Power - Say when you first start your site, you have 10 posts and you make 10 backlinks. If you add 100 more posts and make 100 backlinks from the same 5 article directories, Google won't see those as 100 backlinks. The algorithm will see them as 10 good links and 90 lower value duplicate backlinks from the same domains and IPs. Now you have 110 articles sharing the same link power from your home page instead of 10 articles and everything drops in the SERPS. As you add fresh content, you have to create fresh DIVERSE backlinks to support that content.
You will hear people claim that your new site is in something called the Google Sandbox. The theory is that a new site that has too many backlinks created will get put in a box by Google for 6-12 months and not appear high up in the search results.
Matt Cutts, head of the Quality Team at Google has stated publicly that there IS NO GOOGLE SANDBOX, but there are parts of the Google algorithm that may lead people to believe that such a thing exists.
Google wants a quality experience for visitors of its search engine. People like new and fresh content. Putting new sites that go viral "in a sandbox" is NOT in Google's best interest.
The most likely explanation for The Sandbox effect is this:
- New unique content site gets created and gets a giant fresh content bonus (called QDF or Query Deserves Freshness) since everything is brand new.
- User builds tons of keyword stuffed spammy low quality backlinks to said new site. The link profile for that site looks unnatural to the algorithm. The Google algorithm very likely then IGNORES these low quality links for new sites.
- A month or two later, the site is Sunk to the depths of the search rankings once it loses its QDF bonus and only has ignored backlinks. Site owner thinks they have been sandboxed and stops building links, which makes the problem worse.
The lessons here?
- Make sure your backlinks look natural (links to words like "here" and "this site" and full URLs like "http://www.examplesite.com/article1"
- Make sure you create enough high quality links to any new site to counteract these effects.
Do NOT submit your site for Reconsideration unless you have identified what is wrong first and fixed it. You are basically asking for a Google Quality Engineer to come take a manual look at your site. Have you ever bought any links? Sold any links? Is your site keyword stuffed? Is your site mostly to earn you affiliate commissions? I'd guess 95% of the people on this forum do not really want a Google engineer taking a manual look at their site. A reconsideration request is like asking for an IRS audit without being 100% sure you are owed money. It will just end badly. If anyone with an affiliate site would like to share an example of a successful reconsideration request with us, I'd love to hear it. There is a first time for everything.
If you have received a manual action, you can usually tell. Use the Google toolbar and check if your PageRank is N/A or is grayed out. Check the Google Webmaster report for your site.
If you have been penalized/deindexed/sunk because the Google algorithm had an issue with your site, asking for reconsideration is just asking for trouble. Google will NEVER tell you what is wrong with your site and will just get you a manual review that will end badly. Figure it out using the lists above, fix it, then wait a few weeks before even considering submitting a reconsideration request.
Google Basics - Webmaster Tools Help - Google
Webmaster guidelines - Webmaster Tools Help
Search Engine Optimization (SEO) - Webmaster Tools Help - Google
Webmaster Guidelines - Webmaster Tools Help - Google
Link schemes - Webmaster Tools Help - Google
Paid links - Webmaster Tools Help - Google
Requesting reconsideration of your site - Webmaster Tools Help
Wikipedia: Sandbox Effect
Query Deserves Freshness and Other Temporal Tales | Search News Central
If you find this FAQ useful, please click the THANKS Button -------------------------------------VVVVV