How to remove from google index auto generated urls

valdonatas · 11-21-2011, 06:26 AM

Hi everyone,

I suspect I have a problem with duplicated content - and that's making my SEO efforts worthless. For some reason I find 1050 pages on google index, while in reality there should be no more than 50 to 100 pages in total. I found that google has indexed pages like azbuilders.co.uk/index.php?rating75758=3 - there are many hundreds of pages like this. I am not sure how these pages have appeared in my website - probably some plugin or component created them, just can't figure out what. I have put a line disallow in robots.txt for pages like this - but this will only stop bots from crawling - without removing them from index.
So please advice what should I do to get rid of pages like this. Thanks

dhex · 11-21-2011, 07:39 AM

Its easy, if you have already linked your website to GWT, you can request either page by page removal or you can remove the whole directory from Google index.

But before you submit your removal request, you need to make sure that the URLs in question return 404. You should also disallow Google bot in your robots.txt.

While you're at it you should also try to reduce your page size. Now your homepage weighs more than 1MB. Check this for what you need to do about your page size; https://developers.google.com/pagesp...p&mobile=false

valdonatas · 11-21-2011, 08:11 AM

How did you find that my homepage weighs 1MB? I checked it with Webmaster Tools & SEO Tools website speed test - it shows only 25 kB

dhex · 11-21-2011, 08:29 AM

use firebug to see how many times your page makes server request, how much each page's components weigh, and the the time it takes to get img,html,css, etc from server to your browser. Its on "Net" tab. According to my last ran your heaviest img is http://www.azbuilders.co.uk/images/s...c/tmp/van1.png at 115kb and your homepage makes a total of 48 requests.

LinkVariety · 11-21-2011, 08:31 AM

Quote:

Originally Posted by valdonatas

Hi everyone,

I suspect I have a problem with duplicated content - and that's making my SEO efforts worthless. For some reason I find 1050 pages on google index, while in reality there should be no more than 50 to 100 pages in total. I found that google has indexed pages like azbuilders.co.uk/index.php?rating75758=3 - there are many hundreds of pages like this. I am not sure how these pages have appeared in my website - probably some plugin or component created them, just can't figure out what. I have put a line disallow in robots.txt for pages like this - but this will only stop bots from crawling - without removing them from index.
So please advice what should I do to get rid of pages like this. Thanks

There are a couple of options:

1) Add a rule in robots.txt to block those pages
2) Set a 301 redirect up to bounce all those offending urls back to your homepage (best method)

As for how they got indexed, negative SEO is one possibility. It is possible to get large numbers of pages like this indexed on a low quality site and adversly affect its ranking.

11-21-2011, 06:26 AM	#1
valdonatas valdonatas Join Date: Nov 2011 Posts: 15 Thanks: 0 Thanked 2 Times in 2 Posts	How to remove from google index auto generated urls - duplicate content Hi everyone, I suspect I have a problem with duplicated content - and that's making my SEO efforts worthless. For some reason I find 1050 pages on google index, while in reality there should be no more than 50 to 100 pages in total. I found that google has indexed pages like azbuilders.co.uk/index.php?rating75758=3 - there are many hundreds of pages like this. I am not sure how these pages have appeared in my website - probably some plugin or component created them, just can't figure out what. I have put a line disallow in robots.txt for pages like this - but this will only stop bots from crawling - without removing them from index. So please advice what should I do to get rid of pages like this. Thanks
	builders london:Quality renovations, refurbishments in North and East London areas bathroom fitters London: quality bathroom installations in London

11-21-2011, 07:39 AM	#2
dhex Active Warrior Join Date: Nov 2009 Posts: 66 Thanks: 19 Thanked 3 Times in 3 Posts	Re: How to remove from google index auto generated urls - duplicate content Its easy, if you have already linked your website to GWT, you can request either page by page removal or you can remove the whole directory from Google index. But before you submit your removal request, you need to make sure that the URLs in question return 404. You should also disallow Google bot in your robots.txt. While you're at it you should also try to reduce your page size. Now your homepage weighs more than 1MB. Check this for what you need to do about your page size; https://developers.google.com/pagesp...p&mobile=false

11-21-2011, 08:11 AM	#3
valdonatas valdonatas Join Date: Nov 2011 Posts: 15 Thanks: 0 Thanked 2 Times in 2 Posts	Re: How to remove from google index auto generated urls - duplicate content How did you find that my homepage weighs 1MB? I checked it with Webmaster Tools & SEO Tools website speed test - it shows only 25 kB
	builders london:Quality renovations, refurbishments in North and East London areas bathroom fitters London: quality bathroom installations in London

11-21-2011, 08:29 AM	#4
dhex Active Warrior Join Date: Nov 2009 Posts: 66 Thanks: 19 Thanked 3 Times in 3 Posts	Re: How to remove from google index auto generated urls - duplicate content use firebug to see how many times your page makes server request, how much each page's components weigh, and the the time it takes to get img,html,css, etc from server to your browser. Its on "Net" tab. According to my last ran your heaviest img is http://www.azbuilders.co.uk/images/s...c/tmp/van1.png at 115kb and your homepage makes a total of 48 requests.