What's the best way to prevent Google indexing a bunch of duplicate pages?

Google instructs how to prepare duplicate content on this page: Duplicate content - Webmaster Tools Help

thanks

Chris

I'll use
<meta name="robots" content="noindex">

on the pages I don't want indexed, thanks

Chris

use this on your duplicate pages

<meta name="canonical" content="http://your-main-page" />

Can anyone say which of the two methods (canonical or noindex) is better, and exactly why?

Chris

Why not just hide them in your robots.txt?

Chris, on pages that are very similiar, I would recommend implementing the canonical tag.

For example, lets say you have the two following pages, that are basically identical, except of course the trailing querystring values are different.

www.yoursite.com/product.php?ID=345
www.yoursite.com/product.php?ID=290

It would be best to simply use the canonical tag and set it to...


As a precaution, I always ensure that the page specified within the canonical tag will load without any issues, as this might cause problems, should Google index a page that fails to load. (This might be the case if the querystrings are essential to load given information on the page via an embedded SQL query, or stored procedure.)

As for restricting access to certain pages, I would do this via your robots.txt file. You'll find its much easier to do this rather than updating multiple pages across your site.

Because of course, you'll only need to change the one file.

Hope I havent missed anything here.

Best of luck

So, between

<meta name="canonical" content="http://your-main-page" />

and

<linkrel="canonical" href="www.yoursite.com/product.php">

what's the difference, which is better and why?

and, why is that better (or not) than hiding them in Robots.txt ?

thanks


Chris

If you can create one page and use code to modify it based on the query string, that would be the best way to go. Then you can just tell Google and Bing to ignore the parameter.

It sounds like all the different pages are the same except for a few details. You can have someone write code that will display the unique graphics and/or text that goes with the specific query string.

I do this with my PPC adgroups. If you click on one of my ads that promotes auto insurance in Michigan, the page will say "Michigan Auto Insurance Quotes." Instead of having 51 different pages, I just pull the state from the query string.

If you need to include more data than just the state name, you would put the line number (primary key) or some other unique value like an affiliate code of a table in the query string. The code would grab the data from that row of your table and add it to the page.

This might be the best long-term solution, but I program my own sites. It may be easier to use one of the above suggestions rather than deal with a programmer.

That being said, it doesn't sound like something a programmer would charge an arm and a leg for.

Thanks John, I've had a quick read of some of the links already. Thanks for your comments.

Thanks Alston, to start with, I'll do things an easier way which is just writing my own scripts to automatically generate all the similar pages in a few seconds. If the site becomes profitable I'll look into better ways of doing everything later, just want to get it going first


Chris

Chris, just some quick advice mate.

If your site is quite large, you'll want to make sure that you get it right the first time round.

It can be a real mess if you screw this up and you're left to try and fix it later. Especially once Google starts indexing pages.

OK, I understand the point

thanks!

Chris