Is Having Google Analytics/Webmaster Tools on Your Site a Bad Idea?

I know that there isn't a definitive yes or no answer to this, I just want people's opinions on something I've heard quite a bit about in my time in online marketing.

The argument is that when Google has access to all of your site's analytics, they know how long people spend on your site, click through rates, etc.; basically information which can be used to measure the quality of a site, particularly against another site in the same niche.

So Google can look at Site A and Site B in a particular niche and recognize that Site B boasts better stats than Site A and therefore must be of greater quality and therefore deserves to rank above Site A. Even without directly comparing two sites, they can take this into consideration when grading and ranking a site.

Without giving Google access to this data, they don't have that extra factor to weave into their Algo. (Just offering this side of the argument)

Some webmasters have made note that some of their sites which have these tools set up perform worse than other sites of theirs without it with most everything else being equal. That being said there are too many factors in SEO to ever narrow it down like this.

I know there is no definite answer one way or the other but it's something I finally decided to make a post about and present to other marketers here, so I'd love to get some thoughts.