IM Tools and Google TOS Breaches

[JimmyR]

I'm trying to get my head around how companies create software that obviously breach various parties terms of service, especially Google's, without being taken down.

A big example is ScrapeBox. As far as I can tell they are a legit company, with an office address. They have written software that obviously breaches the Google TOS, and I wouldn't exactly say they are flying under the radar. Every second IM'er knows what ScrapeBox is, so surely Google does too. If that is the case, why haven't they been taken down yet?

Here are the terms I am talking about:

"You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services."

Scrapebox is hardly the only example. There is Market Samurai, every content scraper around, every rank checker around, and more. Does the big G just not have time to deal with this kind of stuff, or do the software companies somehow pass blame down to the user. Something like 'You acknowledge that by using our tools you may be in breach of Google's TOS. If you choose to use our tool to scrape/check/whatever, you accept the responsibility associated with this ..."

I am building a tool that needs to be able to scrape Google. I don't want to go against the TOS, but every second IM tool out there must be in breach of this. So I guess the question is, how are they getting away with it? Even the Google Custom Search API terms forbid scraping...

[Johnny Optimo]

People DO get banned/penalized when they get caught breaking the rules. These methods are popular for a few years and may even be effective in the short term. Sooner or later though, as technology gets better, many will get what's coming to them.

nothing would hurt more than to have spent years building an IM income on a house of cards only to have it all come tumbling down.. that kind of thing happens ALL THE TIME and it's sad.

[MaxReferrals]

Most get around it by not hammering their sites, and building
within their public API. Namely, balancing the traffic
load they send via their scripting... whether by internal throttling
or even add-ons, such as proxy with the software/scripting.

[JimmyR]

Originally Posted by MaxReferrals

Most get around it by not hammering their sites, and building
within their public API. Namely, balancing the traffic
load they send via their scripting... whether by internal throttling
or even add-ons, such as proxy with the software/scripting.

Hiding the act with proxies or throttling is easy, I am more surprised that the software is even allowed to exist. The way I see it, if Google were to enforce their ToS, they could issue a cease and desist or just sue any company which creates a bit of software which scrapes Google search pages. This would even include things like Market Samurai, as I am pretty sure their rank tracker is interrogating search pages with an 'automated script', which is specifically against the ToS.

[Steven W Johnson]

Originally Posted by JimmyR

"You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers)

When you think about it, crying foul over being scraped when you are a company THAT OUT-SCRAPES ALL OTHER SERVICES ON THE PLANET COMBINED is downright laughable. I wonder what a judge would say.

[jasonl70]

craigslist has won million dollar cases against people who facillitated the violation of their TOS.. it seems google could close these guys down fairly quickly if they chose to.

[JimmyR]

Originally Posted by Steven W Johnson

When you think about it, crying foul over being scraped when you are a company THAT OUT-SCRAPES ALL OTHER SERVICES ON THE PLANET COMBINED is downright laughable. I wonder what a judge would say.

My thoughts exactly. But when I thought about it, they DO abide by your robots.txt, and will not explore your site if you don't allow bots.

That said, I wouldn't be surprised if they did keep all of the info, but just removed it from the SERPs. They aren't exactly the most ethical company ever. Remember the whole wifi scraping by street view cars thing?

[schwagoo]

Interet marketers keep google relevant. We sell rank and we throw around the name 'google' as THE engine to rank on. In a sense, we are selling google and expanding their brand every day.
Yes they could go on a suing frenzy and try to shut down every developer of automation software....but they would still be made and distrubuted from overseas.

Not a battle that google wants to engage in full-force.

[Chris Sweeney]

Originally Posted by JimmyR

I'm trying to get my head around how companies create software that obviously breach various parties terms of service, especially Google's, without being taken down.

A big example is ScrapeBox. As far as I can tell they are a legit company, with an office address. They have written software that obviously breaches the Google TOS, and I wouldn't exactly say they are flying under the radar. Every second IM'er knows what ScrapeBox is, so surely Google does too. If that is the case, why haven't they been taken down yet?

Here are the terms I am talking about:

"You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) and shall ensure that you comply with the instructions set out in any robots.txt file present on the Services."

Scrapebox is hardly the only example. There is Market Samurai, every content scraper around, every rank checker around, and more. Does the big G just not have time to deal with this kind of stuff, or do the software companies somehow pass blame down to the user. Something like 'You acknowledge that by using our tools you may be in breach of Google's TOS. If you choose to use our tool to scrape/check/whatever, you accept the responsibility associated with this ..."

I am building a tool that needs to be able to scrape Google. I don't want to go against the TOS, but every second IM tool out there must be in breach of this. So I guess the question is, how are they getting away with it? Even the Google Custom Search API terms forbid scraping...

Originally Posted by JimmyR

Hiding the act with proxies or throttling is easy, I am more surprised that the software is even allowed to exist. The way I see it, if Google were to enforce their ToS, they could issue a cease and desist or just sue any company which creates a bit of software which scrapes Google search pages. This would even include things like Market Samurai, as I am pretty sure their rank tracker is interrogating search pages with an 'automated script', which is specifically against the ToS.

Ahhh, man I totally agree with you. I posted something similar a few months back because Google makes no sense sometimes. Here's the post if you're interested: If certain types of sites/SEO methods are so bad, then why doesn't Google stop them at the source?