Scrapers, proxies, isp "fears," white hat, black hat -- I'm connnnfused!...

8 replies
Here's what I DO know (I think)...

There are scrapers (bot programs) I can purchase.
I can use them to gather leads (for example).
Some of them require the use of "proxies."
Proxies are... (I'm not sure) a path of using the internet that makes one anonymous and/or seem to be using another computer (?)

So, I guess what I am confused about is...

1) What exactly IS a proxy?
2) Where / how does one "get" a proxy?
3) How long are they "good" for?
4) Is using a proxy white or black hat?
5) WHY are proxies needed in some cases (and not in others)?
6) WHY are proxies needed EVER? I mean WHY would my isp CARE what I am "scraping" or how much I am doing it?
7) Is it the scraping that could trigger something bad happening (to me), or is it the mass contacting of the resulting leads that would trigger something bad?
8) WHAT EXACTLY is the "something bad" that could happen to me, if I use the bot in the "wrong" way, or without the proper "disguise?"

I really wish someone could clear all this up for me! It is quite confusing.
If I delve into a "dark" area, I wanna know what the risks are, and how to avoid them. If I wanna keep it all white hat, I wanna know exactly where the BORDERLINES are!!!

As it stands now, I am confused!!

-- TW
#black #connnnfused #fears #hat #isp #proxies #scrapers #white
  • Profile picture of the author monstermoney
    1) What exactly IS a proxy?
    It's just like an IP except you use it to hide your original home ip for whatever purposes.
    2) Where / how does one "get" a proxy?
    You can buy, scrape your own, or find them free on some websites.
    3) How long are they "good" for?
    Depends if you pay for them they can last forever cause you own them. Scraped ones don't last long.
    4) Is using a proxy white or black hat?
    It's generally used for black hat tactics.
    5) WHY are proxies needed in some cases (and not in others)?
    People use proxies only when they are trying to hide their home ip from certain websites for whatever reason.
    6) WHY are proxies needed EVER? I mean WHY would my isp CARE what I am "scraping" or how much I am doing it?
    Your ISP won't care most likely. But if you scrape from google constantly you will get temporarily banned by google in a heartbeat. Thats why you would need more proxies.
    7) Is it the scraping that could trigger something bad happening (to me), or is it the mass contacting of the resulting leads that would trigger something bad?
    Usually scraping google results only result in a temporary ban. other sites have other rules.
    8) WHAT EXACTLY is the "something bad" that could happen to me, if I use the bot in the "wrong" way, or without the proper "disguise?"
    If you use a bot with only your home ip, usually whatever's role that program is used for will not yield as much success rate after the first run because the ip from your home is already banned. Hence reasons for multiple proxies.

    Hope this helps.

    MM-
    {{ DiscussionBoard.errors[7639764].message }}
  • Profile picture of the author TimothyW
    Aha! So, it's being banned from google that we're trying to avoid, yes?
    Is there a known threshold (level of scraping) one can avoid surpassing, to avoid being banned?
    (so one does not have to use proxies)
    If so, what (in general) is that threshold?
    If I do use proxies, do I have to buy them?
    Do I use them once, then throw them away?
    Is it "done all the time," even though it's not technically kosher?

    Just trying to get a feel for the level of "darkness" I am potentially entering.
    (or is it even too taboo to discuss?)
    PM me, if need be.

    -- TW (what can I tell you, I have the mind of a boy scout)
    {{ DiscussionBoard.errors[7639995].message }}
  • Profile picture of the author Warrior X
    Just trying to get a feel for the level of "darkness" I am potentially entering.
    If you're talking about scraping email addresses, then pretty dark.

    And by dark, I mean federal offense.

    But you're not talking about doing that. Right?

    Jermy
    Signature
    #1 In WHITEBOARD VIDEOS - Great Way To Tell Your Story!
    Available Here
    {{ DiscussionBoard.errors[7640037].message }}
  • Profile picture of the author TimothyW
    What about scraping phone #'s?
    Or website addresses?
    Offline small biz leads -- that's the basic goal.

    Re: email addrs... I'm not sure why scraping them should be so bad -- if they exist, and they can be culled through automation, then why not? In other words, picking them out one-by-one is fine, but automating that procedure, that's no good?!? What difference should that make?

    Now, the bulk MAILING to those addrs, THAT I can see as being bad (no matter HOW the addrs were compiled).
    {{ DiscussionBoard.errors[7640056].message }}
  • Profile picture of the author drewfioravanti
    There are much more ethical ways to go about building a business.

    You could do what you are talking about and constantly be scamming, hiding and trying to find ways back in to the networks you get kicked out of.

    Or you could just slowly build a legitimate business and not have to worry and build long term, consistent income.
    {{ DiscussionBoard.errors[7640064].message }}
  • Profile picture of the author Warrior X
    What about scraping phone #'s?
    Or website addresses?
    Offline small biz leads -- that's the basic goal.
    Google places, for example, has all of that information.

    I guess if you were trying to build a big list of phone #'s to telemarket to, then bots would be a way to do it.

    Do you need so many #'s that you can't get them the old fashioned way?

    Now, the bulk MAILING to those addrs, THAT I can see as being bad (no matter HOW the addrs were compiled).
    Kind of splitting hairs, why collect addresses if you can't email them?

    Jeremy
    Signature
    #1 In WHITEBOARD VIDEOS - Great Way To Tell Your Story!
    Available Here
    {{ DiscussionBoard.errors[7640081].message }}
  • Profile picture of the author TimothyW
    There's nothing wrong with pushing things to the limit -- as long as you know in advance what that limit is.
    {{ DiscussionBoard.errors[7640090].message }}
    • Profile picture of the author TerranceCharles
      Sometimes all of these methods can sound exciting, giving you a quick method to do something. However, that usually is the case - it's just a quick method that can get you into trouble long-term such as getting banned, or labeled as a spammer etc. That is not a real way to build a business, I have been a victim of some of those services such as back-links and article spinning etc, in the end it got me nowhere and had my website rank lower and labeled, so I would stay away from theses types of "Instant Results" methods.
      {{ DiscussionBoard.errors[7640219].message }}

Trending Topics