14 replies
  • SEO
  • |
If i'm buying a proxy package to scrape URLs on Scrapebox how many IPs should be in the package if I want to scrape 3/4 times in a month?

Some of these packages have just 10-20 IPs a month, but won't they all get used up in the first scrape I make?
#proxies #scrapebox
  • Profile picture of the author JSProjects
    Scraping with the URL harvester will blow through proxies REALLY fast. So it's best to find decent public proxies for this purpose. (Not the ones built into SB itself.)

    For most everything else you can get by with 5-20 depending on how heavily you use the program.
    {{ DiscussionBoard.errors[5881091].message }}
    • Profile picture of the author Webpals
      I've been using the public proxies, but even after testing a fresh batch of 2000 you only end up with 30-50 good proxies which when harvesting will scrape around 5 or 6 sites before they get burnt up.

      How do people manage to scrape hundreds of sites using scrapebox in one go?
      {{ DiscussionBoard.errors[5881526].message }}
  • Profile picture of the author kingtana1
    You can actually do great using private proxies for this. Using only public proxies for harvesting is the common practice I know, but i have found I can scrape forever with my elite proxies, you just have to make sure you get the connections right in scrapebox.

    No matter what engine you use to harvest, make sure you only have 5 connections running. For time out settings, 30 seconds will do.

    You need speed and reliability, the kind of attributes you will only find with elite proxies. If you do happen to get them burned in the engine's, within a few hours you will be able to use them again.

    With the ton of urls i end up harvesting with my elite's, I spend more time posting than scraping.

    Best of Luck!
    {{ DiscussionBoard.errors[5996144].message }}
  • Profile picture of the author patco
    Don't use private proxies for scraping. You can also use public proxies or the proxies that are in Scrapebox (although the 1st option is better!). And ... Public proxies are FREE!
    Signature

    A blog that will show you How to Lose Weight with a cool Quick Weight Loss guide...
    Also enjoy some of my favorite Funny pictures and photos that will make you smile :)

    {{ DiscussionBoard.errors[5996782].message }}
    • Profile picture of the author JSProjects
      Originally Posted by patco View Post

      Don't use private proxies for scraping. You can also use public proxies or the proxies that are in Scrapebox (although the 1st option is better!). And ... Public proxies are FREE!
      I haven't tried the public proxies found in SB in awhile. I remember them being HORRIBLE a long time ago, though.
      {{ DiscussionBoard.errors[5997067].message }}
  • Profile picture of the author GuerrillaIM
    Add a courtesy timeout to the request and you wont burn through your private proxies so fast. Also never run more threads than you have proxies.
    {{ DiscussionBoard.errors[5997077].message }}
  • Profile picture of the author StevenJones
    Buy proxies for scrapebox, xrumer| where to buy proxies perhaps this is something for you? When I was doing heavy thing with scrapebox just 5 private proxies did the trick.
    {{ DiscussionBoard.errors[5997139].message }}
  • Profile picture of the author cypherslock
    There's a WSO on here for software called Proxy Goblin. $30 and you can harvest and filter solid proxies and set the program on a schedule to email you the results. When I'm doing something big I have it scrape and filter once per day.
    {{ DiscussionBoard.errors[5997232].message }}
  • Profile picture of the author BCRed
    Proxy Goblin and BuyProxies.org both work well in my experience
    {{ DiscussionBoard.errors[5997960].message }}
  • Profile picture of the author John K Glenn
    If you are Harvesting Proxies then first thing you should check out is to find how frequently the Website you are Using to Harvest Proxy is getting updated. If fresh proxies are scraped , success rate would be high.
    {{ DiscussionBoard.errors[6001708].message }}
  • We have a couple hundred private proxies available right now that are guaranteed to work with Scrapebox. Let me know if you're interested.

    Signature
    Philly Web Services - SEO, Internet Traffic, Reputation Management, Website Design, & PPC Management.
    {{ DiscussionBoard.errors[6348230].message }}

Trending Topics