Pulling Data From Wholesale Website

by 5 replies
6
Hello,

I'm curious as to how I'd go about pulling data directly from a wholesalers website. Normally I make use of my suppliers having an ftp data feed but in this instance they do not. Instead I need to somehow pull the necessary data (sku, upc, cost, inventory) directly from their website.

I've heard of this being done before but have no idea what to call it or how to proceed. Any ideas or info would be greatly appreciated!
#programming #data #pulling #website #wholesale
  • PHP Code:
    jQuery.ajax({
                url: "http://DROPSHIPPERPAGE.COM/",
                success: function(result) {
                    html = jQuery(result);

                    alert(html.find("div#a").attr("id"));
                    alert(html.find("div#a").html());
                    alert(html.find("div#a"));

            },
        }); 
    With AJAX, you can get information from a certain URL and in this case alert value of each <div#a>; or in your case append it to your website.
    • [2] replies
    • Is it possible to save that data to an FTP as a CSV file? I'd like to try to automate the process where inventory quantities are being constantly updated every hour or so.
    • This won't work unless they have CORS set up to allow your site to query theirs. Otherwise, your browser will prevent the AJAX from loading in order to guard against XSS attacks.

      What would work would be to write a script that uses cURL (or similar) to automatically fetch and parse the data from the wholesaler's website. The data that you parse could be saved as CSV or any format you like. But you'd need a custom program to do all this and you should check with your wholesaler to make sure that they wouldn't mind you doing this.
  • does the website require login? are you able to run PHP?

    if no login is required to access the data you just need to make a script to fetch the page (as previously mentioned using cURL or something) and parse it with Simple_HTML_Dom. write the results to a file locally and then use PHP's FTP to upload it wherever you want.
    • [1] reply
    • Using python and BS:
      the raw code will be as:
      table=xxx.php
      do while (time==end)
      url="xxx"
      content = urllib2.urlopen(url).read()
      soup = BeautifulSoup(content)
      soup.find_all("p")
      write("p")=table
      wait 1hour
      If you are not familiar with php and python, it may take you some time to catch up.
Join the discussion

Next Topics on Trending Feed

  • 6

    Pulling Data From Wholesale Website

    Hello, I'm curious as to how I'd go about pulling data directly from a wholesalers website. Normally I make use of my suppliers having an ftp data feed but in this instance they do not. Instead I need to somehow pull the necessary data (sku, upc, cost, inventory) directly from their website.