RESOLVED-Accessing site via a proxy server with PHP file_get_contents

by 12 replies
23
ALL NOW SORTED THANKS KIRKMCD & WAYFARER
Hi Guys,
Looking for an answer to this isse that's had me going around in circles for 2 days and now I'm at a brick wall. There are number of resources that purport to have the solution but they all look like they copied the same script so i am now not convinced it is correct.

Hereis the short test script I have used to test this but I get the following error.
Warning: file_get_contents(The AA: Breakdown Cover, Car Insurance, Home Insurance, Loans, Motoring Advice, Travel and Maps) [function.file-get-contents]: failed to open stream: Connection timed out in /home5/XXXXXXXX/public_html/mkm/tools/test.php on line 11


The script is:
_____________

<?php

function file_get_contents_proxy($url,$proxy){

// Create context stream
$context_array = array('http'=>array('proxy'=>$proxy,'request_fullu ri'=>true));
$context = stream_context_create($context_array);

// Use context stream with file_get_contents
$data = file_get_contents($url,false,$context);

// Return data via proxy
return $data;

}



$url = "http://www.theaa.com";
$proxy = "tcp://132.254.101.52:3128";

file_get_contents_proxy($url,$proxy);
Echo $data

?>
________

$url is an arbitary wesite
$proxy is a randomly chosen public proxy that I have tested both with an online checker and by adding it to my browser config.

If I just use the file_get_contents I can access the file directly but I cannot access it via the proxy.

Anyone familiar with accessing sites/site files via a proxy server?
Really appreciate your help

Will definitely find some kind of "reward" for anyone who gives me the solution.
Or tell me how much

Thanks
Tony
#programming #accessing #filegetcontents #php #proxy #server #site
  • Try setting proxy to
    $proxy = "132.254.101.52:3128";
    instead of
    $proxy = "tcp://132.254.101.52:3128";

    I think I read somewhere people having problems with that.
    It could also be that the function is blocked for external urls.

    With all that being said, I usually use CURL for reading external web pages.
    • [1] reply
    • Many thanks Kirk,

      Unfortunately that has made no difference.
      The main script that I created has been working fine on two differnt servers/hostings before I tried to add the proxy (and still does), So the file_get_content is definetly enabled.

      I haven't used curl as I eventually want to distribute the script and I understand that it isn't always enabled on servers.

      But if I can't resolve this then curl may be a good option. Do have know of any resources that would help me sort this in curl or id there something you feel might be a possible solution?

      many thanks
      Tony
      • [1] reply
  • If it was working on other servers, it just may be the current server does not have the allow_url_fopen option enabled in php.ini ... check that.
    • [1] reply
    • Hi Wayfarer,

      To clarify. I have tried on two servers and both have exactly the same results.
      opening the url with either fopen or file-get_contents works fine.

      It only fails to open when I add in the stream_context_create to try and open it via the proxy server.

      cheers
      Tony
  • There's an alternative solution that you may not have considered, one that I've used to scrape public information with: an HTTP proxy. An HTTP proxy is free to set up, and can be done with Google's AppEngine. I own 5 proxies that I modified for my purposes, which came from a free Python script. This is useful if all you need to do is read content from external websites, since everything is done with plain HTTP requests (file_get_contents or cURL requests, without using the proxy argument).

    The proxy script I used was this: mirrorrr - Mirrorrr - Web Cache - Google Project Hosting ... If you need help, PM me. Or, we could work out a deal where you use my proxies. They're super fast and reliable, since they're on Google's cloud-servers.
    • [1] reply
    • Hi Wayfarer,

      Many thanks for the tip and the offer.

      This script is part of a larger script that will become a distributed product so unfortunately I would need a set of proxies for every copy which would not be viable.

      It looks an interesting idea though and I can see a lot of advantages of having that setup.

      I'll keep the idea in mind as I am sure it will be resolution to something else I have in mind.

      I have to get this project finished first but I may well take you up on your offer of help later.

      thanks
      Tony
      • [1] reply
  • [DELETED]
Join the discussion

Next Topics on Trending Feed