I have a question - I'm using CURL and PROXY. I'm getting list with proxy addresses from paid web service. The list looks like this (example):
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
123.456.789.012:1234
Let's say, that the list mentioned above is available here: http://example.com/list_with_the_proxy
My PHP code looks like this:
<?php
$url = 'http://connect.to.another.example.net/'; //I want to open this url using one of the proxy address from the list
$proxy_url = 'http://address.of.proxy.example.org';
function RandomLine($filename) {
$lines = file($filename) ;
return $lines[array_rand($lines)] ;
}
$random_proxy = RandomLine($proxy_url);
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, $random_proxy);
$data = curl_exec($ch);
curl_close($ch);
echo $data;
?>
I have noticed some problem - sometimes some proxy is unavailable, and the site is loading & loading & loading ... Is there any possibility, to reload the script and get new random proxy, if the site is loading too long ? For example if proxy is not responding in 5 seconds, script will load again ?
Look into CURLOPT_TIMEOUT
, reload in every case when you don't get any data in a reasonable time. Also are you sure that the data from the proxy list site is coming through correctly and its not hanging because of that?
Actually you're just facing a common problem that - regardless of PHP or curl - is not going to be solved automatically. That's for many reasons, but more generally because you can not solve that automatically.
You want to make a connection via some proxy list. But on the other hand, you're not in control of the proxies in that list. So you might get a proxy that is not working.
By the definition what a proxy is, you either need to trust the address or drop it. However if you trust it but your decision is wrong, you can be fooled. Because you make use of automated measures but have not told with your question whether you can trust that proxy list source to be reliable or not, I assume that source is not really reliable. Therefore you run into problems you wish to have easily solved, but actually they are not.
I can not judge if this is really a problem. However what I can say is, that stackoverflow.com probably is not the right site to ask your question.
So this is not really an answer but more a lengthy comment. Answer-wise this is subjective and even not constructive as you have not shared the address of your proxy províder.
Take care. Not all of these proxy lists are safe to use. But more generally do not expect that all of these proxies actually work. Those lists are just subject to change. You get what you pay for might one say or, when you look for a reliable proxy, why don't you run one on your own?