i am learning about CURLOPT_CONNECTTIMEOUT, i am using a proxy in this test with 25 seconds timeout setting, but it never timed out, i dont know why. here is my simple code
$proxy="124.206.54.251:3128";
$timeout = 25;
$url = "http://google.com";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_HTTP);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
$headerSent = curl_getinfo($ch, CURLINFO_HEADER_OUT );
$returned_msg = curl_exec($ch);
$error = curl_error($ch);
$info= curl_getinfo($ch);
curl_close($ch);
echo "<br>".$returned_msg."<br>".$info['total_time']."<br>".$error;
can someone plz tell why it never timed out? i only want to wait 25 second and it is unable to for any reason such as proxy is dead, then it should timed out.
CURLOPT_CONNECTTIMEOUT only sets the timeout for the amount of time it takes to connect. If you want to limit the amount of time the entire request is allowed to take, use CURLOPT_TIMEOUT instead.
Related
I have a curl set up like this:
function file_get_contents_curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 400);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch,CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
$data = curl_exec($ch);
return $data;
}
When I call it with the following:
$html = file_get_contents_curl("https://training.sap.com/service-id/module.php/core/as_login.php?AuthId=sf-sp&ReturnTo=%2Fsuccessfactors-community%2Fpermission-check");
You can see that link has an authentication, and I do not want to spend time on it. I just want that curl session to die after maybe 3 seconds or x seconds, or determine before hand if it is going to be a never ending loop on the curl for one reason such a SSL or password requirement etc.
I've increased the timeout but it doesn't work.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://localhost.com/test.php");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 999);
$response = curl_exec($ch);
print_r($response);
You need to add CURLOPT_CONNECTTIMEOUT option to it.
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 600); //10 minutes
CURLOPT_VERBOSE explains, the system takes time for connecting to PROXY before it actually makes a request to a web link.
$ch = curl_init("$url");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_ENCODING, "gzip");
curl_setopt($ch, CURLOPT_PROXY, "$ip");
curl_setopt($ch, CURLOPT_PROXYPORT, "$po");
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_close($ch);
I want to execute multiple links without closing curl to reduce latency. Is there any way to keep the PROXY live and do multiple requests to links?
When curl uses a proxy, the connection keeps open until the call of curl_close.
Two important points,
First/initial proxy connection always gonna take time (latency)
When the connection is established, don't close the connection, simply adjust parameters according to the requirements.
For an example:
//$ch = curl_init("$url");
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 0);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_ENCODING, "gzip");
curl_setopt($ch, CURLOPT_PROXY, "$ip");
curl_setopt($ch, CURLOPT_PROXYPORT, "$po");
curl_setopt($context, CURLOPT_URL, $url); // Request One
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_setopt($context, CURLOPT_URL, $url); // Request Two
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_setopt($context, CURLOPT_URL, $url); // Request Three
$response = curl_exec($ch);
$errmsg = curl_error($ch);
$cInfo = curl_getinfo($ch);
curl_close($ch); //Closing after all requests
The first request will take some time in connection establishment to the proxy which depends on multiple factors. After the first request, all requests will be faster and will happen based on actual proxy speed(provided server script >= proxy speed).
I think we can't do the persistent connection. If you know how to do that, please share :-)
i have a database with reciprocal urls and check it daily for availability, so i can sort available links with unavailable.
i do it in such way:
<?php
foreach($urls as $url) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible;)');
curl_setopt($ch, CURLOPT_URL,$url );
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_NOBODY, TRUE);
curl_setopt($ch, CURLOPT_FAILONERROR, TRUE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT_MS, 60000);
curl_setopt($ch, CURLOPT_PROXY, '127.0.0.1:107');
$page = curl_exec($ch);
$httpcode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if($httpcode>=100 && $httpcode<600) echo $httpcode; else echo '0';
}
?>
It output the header code. How can i use same function in more sleep way? because in this way to execute this script take about 15 minutes for 100 links. Thank you for any ideas and suggestions. I also tried to use multi_curl, but cant understand how to manage outputs.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy); // $proxy is ip of proxy server
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$httpCode = curl_getinfo($ch , CURLINFO_HTTP_CODE); // this results 0 every time
echo '<br> curl'.$ch; //this line outputs resource id#5
$exec = stripslashes(curl_exec($ch));
echo '<br> exec'.curl_exec($ch); //this results blank
i am confused why $exec does not return anything ,i am new to curl please help, thanks in advance
curl_exec will return false when the request failed. Adjust your function to this :
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy); // $proxy is ip of proxy server
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
$httpCode = curl_getinfo($ch , CURLINFO_HTTP_CODE); // this results 0 every time
$response = curl_exec($ch);
if ($response === false)
$response = curl_error($ch);
echo stripslashes($response);
curl_close($ch);
This way u'll see the curl error
You're trying to access a HTTP response code before you actually make the HTTP call. Reverse the execution as follows:
$response = curl_exec($ch);
$httpCode = curl_getinfo($ch , CURLINFO_HTTP_CODE); // this results 0 every time
Try:
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
Maybe the response is longer than 10.
I had the same issue, I solved like this.
The result return 0 mean that you can not connect to the server so please recheck your proxy and increase the timeout :)