How to capture curl_errno in curl_multi mode? - php

For some reason curl_errno($value) always returns 0 instead of 6 when I try url like stkovrflow.com. This is a non existing domain. So curl supposed to return 6. But i'm getting 0.
Can someone tell me what's wrong with my code?
This is how I check curl error
if (curl_errno($value) !== 0)
{
$handles[$key]['error_code'] = curl_errno($value);
}
Here is my full code
<?php
protected function curl($url)
{
$mh = curl_multi_init();
$handles = array();
foreach ($url as $link)
{
$handles[$link] = curl_init($link);
curl_setopt($handles[$link], CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($handles[$link], CURLOPT_HEADER, true);
curl_setopt($handles[$link], CURLOPT_RETURNTRANSFER, true);
curl_setopt($handles[$link],CURLOPT_FAILONERROR,true);
curl_setopt($handles[$link], CURLOPT_FOLLOWLOCATION, $this->curlFollowLocation);
curl_setopt($handles[$link], CURLOPT_MAXREDIRS, $this->curlMaxRedirects);
curl_setopt($handles[$link], CURLOPT_TIMEOUT, $this->curlTimeout);
curl_setopt($handles[$link], CURLOPT_USERAGENT, $this->curlUserAgent);
curl_setopt($handles[$link], CURLOPT_AUTOREFERER, true);
curl_multi_add_handle($mh, $handles[$link]);
}
$running = null;
do {
curl_multi_exec($mh, $running);
usleep(200000);
} while ($running > 0);
foreach ($handles as $key => $value)
{
$handles[$key] = false;
$handles[$key]['error_code'] = false;
if (curl_errno($value) !== 0)
{
$handles[$key]['error_code'] = curl_errno($value);
} else {
$response = curl_multi_getcontent($value);
$httpCode = curl_getinfo($value, CURLINFO_HTTP_CODE);
if ( $httpCode != 200 ) {
$handles[$key]['error_code'] = $httpCode;
} else {
$handles[$key]['html'] = $response;
}
}
curl_multi_remove_handle($mh, $value);
curl_close($value);
}
curl_multi_close($mh);
return $handles;
}
Update:
Looks like curl_errno does not work (see bug report) in curl multi mode. Instead we should use curl_multi_info_read. When I use curl_multi_info_read like this
$e_code = curl_multi_info_read($mh);
var_dump($e_code);
this is my var_dump output.
array (size=3)
'msg' => int 1
'result' => int 6
'handle' => resource(7, curl)
As you can it returns 6 correctly. But php doc says
The data the returned resource points to will not survive calling curl_multi_remove_handle().
Unfortunately my script depends on curl_multi_remove_handle(). Any solution?
=======

Related

How to turn Curl response into Json so I can use it on the front-end?

I have a function which uses curl_multi to make several GET requests. However, I can't figure out how to turn the response to JSON so I could access it on the front end. Usually I use json_decode(), however, it is not working this time. Maybe it's because I'm trying to decode a whole array of strings instead of a singular string?. Currently the response looks like this on the front-end:
And here's my function:
public function getVisited($username){
$user = User::where('username', $username)->first();
$visitedPlaces = $user->visitedPlaces;
$finalPlaces = [];
foreach ($visitedPlaces as $visitedPlace) {
$url = "https://maps.googleapis.com/maps/api/place/details/json?placeid=" . $visitedPlace->place_id . "&key=AIzaSyDQ64lYvtaYYYNWxLzkppdN-n0LulMOf4Y";
array_push($finalPlaces, $url);
}
$result = array();
$curly = array();
$mh = curl_multi_init();
foreach ($finalPlaces as $id => $d) {
$curly[$id] = curl_init();
$url = $d;
curl_setopt($curly[$id], CURLOPT_URL, $url);
curl_setopt($curly[$id], CURLOPT_HEADER, 0);
curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curly[$id], CURLOPT_SSL_VERIFYPEER, 0); // Skip SSL Verification
curl_multi_add_handle($mh, $curly[$id]);
}
$running = null;
do {
curl_multi_exec($mh, $running);
} while($running > 0);
foreach($curly as $id => $c) {
$result[$id] = curl_multi_getcontent($c);
curl_multi_remove_handle($mh, $c);
}
curl_multi_close($mh);
return response()->json([
'sights' => $result
], 201);
}
If you are expecting a json string from the curl_multi_getcontent($c) then you should decode it:
$result[$id] = json_decode(curl_multi_getcontent($c));
Then it should be encoded in the response appropriately.

Checking Multi URL status

I want a lot of domain address for checking the status.
I try multi curl but its to slow
class BotCronJobs extends Controller {
public function __construct() {
}
public function index() {
$Query = servers::all();
$urls = [];
foreach ($Query as $item){
$urls[$item->id] = $item->serverUrl;
}
var_dump($this->test($urls));
}
public function test($urls = []) {
$status = [];
$mh = curl_multi_init();
foreach($urls as $key => $value){
$ch[$key] = curl_init($value);
curl_setopt($ch[$key], CURLOPT_NOBODY, true);
curl_setopt($ch[$key], CURLOPT_HEADER, true);
curl_setopt($ch[$key], CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch[$key], CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch[$key], CURLOPT_SSL_VERIFYHOST, false);
curl_multi_add_handle($mh,$ch[$key]);
}
do {
curl_multi_exec($mh, $running);
curl_multi_select($mh);
} while ($running > 0);
foreach(array_keys($ch) as $key){
$status[$key][] = curl_getinfo($ch[$key], CURLINFO_HTTP_CODE);
$status[$key][] = curl_getinfo($ch[$key], CURLINFO_EFFECTIVE_URL);
curl_multi_remove_handle($mh, $ch[$key]);
}
curl_multi_close($mh);
return $status;
}
}
I just need to check is server online or not and every server have an id it's important for me to understand which server is offline.
is there any faster way?
your function looks nearly optimal to me. i guess its slow because some domains are not responding, and the default CURLOPT_CONNECTTIMEOUT is too high. try setting CURLOPT_CONNECTTIMEOUT to 1 and CURLOPT_TIMEOUT to 2, that should make it stall for max 2 seconds on not-responding domains.
also, if you don't actually need the http response code, but is ok with just checking if the server is actually accepting connections or not, maybe using the socket_ api would be faster.

curl_multi_exec Maximum execution time of 30 seconds exceeded

I have the following function, running an array of URLs with cURL:
function multiRequest($urls){
$res = array();
// Create get requests for each URL
$mh = curl_multi_init();
foreach($urls as $i => $url)
{
$ch[$i] = curl_init($url);
curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch[$i], CURLOPT_HEADER, "Accept: application/json");
curl_setopt($ch[$i], CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
curl_setopt ($ch[$i], CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch[$i], CURLOPT_SSL_VERIFYPEER, 0);
curl_multi_add_handle($mh, $ch[$i]);
}
// Start performing the request
do {
$execReturnValue = curl_multi_exec($mh, $runningHandles);
} while ($execReturnValue == CURLM_CALL_MULTI_PERFORM);
// Loop and continue processing the request
while ($runningHandles && $execReturnValue == CURLM_OK) {
// Wait forever for network
$numberReady = curl_multi_select($mh);
if ($numberReady != -1) {
// Pull in any new data, or at least handle timeouts
do {
$execReturnValue = curl_multi_exec($mh, $runningHandles);
} while ($execReturnValue == CURLM_CALL_MULTI_PERFORM);
}
}
// Check for any errors
if ($execReturnValue != CURLM_OK) {
trigger_error("Curl multi read error $execReturnValue\n", E_USER_WARNING);
}
// Extract the content
foreach($urls as $i => $url)
{
// Check for errors
$curlError = curl_error($ch[$i]);
if($curlError == "") {
$res[$i] = curl_multi_getcontent($ch[$i]);
} else {
print "Curl error on handle $i: $curlError\n";
}
// Remove and close the handle
curl_multi_remove_handle($mh, $ch[$i]);
curl_close($ch[$i]);
}
// Clean up the curl_multi handle
curl_multi_close($mh);
// Print the response data
return $res;
}
I get this error: Maximum execution time of 30 seconds exceeded, the line it points to is $numberReady = curl_multi_select($mh);
The code used to work on PHP version 5.3, after updating to 5.5 the code is no longer working.
I tried setting the limit to more than 30 seconds, it didn't help, also tried ini_set('max_execution_time', 0);
Is there anything wrong with my code? Is there another way to debug it?
This alternative function solved it for me:
function multiRequest($nodes){
//function multiple_threads_request($nodes){
$mh = curl_multi_init();
$curl_array = array();
foreach($nodes as $i => $url) {
$curl_array[$i] = curl_init($url);
curl_setopt($curl_array[$i], CURLOPT_RETURNTRANSFER, true);
curl_setopt($curl_array[$i], CURLOPT_HEADER, "Accept: application/json");
curl_setopt($curl_array[$i], CURLOPT_USERPWD, xxxx);
curl_multi_add_handle($mh, $curl_array[$i]);
}
$running = NULL;
do {
usleep(10000);
curl_multi_exec($mh,$running);
} while($running > 0);
$res = array();
foreach($nodes as $i => $url) {
$res[$url] = curl_multi_getcontent($curl_array[$i]);
}
foreach($nodes as $i => $url) {
curl_multi_remove_handle($mh, $curl_array[$i]);
}
curl_multi_close($mh);
return $res;
}

Combining curl multi and strpos() to find text on webpage

EDITED
I am attempting to use curl multi to check the response from a website and additionally check each curl response for a portion of text. I have grouped the data into an array but I cannot figure out if I am using the correct/most efficient method to run the strpos() function using the 'post' text.
$data = array(array());
$data[0]['url'] = 'http://www.google.com';
$data[0]['post'] = 'google text';
$data[1]['url'] = 'http://www.yahoo.com';
$data[1]['post'] = 'yahoo text';
$r = multiRequest($data);
echo '<pre>';
print_r($r);
Here is my function:
function multiRequest($data, $options = array()) {
// array of curl handles
$curly = array();
// data to be returned
$result = array();
// multi handle
$mh = curl_multi_init();
// loop through $data and create curl handles
// then add them to the multi-handle
foreach ($data as $id => $d) {
$curly[$id] = curl_init();
$url = (is_array($d) && !empty($d['url'])) ? $d['url'] : $d;
curl_setopt($curly[$id], CURLOPT_URL, $url);
curl_setopt($curly[$id], CURLOPT_HEADER, 0);
curl_setopt($curly[$id], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curly[$id], CURLOPT_USERAGENT, 'Chrome: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/22.0.1216.0 Safari/537.2');
// extra options?
if (!empty($options)) {
curl_setopt_array($curly[$id], $options);
}
curl_multi_add_handle($mh, $curly[$id]);
}
// execute the handles
$running = null;
do {
curl_multi_exec($mh, $running);
} while($running > 0);
// get content and remove handles
foreach($curly as $id => $c) {
$result[$id][] = curl_getinfo($c, CURLINFO_EFFECTIVE_URL);
$result[$id][] = curl_getinfo($c, CURLINFO_HTTP_CODE);
$result[$id][] = curl_getinfo($c, CURLINFO_CONTENT_TYPE);
$url = curl_getinfo($c, CURLINFO_EFFECTIVE_URL);
// loop data again
foreach ($data as $id => $d){
if($url==$d['url']){ // only check current url data
$text = curl_exec($c);
$result[$id][] = strpos($text, $d['post']);
}
}
curl_multi_remove_handle($mh, $c);
}
// all done
curl_multi_close($mh);
return $result;
}
Can anyone advise on whether my solution is appropriate? Is there a more efficient/better way to perform my strpos() check?
Thanks
Use an array so you can relate all the urls, strings, and curl handles together:
$stuff = array(
0 => array('url' => 'google', 'text' => 'googletext', 'curl' => null)
1 => array('url' => 'yahoo', 'text' => 'yahootext', 'curl' => null)
etc..
);
foreach($stuff as $key => $info) {
$stuff[$key]['curl'] = curl_init($stuff[$key]['url']);
curl_multi_add_handle($mh, $stuff[$key]['curl']);
}
Then do a similar loop when you're processing the results.

PHP curl_multi_exec runs once

I'm having trouble creating multiple xml requests using php's curl_multi_exec.
The problem is that the do...while loop containing the curl_multi_exec command runs only once and then quits.
Resources Used:
http://www.phpied.com/simultaneuos-http-requests-in-php-with-curl/
http://php.net/manual/en/function.curl-multi-exec.php/
http://www.rustyrazorblade.com/2008/02/curl_multi_exec/
Take a look at my code:
//Multi handle curl initialization
$mh = curl_multi_init();
//set url
$url = 'my_url';
foreach($latLng as $id => $l) {
$ch[$id] = curl_init();
//$request previously set
//Initialize and set options
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $request);
//add to multi_handle
curl_multi_add_handle($mh, $ch[$id]);
}
//Execute the handles
$running = null;
do {
$mrc = curl_multi_exec($mh, $running);
$ready=curl_multi_select($mh);
echo "Ran once\n";
} while ($mrc == CURLM_CALL_MULTI_PERFORM && $ready > 0);
while ($active && $mrc == CURLM_OK) {
if ($curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $running);
echo "Ran again\n";
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
foreach ($mh as $c) {
// HTTP response code
$code = curl_getinfo($c, CURLINFO_HTTP_CODE);
// cURL error number
$curl_errno = curl_errno($c);
// cURL error message
$curl_error = curl_error($c);
// output if there was an error
if ($curl_error) {
echo("*** cURL error: ($curl_errno) $curl_error\n");
}
}
//get content and remove handles
foreach ($ch as $c) {
$result[] = curl_multi_getcontent($c);
curl_multi_remove_handle($mh, $c);
}
print_r($result);
//Close curl
curl_multi_close($mh);
}
I know the request is valid because I receive the correct return data when I perform a single curl execution. The problem lies with the curl_multi_exec().
The output I am receiving is "Ran once" followed by the empty arrays of the curl_multi_getcontent() calls. See below:
Ran once
Array
(
[0] =>
[1] =>
[2] =>
[3] =>
[4] =>
[5] =>
[6] =>
[7] =>
[8] =>
)
Any help is greatly appreciated.
You're not setting up the curl options correctly:
Currently, you're setting options on $ch which is your array, you need to be setting the options specifically on the current curl handler, which in your loop is $ch[$id]:
//Initialize and set options
curl_setopt($ch[$id], CURLOPT_URL, $url);
curl_setopt($ch[$id], CURLOPT_HEADER, 0);
curl_setopt($ch[$id], CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch[$id], CURLOPT_POST, 1);
curl_setopt($ch[$id], CURLOPT_POSTFIELDS, $request);
change this:
foreach ($mh as $c) {
$code = curl_getinfo($c, CURLINFO_HTTP_CODE);
to:
for($i=1;$i<=count($array);$i++){
$code = curl_multi_getcontent($ch[$i]);
assuming $array is the array for your multiple $url.

Categories