I'm trying to run this simple piece of code from php.net under my Mac OS X (Mavericks) to try cURL multi exec feature :
<?php
// create both cURL resources
$ch1 = curl_init();
$ch2 = curl_init();
// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://lxr.php.net/");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/");
curl_setopt($ch2, CURLOPT_HEADER, 0);
//create the multiple cURL handle
$mh = curl_multi_init();
//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$active = null;
//execute the handles
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
//close the handles
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);
?>
That does not work as the only output I got is :
PHP Fatal error: Maximum execution time of 30 seconds exceeded in
This is my environment: Mac OS X 10.9, PHP 5.4.17, Apache 2.2.24.
cUrl is installed as I my regular curl "single" requests work great.
I think this is an issue with Mac OS but I can't find any fix. Do you have any idea?
EDIT: I tried the same code on a Linux server and everything worked fine.
On php 5.3.18+ be aware that curl_multi_select() may return -1 forever until you call curl_multi_exec().
Try this:
while ($this->active && $mrc == CURLM_OK)
{
// add this line
while (curl_multi_exec($this->mh, $this->active) === CURLM_CALL_MULTI_PERFORM);
if (curl_multi_select($this->mh) != -1)
{
do {
$mrc = curl_multi_exec($this->mh, $this->active);
if ($mrc == CURLM_OK)
{
while($info = curl_multi_info_read($this->mh))
{
$this->process($info);
}
}
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
See https://bugs.php.net/bug.php?id=63411 or http://marchtea.com/?p=109 for more information.
Related
How can I stop my function from reciving responses when I send cURL requests? I developed this function based on http://php.net/manual/en/function.curl-multi-exec.php documentation. However when I try send over 50 messages the app I am developing freezes at the send button for over 20 seconds. Is there a way to send multiple cURL and force the code to not wait for a response?
function send_multi_data($url,$data){
//create the multiple cURL handle
$mh = curl_multi_init();
$active = null;
$count=0;
// set URL and other appropriate options
foreach($data as $message){
$count++;
$ch[$count] = curl_init();
curl_setopt($ch[$count], CURLOPT_URL, $url);
curl_setopt($ch[$count], CURLOPT_HEADER, 0);
curl_setopt ($ch[$count], CURLOPT_POSTFIELDS, $message);
curl_multi_add_handle($mh,$ch[$count]);
}
//execute the handles
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
// Close the handle
$count1=0;
foreach($data as $message){
$count1++;
curl_multi_remove_handle($mh, $ch[$count1]);
}
curl_multi_close($mh);
}
I want to user curl for login to other websites. I use curl multi threading for it.
code 1:
<?php
$ch1 = curl_init();
$ch2 = curl_init();
curl_setopt($ch1, CURLOPT_URL, $url);
curl_setopt($ch2, CURLOPT_URL, $url);
curl_setopt($ch1, CURLOPT_COOKIE, $post);
curl_setopt($ch2, CURLOPT_COOKIE, $post2)
$mh = curl_multi_init();
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$active = null;
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
?>
code 2 :
<?php
$mh = curl_multi_init();
foreach($data as $key => $value)
{
$key = curl_init();
curl_setopt($key, CURLOPT_URL, $value["url"]);
curl_setopt($key, CURLOPT_COOKIE, $value[$key]);
curl_multi_add_handle($mh,$key);
}
$active = null;
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
?>
Code 1 can return me result that i want but not very dynamic. Code 2 just return me ‹‹. Anyone can explain to me why like this?
Could you provide an example of $data you are using in the second example. The first loop there makes no sense for me:
foreach($data as $key => $value) //assuming $key here is a scalar
{
$key = curl_init(); // after this line $key is a resource or false
curl_setopt($key, CURLOPT_URL, $value["url"]);
curl_setopt($key, CURLOPT_COOKIE, $value[$key]); // likely $value[$key] is empty, and there is a notice in the log
curl_multi_add_handle($mh,$key);
}
Apart from that, you probably want to close handlers after the final loop, and output some results. I don't see at which point Code 2 just return anything.
I created a php script to run some google queries in order to get acquainted with the concept of multiple parallel requests in curl. As a basis, I used example #1 on this page: http://php.net/manual/en/function.curl-multi-exec.php
I found that curl_multi_select in the provided example always returns -1. The documentation states that this indicates some error happened (by the underlying system call), but there seems to be no way to deduce what went wrong.
Code
$queries = array("Mr.", "must", "my", "name", "never", "new", "next", "no", "not", "now");
$mh = curl_multi_init();
$handles = array();
foreach($queries as $q)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://www.google.nl/#q=" . $q);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_TIMEOUT, 30);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_multi_add_handle($mh,$ch);
$handles[] = $ch;
}
echo "created\n";
$active = null;
$mrc = curl_multi_exec($mh, $active);
if ($mrc != CURLM_OK)
throw new Exception("curl_multi_exec failed");
while ($active && $mrc == CURLM_OK) {
$select = curl_multi_select($mh);
if ($select != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
} else throw new Exception("curl_multi_select failed (it returned -1)");
}
// removed cleanup code for briefety.
Question
How can I find out why curl_multi_select returns -1 or why does it return -1? Do I need some special configuration of php to allow for this multithreading?
As a comment says at http://php.net/manual/en/function.curl-multi-init.php#115055, there's a problem on the official document.
I don't know why and don't have enough knowledge on libcurl, but I understand that curl_multi_select($mh) always have a chance to return -1;
So, this snippets (from the above url) works for me.
<?php
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) == -1) {
usleep(100);
}
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
?>
I am getting only the result for the first script. But I want the response for both PHP scripts to be in one output. In my program I am developing, I am querying several remote servers. I have to search information on different servers and return the result in one response.
<?php
// build the individual requests as above, but do not execute them
$ch_1 = curl_init('http://lifesaver.netai.net/example/pharm.php');
$ch_2 = curl_init('http://192.168.1.2/example/pharm.php');
curl_setopt($ch_1, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch_2, CURLOPT_RETURNTRANSFER, false);
curl_setopt($ch_1, CURLOPT_POST,1);
curl_setopt($ch_2, CURLOPT_POST,1);
curl_setopt($ch_1, CURLOPT_POSTFIELDS, "name=$value");
curl_setopt($ch_2, CURLOPT_POSTFIELDS, "name=$value");
// $_POST["name"]
// build the multi-curl handle, adding both $ch
$mh = curl_multi_init();
curl_multi_add_handle($mh, $ch_1);
curl_multi_add_handle($mh, $ch_2);
// execute all queries simultaneously, and continue when all are complete
$running = null;
do {
curl_multi_exec($mh, $running);
} while ($running);
// all of our requests are done, we can now access the results
$response_1 = curl_multi_getcontent($ch_1);
$response_2 = curl_multi_getcontent($ch_2);
echo $response_1;
echo $response_2;
?>
According to the documentation of curl_multi_getcontent, you need to set CURLOPT_RETURNTRANSFER to true instead of false.
Also, the documentation of curl_multi_init shows an example which is slightly different than yours when it comes to the execution loop:
//execute the handles
$running = null;
do {
$mrc = curl_multi_exec($mh, $running);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($running && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $running);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
You could try it that way.
I found this script on php.net and lets say I wanted to get only the info from part of the page. How would one go about doing this, I know how to do it with curl_init, but the multi seems much more efficent.
For example:
from php.net
<?php
// create both cURL resources
$ch1 = curl_init();
$ch2 = curl_init();
// set URL and other appropriate options
curl_setopt($ch1, CURLOPT_URL, "http://lxr.php.net/");
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/");
curl_setopt($ch2, CURLOPT_HEADER, 0);
//create the multiple cURL handle
$mh = curl_multi_init();
//add the two handles
curl_multi_add_handle($mh,$ch1);
curl_multi_add_handle($mh,$ch2);
$active = null;
//execute the handles
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
while ($active && $mrc == CURLM_OK) {
if (curl_multi_select($mh) != -1) {
do {
$mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);
}
}
//close the handles
curl_multi_remove_handle($mh, $ch1);
curl_multi_remove_handle($mh, $ch2);
curl_multi_close($mh);
?>
Id like to just get the info below from the request:
<b>Key enhancements in PHP 5.3.3 include:</b>
</p>
<ul>
<li>Upgraded bundled sqlite to version 3.6.23.1.</li>
<li>Upgraded bundled PCRE to version 8.02.</li>
<li>Added FastCGI Process Manager (FPM) SAPI.</li>
<li>Added stream filter support to mcrypt extension.</li>
<li>Added full_special_chars filter to ext/filter.</li>
<li>Fixed a possible crash because of recursive GC invocation.</li>
<li>Fixed bug #52238 (Crash when an Exception occured in iterator_to_array).</li>
<li>Fixed bug #52041 (Memory leak when writing on uninitialized variable returned from function).</li>
<li>Fixed bug #52060 (Memory leak when passing a closure to method_exists()).</li>
<li>Fixed bug #52001 (Memory allocation problems after using variable variables).</li>
<li>Fixed bug #51723 (Content-length header is limited to 32bit integer with Apache2 on Windows).</li>
<li>Fixed bug #48930 (__COMPILER_HALT_OFFSET__ incorrect in PHP >= 5.3).</li>
</ul>