I am very new to using Guzzle, and my php feet are still quite soft. Nevertheless, I found Guzzle to be quite forgiving. I followed the following example, tweaking here and there:
use GuzzleHttp\Pool;
use GuzzleHttp\Client;
use GuzzleHttp\Psr7\Request;
require __DIR__ . '/vendor/autoload.php';
$client = new GuzzleHttp\Client();
$requests = function ()
{
...[generate url]..
yield $index => new Request('GET', $theURL);
};
$pool = new Pool($client, $requests(), [
'concurrency' => 50,
'fulfilled' => function ($response, $index)
{
$content = $response->getBody()
->getContents();
file_put_contents('storage/' . $index, $content);
print 'fulfilled index:' . $index . PHP_EOL;
},
'rejected' => function ($reason, $index)
{
print 'rejected index:' . $index . PHP_EOL;
},
]);
$promise = $pool->promise();
$promise->wait();
This works well, but there is a case in which I need to know the URL. It's failing, but I cannot tell /which/ of the 1000 URLs I am feeding it is failing. I can see the 'index', but I cannot understand how to use the index to tell me something meaningful (specifically, the URL).
var_dump( $response ); shows me the response but not the URL.
var_dump( $index ); shows me an index, in this case int(1100), but I've no clue how to use that to my benefit.
Any help? Thanks!
Related
Guzzle provides a mechanism to send concurrent requests: Pool. I used the example from the docs: http://docs.guzzlephp.org/en/stable/quickstart.html#concurrent-requests. It works quite fine, sends concurrent requests and everything is awesome except one thing: it seems Guzzle ignores HTTP/2 in this case.
I've prepared a simplified script that sends two requests to https://stackoverflow.com, the first one is using Pool, the second one is just a regular Guzzle request. Only the regular request connects via HTTP/2.
<?php
include_once 'vendor/autoload.php';
use GuzzleHttp\Client;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;
$client = new Client([
'version' => 2.0,
'debug' => true
]);
/************************/
$requests = function () {
yield new Request('GET', 'https://stackoverflow.com');
};
$pool = new Pool($client, $requests());
$promise = $pool->promise();
$promise->wait();
/************************/
$client->get('https://stackoverflow.com', [
'version' => 2.0,
'debug' => true,
]);
Here is an output: https://pastebin.com/k0HaDWt6 (I highlighted important parts with "!!!!!")
Does anybody know why Guzzle does this and how to make Pool work with HTTP/2?
Found what was wrong: new Client() doesn't actually accept 'version' as an option if passed to Pool requests are created as new Request(). Either the protocol version must be provided as an option of every request or the requests must be created as $client->getAsync() (or ->postAsync or whatever).
See the corrected code:
...
$client = new Client([
'debug' => true
]);
$requests = function () {
yield new Request('GET', 'https://stackoverflow.com', [], null, '2.0');
};
/* OR
$client = new Client([
'version' => 2.0,
'debug' => true
]);
$requests = function () use ($client) {
yield function () use ($client) {
return $client->getAsync('https://stackoverflow.com');
};
};
*/
$pool = new Pool($client, $requests());
$promise = $pool->promise();
$promise->wait();
...
I'm trying to use guzzle 6 which works fine but I'm lost when it comes to how to log all the api calls. I would like to simply log timing, logged in user from session, url and any other usual pertinent info that has to do with the API call. I can't seem to find any documentation for Guzzle 6 that refers to this, only guzzle 3 (Where they've changed the logging addSubscriber call). This is how my current API calls are:
$client = new GuzzleHttp\Client(['defaults' => ['verify' => false]]);
$res = $client->get($this->url . '/api/details', ['form_params' => ['file' => $file_id]]);
You can use any logger which implements PSR-3 interface with Guzzle 6
I used Monolog as logger and builtin middleware of Guzzle with MessageFormatter in below example.
use GuzzleHttp\HandlerStack;
use GuzzleHttp\Middleware;
use GuzzleHttp\MessageFormatter;
use Monolog\Logger;
$stack = HandlerStack::create();
$stack->push(
Middleware::log(
new Logger('Logger'),
new MessageFormatter('{req_body} - {res_body}')
)
);
$client = new \GuzzleHttp\Client(
[
'base_uri' => 'http://httpbin.org',
'handler' => $stack,
]
);
echo (string) $client->get('ip')->getBody();
The details about the log middleware and message formatter has not well documented yet. But you can check the list which variables you can use in MessageFormatter
Also there is a guzzle-logmiddleware which allows you to customize formatter etc.
#KingKongFrog This is the way to specify the name of the log file
$logger = new Logger('MyLog');
$logger->pushHandler(new StreamHandler(__DIR__ . '/test.log'), Logger::DEBUG);
$stack->push(Middleware::log(
$logger,
new MessageFormatter('{req_body} - {res_body}')
));
For Guzzle 7 I did this::
require './guzzle_7.2.0.0/vendor/autoload.php';
require './monolog/vendor/autoload.php';
use GuzzleHttp\Client;
use GuzzleHttp\Exception\RequestException;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\HandlerStack;
use GuzzleHttp\Middleware;
use GuzzleHttp\MessageFormatter;
use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use GuzzleHttp\TransferStats;
//$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
$logger = null;
$messageFormat =
//['REQUEST: ', 'METHOD: {method}', 'URL: {uri}', 'HTTP/{version}', 'HEADERS: {req_headers}', 'Payload: {req_body}', 'RESPONSE: ', 'STATUS: {code}', 'BODY: {res_body}'];
'REQUEST: urldecode(req_body)';
$handlerStack = \GuzzleHttp\HandlerStack::create();
$handlerStack->push(createGuzzleLoggingMiddleware($messageFormat));
function getLogger() {
global $logger;
if ($logger==null) {
$logger = new Logger('api-consumer');
$logger->pushHandler(new \Monolog\Handler\RotatingFileHandler('./TataAigHealthErrorMiddlewarelog.txt'));
}
var_dump($logger);
return $logger;
}
function createGuzzleLoggingMiddleware(string $messageFormat){
return \GuzzleHttp\Middleware::log(getLogger(), new \GuzzleHttp\MessageFormatter($messageFormat));
}
function createLoggingHandlerStack(array $messageFormats){
global $logger;
$stack = \GuzzleHttp\HandlerStack::create();
var_dump($logger);
collect($messageFormats)->each(function ($messageFormat) use ($stack) {
// We'll use unshift instead of push, to add the middleware to the bottom of the stack, not the top
$stack->unshift(createGuzzleLoggingMiddleware($messageFormat) );
});
return $stack;
}
//$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
$client = new Client(['verify' => false, 'handler' => $tapMiddleware($handlerStack)]);
WOW !!
unshift() is indeed better than push() in reverse order ...
$handlers = HandlerStack::create();
$logger = new Logger('Logger');
$templates = [
'{code} >> {req_headers}',
'{code} >> {req_body}',
'{code} << {res_headers}',
'{code} << {res_body}'
];
foreach ($templates as $template) {
$handlers->unshift($this->getMiddleware($logger, $template));
}
$client = new Client([
RequestOptions::DEBUG => false,
'handler' => $handlers
]);
Using this function to obtain the Middleware:
private function getMiddleware(Logger $logger, string $template): callable {
return Middleware::log($logger, new MessageFormatter($template));
}
Logger comes from "monolog/monolog": "^1.27.1".
And these are all supported variable substitutions.
I'm searching to retrieve the request total time in Guzzle 6, just after a simple GET request :
$client = new GuzzleHttp\Client();
$response = client->get('http://www.google.com/');
But can't find anything in the docs about that. Any idea ?
Thanks a lot.
In Guzzle 6.1.0 You can use the 'on_stats' request option to get transfer time etc.
More information can be found at Request Options - on_stats
https://github.com/guzzle/guzzle/releases/tag/6.1.0
You can use setter and getter.
private $totaltime = 0;
public function getTotaltime(){
return $this->totaltime;
}
public function setTotaltime($time){
$this->totaltime = $time;
}
$reqtime= new self();
$response = $client->post($endpointLogin, [
'json' => $payload,
'headers' => $this->header,
'on_stats' => function (TransferStats $stats) use ($reqtime) {
$stats->getTransferTime();
//** set it here **//
$reqtime->setTotaltime($stats->getTransferTime());
}
]);
dd($reqtime->getTotaltime());
An specific example based on the #Michael post.
$client = new GuzzleHttp\Client();
$response = $client->get('http://www.google.com/', [
'on_stats' => function (\GuzzleHttp\TransferStats $stats) {
echo $stats->getEffectiveUri() . ' : ' . $stats->getTransferTime();
}
]);
$client = new GuzzleHttp\Client();
$one = microtime(1);
$response = $client->get('http://www.google.com/');
$two = microtime(1);
echo 'Total Request time: '. ( $two - $one );
I had a similar problem although it's still Guzzle 5.3.
See Guzzle 5.3 - Get request duration for asynchronous requests
Maybe listening to an event in Guzzle6 and retrieving the TransferInfo will do the trick for you too.
This works for synchronous and asynchronous requests alike.
I need to send multiple requests so I want to implement a batch request.
How can we do it in Guzzle6?
Using the the old way:
$client->send(array(
$client->get($courses), //api url
$client->get($job_categories), //api url
));
is giving me the error:
GuzzleHttp\Client::send() must implement interface Psr\Http\Message\RequestInterface, array given
try something like this
$client = new Client();
foreach ($links as $link) {
$requests[] = new Request('GET', $link);
}
$responses = Pool::batch($client, $requests, array(
'concurrency' => 15,
));
foreach ($responses as $response) {
//do something
}
don't forget
use GuzzleHttp\Pool;
use GuzzleHttp\Client;
use GuzzleHttp\Psr7\Request;
I'm using Guzzle (http://guzzlephp.org) to GET a large number of urls (~300k) . The urls are retrieved from an Elastic Search instance, and I would like to keep adding urls to a Pool so the Pool stays rather small instead of adding them all at once.
Is this possible? I looked at the Pool.php, but did not find a way to do this. Is there a way?
Use while and generator (yield).
$client = new GuzzleHttp\Client();
$client = new Client();
$requests = function () {
$uris = ['http://base_url'];
$visited_uris = []; // maybe database instead of array
while(len($uris)>0)
yield new Request('GET', array_pop($uris));
}
};
$pool = new Pool($client, $requests(), [
'concurrency' => 5,
'fulfilled' => function ($response, $index) {
$new_uri = get_new_uri(); // implement function to get new $uri
if(in_array($new_uri, $visited_uris)) {
array_push($uris, $uri);
}
array_push($visited_uris, $uri);
}
]);
$promise = $pool->promise();
$promise->wait();