How to access Guzzle's fullfilled response outside its scope? - php

I am new to guzzle package and I am sending async post requests through it using pool-promise method. Everything is well and good but once the request is fulfilled and response is received, I am trying to store some part of json response in an array $arr.
$client = new Client();
$arr = [];
$requests = function ($total) use ($client) {
$request_headers = [
'api_key' => config('app.wallet_server_api_key'),
'Content-Type' => 'application/x-www-form-urlencoded'
];
$form_params = [
'accounts' => 0,
'totalaccounts' => 100,
];
$uri = 'MY_REQUEST_URL_CAN_NOT_DISCLOSE';
for ($i = 0; $i < $total; $i++) {
// yield new Request('POST', $uri, $request_headers, http_build_query($form_params, null, '&'));
yield function () use ($client, $uri, $request_headers, $form_params) {
return $client->postAsync($uri, [
'headers' => $request_headers,
'form_params' => $form_params
]);
};
}
};
$pool = new Pool($client, $requests(2), [
'concurrency' => 5,
'fulfilled' => function (Response $response, $index) use ($arr) {
// Response is logged successfully
Log::info(json_decode($response->getBody()->getContents(), true)['message']);
// I am pushing the message key from json response received but it is not taking
$arr[] = json_decode((string)$response->getBody()->getContents(), true)['message'];
},
'rejected' => function (RequestException $reason, $index) {
// this is delivered each failed request
Log::warning(json_encode($reason->getMessage()));
},
]);
// Initiate the transfers and create a promise
$promise = $pool->promise();
// Force the pool of requests to complete.
$promise->wait();
dd($arr); // Displays as null
Please help me understand its working.
EDIT: using postman, I am getting response in json format like below:
{"status":true,"message":"Mnemonics fetched successfully",....SOME OTHER KEYS ETC }

Answering my own question as I found the solution.
If you want to set variable in anonymous function, use &$VariableName as in my case it is:
'fulfilled' => function (Response $response, $index) use (&$arr) { // <-- see the & sign here
// Response is logged successfully
Log::info(json_decode($response->getBody()->getContents(), true)['message']);
// I am pushing the message key from json response received but it is not taking
$arr[] = json_decode((string)$response->getBody()->getContents(), true)['message'];
},
I hope someone like me will fond this answer useful.

Related

How to know the requested url in guzzle?

I try load many urls async but can't think of anything how can I know proccess url..Code below:
$requests = function ($total, $urls) {
for ($i = 0; $i < $total; $i++) {
yield new Request('GET', $urls[$i]);
}
};
$pool = new Pool($client, $requests(50, $urls), [
'concurrency' => 5,
'fulfilled' => function (Response $response, $index) {
// How i can this know what url was proccessed?
},
'rejected' => function (RequestException $reason, $index) {
// And here too..
},
]);
$promise = $pool->promise();
$promise->wait();
Please help, any ideas..

How to get requested url for a guzzle async request?

I'm using guzzle in a loop to get an array of promises. This string in the loop:
$rpomises[] = $this->client->getAsync($url, $options);
Next, I make:
$res = collect(Promise\settle($promises)->wait());
One of the items from the result is:
As you can see it's just an array with string field and GuzzleHttp\Rsr7\Response object. So how I can get requested URL from this construction?
Thank you for any help!
I ran into the same issue.
My uses:
use GuzzleHttp\Client;
use GuzzleHttp\Promise\EachPromise;
use GuzzleHttp\Psr7\Response;
use Requests;
Extracted function from class:
public function getUrlsInParallelRememberingSource($urls,
$numberThreads = 10)
{
$client = new Client();
$responsesModified = [];
$promises = (function () use ($urls, $client, &$responsesModified)
{
foreach ($urls as $url) {
yield $client->getAsync($url)->then(function($response) use ($url, &$responsesModified)
{
$data = [
'url' => $url,
'body' => 'res' // pass here whatever you want
];
$responsesModified[] = $data;
return $response;
});
}
})();
$eachPromise = new EachPromise($promises,
[
'concurrency' => $numberThreads,
'fulfilled' => function (Response $response)
{
},
'rejected' => function ($reason)
{
}
]);
$eachPromise->promise()->wait();
return $responsesModified;
}
This gives follwing result:
http://i.kagda.ru/5001133750442_01-11-2020-00:34:55_5001.png

How to stop sending asynchronous requests in Guzzle on certain condition?

When I catch many exceptions I want to stop sending requests in Guzzle. Does anybody know how can do that?
Here my snippet of code:
protected function parseAsyncCustomers($urls)
{
$promises = (function () use ($urls) {
do {
$uri = new Uri(current($urls));
$request = new Request('GET', $uri, ['User-Agent' => UserAgent::random()]);
yield $this->httpClient->sendAsync($request, [
'timeout' => 15,
'connect_timeout' => 15,
]);
} while (next($urls) !== false);
})();
(new \GuzzleHttp\Promise\EachPromise($promises, [
// Multiple Concurrent HTTP Requests
'concurrency' => 10,
'fulfilled' => function (ResponseInterface $response, $index) {
$content = $response->getBody()->getContents();
$this->parseCustomerContent($content, $index);
},
'rejected' => function ($reason, $index) {
// This is delivered each failed request
if ($reason instanceof GuzzleException) {
if ($this->reject++ > 30) {
// how can stop sending next requests?
}
}
},
]))->promise()->wait();
}
There is a fourth parameter to the rejected callback which represents the whole EachPromise. You can reject it in your condition, and it will stop the execution flow.
'rejected' => function ($reason, $index, $idx, $aggregate) {
// This is delivered each failed request
if ($reason instanceof GuzzleException) {
if ($this->reject++ > 30) {
$aggregate->reject('Attempts limit exceeded')
}
}
},

Multi-threaded downloading files with Guzzle HTTP client: EachPromises vs Pool objects

for testing purposes, I have an array of 2000 image URIs (strings) with I download asynchronously with this functions. After some googling & testing & trying I've come up with 2 functions that both of them work (well to be honest downloadFilesAsync2 throws a InvalidArgumentException at the last line).
The function downloadFilesAsync2 is based on the class GuzzleHttp\Promise\EachPromise and downloadFilesAsync1 is based on the GuzzleHttp\Pool class.
Both functions download pretty well the 2000 files asynchronously, with the limit of 10 threads at the same time.
I know that they work, but nothing else. I wonder if someone could explain both aproaches, if one is better than the other, implications, etc.
// for the purpose of this question i've reduced the array to 5 files!
$uris = array /
"https://cdn.enchufix.com/media/catalog/product/u/n/unix-48120.jpg",
"https://cdn.enchufix.com/media/catalog/product/u/n/unix-48120-01.jpg",
"https://cdn.enchufix.com/media/catalog/product/u/n/unix-48120-02.jpg",
"https://cdn.enchufix.com/media/catalog/product/u/n/unix-48120-03.jpg",
"https://cdn.enchufix.com/media/catalog/product/u/n/unix-48120-04.jpg",
);
function downloadFilesAsync2(array $uris, string $dir, $overwrite=true) {
$client = new \GuzzleHttp\Client();
$requests = array();
foreach ($uris as $i => $uri) {
$loc = $dir . DIRECTORY_SEPARATOR . basename($uri);
if ($overwrite && file_exists($loc)) unlink($loc);
$requests[] = new GuzzleHttp\Psr7\Request('GET', $uri, ['sink' => $loc]);
echo "Downloading $uri to $loc" . PHP_EOL;
}
$pool = new \GuzzleHttp\Pool($client, $requests, [
'concurrency' => 10,
'fulfilled' => function (\Psr\Http\Message\ResponseInterface $response, $index) {
// this is delivered each successful response
echo 'success: '.$response->getStatusCode().PHP_EOL;
},
'rejected' => function ($reason, $index) {
// this is delivered each failed request
echo 'failed: '.$reason.PHP_EOL;
},
]);
$promise = $pool->promise(); // Start transfers and create a promise
$promise->wait(); // Force the pool of requests to complete.
}
function downloadFilesAsync1(array $uris, string $dir, $overwrite=true) {
$client = new \GuzzleHttp\Client();
$promises = (function () use ($client, $uris, $dir, $overwrite) {
foreach ($uris as $uri) {
$loc = $dir . DIRECTORY_SEPARATOR . basename($uri);
if ($overwrite && file_exists($loc)) unlink($loc);
yield $client->requestAsync('GET', $uri, ['sink' => $loc]);
echo "Downloading $uri to $loc" . PHP_EOL;
}
})();
(new \GuzzleHttp\Promise\EachPromise(
$promises, [
'concurrency' => 10,
'fulfilled' => function (\Psr\Http\Message\ResponseInterface $response) {
// echo "\t=>\tDONE! status:" . $response->getStatusCode() . PHP_EOL;
},
'rejected' => function ($reason, $index) {
echo 'ERROR => ' . strtok($reason->getMessage(), "\n") . PHP_EOL;
},
])
)->promise()->wait();
}
First, I will address the InvalidArgumentException within the downloadFilesAsync2 method. There are actually a pair of issues with this method. Both relate to this:
$requests[] = $client->request('GET', $uri, ['sink' => $loc]);
The first issue is the fact that Client::request() is a synchronous utility method which wraps $client->requestAsync()->wait(). $client->request() will return an instance of Psr\Http\Message\ResponseInterface, as a result $requests[] will actually be populated with ResponseInterface implementations. This is what, ultimately causes the InvalidArgumentException as the $requests does not contain any Psr\Http\Message\RequestInterface's, and the exception is thrown from within Pool::__construct().
A corrected version of this method should contain code which looks more like:
$requests = [
new Request('GET', 'www.google.com', [], null, 1.1),
new Request('GET', 'www.ebay.com', [], null, 1.1),
new Request('GET', 'www.cnn.com', [], null, 1.1),
new Request('GET', 'www.red.com', [], null, 1.1),
];
$pool = new Pool($client, $requests, [
'concurrency' => 10,
'fulfilled' => function(ResponseInterface $response) {
// do something
},
'rejected' => function($reason, $index) {
// do something error handling
},
'options' => ['sink' => $some_location,],
]);
$promise = $pool->promise();
$promise->wait();
To answer your second question, "What is the difference between these two methods", the answer is simply, there is none. To explain this, let me copy and paste Pool::__construct():
/**
* #param ClientInterface $client Client used to send the requests.
* #param array|\Iterator $requests Requests or functions that return
* requests to send concurrently.
* #param array $config Associative array of options
* - concurrency: (int) Maximum number of requests to send concurrently
* - options: Array of request options to apply to each request.
* - fulfilled: (callable) Function to invoke when a request completes.
* - rejected: (callable) Function to invoke when a request is rejected.
*/
public function __construct(
ClientInterface $client,
$requests,
array $config = []
) {
// Backwards compatibility.
if (isset($config['pool_size'])) {
$config['concurrency'] = $config['pool_size'];
} elseif (!isset($config['concurrency'])) {
$config['concurrency'] = 25;
}
if (isset($config['options'])) {
$opts = $config['options'];
unset($config['options']);
} else {
$opts = [];
}
$iterable = \GuzzleHttp\Promise\iter_for($requests);
$requests = function () use ($iterable, $client, $opts) {
foreach ($iterable as $key => $rfn) {
if ($rfn instanceof RequestInterface) {
yield $key => $client->sendAsync($rfn, $opts);
} elseif (is_callable($rfn)) {
yield $key => $rfn($opts);
} else {
throw new \InvalidArgumentException('Each value yielded by '
. 'the iterator must be a Psr7\Http\Message\RequestInterface '
. 'or a callable that returns a promise that fulfills '
. 'with a Psr7\Message\Http\ResponseInterface object.');
}
}
};
$this->each = new EachPromise($requests(), $config);
}
now if we compare that to an a simplified version of the code within the downloadFilesAsync1 method:
$promises = (function () use ($client, $uris) {
foreach ($uris as $uri) {
yield $client->requestAsync('GET', $uri, ['sink' => $some_location]);
}
})();
(new \GuzzleHttp\Promise\EachPromise(
$promises, [
'concurrency' => 10,
'fulfilled' => function (\Psr\Http\Message\ResponseInterface $response) {
// do something
},
'rejected' => function ($reason, $index) {
// do something
},
])
)->promise()->wait();
In both examples, there is a generator which yields promises that resolve to instances of ResponseInterface and that generator along with the configuration array (fulfilled callable, rejected callable, concurrency) is also fed into a new instance of EachPromise.
In summary:
downloadFilesAsync1 is functionally the same thing as using Pool only without the error checking that has been built into Pool::__construct().
There are a few errors within downloadFilesAsync2 which will cause the files to be downloaded in a synchronous fashion prior to receiving an InvalidArgumentException when the Pool is instantiated.
My only recommendation is: use whichever feels more intuitive for you to use.

How to get transfer time for Pool requests in guzzle6?

I want to get transfer time for each request.
How I can use on_stats option for async requests?
http://docs.guzzlephp.org/en/latest/request-options.html#on-stats
My code:
<?php
use GuzzleHttp\{Pool, Client};
use GuzzleHttp\Psr7\{
Request, Response
};
$httpClient = new Client();
foreach ($items as $request) {
$requests[] = new Request(...);
}
$responses = Pool::batch($httpClient, $requests, ['fulfilled' => function($response, $index) {
});
Solution:
$responses = Pool::batch($httpClient, $requests, ['fulfilled' => function($response, $index) {
}, 'options' => ['on_stats' => function(TransferStats $stats) {
//..
}]]);

Categories