How to perform concurrent requests with GuzzleHttp - php

How do I use Guzzle 6 to create 5 async requests with the following conditions:
All requests start at the same time
I want a 500ms timeout value for all requests. If a request times out I DONT want it to interrupt other requests
If a request returns non-200 I DONT want it to interrupt other requests.
All requests are on different domains... (so I'm not sure how that fits in with the base_uri setting...
If all 5 requests return 200OK < 500ms then I want to be able to loop through their responses...
BUT, if say 2 of them have non-200 and 1 of them times out (over 500ms), I want to still be able to access the responses for the 2 successful ones.
EDIT So far everything works except timeouts are still raising an exception
Here is what I had so far:
<?php
require __DIR__.'/../vendor/autoload.php';
use GuzzleHttp\Client;
use GuzzleHttp\Promise;
$client = new Client([
'http_errors' => false,
'connect_timeout' => 1.50, //////////////// 0.50
'timeout' => 2.00, //////////////// 1.00
'headers' => [
'User-Agent' => 'Test/1.0'
]
]);
// initiate each request but do not block
$promises = [
'success' => $client->getAsync('https://httpbin.org/get'),
'success' => $client->getAsync('https://httpbin.org/delay/1'),
'failconnecttimeout' => $client->getAsync('https://httpbin.org/delay/2'),
'fail500' => $client->getAsync('https://httpbin.org/status/500'),
];
// wait on all of the requests to complete. Throws a ConnectException if any
// of the requests fail
$results = Promise\unwrap($promises);
// wait for the requests to complete, even if some of them fail
$results = Promise\settle($promises)->wait();

Guzzle provides fulfilled and rejected callabcks in the pool. here I performed a test by your values, read more at Guzzle docs:
$client = new Client([
'http_errors' => false,
'connect_timeout' => 0.50, //////////////// 0.50
'timeout' => 1.00, //////////////// 1.00
'headers' => [
'User-Agent' => 'Test/1.0'
]
]);
$requests = function ($total) {
$uris = [
'https://httpbin.org/get',
'https://httpbin.org/delay/1',
'https://httpbin.org/delay/2',
'https://httpbin.org/status/500',
];
for ($i = 0; $i < count($uris); $i++) {
yield new Request('GET', $uris[$i]);
}
};
$pool = new Pool($client, $requests(8), [
'concurrency' => 10,
'fulfilled' => function ($response, $index) {
// this is delivered each successful response
print_r($index."fulfilled\n");
},
'rejected' => function ($reason, $index) {
// this is delivered each failed request
print_r($index."rejected\n");
},
]);
// Initiate the transfers and create a promise
$promise = $pool->promise();
// Force the pool of requests to complete.
$promise->wait();
response
0fulfilled
3fulfilled
1rejected
2rejected
if you want to use your code above you can also pass response status in your $promises, here is an example:
use Psr\Http\Message\ResponseInterface;
use GuzzleHttp\Exception\RequestException;
....
$client = new Client([
'http_errors' => false,
'connect_timeout' => 1.50, //////////////// 0.50
'timeout' => 2.00, //////////////// 1.00
'headers' => [
'User-Agent' => 'Test/1.0'
]
]);
$promises = [
'success' => $client->getAsync('https://httpbin.org/get')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
)
,
'success' => $client->getAsync('https://httpbin.org/delay/1')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
),
'failconnecttimeout' => $client->getAsync('https://httpbin.org/delay/2')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
),
'fail500' => $client->getAsync('https://httpbin.org/status/500')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
),
];
$results = Promise\settle($promises)->wait();

for send multiple https request with concuurrency then you can do it as like below
see below sample code
<?php
$client = new \GuzzleHttp\Client(['base_uri' => 'http://test.com']);
$requests = function () use ($client, $product) {
foreach ($product as $val) {
$methodType = $val['method_type']; // Get, Post, Put
$urlPath = $val['url_path']; // url path eg. /search/car/{abc}
$payload = json_decode($val['payload'], true); // your data if you wish to send in request
yield function() use ($client, $methodType, $urlPath) {
return $client->requestAsync($methodType, $urlPath, []);
// return $client->requestAsync($methodType, $urlPath, [\GuzzleHttp\RequestOptions::JSON => $payload]); // for send data as a Json
// return $client->requestAsync($methodType, $urlPath, [\GuzzleHttp\RequestOptions::QUERY => $payload]); // for send data as a Query String in url
};
}
};
$pool = new \GuzzleHttp\Pool($client, $requests(), [
'concurrency' => 3,
'fulfilled' => function (Response $response, $index) use (&$responses, &$successCount) {
if ($response->getStatusCode() == 200) {
// http status ok response
$responses[] = json_decode($response->getBody(), true);
$successCount++;
} else {
// do perform your logic for success response without 200 HTTP Status code
}
},
'rejected' => function (\GuzzleHttp\Exception\RequestException $reason, $index) use (&$failedCount) {
// error response handle here
if ($reason->hasResponse()) {
$response = $reason->getResponse();
$httpstatuscode = $response->getStatusCode();
}
Log::error($reason->getMessage());
$failedCount++;
},
]);
$pool->promise()->wait();
var_dump($responses);
for more detailed see here

Related

HTTP Get time for response

I'm playing around with guzzle and trying to build a simple page, that i can add my domains to - and have it check if they are currently online/accessible.
I can currently check if an array/list of domain's is online, or if it gets rejected for some reason. I would love to also be able to see in my log/DB how long it takes from i send a the HTTP request to [mydomain.com] until the response arrives back.
Current Code:
use GuzzleHttp\Client;
use GuzzleHttp\Exception\RequestException;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\Psr7\Response;
if(!empty($aDomains))
{
$oClient = new Client(['expect' => false]);
$aAcceptedResponse = [];
$aRejectedResponses = [];
$aCreatedRequests = [];
foreach ($aDomains as $iDomainKey => $sDomain)
{
array_push($aCreatedRequests, new Request('GET', $sDomain));
}
$pool = new Pool($oClient, $aCreatedRequests,
[
'concurrency' => 50,
'options' => ['timeout' => 10],
'fulfilled' => function ($response, $index) use (&$aAcceptedResponse)
{
$aAcceptedResponse[] = $index;
},
'rejected' => function ($reason, $index) use(&$aRejectedResponses)
{
$aRejectedResponses[] = $index;
},
]);
$promise = $pool->promise();
$promise->wait();
}
I figured i would be able to find something in the guzzle response object, but so far i have been unable to find anything - am i blind or is it not possible to see this?
Thanks to El_Vanja's answer i figured it out by just using a global timestamp:
$iStartExecutionTime = microtime(true);
$oClient = new Client(['expect' => false]);
$aAcceptedResponse = [];
$aRejectedResponses = [];
$aCreatedRequests = [];
foreach ($aDomains as $iDomainKey => $oDomain)
{
array_push($aCreatedRequests, new Request('GET', $oDomain->sDomainName));
update_domain_uptime_monitor($oDomain->iID, 1, date('Y-m-d H:i:s', strtotime('NOW')+$oDomain->iInterval), date('Y-m-d H:i:s', strtotime('NOW')));
}
$pool = new Pool($oClient, $aCreatedRequests,
[
'concurrency' => 50,
'options' => ['timeout' => 10],
'fulfilled' => function ($response, $index) use (&$aAcceptedResponse)
{
$aAcceptedResponse[$index] = (microtime(true)-$GLOBALS['iStartExecutionTime']);
},
'rejected' => function ($reason, $index) use(&$aRejectedResponses)
{
$aRejectedResponses[] = $index;
},
]);
$promise = $pool->promise();
$promise->wait();

HTTP Guzzle not returning all data

I have created a function that contacts a remote API using Guzzle but I cannot get it to return all of the data available.
I call the function here:
$arr = array(
'skip' => 0,
'take' => 1000,
);
$sims = api_request('sims', $arr);
And here is the function, where I have tried the following in my $response variable
json_decode($x->getBody(), true)
json_decode($x->getBody()->getContents(), true)
But neither has shown any more records. It returns 10 records, and I know there are over 51 available that it should be returning.
use GuzzleHttp\Client;
function api_request($url, $vars = array(), $type = 'GET') {
$username = '***';
$password = '***';
//use GuzzleHttp\Client;
$client = new Client([
'auth' => [$username, $password],
]);
$auth_header = 'Basic '.$username.':'.$password;
$headers = ['Authorization' => $auth_header, 'Content-Type' => 'application/json'];
$json_data = json_encode($vars);
$end_point = 'https://simportal-api.azurewebsites.net/api/v1/';
try {
$x = $client->request($type, $end_point.$url, ['headers' => $headers, 'body' => $json_data]);
$response = array(
'success' => true,
'response' => // SEE ABOVE //
);
} catch (GuzzleHttp\Exception\ClientException $e) {
$response = array(
'success' => false,
'errors' => json_decode($e->getResponse()->getBody(true)),
);
}
return $response;
}
By reading the documentation on https://simportal-api.azurewebsites.net/Help/Api/GET-api-v1-sims_search_skip_take I assume that the server is not accepting your parameters in the body of that GET request and assuming the default of 10, as it is normal in many applications, get requests tend to only use query string parameters.
In that function I'd try to change it in order to send a body in case of a POST/PUT/PATCH request, and a "query" without json_encode in case of a GET/DELETE request. Example from guzzle documentation:
$client->request('GET', 'http://httpbin.org', [
'query' => ['foo' => 'bar']
]);
Source: https://docs.guzzlephp.org/en/stable/quickstart.html#query-string-parameters

Guzzle Pool doesn't respect timeout

I have some problem with my guzzle client. I set timeout for example 1.0 and in some route I do sleep(5). Guzzle anyway wait on response when should just throw exception.
client:
$requests[] = new Request('GET', $path, [
'timeout' => 1,
'connect_timeout' => 1
]);
$pool = new Pool($this->client, $requests, [
'concurrency' => 5,
'fulfilled' => function ($response, $index) use ($response_merger) {
$response_merger->fulfilled($response);
},
'rejected' => function ($reason, $index) use ($response_merger) {
$response_merger->error($reason);
}
]);
and my route with delay:
$app->get('/timeout', function() use ($app) {
sleep(5);
return (new JsonResponse())->setData([ 'error' => 'My timeout exception.' ])->setStatusCode(504);
});
I always get 504 with My timeout exception, when I should not get it because timeout is set.
I did it with set client, but it is not a solution for me because I need custom timeout for certain request, not client.
$this->client = new Client([
'timeout' => 3.0,
'connect_timeout' => 1.0
]);
I think you've got the wrong signature in mind for new Request(). From the docs:
// Create a PSR-7 request object to send
$headers = ['X-Foo' => 'Bar'];
$body = 'Hello!';
$request = new Request('HEAD', 'http://httpbin.org/head', $headers, $body);
The third parameter is for HTTP headers, not options.
You should pass timeout as an option when constructing the Pool:
$pool = new Pool($this->client, $requests, [
'concurrency' => 5,
'options' => ['timeout' => 10],
'fulfilled' => function ($response, $index) use ($response_merger) {
$response_merger->fulfilled($response);
},
'rejected' => function ($reason, $index) use ($response_merger) {
$response_merger->error($reason);
}
]);
Found this in comments of the Pool code here.

Changing Header for Concurrent Requests (Guzzle)

I am unable to change the request headers when making async requests.
$requests = function ($total) {
$uri = 'https://www.example.com';
$headers = [
'User-Agent' => 'testing/1.0',
'Accept' => 'application/json',
'X-Foo' => ['Bar', 'Baz']
];
for ($i = 0; $i < $total; $i++) {
yield new Request('GET', $uri, $headers); //Does not work
}
};
$pool = new Pool($client, $requests(2), [
'concurrency' => 5,
'fulfilled' => function ($response, $index) {
// this is delivered each successful response
},
'rejected' => function ($reason, $index) {
// this is delivered each failed request
},
]);
I believe I have tried all of the examples provided in the documentation and have been able to change the headers for all of them except the concurrent examples. Any help would be appreciated.
I may have gotten this to work, but not under the conditions that I was hoping for.
$headers = ['User-Agent' => 'Bar'];
$client = new Client(['base_uri' => 'https://www.example.com/']);
$promises = [
'image' => $client->getAsync('/page1',
['User-Agent' => "test"]
),
'png' => $client->getAsync('/page2'),
];
$results = Promise\unwrap($promises);
$results = Promise\settle($promises)->wait();
I believe this works, because in this instance I am using the Client object rather than creating my own request (as I did in the other one). I'm not entirely sure about this. So if anyone can provide any further insight, that would be helpful.

Guzzle PUT request auth error

I have the following code to save content using API from another system. I have added the credentials but it showing wrong credentials error. It is working perfectly in postman.
$client = new GuzzleHttpClient();
try {
$request = new \GuzzleHttp\Psr7\Request('PUT', config('cms.api.backend') .'/products/'. $nid,
[
'auth' => [config('cms.api.user'), config('cms.api.password')],
'form_params' => [
'copywrite' => Input::get('copywrite'),
'status' => $status
],
]);
$promise = $client->sendAsync($request)->then(function ($response) {});
$promise->wait();
}
catch (RequestException $e) {
$this->logHttpError($e->getResponse()->getStatusCode(), $e->getResponse()->getBody(true));
}
What could be wrong in the above code?
Following is the exported code from postman.
$request = new HttpRequest();
$request->setUrl('http://mybackend/api/products/74371');
$request->setMethod(HTTP_METH_PUT);
$request->setHeaders(array(
'postman-token' => 'e0ddcaea-4787-b2c5-0c52-9aaee860ceac',
'cache-control' => 'no-cache',
'authorization' => 'Basic authenticationcode',
'content-type' => 'application/x-www-form-urlencoded'
));
$request->setContentType('application/x-www-form-urlencoded');
$request->setPostFields(array(
'copywrite' => 'date to be saved'
));
try {
$response = $request->send();
echo $response->getBody();
} catch (HttpException $ex) {
echo $ex;
}
Third argument in \GuzzleHttp\Psr7\Request is for headers array only, so you won't send request body (4th arg) this way. Easiest way would be passing this array as second argument to sendAsync() method. It will recognise them and form_params option will be parsed as Content-Type: application/x-www-form-urlencoded header and create a valid stream for your request (it uses http_build_query() function if you want to do it directly in request constructor):
$request = new \GuzzleHttp\Psr7\Request('PUT', config('cms.api.backend') .'/products/'. $nid);
$options = [
'auth' => [config('cms.api.user'), config('cms.api.password')],
'form_params' => [
'copywrite' => Input::get('copywrite'),
'status' => $status
],
];
$promise = $client->sendAsync($request, $options)->then(function ($response) {});
$promise->wait();

Categories