I'm playing around with guzzle and trying to build a simple page, that i can add my domains to - and have it check if they are currently online/accessible.
I can currently check if an array/list of domain's is online, or if it gets rejected for some reason. I would love to also be able to see in my log/DB how long it takes from i send a the HTTP request to [mydomain.com] until the response arrives back.
Current Code:
use GuzzleHttp\Client;
use GuzzleHttp\Exception\RequestException;
use GuzzleHttp\Pool;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\Psr7\Response;
if(!empty($aDomains))
{
$oClient = new Client(['expect' => false]);
$aAcceptedResponse = [];
$aRejectedResponses = [];
$aCreatedRequests = [];
foreach ($aDomains as $iDomainKey => $sDomain)
{
array_push($aCreatedRequests, new Request('GET', $sDomain));
}
$pool = new Pool($oClient, $aCreatedRequests,
[
'concurrency' => 50,
'options' => ['timeout' => 10],
'fulfilled' => function ($response, $index) use (&$aAcceptedResponse)
{
$aAcceptedResponse[] = $index;
},
'rejected' => function ($reason, $index) use(&$aRejectedResponses)
{
$aRejectedResponses[] = $index;
},
]);
$promise = $pool->promise();
$promise->wait();
}
I figured i would be able to find something in the guzzle response object, but so far i have been unable to find anything - am i blind or is it not possible to see this?
Thanks to El_Vanja's answer i figured it out by just using a global timestamp:
$iStartExecutionTime = microtime(true);
$oClient = new Client(['expect' => false]);
$aAcceptedResponse = [];
$aRejectedResponses = [];
$aCreatedRequests = [];
foreach ($aDomains as $iDomainKey => $oDomain)
{
array_push($aCreatedRequests, new Request('GET', $oDomain->sDomainName));
update_domain_uptime_monitor($oDomain->iID, 1, date('Y-m-d H:i:s', strtotime('NOW')+$oDomain->iInterval), date('Y-m-d H:i:s', strtotime('NOW')));
}
$pool = new Pool($oClient, $aCreatedRequests,
[
'concurrency' => 50,
'options' => ['timeout' => 10],
'fulfilled' => function ($response, $index) use (&$aAcceptedResponse)
{
$aAcceptedResponse[$index] = (microtime(true)-$GLOBALS['iStartExecutionTime']);
},
'rejected' => function ($reason, $index) use(&$aRejectedResponses)
{
$aRejectedResponses[] = $index;
},
]);
$promise = $pool->promise();
$promise->wait();
Related
I am new to using Laravel http and changed my code to use a async request. Can someone help me expand on my current code to access the result of each request. I am a little unsure how to use the following
fn (Response|TransferException $result) =>
$this->handleResult($result)
$date = new DateTime;
$date->modify('-15 minutes');
$formatted_date = $date->format('Y-m-d H:i:s');
$twitter_accounts = TwitterAccount::where('last_tweet_fetch','<=',$formatted_date)
->orWhere('last_tweet_fetch', '=', null)
->orderBy('id')
->limit(40)
->get();
foreach ($twitter_accounts as $twitter_account) {
$promise = Http::withToken('__TOKEN__')
->async()->get("https://api.twitter.com/2/users/".urlencode($twitter_account->twitter_id)."/tweets",[
'expansions' => 'attachments.media_keys,author_id,referenced_tweets.id',
'media.fields' => 'type,preview_image_url,url',
'user.fields' => 'profile_image_url,url,description,public_metrics,verified',
'tweet.fields' => 'created_at,text',
'max_results' => '5',
'exclude' => 'replies',
'since_id' => $twitter_account->last_tweet_id
])
->then(
fn (Response|TransferException $result) => $this->handleResult($result)
);
}
Make use of Guzzle Client request
$headers = [];
$body = [
'expansions' => 'attachments.media_keys,author_id,referenced_tweets.id',
'media.fields' => 'type,preview_image_url,url',
'user.fields' => 'profile_image_url,url,description,public_metrics,verified',
'tweet.fields' => 'created_at,text',
'max_results' => '5',
'exclude' => 'replies',
'since_id' => $twitter_account->last_tweet_id
];
// Send an asynchronous request.
$request = new \GuzzleHttp\Psr7\Request('GET', "https://api.twitter.com/2/users/".urlencode($twitter_account->twitter_id)."/tweets", $headers, $body);
$promise = $client->sendAsync($request)->then(function ($response) {
echo 'I completed! ' . $response->getBody();
});
$promise->wait();
I would like to refactor my code and use an outside function for the anonymous fullfilled.
For instance i want to refactor the code below:
to something more readable (without the anonymous function). like
$fulfilled = New Fulfilled($phones);
$rejected = New Rejected();
$pool = new Pool(..., [
'concurrency' => 15,
'fulfilled' => $Fulfilled,
'rejected' => $Rejected,
]);
I created a class for fulfilled like this
But how can i get back my &$result ?
Guzzle is a puzzle.
[EDIT]
As said from Abiux, a class is not so much readable. I tried to extract the anonymous function outside the $pool like this
public function fulfilled($response,$index){
$body=$response->getBody();
$status=$response->getStatusCode();
$reason=$response->getReasonPhrase();
$result[] = [
"userNumber" => $this->phones[$index],
"status" => $status,
"reason" => $reason,
"body" => json_decode($body,true)];
return $result;
}
// -------------------------------------
public function get_async($phones){
...
$client = new Client();
$requests = function ($phones) use ($filter){
foreach ($phones as $phone){
$url="contexts?userNumber=$phone&$filter";
yield new Request('GET', $url);
}
};
$rejected = new Rejected();
$pool = new Pool($client, $requests($phones), [
'concurrency' => 2,
'options' => $options,
'fulfilled' => ['this','fulfilled',&$result],
'rejected' => $rejected,
]);
// Initiate the transfers and create a promise
$promise = $pool->promise();
// Force the pool of requests to complete.
$promise->wait();
return $result;
}
but it fails also
First, don't replace anonymous function with a class. It's not more readable and it doesn't serve any purpose (as far as you have expalined).
Second, you can pass $result as part of the object instantiation to __construct() and then reference it within the __invoke function.
How do I use Guzzle 6 to create 5 async requests with the following conditions:
All requests start at the same time
I want a 500ms timeout value for all requests. If a request times out I DONT want it to interrupt other requests
If a request returns non-200 I DONT want it to interrupt other requests.
All requests are on different domains... (so I'm not sure how that fits in with the base_uri setting...
If all 5 requests return 200OK < 500ms then I want to be able to loop through their responses...
BUT, if say 2 of them have non-200 and 1 of them times out (over 500ms), I want to still be able to access the responses for the 2 successful ones.
EDIT So far everything works except timeouts are still raising an exception
Here is what I had so far:
<?php
require __DIR__.'/../vendor/autoload.php';
use GuzzleHttp\Client;
use GuzzleHttp\Promise;
$client = new Client([
'http_errors' => false,
'connect_timeout' => 1.50, //////////////// 0.50
'timeout' => 2.00, //////////////// 1.00
'headers' => [
'User-Agent' => 'Test/1.0'
]
]);
// initiate each request but do not block
$promises = [
'success' => $client->getAsync('https://httpbin.org/get'),
'success' => $client->getAsync('https://httpbin.org/delay/1'),
'failconnecttimeout' => $client->getAsync('https://httpbin.org/delay/2'),
'fail500' => $client->getAsync('https://httpbin.org/status/500'),
];
// wait on all of the requests to complete. Throws a ConnectException if any
// of the requests fail
$results = Promise\unwrap($promises);
// wait for the requests to complete, even if some of them fail
$results = Promise\settle($promises)->wait();
Guzzle provides fulfilled and rejected callabcks in the pool. here I performed a test by your values, read more at Guzzle docs:
$client = new Client([
'http_errors' => false,
'connect_timeout' => 0.50, //////////////// 0.50
'timeout' => 1.00, //////////////// 1.00
'headers' => [
'User-Agent' => 'Test/1.0'
]
]);
$requests = function ($total) {
$uris = [
'https://httpbin.org/get',
'https://httpbin.org/delay/1',
'https://httpbin.org/delay/2',
'https://httpbin.org/status/500',
];
for ($i = 0; $i < count($uris); $i++) {
yield new Request('GET', $uris[$i]);
}
};
$pool = new Pool($client, $requests(8), [
'concurrency' => 10,
'fulfilled' => function ($response, $index) {
// this is delivered each successful response
print_r($index."fulfilled\n");
},
'rejected' => function ($reason, $index) {
// this is delivered each failed request
print_r($index."rejected\n");
},
]);
// Initiate the transfers and create a promise
$promise = $pool->promise();
// Force the pool of requests to complete.
$promise->wait();
response
0fulfilled
3fulfilled
1rejected
2rejected
if you want to use your code above you can also pass response status in your $promises, here is an example:
use Psr\Http\Message\ResponseInterface;
use GuzzleHttp\Exception\RequestException;
....
$client = new Client([
'http_errors' => false,
'connect_timeout' => 1.50, //////////////// 0.50
'timeout' => 2.00, //////////////// 1.00
'headers' => [
'User-Agent' => 'Test/1.0'
]
]);
$promises = [
'success' => $client->getAsync('https://httpbin.org/get')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
)
,
'success' => $client->getAsync('https://httpbin.org/delay/1')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
),
'failconnecttimeout' => $client->getAsync('https://httpbin.org/delay/2')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
),
'fail500' => $client->getAsync('https://httpbin.org/status/500')->then(
function (ResponseInterface $res) {
echo $res->getStatusCode() . "\n";
},
function (RequestException $e) {
echo $e->getMessage() . "\n";
echo $e->getRequest()->getMethod();
}
),
];
$results = Promise\settle($promises)->wait();
for send multiple https request with concuurrency then you can do it as like below
see below sample code
<?php
$client = new \GuzzleHttp\Client(['base_uri' => 'http://test.com']);
$requests = function () use ($client, $product) {
foreach ($product as $val) {
$methodType = $val['method_type']; // Get, Post, Put
$urlPath = $val['url_path']; // url path eg. /search/car/{abc}
$payload = json_decode($val['payload'], true); // your data if you wish to send in request
yield function() use ($client, $methodType, $urlPath) {
return $client->requestAsync($methodType, $urlPath, []);
// return $client->requestAsync($methodType, $urlPath, [\GuzzleHttp\RequestOptions::JSON => $payload]); // for send data as a Json
// return $client->requestAsync($methodType, $urlPath, [\GuzzleHttp\RequestOptions::QUERY => $payload]); // for send data as a Query String in url
};
}
};
$pool = new \GuzzleHttp\Pool($client, $requests(), [
'concurrency' => 3,
'fulfilled' => function (Response $response, $index) use (&$responses, &$successCount) {
if ($response->getStatusCode() == 200) {
// http status ok response
$responses[] = json_decode($response->getBody(), true);
$successCount++;
} else {
// do perform your logic for success response without 200 HTTP Status code
}
},
'rejected' => function (\GuzzleHttp\Exception\RequestException $reason, $index) use (&$failedCount) {
// error response handle here
if ($reason->hasResponse()) {
$response = $reason->getResponse();
$httpstatuscode = $response->getStatusCode();
}
Log::error($reason->getMessage());
$failedCount++;
},
]);
$pool->promise()->wait();
var_dump($responses);
for more detailed see here
I have some problem with my guzzle client. I set timeout for example 1.0 and in some route I do sleep(5). Guzzle anyway wait on response when should just throw exception.
client:
$requests[] = new Request('GET', $path, [
'timeout' => 1,
'connect_timeout' => 1
]);
$pool = new Pool($this->client, $requests, [
'concurrency' => 5,
'fulfilled' => function ($response, $index) use ($response_merger) {
$response_merger->fulfilled($response);
},
'rejected' => function ($reason, $index) use ($response_merger) {
$response_merger->error($reason);
}
]);
and my route with delay:
$app->get('/timeout', function() use ($app) {
sleep(5);
return (new JsonResponse())->setData([ 'error' => 'My timeout exception.' ])->setStatusCode(504);
});
I always get 504 with My timeout exception, when I should not get it because timeout is set.
I did it with set client, but it is not a solution for me because I need custom timeout for certain request, not client.
$this->client = new Client([
'timeout' => 3.0,
'connect_timeout' => 1.0
]);
I think you've got the wrong signature in mind for new Request(). From the docs:
// Create a PSR-7 request object to send
$headers = ['X-Foo' => 'Bar'];
$body = 'Hello!';
$request = new Request('HEAD', 'http://httpbin.org/head', $headers, $body);
The third parameter is for HTTP headers, not options.
You should pass timeout as an option when constructing the Pool:
$pool = new Pool($this->client, $requests, [
'concurrency' => 5,
'options' => ['timeout' => 10],
'fulfilled' => function ($response, $index) use ($response_merger) {
$response_merger->fulfilled($response);
},
'rejected' => function ($reason, $index) use ($response_merger) {
$response_merger->error($reason);
}
]);
Found this in comments of the Pool code here.
I am unable to change the request headers when making async requests.
$requests = function ($total) {
$uri = 'https://www.example.com';
$headers = [
'User-Agent' => 'testing/1.0',
'Accept' => 'application/json',
'X-Foo' => ['Bar', 'Baz']
];
for ($i = 0; $i < $total; $i++) {
yield new Request('GET', $uri, $headers); //Does not work
}
};
$pool = new Pool($client, $requests(2), [
'concurrency' => 5,
'fulfilled' => function ($response, $index) {
// this is delivered each successful response
},
'rejected' => function ($reason, $index) {
// this is delivered each failed request
},
]);
I believe I have tried all of the examples provided in the documentation and have been able to change the headers for all of them except the concurrent examples. Any help would be appreciated.
I may have gotten this to work, but not under the conditions that I was hoping for.
$headers = ['User-Agent' => 'Bar'];
$client = new Client(['base_uri' => 'https://www.example.com/']);
$promises = [
'image' => $client->getAsync('/page1',
['User-Agent' => "test"]
),
'png' => $client->getAsync('/page2'),
];
$results = Promise\unwrap($promises);
$results = Promise\settle($promises)->wait();
I believe this works, because in this instance I am using the Client object rather than creating my own request (as I did in the other one). I'm not entirely sure about this. So if anyone can provide any further insight, that would be helpful.