I'm trying to make 4 async search calls (POST HTTP request) to search 4 database tables at the same time, then wait for all of them to finish (asynchronously), combine the results and return them to the user.
Here is my code:
public static function async_search($search_words)
{
$curl = new CurlMultiHandler();
$client = new Client([
'base_uri' => 'https://mytestapi.com/',
'timeout' => 0,
]);
$finished_promisesArr = array(
'search1' => 0,
'search2' => 0,
'search3' => 0,
'search4' => 0,
);
$bodyArr = array(
'search_words' => $search_words,
);
$search1_promise = $client->requestAsync('POST', 'search1', array(
'form_params' => $bodyArr,
))->then(
function(ResponseInterface $response) {
echo 'got response';
$finished_promisesArr['search1'] = 1;
},
function (RequestException $e) {
$finished_promisesArr['search1'] = 1;
}
);
$search2_promise = $client->requestAsync('POST', 'search2', array(
'form_params' => $bodyArr,
))->then(
function(ResponseInterface $response) {
$finished_promisesArr['search2'] = 1;
},
function (RequestException $e) {
$finished_promisesArr['search2'] = 1;
}
);
$search3_promise = $client->requestAsync('POST', 'searchCompanies', array(
'form_params' => $bodyArr,
))->then(
function(ResponseInterface $response) {
$finished_promisesArr['search3'] = 1;
},
function (RequestException $e) {
$finished_promisesArr['search3'] = 1;
}
);
$search4_promise = $client->requestAsync('POST', 'searchCompaniesByIndustries', array(
'form_params' => $bodyArr,
))->then(
function(ResponseInterface $response) {
$finished_promisesArr['search4'] = 1;
},
function (RequestException $e) {
$finished_promisesArr['search4'] = 1;
}
);
$promisesAggregate = GuzzleHttp\Promise\all([$search1_promise, $search2_promise, $search3_promise, $search4_promise]);
foreach ($promisesAggregate as $agg) {
$curl->tick();
}
$keep_running = true;
while ($keep_running) {
$all_processes_finished = true;
foreach ($finished_promisesArr as $promise => $status) {
if (!$status) {
$all_processes_finished = false;
}
}
if ($all_processes_finished) {
$keep_running = false;
}
}
return array();
}
Please ignore the empty array in the result and the fact that I'm not doing anything with the response.
I've got logs on the server methods and they're not even being called, and the loop continues to run infinitely.
Note that when I use the request method instead of requestAsync I do get the correct result.
Any ideas here?
Thank's!
Just call $promisesAggregate->wait() after you get it from GuzzleHttp\Promise\all function.
It means the same as your code, but does it right (your wait code calls ->tick() only once and this is the mistake).
Related
I have a problem, I am currently doing a search by date range of maximum 10 days and the issue is that if there are more than 10 messages or there is a total of 38 messages it takes 4 to 5 minutes to load the data, so I wanted to know if there is a way to bring it paginated directly when doing the query in Horde.
public function connect($username, $password, $hostname, $port, $ssl, $folder, $debug = 0, $cache = 0)
{
$opt = [
'username' => $username,
'password' => $password,
'hostspec' => $hostname,
'port' => $port,
'secure' => $ssl,
];
if ($debug) $opt['debug'] = Log::debug("message");
if ($cache) $opt['cache'] = array(
'backend' => new \Horde_Imap_Client_Cache_Backend_Cache(array(
'cacheob' => new Horde_Cache(new Horde_Cache_Storage_File(array(
'dir' => '/tmp/hordecache'
)))
))
);
static::$folder = 'INBOX';
try {
return static::$client = new \Horde_Imap_Client_Socket($opt);
} catch (\Horde_Imap_Client_Exception $e) {
// Any errors will cause an Exception.
dd('Error');
}
}
public function searchSince($date_string)
{
try {
$query = new Horde_Imap_Client_Search_Query();
$query->dateSearch(
new Horde_Imap_Client_DateTime($date_string), Horde_Imap_Client_Search_Query::DATE_SINCE
);
$results = static::$client->search(static::$folder,$query);
} catch (\Exception $th) {
dd('Since Mail');
}
if ($results) {
static::$var = $results['match']->ids;
return true;
}
return false;
}
//I get the header data of each message (subject, date, uid, from, client name)
public function next(){
if ($var = next(static::$var)) {
static::$id = $var;
$headers = HordeImap::fetchHeaders(static::$id);
static::$clientName = $headers->getHeader('From')->getAddressList(true)->first()->__get('personal');
static::$fromEmail = $headers->getHeader('From')->getAddressList(true)->first()->__get('bare_address');
static::$subject = $headers->getHeader('subject')->__get('value');
$emailDate = $headers->getHeader('Date')->__get('value');
$unixTimestamp = strtotime($emailDate);
static::$emailDate = date('Y-m-d H:i:s',$unixTimestamp);
return true;
}
return false;
}
Controller
$mailb = new HordeImap();
$mailb->connect($userMail->client,$userMail->secret_token,$userMail->server,$userMail->port,'ssl','INDEX',1);
$msgs = $mailb->searchSince(date('d F Y', strtotime(date("Y-m-d", time()) . " -10 days")));
static::$mailbox = [];
if($msgs) while($mailb->next()){
static::$mailbox[] = [
'uid' => $mailb::$id,
'from' => $mailb::$fromEmail,
'client'=> $mailb::$clientName,
'date' => $mailb::$emailDate,
'subject'=> $mailb::$subject,
];
}
// I page the array once I have obtained all the contents.
$page = null;
$page = $page ?: (Paginator::resolveCurrentPage() ?: 1);
$items = static::$mailbox instanceof Collection ? static::$mailbox : Collection::make(static::$mailbox);
dd(new LengthAwarePaginator($items->forPage($page, 5), $items->count(), 5, null, []));
I have problem when i try download many images using Guzzle. Some of them probably invalid, so when i made AsyncRequest i cant handle exception of every images.
I tried to make validation method, which must work with downloaded files, but it doesnt work on-time.
$requests = function () use ($client, $imagesMetaData) {
foreach ($imagesMetaData as $index => $image) {
yield $index => function () use ($client, $image) {
try {
$resource = fopen($image['savePath'], 'w+b');
if (!$resource) {
// do smth
}
$opts = ['sink' => $resource];
$result = $client->requestAsync('GET', $image['downloadUri'], $opts);
return ($result);
} catch (\Exception $e) {
$image['writeLog'](
// do smth
);
return new Response(404);
}
};
}
};
Pool::batch($client, $requests(), [
'concurrency' => 5,
'fulfilled' => function (Response $response, $index) use ($me, $imagesMetaData, &$fulfilled, &$rejected) {
$image = $imagesMetaData[$index];
foreach ($imagesMetaData as $image) {
if ($me->validateDownloadedImage($image)) {
$fulfilled[] = $image;
} else {
$rejected[] = $image;
}
}
},
'rejected' => function ($reason, $index) use ($me, $imagesMetaData, &$rejected) {
$image = $imagesMetaData[$index];
$me->rejectImage($image, $reason);
$rejected[] = $image;
}
]);
return [$fulfilled, $rejected];
If files images are fine - everything work as expected, but if one of them failed - i have next error
Catalog import exception of type RuntimeException thrown in file /var/www/my.site/localhost/core/libraries/guzzlehttp/psr7/src/Stream.php at line 250. Unable to write to stream.
https://i.imgur.com/SBLpni5.png
If someone know this, please help.
I fixed this with opening stream:
$stream = stream_for($resource);
and pass this stream to 'sink'
:
'sink' => $stream
final code looks like this:
$requests = function () use ($client, $imagesMetaData) {
foreach ($imagesMetaData as $index => $image) {
yield $index => function () use ($client, $image) {
try {
$resource = fopen($image['savePath'], 'w+b');
$stream = stream_for($resource);
if (!$resource) {
// do smth
}
$opts = ['sink' => $stream];
$result = $client->requestAsync('GET', $image['downloadUri'], $opts);
return ($result);
} catch (\Exception $e) {
$image['writeLog'](
// do smth
);
return new Response(404);
}
};
}
};
Pool::batch($client, $requests(), [
'concurrency' => 5,
'fulfilled' => function (Response $response, $index) use ($me, $imagesMetaData, &$fulfilled, &$rejected) {
$image = $imagesMetaData[$index];
foreach ($imagesMetaData as $image) {
if ($me->validateDownloadedImage($image)) {
$fulfilled[] = $image;
} else {
$rejected[] = $image;
}
}
},
'rejected' => function ($reason, $index) use ($me, $imagesMetaData, &$rejected) {
$image = $imagesMetaData[$index];
$me->rejectImage($image, $reason);
$rejected[] = $image;
}
]);
return [$fulfilled, $rejected];
Here is my code:
public static function test(){
try{
$apiContext = ApiContext::create(
'test', 'bcy', 'v1',
new SimpleTokenCredential('my_token'),
array( 'mode' => 'sandbox','log.LogEnabled' => false, 'log.FileName' => 'BlockCypher.log', 'log.LogLevel' => 'DEBUG') );
$input = new \BlockCypher\Api\TXInput();
$input->addAddress("input_address");
$output = new \BlockCypher\Api\TXOutput();
$output->addAddress("output_address ");
$output->setValue(1000); // Satoshis
/// Tx
$tx = new \BlockCypher\Api\TX();
$tx->addInput($input);
$tx->addOutput($output);
$request = clone $tx;
$txClient = new TXClient($apiContext);
try {
$output = $txClient->create($tx);
} catch (Exception $ex) {
dd("Created TX", "TXSkeleton", null, $request, $ex);
exit(1);
}
dd("Created TX", "TXSkeleton", $output->getTx()->getHash(), $request, $output);
return $output;
}
catch (\BlockCypher\Exception\BlockCypherConnectionException $ex) {
echo $ex->getData();
die;
}
}
This is what I use to create CreateTransaction api but when I change the mode from bcy to btc it gives error for checking url get/post
code source :: click here
And here the response I'm getting also it came in catch so it's a error I have create api for generate address and create input address from there and make account on block.io and make a address for out from there to use in this api beside from these my account on blockcypher in free and nothing purchase in it
{
"errors":[
{
"error":"Unable to find a transaction to spend for address CCrB7dvBT1bqNfWxupKPH9v8yN7xukmqUF."
},
{
"error":"Error building transaction: Address 33cjwDAyNeAPVUMWqh9hdRxdmwdTE4kyTx is of unknown size.."
},
{
"error":"Not enough funds after fees in 0 inputs to pay for 0 outputs, missing -22200."
},
{
"error":"Error validating generated transaction: Transaction missing input or output."
}
],
"tx":{
"block_height":-1,
"block_index":-1,
"hash":"d21633ba23f70118185227be58a63527675641ad37967e2aa461559f577aec43",
"addresses":[
],
"total":0,
"fees":0,
"size":10,
"preference":"low",
"relayed_by":"116.193.163.150",
"received":"2017-11-14T10:20:43.757719705Z",
"ver":1,
"double_spend":false,
"vin_sz":0,
"vout_sz":0,
"confirmations":0,
"inputs":[
],
"outputs":[
]
}
}
I am working it on test purpose so use test main
I have installed it from github
I find one way of doing this thing here is my code
<?php
try
{
$apiContext = ApiContext::create(env('BlockCypher_net') , env('BlockCypher_cn') , env('BlockCypher_v') , new SimpleTokenCredential(env('BlockCypher_key')) , array(
'log.LogEnabled' => true,
'log.FileName' => 'BlockCypher.log',
'mode' => 'sandbox',
'log.LogLevel' => 'DEBUG'
));
$input = new BlockCypherApiTXInput();
$input->addAddress($user['address']);
$output = new BlockCypherApiTXOutput();
$output->addAddress($data['address12']);
$value_btc = 100000000 * ($data['btc12'] + 1 * ($data['btc12'] / 100));
// dd($value_btc);
$output->setValue($value_btc);
$tx = new BlockCypherApiTX();
$tx->addInput($input);
$tx->addOutput($output);
$request = clone $tx;
$params = array(
'includeToSignTx' => 1,
'script_type' => 'mutlisig-n-of-m',
);
$txClient = new TXClient($apiContext);
try
{
$txSkeleton = $txClient->create($tx, $params);
$privateKeys = array(
$user['private']
);
$txSkeleton = $txClient->sign($txSkeleton, $privateKeys);
$txSkeleton = $txClient->send($txSkeleton);
return array(
'success' => 0
);
// dd($txSkeleton->getTx()->getHash());
}
catch(BlockCypherExceptionBlockCypherConnectionException $ex)
{
return array(
'success' => 0,
'msg' => $ex->getData()
);
}
return $txSkeleton->getTx()->getHash();
}
catch(BlockCypherExceptionBlockCypherConnectionException $ex)
{
return array(
'success' => 0,
'msg' => $ex->getData()
);
}
it's work for me hope it will help you drop comment if get any error.
I try to use GuzzleHttp with Magento to update my product catalog.
I use a pool request to get data from database and I try to post them into magento trought API
$apiUrl = 'http://xxx.xxxxx.xxx';
$middleware = new Oauth1([
'consumer_key' => '-------------------------------',
'consumer_secret' => '-------------------------------',
'token' => '-------------------------------',
'token_secret' => '-------------------------------'
]);
$stack->push($middleware);
$client = new Client();
$clientUpdate = new Client([
'base_uri' => $apiUrl,
'handler' => $stack,
'auth' => 'oauth'
]);
$mapper = new JsonMapper();
$requests = function ($total) {
$uri = 'http://10.0.0.114:15021/myservice';
for ($i = 1; $i <= $total; $i++) {
yield new Request('GET', $uri.$i);
}
};
$pool = new Pool($client, $requests(3), [
'concurrency' => 3,
'fulfilled' => function ($response, $index) {
global $clientUpdate, $mapper;
$productsArray = $mapper->mapArray(
json_decode($response->getBody($asString = TRUE)), new ArrayObject(), 'ProductGo'
);
$product = new Product();
$stock = new StockData();
foreach($productsArray as &$value){
$product->type_id = "simple";
$product->attribute_set_id = 4;
$product->sku = $value->xxxxxxxxxxxxxxx;
$product->weight = 1;
$product->name = $value->xxxxxxxx;
$product->price = $value->xxxxxxxx;
$product->status = 1;
$product->visibility =4;
$product->tax_class_id = 4;
$product->description = $value->xxxxxxx;
$product->short_description = $value->xxxxxx;
$stock->qty = $value->xxxxx;
$stock->min_qty = "0";
$stock->is_qty_decimal = 0;
$stock->is_in_stock = 1;
$product->stock_data = (object) array_filter((array) $stock);
$productIn = (object) array_filter((array) $product);
try{
$response = $clientUpdate->request('POST', '/api/rest/products', ['json' => json_encode($productIn)]);
echo $response;
}catch (RequestException $e) {
echo GuzzleHttp\Psr7\str($e->getRequest());
if ($e->hasResponse()) {
echo GuzzleHttp\Psr7\str($e->getResponse());
}
}
}
},
'rejected' => function ($reason, $index) {
echo $reason;
},
]);
// Initiate the transfers and create a promise
$promise = $pool->promise();
// Force the pool of requests to complete.
$promise->wait();
Pool request works ok. If I debug I will put a break point where I make a request but after I get ever the same error
PHP Fatal error: Uncaught exception 'GuzzleHttp\Exception\ServerException' with message 'Server error: 500' in /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/guzzle/src/Middleware.php:68
Stack trace:
#0 /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/promises/src/Promise.php(199): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Response))
#1 /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/promises/src/Promise.php(152): GuzzleHttp\Promise\Promise::callHandler(1, Object(GuzzleHttp\Psr7\Response), Array)
#2 /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/promises/src/TaskQueue.php(60): GuzzleHttp\Promise\Promise::GuzzleHttp\Promise\{closure}()
#3 /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(96): GuzzleHttp\Promise\TaskQueue->run()
#4 /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php(123): GuzzleHttp\Handler\CurlMultiHandler->tick()
#5 /home/xxxxxxxxxxx/PhpstormPr in /home/xxxxxxxxxxx/PhpstormProjects/xxxxxxxxxxx/vendor/guzzlehttp/guzzle/src/Middleware.php on line 68
I can't understand what's the problem, I've tried to post with postman this json as product and it works.
Any suggestions???
http://docs.guzzlephp.org/en/latest/request-options.html#jso
Summary:
The json option is used to easily upload JSON encoded data as the body
of a request. A Content-Type header of application/json will be added
if no Content-Type header is already present on the message.
Here's an example on how to set up the header on request:
$client->request('POST', $url, [
'headers' => [
'Accept' => 'application/json',
],
'body' = $body
]);
Updating my collection record field with 'MongoBinData', exception is triggered:
"document fragment is too large: 21216456, max: 16777216"
I find some web discussion about 'allowDiskUse:true' for aggregate, but nothing ubout 'update'.
Here a part of code in PHP:
try {
$criteria = array( '_id' => $intReleaseId);
$fileData = file_get_contents( $_FILES[ $fileKey]["tmp_name"]);
$mongoBinData = new MongoBinData( $fileData, MongoBinData::GENERIC)
$docItem['data'] = $mongoBinData;
$docItem['fileType'] = $strFileType;
$docItem['fileSize'] = $intFileSize;
$docItem['fileExtension'] = $strFileExtension;
$docItem['fileName'] = $strFileName;
$options = array( "upsert" => true,
'safe' => true, 'fsync' => true,
'allowDiskUse' => true ); // this option doesn't change anything
$reportJson = self::GetCollection('releases')->update( $criteria, $docItem, $options);
...
MongoDb release is db version v3.0.6
Some idea ?
Self resolved.
Use gridFS is immediate and simple.
$_mongo = new MongoClient();
$_db = $_mongo->selectDB($_mDbName);
$_gridFS = $_db->getGridFS();
/* */
function saveFileData($intReleaseId, $binData) {
$criteria = array( '_id' => $intReleaseId);
// if exist or not, remove previous value
try {
$_gridFS->remove( $criteria);
}
catch(Exception $e) {}
// store new file content
$storeByteCompleted = false;
try {
$reportId = $_gridFS->storeBytes(
$binData,
array("_id" => $intReleaseId));
if ($reportId == $intReleaseId) {
$storeByteCompleted = true;
}
catch(Exception $e) {}
return $storeByteCompleted;
}
function loadFileData($intReleaseId) {
$gridfsFile = null;
$binData = null;
try {
$gridfsFile = $_gridFS->get($intReleaseId);
}
catch(Exception $e) {}
if ($gridfsFile != null) {
$binData = $gridfsFile->getBytes()
}
return $binData;
}
That's all.