I am trying to get around the 30 second cursor time limit and avoid a MongoCursorTimeoutException by using DB->command method as seen here: MongoCursorTimeoutException for aggregate function
When I try to aggregate, command instantly returns:
array(3) { ["ok"]=> float(0) ["errmsg"]=> string(42) "Invalid input resource, " ["code"]=> int(17138) } There is pretty much no documentation on this and the only think I could find is the source code where the error takes place https://github.com/mongodb/mongo/blob/master/src/mongo/db/pipeline/pipeline.cpp#L327
Can someone please help? The pipeline works fine if I run collection->aggregate(pipeline, options) so I don't think it is the pipeline. My code is below:
$connection = new MongoClient( 'mongodb://user:pass#mydb.biz' );
$summaryDB = $connection->selectDB('Summary');
$summaryCollection = $summaryDB->selectCollection('hitSummary');
//agg pipeline
$pipeline =
[
[
'$match' => [
'date' => [
'$gte' => $weekAgo,
'$lt' => $today,
]
]
], [
'$group' => [
'_id' => [
'client' => '$client',
'date' => '$date',
],
//conversions
'z1is' => [ '$sum' => '$actions.sales.import.z1.count' ],
'z2is' => [ '$sum' => '$actions.sales.import.z2.count' ],
'z3is' => [ '$sum' => '$actions.sales.import.z3.count' ],
'z1ps' => [ '$sum' => '$actions.sales.pixel.z1.count' ],
'z2ps' => [ '$sum' => '$actions.sales.pixel.z2.count' ],
'z3ps' => [ '$sum' => '$actions.sales.pixel.z3.count' ],
'z1as' => [ '$sum' => '$actions.sales.apiOnly.z1.count' ],
'z2as' => [ '$sum' => '$actions.sales.apiOnly.z2.count' ],
'z3as' => [ '$sum' => '$actions.sales.apiOnly.z3.count' ],
'z1ss' => [ '$sum' => '$actions.sales.s2s.z1.count' ],
'z2ss' => [ '$sum' => '$actions.sales.s2s.z2.count' ],
'z3ss' => [ '$sum' => '$actions.sales.s2s.z3.count' ],
//clicks
'z1ic' => [ '$sum' => '$actions.click.import.z1.count' ],
'z2ic' => [ '$sum' => '$actions.click.import.z2.count' ],
'z3ic' => [ '$sum' => '$actions.click.import.z3.count' ],
'z1pc' => [ '$sum' => '$actions.click.pixel.z1.count' ],
'z2pc' => [ '$sum' => '$actions.click.pixel.z2.count' ],
'z3pc' => [ '$sum' => '$actions.click.pixel.z3.count' ],
'z1ac' => [ '$sum' => '$actions.click.apiOnly.z1.count' ],
'z2ac' => [ '$sum' => '$actions.click.apiOnly.z2.count' ],
'z3ac' => [ '$sum' => '$actions.click.apiOnly.z3.count' ],
'z1sc' => [ '$sum' => '$actions.click.s2s.z1.count' ],
'z2sc' => [ '$sum' => '$actions.click.s2s.z2.count' ],
'z3sc' => [ '$sum' => '$actions.click.s2s.z3.count' ],
//impressions
'z1ii' => [ '$sum' => '$actions.display.import.z1.count' ],
'z2ii' => [ '$sum' => '$actions.display.import.z2.count' ],
'z3ii' => [ '$sum' => '$actions.display.import.z3.count' ],
'z1pi' => [ '$sum' => '$actions.display.pixel.z1.count' ],
'z2pi' => [ '$sum' => '$actions.display.pixel.z2.count' ],
'z3pi' => [ '$sum' => '$actions.display.pixel.z3.count' ],
'z1ai' => [ '$sum' => '$actions.display.apiOnly.z1.count' ],
'z2ai' => [ '$sum' => '$actions.display.apiOnly.z2.count' ],
'z3ai' => [ '$sum' => '$actions.display.apiOnly.z3.count' ],
'z1si' => [ '$sum' => '$actions.display.s2s.z1.count' ],
'z2si' => [ '$sum' => '$actions.display.s2s.z2.count' ],
'z3si' => [ '$sum' => '$actions.display.s2s.z3.count' ],
]
], [
'$project' => [
'impressions' => [ '$add' => ['$z1ii', '$z2ii', '$z3ii',
'$z1pi', '$z2pi', '$z3pi',
'$z1ai', '$z2ai', '$z3ai',
'$z1si', '$z2si', '$z3si'] ],
'clicks' => [ '$add' => ['$z1ic', '$z2ic', '$z3ic',
'$z1pc', '$z2pc', '$z3pc',
'$z1ac', '$z2ac', '$z3ac',
'$z1sc', '$z2sc', '$z3sc'] ],
'importSales' => [ '$add' => ['$z1is', '$z2is', '$z3is'] ],
'pixelSales' => [ '$add' => ['$z1ps', '$z2ps', '$z3ps'] ],
'apiSales' => [ '$add' => ['$z1as', '$z2as', '$z3as'] ],
's2sSales' => [ '$add' => ['$z1ss', '$z2ss', '$z3ss'] ],
]
]
];
//do something with this
$options = [ 'timeout' => -1 ];
$result = $summaryDB->command(
[
'aggregate' => $summaryCollection,
'pipeline' => $pipeline,
],
$options
);
var_dump($result);
When you run DB command, the first argument is the name of the command as the key and the value is the name of the collection.
In your case you should be passing "hitSummary" as the value of 'aggregate' and not the collection object.
See simple command example here: http://php.net/manual/en/mongodb.command.php
Related
I want to know how can I propery filter field from another collection that is linked from main collection.
Here is my filter values
if(!empty($filterDataBuyer['price'])){
$findArray['price'] = intval($filterDataBuyer['price']);
}
if(!empty($filterDataBuyer['search'])){
$findArray['full_name'] = $filterDataBuyer['search'];
}
}
$findArray['type'] = 'buyer';
The price and type works fine but the full_name does not work
And here is for getting the data
$getData = [
[
'$match' => $findArray
],
[
'$project' => [
'_id' => ['$toString' => '$_id'],
'type' => '$type',
'created_date' => '$created_date',
'offer_id' => '$offer_id',
'buyer_id' => '$buyer_id',
'traveler_id' => '$raveler_id',
'order_id' => '$order_id',
'price' => '$price'
]
],
[
'$lookup' => [
'from' => 'users',
'let' => [
'admin_id' => ['$toObjectId' => '$buyer_id'],
],
'pipeline' => [
[
'$match' => [
'$expr' => [
'$eq' => [
'$_id',
'$$admin_id'
]
],
],
],
[
'$project' => [
'_id' => ['$toString' => '$_id'],
'profile_image' => '$profile_image',
'full_name' => '$full_name'
]
],
],
'as' => 'profileData'
]
],
What I want to do is if full_name field is available, it could also filter the result by it, price and type can filter successfully but full_name is not. Result returns only if full_name has value on it.
My PHP version is 7.4.27 and mongodb extension is 1.12.0
This is just two collections, payment_details and users where payment_details has buyer_id field that is reference to users id field
From mongocompass:
payment_details:
users:
Full Code update:
if(!empty($filterDataBuyer['price'])){
/* FLIGHT-31 fix */
$findArray['price'] = intval($filterDataBuyer['price']);
}
if(!empty($filterDataBuyer['search'])){
$findArray['full_name'] = $filterDataBuyer['search'];
}
$findArray['type'] = 'buyer';
$buyer_trasections = $db->payment_details->find($findArray);
$buyerRes_trasections = iterator_to_array($buyer_trasections);
$getData = [
[
'$match' => $findArray
],
[
'$project' => [
'_id' => ['$toString' => '$_id'],
'type' => '$type',
'created_date' => '$created_date',
'offer_id' => '$offer_id',
'buyer_id' => '$buyer_id',
'traveler_id' => '$raveler_id',
'order_id' => '$order_id',
'price' => '$price'
]
],
[
'$lookup' => [
'from' => 'users',
'let' => [
'admin_id' => ['$toObjectId' => '$buyer_id'],
],
'pipeline' => [
[
'$match' => [
'$expr' => [
'$eq' => [
'$_id',
'$$admin_id'
]
],
],
],
[
'$project' => [
'_id' => ['$toString' => '$_id'],
'profile_image' => '$profile_image',
'full_name' => '$full_name'
]
],
],
'as' => 'profileData'
]
],
[
'$skip' => $page
],
[
'$limit' => $config['per_page'],
],
[
'$sort' => [ 'created_date'=> -1]
]
];
$buyer_trasec = $db->payment_details->aggregate($getData);
$buyerRes_trasec = iterator_to_array($buyer_trasec);
echo "<pre>";print_r($buyerRes_trasec);exit;
And this is the sample, first result for echo "<pre>";print_r($buyerRes_trasec);exit;
I am new to Elasticsearch and I am using REST API for PHP to play around with data returned. I am using following code to retrieve data.
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
Above code returns data perfectly matching "validated=>true" and "image=>true".
Further I want to add open text search like we use /_search/?q=Apple macbook. I have tried to use match, multi_match, query_string options, but couldn't get success.
So, I want to retrieve results from ES that have "validated=>true", "image=>true" and matches with text "Apple macbook".
Thanks in advance.
You can try with query_string or simple_query_string
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ],
[ 'query_string' => [ 'query' => 'Apple macbook' ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ],
[ 'simple_query_string' => [ 'query' => 'Apple macbook' ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
you can also do this by
enabling all_field mapping for your index, you can do that by following the below URL
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-all-field.html
and then use the below ES query:
$params = [
'index' => 'my_search',
'type' => 'mytype',
'from' => 0,
'size' => 10,
'body' => [
'query' => [
'bool' => [
'must' => [
[ 'match' => [ '_all' => 'Apple macbook' ] ],
[ 'match' => [ 'validated' => true ] ],
[ 'match' => [ 'image' => true ] ]
]
]
],
'sort' => [
'created_at' => [ 'order' => 'asc']
]
]
];
I have a data of stores, report_type is every type of data
- type 3 stored how much money that clients buy and how many transaction.
- type 5 stored how many people and average age of them.
I need an array like this:
"report_type3"
(
[_id] => 2441
[amountclient] => 3749078452,
[trans] => 1000
),
(
[_id] => 2442
[amountclient] => 15533754,
[trans] => 900
)
,
"report_type5"
(
[_id] => 2441
[people] => 120,
[avgage] => 23,
),
(
[_id] => 2441
[people] => 120,
[avgage] => 23,
)
My query looks like this, but it's not working
$cursor = $collection->aggregate([
['$match' => [ 'id_station' => ['$in' => $store ], 'report_type' => 3,
'date' => ['$gte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-07-01")*1000), '$lte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-08-05")*1000)]
]
],
['$group' => [
'_id' => [ 'id_station' => '$id_station', 'receiptid' => '$receiptid' ],
'amount' => [ '$sum' => '$amount' ]
]], .... *I group by receiptid because there are many items in one clients.*
[ '$group' => [
'_id' => '$_id.id_station',
'amount' => [ '$sum' => '$amount' ],
'trans' => [ '$sum' => '$trans' ]
]],
[
'$project' => [
'report_type3' => ['_id','$amountclient','$trans']
]
],
['$match' => [ 'id_station' => ['$in' => $store ], 'report_type' => 5,
'date' => ['$gte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-07-01")*1000), '$lte'=> new MongoDB\BSON\UTCDateTime(strtotime("2017-08-05")*1000)]
]
],
[ '$group' => [
'_id' => ['$_id.id_station', 'area_type' => '$area_type'],
'people' => [ '$sum' => '$people' ],
'avgage' => [ '$avg' => '$age' ]
]],
[
'$project' => [
'report_type5' => [ '_id', 'people', 'avgage' ]
]
]
]);
Could you please help me on this?
I am trying to add multiple criterion to ElasticSearch PHP in must clause, but it is giving me a 500 Error.
My params are:
$params = ['index' => 'smartjn',
'type' => 'feed_details',
'size' => 10,
'body' => [
'sort' => [
'sorter' => [
'order' => 'desc',
'mode' => 'max'
]
],
'query' => ['bool' => [
'must' => [
['match' => ['approvalStatus' => 'approved']],
[ '_id' => [
'$lt' => new MongoDB\BSON\ObjectId($documentid)
]
]
],
'must_not' => ['term' => ['muteFeedUserIds' => $userID]],
'must_not' => ['term' => ['muteUserIds' => $userID]]
]
]
]
];
Please note that this works if I remove the second criteria in must clause i.e.
$params = ['index' => 'smartjn',
'type' => 'feed_details',
'size' => 10,
'body' => [
'sort' => [
'sorter' => [
'order' => 'desc',
'mode' => 'max'
]
],
'query' => ['bool' => [
'must' => [
['match' => ['approvalStatus' => 'approved']]
],
'must_not' => ['term' => ['muteFeedUserIds' => $userID]],
'must_not' => ['term' => ['muteUserIds' => $userID]]
]
]
]
];
Any idea what am I missing?
Thanks
check this out.
'query' => ['bool' => [
'must' => [
['match' => ['approvalStatus' => 'approved']],
[ 'ids' => [
'values' => [new MongoDB\BSON\ObjectId($documentid)]
]
]
],
'must_not' => ['term' => ['muteFeedUserIds' => $userID]],
'must_not' => ['term' => ['muteUserIds' => $userID]]
]
If that doesnt work i am giving you the raw as i am not good with elasticphp
{
"query": {
"bool": {
"must": [
{
"ids": {
"type": "type_of_index"
"values": ["AVbqtjXfoKQ0pubF_-NA"]
}
}
]
}
}
}
type is optional
Just add this ids part under must area.
I'm trying to run a mongo query in my laravel app but receiving the error listed in the title of this post. I think I need to set useCursor to true but am having trouble getting it to work. Here's my code:
$email_duplicates = User::raw(function ($collection) {
return $collection->aggregate(
[
[
'$group' => [
'_id' => [
'email' => '$email',
],
'uniqueIds' => [
'$addToSet' => '$_id',
],
'count' => [
'$sum' => 1,
],
],
],
[
'$match' => [
'count' => [
'$gt' => 1,
],
],
],
],
[
'allowDiskUse' => true,
],
[
'useCursor' => true,
]
);
});
anyone know what I'm doing wrong?
Thanks!