I have the following query working in mongoDB but its not working in PHP.
MongoDB Query
db.energy_meter.aggregate(
{
$unwind: {
path:"$KeyValues",
includeArrayIndex:"arrayIndex",
preserveNullAndEmptyArrays:true
}
},
{
$project: {
timestamp:{
"$add":["$EventTS",{"$multiply":[60000,"$arrayIndex"]}]
} ,
"RPhaseVoltage":"$KeyValues.RPhaseVoltage",
arrayIndex:1,
}
}
);
Above query is converted to PHP
$cursor = DB::collection('energy_meter')->raw(function($collection)
{
return $collection->aggregate([
[
'$unwind' =>
['path' => '$KeyValues'],
['includeArrayIndex' => 'arrayIndex'],
['preserveNullAndEmptyArrays' => 'true']
],
[
'$project' =>
[
'timestamp' => [
'$add' => [
'$EventTS',
['$multiply' => [60000, '$arrayIndex']]
]
]
],
[
'MainsInputVoltagev' => ['$KeyValues.MainsInputVoltagev']
],
[
'arrayIndex' => 1
]
]
]);
});
I am getting following error
RuntimeException in Aggregate.php line 168: A pipeline stage specification object must contain exactly one field.
What is problem in my converted php query? Please suggest resolution of above problem.
You should always convert normal query to array decode. json_decode should make query for PHP driver and json_encode should give query mongodb query parameters.
(
{
$unwind: {
path:"$KeyValues",
includeArrayIndex:"arrayIndex",
preserveNullAndEmptyArrays:true
}
},
{
$project: {
timestamp:{
"$add":["$EventTS",{"$multiply":[60000,"$arrayIndex"]}]
} ,
"RPhaseVoltage":"$KeyValues.RPhaseVoltage",
arrayIndex:1,
}
}
)
Like this :
array(
array(
'$unwind' => array(
'path' => '$KeyValues',
'includeArrayIndex' =>"arrayIndex",
'preserveNullAndEmptyArrays'=> true
)
),
array(
'$project' => array(
'timestamp' => array(
'$add'=>[ '$EventTS',array('$multiply'=>[60000,'$arrayIndex'])]
) ,
"RPhaseVoltage" => '$KeyValues.RPhaseVoltage',
'arrayIndex' =>1,
)
)
)
If you have at least PHP5.4, you can use simpler array syntax. Replace array( with [ and ) with ] for array.
[
[
'$unwind' => [
'path' => '$KeyValues',
'includeArrayIndex' => 'arrayIndex',
'preserveNullAndEmptyArrays' => 'true'
]
],
[
'$project' => [
'timestamp' => [
'$add' => [
'$EventTS',
[ '$multiply' => [60000, '$arrayIndex'] ]
]
],
'MainsInputVoltagev' => '$KeyValues.MainsInputVoltagev',
'arrayIndex' => 1
]
]
]
Related
MongoDB 3.4
I have a project and project_permission collections. The project_permission collection contains permissions to the projects for some users. A single user can have multiple different permissions to the project.
[
'$lookup' => [
'from' => ProjectPermission::collectionName(),
'localField' => '_id',
'foreignField' => 'project_id',
'as' => 'project_permissions'
]
],
[
'$project' => [
// ... irrelevant fields here
'permissions' => '$project_permissions'
]
],
this is how the project query results looks like without filtering:
// other project results
// ... other fields
'permissions' => [
0 => [
'_id' => '5d2873aafa873b2b7c000fad'
'project_id' => '56a9e5c5d18cacc72a485839'
'user_id' => '562f6bfc05dfe9570fb6e427'
'permission' => 'read'
'created_at' => 1562932138
'updated_at' => 1562932139
]
1 => [
'_id' => '5d2879fdfa873b2b7c000fbd'
'project_id' => '56a9e5c5d18cacc72a485839'
'user_id' => '562f6bfc05dfe9570fb6e427'
'permission' => 'write'
'created_at' => 1562932139
'updated_at' => 1562932140
]
2 => [
'_id' => '5db960b5fa873b1604005e8e'
'project_id' => '56a9e5c5d18cacc72a485839'
'user_id' => '582b30dd1e634e6362e1b504'
'permission' => 'write'
'created_at' => 1572430005
'updated_at' => 1572430005
]
]
What I would like to achieve is to return with only those projects where the client - who requested the query - has a specific permission to the project, for example write.
The way I tried it:
pipeline: [
0 => [
'$match' => [
// not related to the problem
]
]
1 => [
'$match' => [
'$and' => [
0 => [
'shared_permissions' => [
'$eq' => true
]
]
1 => [
'$or' => [
0 => [
'project_permissions' => [
'$exists' => true
'$ne' => []
]
]
1 => [
'owner_id' => [
'$ne' => MongoDB\BSON\ObjectId#1
(
[oid] => '582b30dd1e634e6362e1b504'
)
]
]
]
]
]
]
]
2 => [
'$lookup' => [
'from' => 'project_permission'
'localField' => '_id'
'foreignField' => 'project_id'
'as' => 'project_permissions'
]
]
3 => [
'$project' => [
// more not important fields here
'shared_permissions' => 1
'permissions' => [
'$map' => [
'input' => [
'$filter' => [
'input' => '$project_permissions'
'as' => 'project_permission'
'cond' => [
'$and' => [
0 => [
'$eq' => ['$$project_permission.user_id', MongoDB\BSON\ObjectId#1
(
[oid] => '582b30dd1e634e6362e1b504'
)
]
1 => [
'$eq' => ['$$project_permission.permission', 'write']
]
]
]
]
]
'as' => 'project_permission'
'in' => [
'user_id' => '$$project_permission.user_id'
'permission' => '$$project_permission.permission'
]
]
]
]
]
]
For this I almost get the correct response:
[
0 => [
'_id' => '56a9e5c5d18cacc72a485839'
'short_id' => 3
'title' => 'Modified title'
'owner_id' => '562f692a05dfe9560fb6e428'
'updated_at' => 1572435428
'owner_name' => 'Borat Sagdiyev'
'shared_permissions' => true
'permissions' => [
0 => [
'user_id' => '582b30dd1e634e6362e1b504'
'permission' => 'write'
]
]
]
1 => []
]
The problem with this is that empty array, where the result was filtered out - and it wouldn't be a problem if the empty array wouldn't be in the result, because if I use the pagination, then it says two results, instead of one. And we know that in the worst case we would get back an array of empty arrays only.
So what I would like to achieve is this last example results without empty arrays in a way where the pagination will be fine with it too.
ps.: unwind is not an option, because of some structural conventions.
Any ideas?
To perform an equality match between a field from the input documents with a field from the documents of the “joined” collection, the $lookup stage has the following syntax:
{
$lookup:
{
from: <collection to join>,
localField: <field from the input documents>,
foreignField: <field from the documents of the "from" collection>,
as: <output array field>
}
}
I am using Elastic search 5.x and the following code is working fine:
curl -XPOST "http://localhost:9200/test_index/test_info/_delete_by_query" -d'
{
"query": {
"match": {
"category_id": "21"
}
}
}'
But when I am trying the same in my php code, its not working:
$client->deleteByQuery([
'index' => 'test_index',
'type' => 'test_info',
'query' => [
'match' => [
['category_id' => 21]
]
]
]);
You need to provide your query array inside body array of your parameters:
$client->deleteByQuery([
'index' => 'test_index',
'type' => 'test_info',
'body' => [
'query' => [
'match' => [
['category_id' => 21]
]
]
]
]);
this an old question, previous comments don't work anymore in 2020 :
$client->deleteByQuery([
'index' => 'test_index',
(there were a type here) 'type' => 'test_info',
'body' => [
'query' => [
'match' => [
(there were an array here) ['category_id' => 21]
]
]
]
]);
So the final code is :
$client->deleteByQuery([
'index' => 'test_index',
'body' => [
'query' => [
'match' => [
'category_id' => 21
]
]
]
I'm triying to get all the logs around a expecific geopoint, and group them by a subfield (context.id) but I'm having problems. I tried the nested aggregation, etc.. but I'm having no luck. I'm using the PHP library so I wrote the query as a php array. All is working until addign the aggregation query.
The exception throw is :
illegal_state_exception: Field data loading is forbidden on [context.id]
$params = [
'index' => 'logstash-*',
'type' => 'INFO',
'body' => [
'query' => [
'bool' => [
"must" => [
["term" => ["tags" => "producer"]],
["term" => ["tags" => "statistics"]],
["term" => ["message" => "view"]],
],
"filter" => [
"geo_distance" => [
"distance" => "10km",
"distance_type" => "plane",
"geoip.location" => [
"lat" => 40.4326058,
"lon" => -3.6996032
]
]
]
]
],
"aggs" => [
"context" => [
"nested" => [
"path" => "context"
],
"aggs" => [
"group_by_id" => [
"terms" => ["field" => "context.id"]
]
]
]
],
]
];
Can someone point me to the right query?
Finally, the problem here was the field. It was a indexed field I need to use the "context.id.raw" field instead.
I have this code what executes an query.
I am using the official php elastic search library.
https://github.com/elastic/elasticsearch-php
The field "Tijdsperiode" is in this format : ( 2016-01-30 00:00:00 ) ( YY-MM-DD HH:MM:SS )
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'body' => [
'query' => [
'match_all' => [],
'filter' => [
'range' => [
'Tijdsperiode' => [
'gte' => '2016-01-30 01:00:00',
'lte' => '2016-01-30 08:00:00'
]
]
]
],
],
];
$response = $client->search($params);
var_dump($response);
I just wonder what the format need to be to get results between the 2 dates.
When i do this :
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'body' => [
'query' => [
'match_all' => [],
],
],
];
It's working fine but i need the results between the 2 dates!
I also tried this but with no result :
$json = '{
"query": {
"bool": {
"must": [
{
"range": {
"Tijdsperiode": {
"gt": "2016-01-30 07:00:00",
"lt": "2016-01-30 09:00:00"
}
}
}
]
}
}
}';
$client = ckan_graphmapper_client();
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'body' => $json
];
$response = $client->search($params);
I also tried it with a mapping :
$params = [
'index' => 'veenendaal2',
'type' => 'passanten2',
'size' => $size,
'body' => [
'query' => [
"match_all" => [],
'filter' => [
'range' => [
'Tijdsperiode' => [
'gte' => '2016-01-30 01:00:00',
'lte' => '2016-01-30 08:00:00'
]
]
]
],
'mappings' => [
'_default_' => [
'properties' => [
'Tijdsperiode' => [
'type' => 'date',
'format' => 'yyyy-MM-dd HH:mm:ss'
]
]
]
]
]
];
How to do the right syntax for choosing between 2 dates?. Both string formats ( 2016-01-30 00:00:00 ) ( YY-MM-DD HH:MM:SS )
Thanks!
So it turns out my import of data was not mapping the correct way.
The date field was recognised as a string value.
When i finnaly had the correct mapping in elastic i could do this query :
$query = [
'query' => [
'filtered' => [
'query' => [
'match_all' => []
],
'filter' => [
'range' => [
'Tijdsperiode' => [
'gte' => $start,
'lte' => $end
]
]
]
]
]
];
I have the following (working) MongoDB query to generate a list of the hashtag count.
db.twitter.aggregate([
{
$group: {
_id: "$status.entities.hashtags.text",
hashtags: {
$addToSet : "$status.entities.hashtags.text"
}
}
},
{ $unwind : "$hashtags" },
{ $unwind : "$hashtags" },
{ $group : { _id : "$hashtags", count: { $sum : 1 } } },
{ $sort : { count : -1, _id : 1 } }
]);
Now I try to convert this query to PHP code (for laravel):
$cursor = DB::collection('twitter')->raw(function($collection)
{
return $collection->aggregate(array(
array(
'$group' => array(
'_id' => '$status.entities.hashtags.text',
'hashtags' => array(
'$addToSet' => '$status.entities.hashtags.text',
),
),
),
array(
'$unwind' => '$hashtags',
),
array(
'$unwind' => '$hashtags',
),
array(
'$group' => array(
'_id' => '$hashtags', '
count' => array(
'$sum => 1',
),
),
),
array(
'$sort' => array(
'count' => '-1',
'_id' => '1',
),
),
));
});
dd($cursor);
What I can derive from the Laravel-MongoDB docs is that the raw query input works the same as in PHP mongodb.
The error returned is this:
MongoResultException (15951)
localhost:27017: exception: the group aggregate field 'count' must be defined as an expression inside an object
You solved this but I can tell you where you was wrong:
'$sum => 1',
Should be:
array('$sum' => 1)
Rewrote the array and now it works:
$cursor = DB::collection('twitter')->raw(function($collection)
{
return $collection->aggregate([
[
'$group' => [
'_id' => '$status.entities.hashtags.text',
'hashtags' => [
'$addToSet' => '$status.entities.hashtags.text'
]
]
],
[ '$unwind' => '$hashtags' ],
[ '$unwind' => '$hashtags' ],
[ '$group' => [ '_id' => [ '$toLower' => '$hashtags' ], 'count' => [ '$sum' => 1 ] ] ],
[ '$sort' => [ 'count' => -1, '_id' => 1 ] ]
]);
});
Just replaced the {} by [] and the : by => and that did the trick!