I have the following (working) MongoDB query to generate a list of the hashtag count.
db.twitter.aggregate([
{
$group: {
_id: "$status.entities.hashtags.text",
hashtags: {
$addToSet : "$status.entities.hashtags.text"
}
}
},
{ $unwind : "$hashtags" },
{ $unwind : "$hashtags" },
{ $group : { _id : "$hashtags", count: { $sum : 1 } } },
{ $sort : { count : -1, _id : 1 } }
]);
Now I try to convert this query to PHP code (for laravel):
$cursor = DB::collection('twitter')->raw(function($collection)
{
return $collection->aggregate(array(
array(
'$group' => array(
'_id' => '$status.entities.hashtags.text',
'hashtags' => array(
'$addToSet' => '$status.entities.hashtags.text',
),
),
),
array(
'$unwind' => '$hashtags',
),
array(
'$unwind' => '$hashtags',
),
array(
'$group' => array(
'_id' => '$hashtags', '
count' => array(
'$sum => 1',
),
),
),
array(
'$sort' => array(
'count' => '-1',
'_id' => '1',
),
),
));
});
dd($cursor);
What I can derive from the Laravel-MongoDB docs is that the raw query input works the same as in PHP mongodb.
The error returned is this:
MongoResultException (15951)
localhost:27017: exception: the group aggregate field 'count' must be defined as an expression inside an object
You solved this but I can tell you where you was wrong:
'$sum => 1',
Should be:
array('$sum' => 1)
Rewrote the array and now it works:
$cursor = DB::collection('twitter')->raw(function($collection)
{
return $collection->aggregate([
[
'$group' => [
'_id' => '$status.entities.hashtags.text',
'hashtags' => [
'$addToSet' => '$status.entities.hashtags.text'
]
]
],
[ '$unwind' => '$hashtags' ],
[ '$unwind' => '$hashtags' ],
[ '$group' => [ '_id' => [ '$toLower' => '$hashtags' ], 'count' => [ '$sum' => 1 ] ] ],
[ '$sort' => [ 'count' => -1, '_id' => 1 ] ]
]);
});
Just replaced the {} by [] and the : by => and that did the trick!
Related
I have the following MongoDB query:
db.crimes.aggregate([
{ $match: {"CrimeLSOAName":/.*Bradford.*/} },
{
$group: {
"_id": "$CrimeType",
"count": {
$sum: 1
}
}
},
{
$sort: {
"count": -1
}
},
{
$limit: 10
}
])
Which outputs the ten most common 'CrimeTypes' where CrimeLSOAName field contains 'Bradford'. I am trying to use the query in PHP, my attempts are below but the query does not run correctly. If someone could give some help it would be appreciated.
Trying to use in PHP:
$top10Crimes = array(
array('$match') => array('CrimeLSOAName:/.*Bradford.*/'),
array(('$group') => array('_id' => '$CrimeType',
'count' => '$sum: 1')),
array('$sort') => array('count' => '-1')
array('$limit' => '10')
);
$result = $collection->aggregate($top10Crimes);
Your array is invalid, this should work. It usually helps to format your array similar to the pipeline you have in the mongo shell.
$top10Crimes = array(
array(
'$match' => array(
'CrimeLSOAName' => '.*Bradford.*/'
)
),
array(
'$group' => array(
'_id' => '$CrimeType',
'count' => array(
'$sum' => 1
)
)
),
array(
'$sort' => array(
'count' => '-1'
)
),
array(
'$limit' => '10'
)
);
I try to sort my documents per name using elastic search & the official php client , how i can proceed ?
$params = [
'index' => $this->index ,
'type' => 'videos',
'from' => $this->uri->segment(2),
'size' => 12,
'body' => [
'query' => [
'filtered' => [
'filter' => [
'term' => [ 'name' => $query ] ,
'term' => [ 'tags' => $query ]
]
]
]
]
];
$data['results'] = $this->client->search($params);
I know this question is over a year old, but the answer is not easy to find on the internet, so I'll answer it anyway.
To specify the field to sort on and the order to sort in, use the following syntax:
$params['sort'] = array('updated_at:desc');
To sort on multiple fields:
$params['sort'] = array('updated_at:desc', 'user_id:asc', ...);
I just saw this post while searching for an answer to the same question, in my case the solution was much simpler and different to the ones I saw here.
$params['body']['sort'] = [ 'id' => 'desc']
this worked fine for me using "elasticsearch/elasticsearch": "^6.1"
Try this
$params = [
'index' => $this->index ,
'type' => 'videos',
'from' => $this->uri->segment(2),
'size' => 12,
'body' => [
'query' => [
'filtered' => [
'filter' => [
'term' => [ 'name' => $query ] ,
'term' => [ 'tags' => $query ]
]
]
],
'sort' => [
'name' => [
'order' => 'asc'
]
]
]
];
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-sort.html
If you have unanalyzed keyword you must use it without enabling fielddata:
$params = [
'index' => $this->index ,
'type' => 'videos',
'from' => $this->uri->segment(2),
'size' => 12,
'body' => [
'query' => [
'filtered' => [
'filter' => [
'term' => [ 'name' => $query ] ,
'term' => [ 'tags' => $query ]
]
]
],
'sort' => [
'name' => [
'order.keyword' => 'asc'
]
]
]
];
But if you haven't the unanalyzed keyword you should reset your index mapping to enable fielddata to name field:
curl -X PUT "localhost:9200/my_index/_mapping" -H 'Content-Type: application/json' -d'
{
"properties": {
"name": {
"type": "text",
"fielddata": true
}
}
}
'
see : https://www.elastic.co/guide/en/elasticsearch/reference/current/fielddata.html
I have the following query working in mongoDB but its not working in PHP.
MongoDB Query
db.energy_meter.aggregate(
{
$unwind: {
path:"$KeyValues",
includeArrayIndex:"arrayIndex",
preserveNullAndEmptyArrays:true
}
},
{
$project: {
timestamp:{
"$add":["$EventTS",{"$multiply":[60000,"$arrayIndex"]}]
} ,
"RPhaseVoltage":"$KeyValues.RPhaseVoltage",
arrayIndex:1,
}
}
);
Above query is converted to PHP
$cursor = DB::collection('energy_meter')->raw(function($collection)
{
return $collection->aggregate([
[
'$unwind' =>
['path' => '$KeyValues'],
['includeArrayIndex' => 'arrayIndex'],
['preserveNullAndEmptyArrays' => 'true']
],
[
'$project' =>
[
'timestamp' => [
'$add' => [
'$EventTS',
['$multiply' => [60000, '$arrayIndex']]
]
]
],
[
'MainsInputVoltagev' => ['$KeyValues.MainsInputVoltagev']
],
[
'arrayIndex' => 1
]
]
]);
});
I am getting following error
RuntimeException in Aggregate.php line 168: A pipeline stage specification object must contain exactly one field.
What is problem in my converted php query? Please suggest resolution of above problem.
You should always convert normal query to array decode. json_decode should make query for PHP driver and json_encode should give query mongodb query parameters.
(
{
$unwind: {
path:"$KeyValues",
includeArrayIndex:"arrayIndex",
preserveNullAndEmptyArrays:true
}
},
{
$project: {
timestamp:{
"$add":["$EventTS",{"$multiply":[60000,"$arrayIndex"]}]
} ,
"RPhaseVoltage":"$KeyValues.RPhaseVoltage",
arrayIndex:1,
}
}
)
Like this :
array(
array(
'$unwind' => array(
'path' => '$KeyValues',
'includeArrayIndex' =>"arrayIndex",
'preserveNullAndEmptyArrays'=> true
)
),
array(
'$project' => array(
'timestamp' => array(
'$add'=>[ '$EventTS',array('$multiply'=>[60000,'$arrayIndex'])]
) ,
"RPhaseVoltage" => '$KeyValues.RPhaseVoltage',
'arrayIndex' =>1,
)
)
)
If you have at least PHP5.4, you can use simpler array syntax. Replace array( with [ and ) with ] for array.
[
[
'$unwind' => [
'path' => '$KeyValues',
'includeArrayIndex' => 'arrayIndex',
'preserveNullAndEmptyArrays' => 'true'
]
],
[
'$project' => [
'timestamp' => [
'$add' => [
'$EventTS',
[ '$multiply' => [60000, '$arrayIndex'] ]
]
],
'MainsInputVoltagev' => '$KeyValues.MainsInputVoltagev',
'arrayIndex' => 1
]
]
]
I have a user collection with following data
[
{
"user_id": "5625c95ac2d34f27148b64fa",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "562744ccc2d34f27148b6eb7",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "56248eb9c2d34f2f148b5a18",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "561f40bac2d34f17148b462c",
"friend_id": "561f3e06c2d34f27148b45f6"
},
{
"user_id": "561f40bac2d34f17148b462c",
"friend_id": "5620de97c2d34f2f148b578f"
},
{
"user_id": "56276b52c2d34f27148b7128",
"friend_id": "561f40bac2d34f17148b462c"
},
{
"user_id": "561f40bac2d34f17148b462c",
"friend_id": "56276b52c2d34f27148b7128"
}
]
i need to fetch the documents in which combination of user_id and friend_id not repeated. i.e in the above example last two documents user_id repeated in friend_id of next document.
I tried with mongo aggrigate and group by but could not reduce it.
In order to do this you basically need to combine both user_id and friend_id values in a uniquely sorted combination. This means creating an array for each document with those members and sorting that array so that the order is always the same.
Then you can $group on that sorted array content to see which documents contain that same combination and then only return those that do not share that same combination.
This leads to this aggregate statement:
db.collection.aggregate([
{ "$project": {
"user_id": 1,
"friend_id": 1,
"combined": {
"$map": {
"input": ["A","B"],
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el", "A" ] },
"$user_id",
"$friend_id"
]
}
}
}
}},
{ "$unwind": "$combined" },
{ "$sort": { "combined": 1 } },
{ "$group": {
"_id": "$_id",
"combined": { "$push": "$combined" },
"user_id": { "$first": "$user_id" },
"friend_id": { "$first": "$friend_id" }
}},
{ "$group": {
"_id": "$combined",
"docs": { "$push": {
"_id": "$_id",
"user_id": "$user_id",
"friend_id": "$friend_id"
}}
}},
{ "$redact": {
"$cond": {
"if": { "$ne": [{ "$size": "$docs" }, 1] },
"then": "$$PRUNE",
"else": "$$KEEP"
}
}}
])
The PHP translation for laravel means to need to access the raw collection object from the manager, where "collection" is the actual name of the collection in MongoDB:
$result = DB::collection("collection")->raw(function($collection) {
return $collection->aggregate(
array(
array(
'$project' => array(
'user_id' => 1,
'friend_id' => 1,
'combined' => array(
'$map' => array(
'input' => array("A","B"),
'as' => 'el',
'in' => array(
'$cond' => array(
array( '$eq' => array( '$el', 'A' ) ),
'$user_id',
'$friend_id'
)
)
)
)
)
),
array( '$unwind' =>'$combined' ),
array( '$sort' => array( 'combined' => 1 ) ),
array(
'$group' => array(
'_id' => '$_id',
'combined' => array( '$push' => '$combined' ),
'user_id' => array( '$first' => '$user_id' ),
'friend_id' => array( '$first' => '$friend_id' )
)
),
array(
'$group' => array(
'_id' => '$combined',
'docs' => array(
'$push' => array(
'_id' => '$_id',
'user_id' => '$user_id',
'friend_id' => 'friend_id'
)
)
)
),
array(
'$redact' => array(
'$cond' => array(
'if' => array( '$ne' => array( array( '$size' => '$docs'), 1) ),
'then' => '$$PRUNE',
'else' => '$$KEEP'
)
)
)
)
);
});
Or if your MongoDB version is less than 2.6, and you lack operators like $map and $redact, then you can still do this, but not as efficiently:
$result = DB::collection("collection")->raw(function($collection) {
return $collection->aggregate(
array(
array(
'$project' => array(
'user_id' => 1,
'friend_id' => 1,
'type' => array( '$const' => array( 'A', 'B' ) )
)
),
array( '$unwind' => '$type' ),
array(
'$group' => array(
'_id' => '$_id',
'user_id' => array( '$first' => '$user_id' ),
'friend_id' => array( '$first' => '$friend_id' ),
'combined' => array(
'$push' => array(
'$cond' => array(
array( '$eq' => array( '$type', 'A' ) ),
'$user_id',
'$friend_id'
)
)
)
)
)
array( '$unwind' =>'$combined' ),
array( '$sort' => array( 'combined' => 1 ) ),
array(
'$group' => array(
'_id' => '$_id',
'combined' => array( '$push' => '$combined' ),
'user_id' => array( '$first' => '$user_id' ),
'friend_id' => array( '$first' => '$friend_id' )
)
),
array(
'$group' => array(
'_id' => '$combined',
'docs' => array(
'$push' => array(
'_id' => '$_id',
'user_id' => '$user_id',
'friend_id' => 'friend_id'
)
),
'count' => array( '$sum' => 1 )
)
),
array( '$match' => array( 'count' => 1 ) )
)
);
});
Where the first three stages mimic what the first stage is doing in the first example listing by putting both values in a single array. Of course the last two stages by "counting" the array members while grouping and then filtering out anything that does not have a "count" of 1.
In either case this leaves you with output that only lists the documents where that combination does not occur in either order:
{
"_id" : [ "561f40bac2d34f17148b462c", "5625c95ac2d34f27148b64fa" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0cc9"),
"user_id" : "5625c95ac2d34f27148b64fa",
"friend_id" : "561f40bac2d34f17148b462c"
}
]
}
{
"_id" : [ "561f3e06c2d34f27148b45f6", "561f40bac2d34f17148b462c" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0ccc"),
"user_id" : "561f40bac2d34f17148b462c",
"friend_id" : "561f3e06c2d34f27148b45f6"
}
]
}
{
"_id" : [ "561f40bac2d34f17148b462c", "56248eb9c2d34f2f148b5a18" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0ccb"),
"user_id" : "56248eb9c2d34f2f148b5a18",
"friend_id" : "561f40bac2d34f17148b462c"
}
]
}
{
"_id" : [ "561f40bac2d34f17148b462c", "5620de97c2d34f2f148b578f" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0ccd"),
"user_id" : "561f40bac2d34f17148b462c",
"friend_id" : "5620de97c2d34f2f148b578f"
}
]
}
{
"_id" : [ "561f40bac2d34f17148b462c", "562744ccc2d34f27148b6eb7" ],
"docs" : [
{
"_id" : ObjectId("56306f6cd2387ad4c95b0cca"),
"user_id" : "562744ccc2d34f27148b6eb7",
"friend_id" : "561f40bac2d34f17148b462c"
}
]
}
You can pretty up the output, but this serves the purpose of showing the ordered combination used along with the original document data.
i am failing to convert the following mongoDB console command:
db.customers.aggregate( [
{ $group : {
_id: {
year : { $year: "$since" },
month : { $month: "$since" }
},
count: { $sum: 1 }
}
}]
);
which works into php
$customers->aggregate(array(
'$group' => array(
'_id' => array( 'year' => array('$year' => '$since'),
'month' => array('$month' => '$since')
)
),
array(
'count' => array( '$sum' => 1 )
),
)
);
which returns exception: A pipeline stage specification object must contain exactly one field.
also already tried '"$since"' with no luck
The count field must be a part of the group.
$customers->aggregate(array(
'$group' => array(
'_id' => array( 'year' => array('$year' => '$since'),
'month' => array('$month' => '$since')
),
'count' => array( '$sum' => 1 )
)
));