Aggregate nested array in Mongodb - php

Question

I have a mongo collection lie this :
{
"_id":ObjectId("55f16650e3cf2242a79656d1"),
"user_id":11,
"push":[
ISODate("2015-09-08T11:14:18.285 Z"),
ISODate("2015-09-08T11:14:18.285 Z"),
ISODate("2015-09-09T11:14:18.285 Z"),
ISODate("2015-09-10T11:14:18.285 Z"),
ISODate("2015-09-10T11:14:18.285 Z")
]
}{
"_id":ObjectId("55f15c78e3cf2242a79656c3"),
"user_id":12,
"push":[
ISODate("2015-09-06T11:14:18.285 Z"),
ISODate("2015-09-05T11:14:18.285 Z"),
ISODate("2015-09-07T11:14:18.285 Z"),
ISODate("2015-09-09T11:14:18.285 Z"),
ISODate("2015-09-09T11:14:18.285 Z"),
ISODate("2015-09-10T11:14:18.285 Z"),
ISODate("2015-09-11T11:14:18.285 Z")
]
}
How can I find user_ids where count of timeStamps < 3 and having date(timestamp) > (currentDate-5) in single query. I will be using php and dont want to bring all the documents in memory.
Explanation:
user_id : date : count
11 : 2015-09-08 : 2
2015-09-09 : 1
2015-09-10 : 2
12 : 2015-09-05 : 1
2015-09-06 : 1
2015-09-07 : 1
2015-09-09 : 2
2015-09-10 : 1
2015-09-11 : 1
If date set to 2015-09-09(user input) it will give 3(count) for user_id 11 and 4(count) for user_id 12. So suppose count is set to 3(user input). The query should return 11(user_id). If count is set to 2, there will be no user_id available and if count is set to 5, it should return both 11 and 12

Answer 1

To solve this you need an aggregation pipeline that first "filters" the results to the "last 5 days" and then essentially "sums the count" of array items present in each qualifying document to then see if the "total" is "less than three".
The $size operator of MongoDB aggregation really helps here, as does $map and some additional filtering via $setDifference for the false results returned from $map, as doing this "in document first" and "within" the $group stage required, is the most efficient way to process this
$result = $collection->aggregate(array(
array( '$match' => array(
'push' => array(
'time' => array(
'$gte' => MongoDate( strtotime('-5 days',time()) )
)
)
)),
array( '$group' => array(
'_id' => '$user_id',
'count' => array(
'$sum' => array(
'$size' => array(
'$setDifference' => array(
array( '$map' => array(
'input' => '$push',
'as' => 'time',
'in' => array(
'$cond' => array(
array( '$gte' => array(
'$$time',
MongoDate( strtotime('-5 days',time()) )
)),
'$time',
FALSE
)
)
)),
array(FALSE)
)
)
)
)
)),
array( '$match' => array(
'count' => array( '$lt' => 3 )
))
));
So the after all of the work to first find the "possible" documents that contain array entries meeting the criteria via $match and then find the "total" size of the matched array items under $group, then the final $match excludes all results that are less than three in total size.
For the largely "JavaScript brains" out there ( like myself, well trained into it ) this is basically this contruct:
db.collection.aggregate([
{ "$match": {
"push": {
"$gte": new Date( new Date().valueOf() - ( 5 * 1000 * 60 * 60 * 24 ))
}
}},
{ "$group": {
"_id": "$user_id",
"count": {
"$sum": {
"$size": {
"$setDifference": [
{ "$map": {
"input": "$push",
"as": "time",
"in": {
"$cond": [
{ "$gte": [
"$$time",
new Date(
new Date().valueOf() -
( 5 * 1000 * 60 * 60 * 24 )
)
]},
"$$time",
false
]
}
}},
[false]
]
}
}
}
}},
{ "$match": { "count": { "$lt": 3 } } }
])
Also, future versions of MongoDB will offer $filter, which simplifies the whole $map and $setDifference statement part:
db.collection.aggregate([
{ "$match": {
"push": {
"$gte": new Date( new Date().valueOf() - ( 5 * 1000 * 60 * 60 * 24 ))
}
}},
{ "$group": {
"_id": "$user_id",
"count": {
"$sum": {
"$size": {
"$filter": {
"input": "$push",
"as": "time",
"cond": {
"$gte": [
"$$time",
new Date(
new Date().valueOf() -
( 5 * 1000 * 60 * 60 * 24 )
)
]
}
}
}
}
}
}},
{ "$match": { "count": { "$lt": 3 } } }
])
As well as noting that the "dates" are probably best calculated "before" the pipeline definition as a separate variable for the best accuracy.