I have a collection with documents that look like this:
{
_id: ObjectId("516eb5d2ef4501a804000000"),
accountCreated: "2013-04-17 16:46",
accountLevel: 0,
responderCount: 0
}
I want to group and count these documents based on the accountCreated date (count per day), but I am stuck with the handling of dates since the date includes time as well.
This is what I have, but it returns the count including the time, witch means lots of entries always with 1 as accounts.
$g = $form->mCollectionUsers->aggregate(array(
array( '$group' => array( '_id' => '$accountCreated', 'accounts' => array( '$sum' => 1 ) ) )
));
Is there a way to rewrite the date to only take day in account and skip the time?
I have found this example but I canĀ“t really get figure out how to adapt it to this example.
If accountCreated is a date you can do it like this (I'll use the mongo shell syntax since I'm not familiar with the php driver):
db.mCollectionUsers.aggregate([
{$project :{
day : {"$dayOfMonth" : "$accountCreated"},
month : {"$month" : "$accountCreated"},
year : {"$year" : "$accountCreated"}
}},
{$group: {
_id : {year : "$year", month : "$month", day : "$day"},
accounts : { "$sum" : 1}
}}
]);
If you want to display the date properly:
db.mCollectionUsers.aggregate([
{
$group: {
_id: { $dateToString: { format: '%Y-%m-%d', date: '$accountCreated' } },
count: { $sum: 1 }
}
},
{
$project: {
_id: 0,
date: '$_id',
count: 1
}
}
])
The result would look like:
[
{
"date": "2020-11-11",
"count": 8
},
{
"date": "2020-11-13",
"count": 3
},
{
"date": "2020-11-16",
"count": 3
},
]
Related
I got an Affiliate model who has a hasMany relationship with Booking.
Now I want to bring all the affiliates with their bookings where the booking date is between 2 given dates and only bring the bookings that comply with that constraint.
now I got this
$affiliate = Affiliate::whereHas('bookings', static function (Builder $builder) use ($params) {
$builder->whereOrAny('date','>=', $params['dateFrom']);
$builder->whereOrAny('date','<=', $params['dateTo']);
})->with('bookings');
and this does a good job filtering the affiliates that contains a booking that has the date between the ones in the params, BUT I want to also filter the bookings so the resulting $affiliates->bookings only contains the bookings that have a date between those given dates
by example, I have something like this
$affiliates = [
{
name : 'Example 1',
bookings : [
{
'date' : 2020-06-01
},
{
'date' : 2019-05-01
},
{
'date' : 2019-04-01
}
]
},
{
name : 'Example 2',
bookings : [
{
'date' : 2020-06-01
},
{
'date' : 2020-05-01
},
{
'date' : 2020-04-01
}
]
},
{
name : 'Example 3',
bookings : [
{
'date' : 2019-06-01
},
{
'date' : 2020-05-01
},
{
'date' : 2020-04-01
}
]
}
]
and If I filter with fromDate = 2019-01-01 and toDate = 2019-31-12 (any date in 2019) return something like this
$affiliatesFiltered = [
{
name : 'Example 1',
bookings : [
{
'date' : 2019-05-01
},
{
'date' : 2019-04-01
}
]
},
{
name : 'Example 3',
bookings : [
{
'date' : 2019-06-01
},
]
}
]
as you can see not only I need to filter the affiliates to only bring the ones who have bookings between those dates but also bring only the bookings between those dates I hope you guys can help me
Ok I got a solution thanks to some of the comments, in case someone is wondering how to do this It would be something like this
$affiliate->with(['bookings' => function($query) use($params) {
$query->where('date','>=', $params['dateFrom'])
->where('date','<=', $params['dateTo'])
;
}])
and it works like a charm :) thanks a lot!
Assuming your bookings model also has the dateFrom and dateTo properties, this should work for you in order to filter your relationship as well:
...
->with(['bookings' => function ($q) use ($params){
$q->where(
[
['bookings.dateFrom','>=',$params['dateFrom']],
['bookings.dateTo','<=',$params['dateTo']],
]
);
}]);
How can I obtain an object from a MongoDB collection where a specific field1 (timestamp or date) is older/newer than another specific field2 (timestamp or date)?
Given the following example object:
// MongoDB 3.2
{
name: 'test',
updated_on: Timestamp(1474416000, 0),
export: {
active: true,
last_exported_on: Timestamp(1474329600, 0)
}
}
This object should match a query like: where export.active is true and updated_on > export.last_exported_on
I've tried it with the aggregation framework, since I've read that $where can be very slow, but without any success.
// PHP 5.4 (and MongoDB PHP lib. http://mongodb.github.io/mongo-php-library)
$collection->aggregate([
['$project' => [
'dst' => ['$cmp' => ['updated_on', 'export.last_exported_on']],
'name' => true
]],
['$match' => ['dst' => ['$gt' => 0], 'export.active' => ['$eq' => true]]],
['$limit' => 1]
]);
I can change timestamps into date or anything else, but I don't see the problem in the type.
Edit: Not all objects have the last_exported_on or the export fields at all. Besides that both can be null or empty or 000000.
That's because after you do the $project you end up only with the dst and _id fields, so you cannot $match on export.active. You need to match on export.active before the projection. After that you need another match on the dst field.
[
{
$match: {
"export.active": true
}
},
{
$project: {
dst: {
$cmp: [
"$updated_on",
"$export.last_exported_on"
]
}
}
},
{
$match: {
dst: 1
}
}
]
Edit
Alternatively, you can make sure to preserve export.active and to spare another $match:
[
{
$project: {
"export.active": 1,
cmp: {
$cmp: [
"$updated_on",
"$export.last_exported_on"
]
}
}
},
{
$match: {
cmp: 1,
"export.active": true
}
}
]
I'm having a dataset which contains datapoints for every 5 seconds per day. This would result in a dataset of 17280 items a day.
This set is way too big and i want it smaller (i'm using these items to draw a graph).
Since the graph's x-axis is over time i decided a gap of 5 minutes per datapoint is good enough. This will return into 288 datapoints a day. A lot less and good enough to make a graph.
My MongoCollection looks like this:
{
"timestamp":"12323455",
"someKey":123,
"someOtherKey": 345,
"someOtherOtherKey": 6789
}
The data gets posted every 5 seconds into the database. So the timestamp will differ 5 seconds for each result.
As my x-axis is divided in 5 minutes sequences I'd love to calculate the average values of someKey, someOtherKey and someOtherOtherkey over these 5 minutes.
This new average will be one of the datapoints in my graph.
How would one get all the datapoints from 1 day with each average 5 minutes apart from eachother? (288 datapoints per day).
As for now i'm selecting every document from midnight this day:
$result = $collection
->createQueryBuilder()
->field('timestamp')->gte($todayMidnight)
->sort('timestamp', 'DSC')
->getQuery()
->execute();
How would one filter this list of data (within the same query) to get the datapoints for every 5 minutes (and the datapoint being an average of the points within these 5 minutes)?
It would be nice to have this query built with doctrine as i'll need it in my symfony application.
EDIT
I've tried to get my query first within the mongoshell working.
As in the comments suggested i should start using aggregation.
The query i've made so far is based upon another question asked here at stackoverflow
This is the current query:
db.Pizza.aggregate([
{
$match:
{
timestamp: {$gte: 1464559200}
}
},
{
$group:
{
_id:
{
$subtract: [
"$timestamp",
{"$mod": ["$timestamp", 300]}
]
},
"timestamp":{"$first":"$timestamp"},
"someKey":{"$first":"$someKey"},
"someOtherKey":{"$first":"$someOtherKey"},
"someOtherOtherKey":{"$first":"$someOtherOtherKey"}
}
}
])
This query will give me the last result for each 300 seconds (5 minutes) from today Midnight.
I want it to get all documents within those 300 seconds and calculate an average over the columns someKey, someOtherKey, someOtherOtherKey
So if we take this example dataset:
{
"timestamp":"1464559215",
"someKey":123,
"someOtherKey": 345,
"someOtherOtherKey": 6789
},
{
"timestamp":"1464559220",
"someKey":54,
"someOtherKey": 20,
"someOtherOtherKey": 511
},
{
"timestamp":"1464559225",
"someKey":654,
"someOtherKey": 10,
"someOtherOtherKey": 80
},
{
"timestamp":"1464559505",
"someKey":90,
"someOtherKey": 51,
"someOtherOtherKey": 1
}
The query should return 2 rows namely:
{
"timestamp":"1464559225",
"someKey":277,
"someOtherKey": 125,
"someOtherOtherKey": 2460
},
{
"timestamp":"1464559505",
"someKey":90,
"someOtherKey": 51,
"someOtherOtherKey": 1
}
The first result is calculated like this:
Result 1 - someKey = (123+54+654)/3 = 277
Result 1 - someOtherKey = (345+20+10)/3 = 125
Result 1 - someOtherOtherKey = (6789+511+80)/3 = 2460
How would one make this calculation within the mongoshell with the aggregation function?
Based on the given answeres here on stackoverflow i've managed to get exactly what i wanted.
This is the big aggregation query i have to make to get all my results back:
db.Pizza.aggregate([
{
$match:
{
timestamp: {$gte: 1464559200}
}
},
{
$group:
{
_id:
{
$subtract: [
'$timestamp',
{$mod: ['$timestamp', 300]}
]
},
timestamp: {$last: '$timestamp'},
someKey: {$avg: '$someKey'},
someOtherKey: {$avg: '$someOtherKey'},
someOtherOtherKey: {$avg: '$someOtherOtherKey'}
}
},
{
$project:
{
_id: 0,
timestamp: '$timestamp',
someKey: '$someKey',
someOtherKey:'$someOtherKey',
someOtherOtherKey:'$someOtherOtherKey'
}
}
])
The Match part is for getting every result after Today Midnight (timestamp of today midnight).
The Group part is the most interesting part. Here we're looping through every document we've found and calculate a modulus for every 300 seconds (5 minutes) then we fill the property timestamp with the last result of the modulus operations.
The Project part is necessary to remove the _id from the actual result as the result doesn't represent something in the database anymore.
Given answeres where this answere is based on:
MongoDB - Aggregate max/min/average for multiple variables at once
How to subtract in mongodb php
MongoDB : Aggregation framework : Get last dated document per grouping ID
Doctrine Solution
$collection->aggregate([
[
'$match' => [
'timestamp' => ['$gte' => 1464559200]
]
],
[
'$group' => [
'_id' => [
'$subtract' => [
'$timestamp',
[
'$mod' => ['$timestamp',300]
]
]
],
'timestamp' => [
'$last' => '$timestamp'
],
$someKey => [
'$avg' => '$'.$someKey
],
$someOtherKey => [
'$avg' => '$'.$someOtherKey
],
$someOtherOtherKey => [
'$avg' => '$'.$someOtherOtherKey
]
]
]
]);
I'm trying to adjust the timezone with date aggregation operators.
I need to make -7 hours adjustment on the $signs.timestamp field.
This is my code:
function statsSignatures() {
$cursor = $this->db->collection->users->aggregate(
array('$unwind' => '$signs'),
array('$project'=>array(
'signs'=>'$signs',
'y'=>array('$year'=>'$signs.timestamp'),
'm'=>array('$month'=>'$signs.timestamp'),
'd'=>array('$dayOfMonth'=>'$signs.timestamp'),
'h'=>array('$hour'=>'$signs.timestamp')
)),
array('$group'=>array(
'_id'=>array('year'=>'$y','month'=>'$m','day'=>'$d','hour'=>'$h'),
'total'=>array('$sum'=>1)
)),
array('$sort'=>array(
'_id.year'=>1,
'_id.month'=>1,
'_id.day'=>1,
'_id.hour'=>1
))
);
return $cursor['result'];
}
I'm using MongoDB version 2.6.3.
Thank you a lot !
You can use $project with $subtract operator to make a -7 hour adjustment to a Date field:
{
$project : {
ts : { $subtract : [ "$signs.timestamp", 25200000 ] }
}
}
// 25200000 == 1000 milis x 60 sec x 60 mins x 7 h
The projected field ts is a Date that's offset by -7 hours.
Edit
This is the correct PHP syntax when using $subtract.
array(
'$project' => array(
'ts' => array('$subtract' => array('$signs.timestamp', 25200000))
)
)
Subtract accepts an array of values, not a key=>value pair.
I'm not sure why, but I'm getting "exception: invalid operator '$signs.timestamp'" if i'm trying to subtract this in php like this code:
$cursor = $app['mdb']->changemi->users->aggregate(
array('$unwind' => '$signs'),
array('$project' => array(
'ts'=>array('$subtract'=>array(
'$signs.timestamp'=> 25200000
))
)),
array('$project'=>array(
'y'=>array('$year'=>'$ts'),
'm'=>array('$month'=>'$ts'),
'd'=>array('$dayOfMonth'=>'$ts'),
'h'=>array('$hour'=>'$ts')
)),
array('$group'=>array(
'_id'=>array('year'=>'$y','month'=>'$m','day'=>'$d','hour'=>'$h'),
'total'=>array('$sum'=>1)
)),
array('$sort'=>array(
'_id.year'=>1,
'_id.month'=>1,
'_id.day'=>1,
'_id.hour'=>1
))
);
So I came with 2 workarounds:
backend php. json_decode
$cursor = $app['mdb']->changemi->users->aggregate(
array('$unwind' => '$signs'),
json_decode('{"$project" : {"ts" : { "$subtract" : [ "$signs.timestamp", 25200000 ] }}}',true),
array('$project'=>array(
'y'=>array('$year'=>'$ts'),
'm'=>array('$month'=>'$ts'),
'd'=>array('$dayOfMonth'=>'$ts'),
'h'=>array('$hour'=>'$ts')
)),
array('$group'=>array(
'_id'=>array('year'=>'$y','month'=>'$m','day'=>'$d','hour'=>'$h'),
'total'=>array('$sum'=>1)
)),
array('$sort'=>array(
'_id.year'=>1,
'_id.month'=>1,
'_id.day'=>1,
'_id.hour'=>1
))
);
frontend javascript (minusHours)
Date.prototype.minusHours= function(h){
this.setHours(this.getHours()-h);
return this;
}
...
"date": new Date({{ i._id.year }}, {{ i._id.month -1 }}, {{ i._id.day }}, {{ i._id.hour }}, 0, 0, 0).minusHours(7),
Here is what worked for me. Instead of doing the timezone conversion in the 'project', I just convert the timestamp while grouping.
group._id = {
year: { $year : [{ $subtract: [ "$timestamp", 25200000 ]}] },
month: { $month : [{ $subtract: [ "$timestamp", 25200000 ]}] },
day: { $dayOfMonth : [{ $subtract: [ "$timestamp", 25200000 ]}] }
};
group.count = {
$sum : 1
};
There's no need to close some objects in array, this way worked for me:
group._id = {
year: { $year : { $subtract: [ "$timestamp", 25200000 ]}},
month: { $month : { $subtract: [ "$timestamp", 25200000 ]}},
day: { $dayOfMonth : { $subtract: [ "$timestamp", 25200000 ]}}
};
group.count = {
$sum : 1
};
I'm using PHP with MongoDB, How can apply below commend inside?
db.event.group({
keyf: function(doc) {
return {
year: doc.created.getFullYear(),
month: doc.created.getMonth() + 1,
day: doc.created.getDate()
}
},
reduce: function(curr, result){
result.count++;
},
initial: {count: 0}
});
I have tried below, but NOT working. Looks like not supprt keyf?
$keyf = 'function(doc){return {year: doc.created.getFullYear(), month: doc.created.getMonth()+1, day: doc.created.getDate()}}';
$initial = array('count' => 0);
$reduce = 'function(curr, result){result.count++;}';
$collection->group($keyf, $initial, $reduce);
It looks like you are basically counting the amount of documents under a date.
It should be noted that the group command has numerous flaws including:
Not officially supporting sharding (warning not to use it)
Is basically JavaScript
Is Basically a Map Reduce
Is extremely slow
that means it has since been "deprecated" in favour of the aggregation framework, which in PHP for you would be:
$db->collection->aggregate(array(
array('$group' => array(
'_id' => array(
'day' => array('$dayOfMonth' => '$created'),
'month' => array('$month' => '$created'),
'year' => array('$year' => '$created')
),
'count' => array('$sum' => 1)
))
));
To understand what operators I used etc you can look here:
http://docs.mongodb.org/manual/reference/operator/aggregation/dayOfMonth/
http://docs.mongodb.org/manual/reference/operator/aggregation/month/#exp._S_month
http://docs.mongodb.org/manual/reference/operator/aggregation/year/
http://docs.mongodb.org/manual/reference/operator/aggregation/sum/
The PHP driver does have the MongoCode class for constructing the JavaScript values that are required.
But you are actually better off using the .aggregate() command to this as it is "native* code and does not rely on the JavaScript engine. So it is much faster at producing results.
db.collection.aggregate([
{ "$group": {
"_id": {
"year": { "$year": "$created" },
"month": { "$month": "$created" },
"day": { "$dayOfMonth": "$created" }
},
"count": { "$sum": 1 }
}}
])
Data Problem
So the aggregate function works are expected, but you seem to have a problem with your test data. Here is cwhat you gave:
db.post.insert({'field':'b', 'created':new Date('2014, 1, 1')});
db.post.insert({'field':'c', 'created':new Date('2014, 1, 1 11:11:11')});
db.post.insert({'field':'d', 'created':new Date('2014, 1, 1 12:00:00')});
db.post.insert({'field':'a', 'created':new Date('2014, 1, 2')});
db.post.insert({'field':'b', 'created':new Date('2014, 1, 2')})
And this produces the data:
{ "field" : "a", "created" : ISODate("2013-12-31T13:00:00Z") }
{ "field" : "b", "created" : ISODate("2013-12-31T13:00:00Z") }
{ "field" : "c", "created" : ISODate("2014-01-01T00:11:11Z") }
{ "field" : "d", "created" : ISODate("2014-01-01T01:00:00Z") }
{ "field" : "a", "created" : ISODate("2014-01-01T13:00:00Z") }
{ "field" : "b", "created" : ISODate("2014-01-01T13:00:00Z") }
So it looks like you were trying to add "hours" in the same day to test the grouping. But the arguments to Date() are not correct. You wanted this:
db.post.insert({'field':'b', 'created':new Date('2014-01-01')});
db.post.insert({'field':'c', 'created':new Date('2014-01-01 11:11:11')});
So the whole date as a string and not the "comma" separated values