How to use "group" in MongoDB with PHP? - php

I'm using PHP with MongoDB, How can apply below commend inside?
db.event.group({
keyf: function(doc) {
return {
year: doc.created.getFullYear(),
month: doc.created.getMonth() + 1,
day: doc.created.getDate()
}
},
reduce: function(curr, result){
result.count++;
},
initial: {count: 0}
});
I have tried below, but NOT working. Looks like not supprt keyf?
$keyf = 'function(doc){return {year: doc.created.getFullYear(), month: doc.created.getMonth()+1, day: doc.created.getDate()}}';
$initial = array('count' => 0);
$reduce = 'function(curr, result){result.count++;}';
$collection->group($keyf, $initial, $reduce);

It looks like you are basically counting the amount of documents under a date.
It should be noted that the group command has numerous flaws including:
Not officially supporting sharding (warning not to use it)
Is basically JavaScript
Is Basically a Map Reduce
Is extremely slow
that means it has since been "deprecated" in favour of the aggregation framework, which in PHP for you would be:
$db->collection->aggregate(array(
array('$group' => array(
'_id' => array(
'day' => array('$dayOfMonth' => '$created'),
'month' => array('$month' => '$created'),
'year' => array('$year' => '$created')
),
'count' => array('$sum' => 1)
))
));
To understand what operators I used etc you can look here:
http://docs.mongodb.org/manual/reference/operator/aggregation/dayOfMonth/
http://docs.mongodb.org/manual/reference/operator/aggregation/month/#exp._S_month
http://docs.mongodb.org/manual/reference/operator/aggregation/year/
http://docs.mongodb.org/manual/reference/operator/aggregation/sum/

The PHP driver does have the MongoCode class for constructing the JavaScript values that are required.
But you are actually better off using the .aggregate() command to this as it is "native* code and does not rely on the JavaScript engine. So it is much faster at producing results.
db.collection.aggregate([
{ "$group": {
"_id": {
"year": { "$year": "$created" },
"month": { "$month": "$created" },
"day": { "$dayOfMonth": "$created" }
},
"count": { "$sum": 1 }
}}
])
Data Problem
So the aggregate function works are expected, but you seem to have a problem with your test data. Here is cwhat you gave:
db.post.insert({'field':'b', 'created':new Date('2014, 1, 1')});
db.post.insert({'field':'c', 'created':new Date('2014, 1, 1 11:11:11')});
db.post.insert({'field':'d', 'created':new Date('2014, 1, 1 12:00:00')});
db.post.insert({'field':'a', 'created':new Date('2014, 1, 2')});
db.post.insert({'field':'b', 'created':new Date('2014, 1, 2')})
And this produces the data:
{ "field" : "a", "created" : ISODate("2013-12-31T13:00:00Z") }
{ "field" : "b", "created" : ISODate("2013-12-31T13:00:00Z") }
{ "field" : "c", "created" : ISODate("2014-01-01T00:11:11Z") }
{ "field" : "d", "created" : ISODate("2014-01-01T01:00:00Z") }
{ "field" : "a", "created" : ISODate("2014-01-01T13:00:00Z") }
{ "field" : "b", "created" : ISODate("2014-01-01T13:00:00Z") }
So it looks like you were trying to add "hours" in the same day to test the grouping. But the arguments to Date() are not correct. You wanted this:
db.post.insert({'field':'b', 'created':new Date('2014-01-01')});
db.post.insert({'field':'c', 'created':new Date('2014-01-01 11:11:11')});
So the whole date as a string and not the "comma" separated values

Related

Mongodb string to date conversion

Below is my sample mongodb collection
{
"_id" : ObjectId("57ed32f4070577ec56a56b9f"),
"log_id" : "180308",
"issue_id" : "108850",
"author_key" : "priyadarshinim_contus",
"timespent" : NumberLong(18000),
"comment" : "Added charts in the dashboard page of the application.",
"created_on" : "2017-08-16T18:22:04.816+0530",
"updated_on" : "2017-08-16T18:22:04.816+0530",
"started_on" : "2017-08-16T18:21:39.000+0530",
"started_date" : "2017-08-02",
"updated_date" : "2017-08-02",
"role" : "PHP",
"updated_at" : ISODate("2017-09-29T15:27:48.069Z"),
"created_at" : ISODate("2017-09-29T15:27:48.069Z"),
"status" : 1.0
}
I need to get records with help of started_date , by default I will give two dates in that i will check $gt and $lt of started date .
$current_date = '2017-08-31';
$sixmonthfromcurrent ='2017-08-01';
$worklogs = Worklog::raw ( function ($collection) use ($issue_jira_id, $current_date, $sixmonthfromcurrent) {
return $collection->aggregate ( [
['$match' => ['issue_id' => ['$in' => $issue_jira_id],
'started_date' => ['$lte' => $current_date,'$gte' => $sixmonthfromcurrent]
]
],
['$group' => ['issue_id' => ['$push' => '$issue_id'],
'_id' => ['year' => ['$year' => '$started_date'],
'week' => ['$week' => '$started_date'],'resource_key' => '$author_key'],
'sum' => array ('$sum' => '$timespent')]
],
[ '$sort' => ['_id' => 1]
]
] );
} );
If I run this query I am getting this type of error:
Can't convert from BSON type string to Date
How to rectify this error?
The only field in your $group that I see as troubling is the field week.
The year you could extract by doing a $project before your $group aggregation:
$project: {
year: { $substr: [ "$started_date", 0, 4 ] },
issue_id: 1,
author_key: 1,
timespent: 1
}
if you know that the date string will always come at this format. Of course you cannot do a substr operation for finding out the week.
It would be easy though if your field started_date would be an actual ISODate(), then you could use exactly what you wrote as you probably already saw in the documentation.
If you need the field week very bad, which I imagine you do, then I'd suggest you convert your field started_date to an ISODate().
You can do that with a bulkWrite:
db = db.getSiblingDB('yourDatabaseName');
var requests = [];
db.yourCollectionName.find().forEach(doc => {
var date = yourFunctionThatConvertsStringToDate(doc.started_date);
requests.push( {
'updateOne': {
'filter': { '_id': doc._id },
'update': { '$set': {
"started_date": date
} }
}
});
if (requests.length === 500) {
db.yourCollectionName.bulkWrite(requests);
requests = [];
}
});
if(requests.length > 0) {
db.yourCollectionName.bulkWrite(requests);
}
Load this script directly on your mongodb server and execute there.
Hope this helps.

PHP & MongoDB show results grouped by date

I have an mongodb collection with following documents:
{
"_id" : ObjectId("547af6aea3f0eba7148b4567"),
"check_id" : "f5d654e7-257d-4a93-ae50-2d59dfeeb451",
"chunks" : NumberLong(200),
"num_hosts" : NumberLong(1000),
"num_rbls" : NumberLong(163),
"owner" : NumberLong(7901),
"created" : ISODate("2014-11-30T10:51:26.924Z"),
"started" : ISODate("2014-11-30T10:51:31.558Z"),
"finished" : ISODate("2014-11-30T10:57:08.512Z")
}
{
"_id" : ObjectId("54db19a858a5d395a18b4567"),
"check_id" : "9660e510-1349-43f3-9d5e-8bf4b06179be",
"chunks" : NumberLong(2),
"num_hosts" : NumberLong(10),
"num_rbls" : NumberLong(166),
"owner" : NumberLong(7901),
"created" : ISODate("2015-02-11T08:58:17.118Z"),
"started" : ISODate("2015-02-11T08:58:18.78Z"),
"finished" : ISODate("2015-02-11T08:58:47.486Z")
}
{
"_id" : ObjectId("54db267758a5d30eab8b4567"),
"check_id" : "9660e510-1349-43f3-9d5e-8bf4b06179be",
"chunks" : NumberLong(2),
"num_hosts" : NumberLong(10),
"num_rbls" : NumberLong(166),
"owner" : NumberLong(7901),
"created" : ISODate("2015-02-11T09:52:55.388Z"),
"started" : ISODate("2015-02-11T09:52:56.109Z"),
"finished" : ISODate("2015-02-11T09:53:22.095Z")
}
What I need is to get the result and produce an array similar to this:
Array
(
[2015-02-11] => array
(
//array with results from 2015-02-11
)
[2014-11-30] => array
(
//array with results from 2014-11-30
)
)
I know that it's possible to just perform simply collection->find and then loop through results and use php logic to achieve my goal but is it possible to make it using mongo? Maybe using aggregation framework?
EDIT: I want to group results by "created" date
Any help will be highly appreciated.
Monogo aggregation mongo aggregation group used for this, so below query may solve your problem
db.collectionName.aggregate({
"$group": {
"_id": "$created",
"data": {
"$push": {
"check_id": "$check_id",
"chunks": "$chunks",
"num_hosts": "$num_hosts",
"num_rbls": "$num_rbls",
"owner": "$owner",
"started": "$started",
"finished": "$finished"
}
}
}
}).pretty()
Or
db.collectionName.aggregate({
"$group": {
"_id": "$created",
"data": {
"$push": "$$ROOT"
}
}
}).pretty()
Also in mongo 2.8 $dateToString provide facility to convert ISO date to string format so below query also work
db.collectionName.aggregate([
{
"$project": {
"yearMonthDay": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created"
}
},
"check_id": "$check_id",
"chunks": "$chunks",
"num_hosts": "$num_hosts",
"num_rbls": "$num_rbls",
"owner": "$owner",
"started": "$started",
"finished": "$finished"
}
},
{
"$group": {
"_id": "$yearMonthDay",
"data": {
"$push": "$$ROOT"
}
}
}
]).pretty()
I have managed to solve this using the aggregation framework. Here is the answer, in case anyone need it.
$op = array(
array(
'$project' => array(
'data' => array(
'check_id' => '$check_id',
'chunks' => '$chunks',
'num_hosts' => '$num_hosts',
'num_rbls' => '$num_rbls',
'owner' => '$owner',
'started' => '$started',
'finished' => '$finished',
),
'year' => array('$year' => '$created' ),
'month' => array('$month' => '$created' ),
'day' => array('$dayOfMonth' => '$created'),
)
),
array(
'$group' => array(
'_id' => array('year' => '$year', 'month' => '$month', 'day' => '$day'),
'reports_data' => array('$push' => '$data'),
)
),
);
$c = $collection->aggregate($op);

Mongodb aggregate group by day (php)

I have a collection with documents that look like this:
{
_id: ObjectId("516eb5d2ef4501a804000000"),
accountCreated: "2013-04-17 16:46",
accountLevel: 0,
responderCount: 0
}
I want to group and count these documents based on the accountCreated date (count per day), but I am stuck with the handling of dates since the date includes time as well.
This is what I have, but it returns the count including the time, witch means lots of entries always with 1 as accounts.
$g = $form->mCollectionUsers->aggregate(array(
array( '$group' => array( '_id' => '$accountCreated', 'accounts' => array( '$sum' => 1 ) ) )
));
Is there a way to rewrite the date to only take day in account and skip the time?
I have found this example but I canĀ“t really get figure out how to adapt it to this example.
If accountCreated is a date you can do it like this (I'll use the mongo shell syntax since I'm not familiar with the php driver):
db.mCollectionUsers.aggregate([
{$project :{
day : {"$dayOfMonth" : "$accountCreated"},
month : {"$month" : "$accountCreated"},
year : {"$year" : "$accountCreated"}
}},
{$group: {
_id : {year : "$year", month : "$month", day : "$day"},
accounts : { "$sum" : 1}
}}
]);
If you want to display the date properly:
db.mCollectionUsers.aggregate([
{
$group: {
_id: { $dateToString: { format: '%Y-%m-%d', date: '$accountCreated' } },
count: { $sum: 1 }
}
},
{
$project: {
_id: 0,
date: '$_id',
count: 1
}
}
])
The result would look like:
[
{
"date": "2020-11-11",
"count": 8
},
{
"date": "2020-11-13",
"count": 3
},
{
"date": "2020-11-16",
"count": 3
},
]

No Results with Text Search MongoDB PHP

I am trying to search a text field in my collection. This is an example document in my collection:
{
"_id" : ObjectId("51f9c432573906141dbc9996"),
"id" : ObjectId("51f9c432573906141dbc9995"),
"body" : "the",
"rank" : 0,
"num_comm" : 0,
"activity" : 1375323186
}
This is how I am searching...
$mongo = new MongoClient("mongodb://127.0.0.1");
$db = $mongo->requestry;
try
{
$search_results = $db->command(array('text' => 'trending', 'search' => '"the"'));
}
catch (MongoCursorException $e)
{
return array('error' => true, 'msg' => $e->getCode());
}
return array('error' => false, 'results' => $search_results);
And this is the result that I get...
{
error: false,
results: {
queryDebugString: "||||the||",
language: "english",
results: [ ],
stats: {
nscanned: 0,
nscannedObjects: 0,
n: 0,
nfound: 0,
timeMicros: 66
},
ok: 1
}
}
Below are my indexes on the collection...
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "requestry.trending",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"ns" : "requestry.trending",
"name" : "body_text",
"weights" : {
"body" : 1
},
"default_language" : "english",
"language_override" : "language",
"textIndexVersion" : 1
}
Any ideas on why I get a blank results array every time?
Thanks in advance for any help!
Nathan
You can not search for "the" because it is a stop-word, and stop-words are not indexed. You can find a list of stop-words at https://github.com/mongodb/mongo/blob/master/src/mongo/db/fts/stop_words_english.txt
You can actually see what is being tried to match in the debug string:
queryDebugString: "||||the||"
The first element is empty here, which means no match is done. If you look what happens for '"cat" AND "purple"', the debug string is:
queryDebugString: "cat|purpl||||cat|purple||"
The first element(s) are now cat|purpl - this shows that stemming has also been applied for purple.
You have nested quotes on your code ('the' string literal):
$search_results = $db->command(array('text' => 'trending', 'search' => '"the"'));
Try not nesting the quotes
$search_results = $db->command(array('text' => 'trending', 'search' => 'the'));

how to make scan and order 0, when sorting on multikeys

I have bunch of documents on mongodb, and this is a dummy insert
array (
'_id' => new MongoId("51a449866803fa680a000002"),
'a' => 'dweddwe',
'b' => 'asdasdad',
'c' =>
array (
'0' => 'car',
),
'u' => '1',
'x' =>
array (
'0' => '51a0c0356803fa890a000003',
'1' => '51a0c0356803fa890a000003',
),
'y' => 'merto',
)
I have more than 100 inserted documents in my mongo database, the problem is, when I use this code and index like this x_1__id_1 or in any other ways, I always get [scanAndOrder] => 1, I have no idea what might be the problem or solution, how can I sort it in an efficient way ? thank you :)
$m->cars->post->find(array("x" => array('$in' => $mendor["t"])))->limit(10)->sort(array("_id" => -1))->explain();
this is $mendor["t"],
't' =>
array (
'0' => '519f2de16803fabd0d000001',
'1' => '51a0bf996803fa890a000001',
'2' => '519f2db96803fad20d000001',
'3' => '519f1cc56803fa960d000001',
),
Compund indexes are not useable for reverse ordering, but have no worries about it will not help in this case probably if you have in $mendor["t"] a real list.
I made a test collection called t with simple documents like :
{ "_id" : ObjectId("51a4c2c75e0733e8428ab2c0"), "x" : [ 1, 2, 3, 4 ] }
{ "_id" : ObjectId("51a4c2c95e0733e8428ab2c1"), "x" : [ 1, 2, 3, 6 ] }
{ "_id" : ObjectId("51a4c2cd5e0733e8428ab2c2"), "x" : [ 1, 4, 3, 6 ] }
I created the index : x_1__id_1
For the query:
db.t.find({x:3}).hint("x_1__id_1").sort({_id:1}).explain()
{
"cursor" : "BtreeCursor x_1__id_1",
"isMultiKey" : true,
"n" : 14,
"nscannedObjects" : 14,
"nscanned" : 14,
"nscannedObjectsAllPlans" : 14,
"nscannedAllPlans" : 14,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"x" : [
[
3,
3
]
],
"_id" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"server" : ""
}
So it works as you like your query to work but :
db.t.find({x:{$in:[3,4]}}).hint("x_1__id_1").sort({_id:1}).explain()
{
"cursor" : "BtreeCursor x_1__id_1 multi",
"isMultiKey" : true,
"n" : 16,
"nscannedObjects" : 28,
"nscanned" : 28,
"nscannedObjectsAllPlans" : 28,
"nscannedAllPlans" : 28,
"scanAndOrder" : true,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"x" : [
[
3,
3
],
[
4,
4
]
],
"_id" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"server" : ""
}
This is sort of reasonable while the multikey index which is used, is store as separated index keys the values in the array. See documentation: http://docs.mongodb.org/manual/core/indexes/#multikey-indexes
So the different parts which where collected from the different keys will be merged by the engine and than sorted. If you use a criteria which is like my first query,so instead of an array just search for one value. Than it will scanAndOrder : false. Probably you have to make an x_1_id-1 index.
There is a workaround but it is ugly a bit. If you use an or clause for the query than every part of the or list will use separately an index usage. So instead of using the in:[] condition, use or:[] and define as many different queries as many values you have in your $mendor["t"] array. In mongoshell did not worked, but i am sure it should, maybe i missed something.
You have a compound index on x, _id. The ordering of the _id is not absolute but within the context of an x value. In your query you are selecting a set of documents for particular x values and then ordering by just _id. So to get you the order it has to actually sort the results based on _id values as opposed to just using index order. That is what you see scan and order to be true. Hope this helps.

Categories