So here is a sample of a document in my mongodb collection:
{
"_id" : ObjectId("561e0de61c9218b7bf9877c3"),
"Date" : NumberLong(20151014),
"Hour" : NumberLong(10),
"ProductId" : ObjectId("5614ba9c2e131caa098b4567"),
"ProductName" : "Test",
"ProducerId" : ObjectId("5617802151f8adf4db329d52"),
"ProducerName" : "Producer",
"ProducerRate" : NumberLong(300),
"ProducerMedium" : "Emailer",
"TotalLead" : NumberLong(6),
"VerifiedLead" : NumberLong(3),
"UnverifiedLead" : NumberLong(2),
"UnQualifiedLead" : NumberLong(1),
"TotalEarning" : NumberLong(660),
"Consumers" : [
{
"ConsumerId" : ObjectId("5617802151f8adf4db329d54"),
"ConsumerName" : "Consumer1",
"ConsumedRate" : NumberLong(120),
"ConsumedLead" : NumberLong(3),
"Earning" : NumberLong(360)
},
{
"ConsumerId" : ObjectId("5617802151f8adf4db329d58"),
"ConsumerName" : "Consumer2",
"ConsumedRate" : NumberLong(100),
"ConsumedLead" : NumberLong(3),
"Earning" : NumberLong(300)
}
]
}
Now i want to get the ConsumedLead grouped by ConsumerId and ProductId from the database in php.
what i have did so far to give me TotalLead and VerifiedLead grouped by product id but have no idea how to get consumerbased results for same:
$keyf = new MongoCode('function(doc) {
return {\'ProductId\': doc.ProductId,\'ProductName\': doc.ProductName};
}');
$initial = array('TotalLead'=>0,'VerifiedLead'=>0);
$reduce = "function(obj, prev) {
prev.TotalLead += obj.TotalLead;
prev.VerifiedLead += obj.VerifiedLead;
}";
$result = $collection->group($keyf, $initial, $reduce);
var_dump($result);
Any Help Please.
EDIT:
expected result wpuld be :
{ [0]=> array(4) { ["ProductId"]=> object(MongoId)#8 (1) { ["$id"]=> string(24) "5614ba9c2e131caa098b4567" } ["ProductName"]=> string(4) "Test" ["ConsumerId"]=> object(MongoId)#8 (1) { ["$id"]=> string(24) "5617802151f8adf4db329d58" } ["ConsumedLead"]=> float(4) } }
The solution is to use the aggregation framework where the operation includes an $unwind operator initial pipeline stage as this will deconstruct the Consumers array field from the input documents and outputs a document for each element. Each output document replaces the array with an element value. This will then make it possible for the $sum group accumulator operator in the $group step to work and thus givies you the required ConsumedLead grouped by ConsumerId and ProductId:
db.collection.aggregate([
{
"$unwind": "$Consumers"
},
{
"$group": {
"_id": {
"ProductId": "$ProductId",
"ConsumerId": "$Consumers.ConsumerId"
},
"TotalConsumedLead": {
"$sum": "$Consumers.ConsumedLead"
}
}
}
])
Running this aggregation operation on the above sample will result:
/* 0 */
{
"result" : [
{
"_id" : {
"ProductId" : ObjectId("5614ba9c2e131caa098b4567"),
"ConsumerId" : ObjectId("5617802151f8adf4db329d58")
},
"TotalConsumedLead" : NumberLong(3)
},
{
"_id" : {
"ProductId" : ObjectId("5614ba9c2e131caa098b4567"),
"ConsumerId" : ObjectId("5617802151f8adf4db329d54")
},
"TotalConsumedLead" : NumberLong(3)
}
],
"ok" : 1
}
So your final working aggregation in PHP should be:
$pipeline = array(
array('$unwind' => '$Consumers'),
array(
'$group' => array(
'_id' => array(
'ProductId' => '$ProductId',
'ConsumerId' => '$Consumers.ConsumerId',
),
'TotalConsumedLead' => array(
'$sum' => '$Consumers.ConsumedLead'
),
)
),
);
$out = $collection->aggregate($pipeline ,$options);
Related
I have Inserted records in mongodb collection/table. everything working fine if i pass data which is available in collection like StoreId = 1
Problem: if i search data which is not available in collection like StoreId=3 then it is taking too long time to response why???? How do i simplify this issue?
Insert Data:
$data = array();
$data[] = array("StoreId" => "1","StoreName" => "abc");
$data[] = array("StoreId" => "2","StoreName" => "xyz");
$db->test->insertMany($data);
Fetch Records:
$arrFind = array(
'$and' => array(
array(
'StoreId' => "1"
),
array(
'StoreName' => array(
'$regex' => '^ab',
'$options' => 'i'
)
)
)
);
$projection = array("_id" => false);
$result = $db->test->find($arrFind,['limit'=>$data['Count'], 'projection' => $projection ]);
$array1 = iterator_to_array($result, false);
echo "<pre>";
print_r($array1);
above example working find returns output very quickly, but if i pass StoreId=3 then it is taking too long time to response blank output.
Mongodb:
db.test.find( { StoreId: "1" } ).pretty() //returns output quickly
db.test.find( { StoreId: "3" } ).pretty() //taking too-long time for blank output.
explain:
db.test.find( { StoreId: "3" } ).explain(true)
{
"queryPlanner" : {
"plannerVersion" : 1,
"namespace" : "mydb.test",
"indexFilterSet" : false,
"parsedQuery" : {
"StoreId" : {
"$eq" : "3"
}
},
"winningPlan" : {
"stage" : "COLLSCAN",
"filter" : {
"StoreId" : {
"$eq" : "3"
}
},
"direction" : "forward"
},
"rejectedPlans" : [ ]
},
"executionStats" : {
"executionSuccess" : true,
"nReturned" : 0,
"executionTimeMillis" : 35083,
"totalKeysExamined" : 0,
"totalDocsExamined" : 6816823,
"executionStages" : {
"stage" : "COLLSCAN",
"filter" : {
"StoreId" : {
"$eq" : "3"
}
},
"nReturned" : 0,
"executionTimeMillisEstimate" : 34087,
"works" : 6816825,
"advanced" : 0,
"needTime" : 6816824,
"needYield" : 0,
"saveState" : 53988,
"restoreState" : 53988,
"isEOF" : 1,
"invalidates" : 0,
"direction" : "forward",
"docsExamined" : 6816823
},
"allPlansExecution" : [ ]
},
"serverInfo" : {
"host" : "Neha-PC",
"port" : 27017,
"version" : "3.6.3",
"gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5"
},
"ok" : 1
}
Hi looking at you question , it examined around 6816823 record , it seems there is no secondary index has been created
you need to create the index on "StoreId" reduced the lookup
u can check what all the index has been created for your collection using following command
db.test.getIndexKeys();
To create index you can use following command
db.test.createIndex( { StoreId: 1 } );
after creating the index on your collection try to run the query , it should be fast
for more info
https://docs.mongodb.com/manual/reference/method/db.collection.createIndex/
Because you don't have any indexes which can be used for this query, so MongoDB need to check every document in the collection. The collection scan ("COLLSCAN") is pretty slow on big collections. You need to create an index from StoreId element.
I have the following command line mongodb query:
db.getCollection('Data').aggregate([
{'$project' : {"_id":"$_id",
"g":"$g",
"value": {'$substr':["$g",0,4]},
}
}])
The result of this query is:
{
"result" : [
{
"_id" : NumberLong(1),
"g" : "1383,09,1,2000",
"value" : "1383"
},
{
"_id" : NumberLong(2),
"g" : "1499,06,1,1",
"value" : "1499"
},
],
"ok" : 1.0000000000000000,
"$gleStats" : {
"lastOpTime" : Timestamp(0, 0),
"electionId" : ObjectId("564d7df200e15758444e9a7d")
}
}
Now I want to use this query in a php file.
Especially how can I use $substr within $project with aggregate in PHP?
You need to use the MongoCollection::aggregate method
$pipeline = array(
'$project' => array(
'g' => '$g',
'value' => array('$substr' => array('$g', 0,4))
)
);
$results = $collection->aggregate($pipeline)
I have an mongodb collection with following documents:
{
"_id" : ObjectId("547af6aea3f0eba7148b4567"),
"check_id" : "f5d654e7-257d-4a93-ae50-2d59dfeeb451",
"chunks" : NumberLong(200),
"num_hosts" : NumberLong(1000),
"num_rbls" : NumberLong(163),
"owner" : NumberLong(7901),
"created" : ISODate("2014-11-30T10:51:26.924Z"),
"started" : ISODate("2014-11-30T10:51:31.558Z"),
"finished" : ISODate("2014-11-30T10:57:08.512Z")
}
{
"_id" : ObjectId("54db19a858a5d395a18b4567"),
"check_id" : "9660e510-1349-43f3-9d5e-8bf4b06179be",
"chunks" : NumberLong(2),
"num_hosts" : NumberLong(10),
"num_rbls" : NumberLong(166),
"owner" : NumberLong(7901),
"created" : ISODate("2015-02-11T08:58:17.118Z"),
"started" : ISODate("2015-02-11T08:58:18.78Z"),
"finished" : ISODate("2015-02-11T08:58:47.486Z")
}
{
"_id" : ObjectId("54db267758a5d30eab8b4567"),
"check_id" : "9660e510-1349-43f3-9d5e-8bf4b06179be",
"chunks" : NumberLong(2),
"num_hosts" : NumberLong(10),
"num_rbls" : NumberLong(166),
"owner" : NumberLong(7901),
"created" : ISODate("2015-02-11T09:52:55.388Z"),
"started" : ISODate("2015-02-11T09:52:56.109Z"),
"finished" : ISODate("2015-02-11T09:53:22.095Z")
}
What I need is to get the result and produce an array similar to this:
Array
(
[2015-02-11] => array
(
//array with results from 2015-02-11
)
[2014-11-30] => array
(
//array with results from 2014-11-30
)
)
I know that it's possible to just perform simply collection->find and then loop through results and use php logic to achieve my goal but is it possible to make it using mongo? Maybe using aggregation framework?
EDIT: I want to group results by "created" date
Any help will be highly appreciated.
Monogo aggregation mongo aggregation group used for this, so below query may solve your problem
db.collectionName.aggregate({
"$group": {
"_id": "$created",
"data": {
"$push": {
"check_id": "$check_id",
"chunks": "$chunks",
"num_hosts": "$num_hosts",
"num_rbls": "$num_rbls",
"owner": "$owner",
"started": "$started",
"finished": "$finished"
}
}
}
}).pretty()
Or
db.collectionName.aggregate({
"$group": {
"_id": "$created",
"data": {
"$push": "$$ROOT"
}
}
}).pretty()
Also in mongo 2.8 $dateToString provide facility to convert ISO date to string format so below query also work
db.collectionName.aggregate([
{
"$project": {
"yearMonthDay": {
"$dateToString": {
"format": "%Y-%m-%d",
"date": "$created"
}
},
"check_id": "$check_id",
"chunks": "$chunks",
"num_hosts": "$num_hosts",
"num_rbls": "$num_rbls",
"owner": "$owner",
"started": "$started",
"finished": "$finished"
}
},
{
"$group": {
"_id": "$yearMonthDay",
"data": {
"$push": "$$ROOT"
}
}
}
]).pretty()
I have managed to solve this using the aggregation framework. Here is the answer, in case anyone need it.
$op = array(
array(
'$project' => array(
'data' => array(
'check_id' => '$check_id',
'chunks' => '$chunks',
'num_hosts' => '$num_hosts',
'num_rbls' => '$num_rbls',
'owner' => '$owner',
'started' => '$started',
'finished' => '$finished',
),
'year' => array('$year' => '$created' ),
'month' => array('$month' => '$created' ),
'day' => array('$dayOfMonth' => '$created'),
)
),
array(
'$group' => array(
'_id' => array('year' => '$year', 'month' => '$month', 'day' => '$day'),
'reports_data' => array('$push' => '$data'),
)
),
);
$c = $collection->aggregate($op);
I have this collection
> db.test.find()
{ "_id" : ObjectId("5398ddf40371cdb3aebca3a2"), "name" : "ahmed", "qte" : 30 }
{ "_id" : ObjectId("5398de040371cdb3aebca3a3"), "name" : "demha", "qte" : 35 }
{ "_id" : ObjectId("5398de140371cdb3aebca3a4"), "name" : "ahmed", "qte" : 50 }
{ "_id" : ObjectId("5398de210371cdb3aebca3a5"), "name" : "ahmed", "qte" : 60 }
i would like to sum "qte" where "name"= "ahmed" and print the sum with php
i know how to do with SQL but i have no idea how it is in mongodb.
Thanks :)
Use the aggregation framework.
Assuming you have an the current collection as $collection
result = $collection->aggregate(array(
array(
'$match' => array(
'name' => 'ahmed'
)
),
array(
'$group' => array(
'_id' => NULL,
'total' => array(
'$sum' => '$qte'
)
)
)
));
The two parts are the $match to meet the criteria, and the $group to arrive at the "total" using $sum
See other Aggregation Framework Operators and the Aggregation to SQL Mapping chart for more examples.
This is done with an aggregate statement:
db.test.aggregate([
{
$match: {
name: "ahmed"
}
},
{
$group: {
_id:"$name",
total: {
$sum: "$qte"
}
}
}
])
I am trying to search a text field in my collection. This is an example document in my collection:
{
"_id" : ObjectId("51f9c432573906141dbc9996"),
"id" : ObjectId("51f9c432573906141dbc9995"),
"body" : "the",
"rank" : 0,
"num_comm" : 0,
"activity" : 1375323186
}
This is how I am searching...
$mongo = new MongoClient("mongodb://127.0.0.1");
$db = $mongo->requestry;
try
{
$search_results = $db->command(array('text' => 'trending', 'search' => '"the"'));
}
catch (MongoCursorException $e)
{
return array('error' => true, 'msg' => $e->getCode());
}
return array('error' => false, 'results' => $search_results);
And this is the result that I get...
{
error: false,
results: {
queryDebugString: "||||the||",
language: "english",
results: [ ],
stats: {
nscanned: 0,
nscannedObjects: 0,
n: 0,
nfound: 0,
timeMicros: 66
},
ok: 1
}
}
Below are my indexes on the collection...
{
"v" : 1,
"key" : {
"_id" : 1
},
"ns" : "requestry.trending",
"name" : "_id_"
},
{
"v" : 1,
"key" : {
"_fts" : "text",
"_ftsx" : 1
},
"ns" : "requestry.trending",
"name" : "body_text",
"weights" : {
"body" : 1
},
"default_language" : "english",
"language_override" : "language",
"textIndexVersion" : 1
}
Any ideas on why I get a blank results array every time?
Thanks in advance for any help!
Nathan
You can not search for "the" because it is a stop-word, and stop-words are not indexed. You can find a list of stop-words at https://github.com/mongodb/mongo/blob/master/src/mongo/db/fts/stop_words_english.txt
You can actually see what is being tried to match in the debug string:
queryDebugString: "||||the||"
The first element is empty here, which means no match is done. If you look what happens for '"cat" AND "purple"', the debug string is:
queryDebugString: "cat|purpl||||cat|purple||"
The first element(s) are now cat|purpl - this shows that stemming has also been applied for purple.
You have nested quotes on your code ('the' string literal):
$search_results = $db->command(array('text' => 'trending', 'search' => '"the"'));
Try not nesting the quotes
$search_results = $db->command(array('text' => 'trending', 'search' => 'the'));