How to do an aggregate query on an embedded document? - php

I am using jenssegers/laravel-mongodb library in a laravel application however I need to show counts of an embedded document. Using a generic example of comment/posts, while I can solve my problem by just pulling all the posts and looping through to get comments to count them but was just was not sure if I could query them.
I did set up my relationships. In my post class I did:
public function comments()
{
return $this->hasMany('App\Comment');
}
and in my comment class:
public function post()
{
return $this->belongsTo('App\Post');
}
Later in code:
$post->comments()->save($comment);
$comment->post()->associate($post);
my document structure:
"posts" : [
{
"_id" : ObjectId("5805a11e2594ee26543ea041"),
"Post_Num" : "166236001010",
"updated_at" : ISODate("2016-10-18T04:12:14.454Z"),
"created_at" : ISODate("2016-10-18T04:12:14.451Z"),
"comments" : [
{
"Comment_Num" : "3333333",
"_id" : ObjectId("5805a11e2594ee26543ea042"),
"post_id" : "5805a11e2594ee26543ea041",
},
{
"Comment_Num" : "3333333",
"_id" : ObjectId("5805a11e2594ee26543ea042"),
"post_id" : "5805a11e2594ee26543ea041",
}
]
},
{
"_id" : ObjectId("5805a11e2594ee26543ea041"),
"Post_Num" : "166236001010",
"comments" : [
{
"Comment_Num" : "3333333",
"_id" : ObjectId("5805a11e2594ee26543ea042"),
"post_id" : "5805a11e2594ee26543ea041",
}
]
}
]
Now when I try getting the comments like:
$post->comments()->count()
or
$post->comments()->get()->count()
or
$post->comments->get()->count()
I get a 0. The same logic works if it is not an embedded document but just was wondering if it was possible to do an aggregate query ? Perhaps is best to just let the code iterate and add everything?
As you can tell I need some minor hand holding. Thank You
UPDATE: I am trying the following
public function commentCount()
{
$commentsCount = Post::raw(function($collection)
{
return $collection->aggregate(['$project' => ['_id' => 1,
'comments_count' => ['$size' => '$comments']],
['$group' => ['_id' => null, 'count' => ['$sum' => '$comments_count']]]]);
});
return $commentsCount;
}
What I get now is:
$pipeline is not a list (unexpected index: "$project")

Just to be clear, you want a list of your posts with the number of comments on each post?
Aggregation has something to offer for that:
https://docs.mongodb.com/manual/reference/operator/aggregation/size/#exp._S_size
I'm not a php dev but this is my shot at it:
Post::raw()->aggregate(
['$project' => ['_id' => 1,
'Post_Num' => 1,
'comments_count' => ['$size' => '$comments']],
['$group' => ['_id' => null, 'count' => ['$sum' => '$comments_count']]]
]);

Related

php mongo upsert issue

I'm using this package https://github.com/jenssegers/laravel-mongodb#mongodb-specific-operations to deal with a mongo on the project I'm working on.
I have collection with such a structure (here is a example of one document)
{
"_id" : ObjectId("5fda3a602279e5262a3ddec6"),
"type" : "type",
"tags" : [
"tag1",
"tag2",
],
"filters" : [
{
"key" : "1",
"label" : "some key1"
},
{
"key" : "2",
"label" : "some key2"
}
],
"updated_at" : ISODate("2020-12-16T16:48:32.000+0000"),
"created_at" : ISODate("2020-12-16T16:48:32.000+0000")
},
what I need to do is check if collection has a document with a same type and exact same tags as provided, if yes I need to update that document if not create it.
Here is a code I wrote:
$this->collection->where('type', $type)
->where('tags', 'all', $tags)
->update(
['type' => $type, 'tags' => $tags, 'filters' => $filters,],
['upsert' => true,]
);
here I'm checking if we have a document with a type equal to $type and tags same as $tags, if such document exists I update it (replace all data with a new one) if not it should create a new document, thanks to ['upsert' => true,]
But when I'm running it I'm getting error cannot infer query fields to set, path 'tags' is matched twice
FYI: when I'm removing ->where('tags', 'all', $tags) query is working, not sure if correct, but it's not failing
Also I tried to do it like this: ->where('tags', ['$all' => $tags]) and issue is the same as above
Updated it's happening only when it trying to insert new field, updates work correctly

MongoDb fetch document subset using PHP

I have a MongoDB document structure like this:
[
{
"locale":"en",
"translations":[
{
"name":"translation1",
"value":"enValue"
},
{
"name":"translation2",
"value":"enValue"
},
{
"name":"translation3",
"value":"enValue"
}
]
},
{
"locale":"ru",
"translations":[
{
"name":"translation1",
"value":"ruValue"
},
{
"name":"translation2",
"value":"ruValue"
},
{
"name":"translation3",
"value":"ruValue"
}
]
}
]
and I need to get the translation with name translation1 for locale en.
The expected result I want is:
{
"_id" : ObjectId("5e845ba1005e625a6237d2e0"),
"translations" : [
{
"name" : "translation1",
"value" : "enValue"
}
]
}
I know how to do this with pure mongo, it should be like this:
db.translations.find({"locale" : "en"},
{ translations: { $elemMatch: { name: "translation1" } } } )
Here is the proof https://gyazo.com/fb9b1a505a898c7137ece5304d715171
but I can't make it work with PHP. I tried code like:
$collection = $this->database->{$group};
$collection->find(
[
'locale' => 'en',
'translations' => ['$elemMatch' => ['name' => 'translation1']
]
);
And Im getting all translations for en instead of only tranlsation1 as a result:
{
"_id" : ObjectId("5e845ba1005e625a6237d2e0"),
"locale" : "en",
"translations" : [
{
"name" : "translation1",
"value" : "enValue"
},
{
"name":"translation2",
"value":"enValue"
},
{
"name":"translation3",
"value":"enValue"
}
]
}
I tried as:
$collection = $this->database->{$group};
$collection->find(
['locale' => 'en'],
[
'translations' => ['$elemMatch' => ['name' => 'translation1']
]
);
also result is the same as above.
Tried like:
$collection = $this->database->{$group};
$collection->find(
[
'locale' => 'en',
[
'translations' => ['$elemMatch' => ['name' => 'translation1']
]
]
);
result is null
As a workaround, for now, I filter result on PHP side, but it extra work
This appears to be a bug in the driver.
This database command should be equivalent to the find you were running, but the command works as expected while the find does not.
$this->$database->command([
'find'=>'CollectionName',
'filter'=>['locale' => 'en'],
'projection'=>['translations' => ['$elemMatch' => ['name' => 'translation1']]]
])

select sum column by with Mongodb and Laravel

I have MongoDB document which contains following data(xxx)
{
"_id" : "48e5f6b1-026f-48b8-98cb-6572cfa0eaa6",
"esti_delivery_time" : "2015-1-1",
"original_price" : NumberInt(4060),
"price":109.2,
"code" : "JS1709137",
"updated_at" : ISODate("2017-09-13T06:01:18.000+0000"),
"created_at" : ISODate("2017-09-13T03:45:54.000+0000")
}
I want to
SELECT "xxx" with SUM "price" and SUM "original_price"
Because you need the sums of the price and original_price elements, it is means that you need to group the documents by some data (one or more elements). For this task you need to use the $group aggregator.
MongoDB PHP Driver documentation
Example:
$con = new MongoDB\Client('mongodb://localhost:27017', [], [
'typeMap' => [
'root' => 'array',
'document' => 'array',
'array' => 'array'
]
]);
$collection = $con->selectDatabase("db_name")->selectCollection("collection_name");
$cursor = $collection->aggregate([
[
'$group' => [
"_id" => '$code',
"sum_price" => ['$sum' => '$price'],
"sum_original_price" => ['$sum' => '$original_price']
]
]
]);
My answer:
enter image description here
Thanks for your reply

Elasticsearch MLT query with Elastica for PHP

Wondering if this from the Elasticsearch official doc:
{
"more_like_this" : {
"fields" : ["name.first", "tweet"],
"like" : [
{
"_index" : "marvel",
"_type" : "quotes",
"doc" : {
"name": {
"first": "Ben",
"last": "Grimm"
},
"tweet": "You got no idea what I'd... what I'd give to be invisible."
}
},
],
"min_term_freq" : 1,
"max_query_terms" : 1
}
}
is yet implemented within the latest release of Elastica?
The bit I am struggling with is the "doc" section of the "like".
My code is as follow:
$moreLikeThis = (new Query\MoreLikeThis())
->setFields([
'name.first',
'tweet'
])
->setLike((new Document())
->setIndex('myIndexName')
->setType('myTypeName')
->setData([
'tweet' => 'Foo',
'name' => [
'first' => 'Bar',
'last' => 'Test'
]
])
)
->setMinTermFrequency(1)
->setMinDocFrequency(1);
But it looks like the query is not generated properly. Here is what I get when I var_dump() Request::toString():
string(398)
"{"path":"myIndexName/myTypeName/_search","method":"GET","data":{"query":{"more_like_this":{"fields":["name.first","tweet"],"like":{"_id":"","_type":"myTypeName","_index":"myIndexName"},"min_term_freq":1,"min_doc_freq":1}}},"query":{"search_type":"count"},"connection":{"config":[],"host":"localhost","port":9200,"enabled":true}}"
The "doc" section if definitely missing? Am I not using it properly?
If anyone is wondering, the feature was actually not fully implemented within Elastica.
It's fixed now.

Yii2 Elasticsearch extension - how do I handle type mapping?

I want to be able to store a json object in my ES index. Here's an example of what I'm trying to store (this a serialized model, a request body that is sent to ES):
"{"id":218,"name":"Test2","category_id":1,"address":"Pushkin street","phone":null,"site":null,"location":{"lat":64,"lon":70},"city":"Heaven","description":"Super company","tags":["#test1","#test2"]}"
When I try to store it (via the extension, of course), here's the error that ES returns:
"{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse [location]"}],"type":"mapper_parsing_exception","reason":"failed to parse [location]","caused_by":{"type":"illegal_argument_exception","reason":"unknown property [lat]"}},"status":400}"
It seems that I am unable to do so without having a specific type mapping, like in the docs:
https://www.elastic.co/guide/en/elasticsearch/reference/1.4/mapping-object-type.html
However, I don't seem to find a way to provide that mapping inside the model. The extension's documentation doesn't really say anything about it.
So, my question is: do I need it at all, and if I do, how?
Appreciate all feedback.
I'll assume your model is \yii\elasticsearch\ActiveRecord. You'll need to describe its attributes:
public function attributes()
{
return [
'name',
'category_id',
'address',
'phone',
'site',
'location',
'city',
'description',
'tags'
];
}
Don't forget to configure index() and type(). In the following example type is my_address.
Then you'll need to create an index with proper field mapping. Here's what your mapping should look like:
"mappings" : {
"my_address" : {
"properties" : {
"name" : { "type" : "string"},
"category_id" : { "type" : "integer"},
"address" : { "type" : "string"},
"phone" : { "type" : "string"},
"site" : { "type" : "string"},
"location" : { "type" : "geo_point"},
"city" : { "type" : "string"},
"description" : { "type" : "string"},
"tags" : { "type" : "string"}
}
}
}
Note three things:
Location is of type geo_point.
Tags are declared as string. This will also allow them to be arrays of strings.
I didn't include the id field. If it's unique, I suggest you just set your yii model's id to the necessary value ($model->primaryKey = '123'). Otherwise your ES model will have its internal id set to something like AVDXmfJ3Ou7LzqD1DDMj and also have an id field which is not very convenient.
I encourage you to take a closer look at the mappings - they are very important when it comes to configuring how exactly the strings are being analyzed.
UPDATE: You don't really describe the mapping anywhere in your model. Do it in a migration - similar to creating tables in SQL.
In case you using ElasticSearch ActiveRecord , you could define a method for setupMapping
Class BookIndex extends yii\elasticsearch\ActiveRecord
{
/**
* sets up the index for this record
*
*/
public static function setUpMapping()
{
$db = static::getDb();
//in case you are not using elasticsearch ActiveRecord so current class extends database ActiveRecord yii/db/activeRecord
// $db = yii\elasticsearch\ActiveRecord::getDb();
$command = $db->createCommand();
/*
* you can delete the current mapping for fresh mapping but this not recommended and can be dangrous.
*/
// $command->deleteMapping(static::index(), static::type());
$command->setMapping(static::index(), static::type(), [
static::type() => [
// "_id" => ["path" => "id", "store" => "yes"],
"properties" => [
'name' => ["type" => "string"],
'author_name' => ["type" => "string"],
'publisher_name' => ["type" => "string"],
'created_at' => ["type" => "long"],
'updated_at' => ["type" => "long"],
'status' => ["type" => "long"],
],
],
]);
}
}
Later on you just need to call this method any time you want to apply the new mapping.

Categories