I need to update multiple embedded docs in mongo using PHP. My layout looks like this:
{
_id: id,
visits : {
visitID: 12
categories: [{
catagory_id: 1,
name: somename,
count: 11,
duration: 122
},
{
catagory_id: 1,
name: some other name,
count: 11,
duration: 122
},
{
catagory_id: 2,
name: yet another name,
count: 11,
duration: 122
}]
}
}
The document can have more than one visit too.
Now i want to update 2 categories, one with id=1 and name=somename and the other with id=1 and name=some_new_name. Both of them should inc "count" by 1 and "duration" by 45.
First document exists, but second does not.
Im thinking of having a function like this:
function updateCategory($id, $visitID,$category_name,$category_id) {
$this->profiles->update(
array(
'_id' => $id,
'visits.visitID' => $visitID,
'visits.categories.name' => $category_name,
'visits.categories.id' => $category_id,
),
array(
'$inc' => array(
'visits.categories.$.count' => 1,
'visits.categories.$.duration' =>45,
),
),
array("upsert" => true)
);
}
But with this i need to call the function for each category i will update. Is there any way to do this in one call?
EDIT:
Changed the layout a bit and made "categories" an object instead of array. Then used a combination of "category_id" and "category_name" as property name. Like:
categories: {
1_somename : {
count: 11,
duration: 122
},
1_some other name : {
count: 11,
duration: 122
},
2_yet another name : {
count: 11,
duration: 122
},
}
Then with upsert and something like
$inc: {
"visits.$.categories.1_somename.d": 100,
"visits.$.categories.2_yet another name.c": 1
}
i can update several "objects" at a time..
Mongodb currently not supporting arrays multiple levels deep updating (jira)
So following code will not work:
'$inc' => array(
'visits.categories.$.count' => 1,
'visits.categories.$.duration' => 123,
),
So there is some solutions around this:
1.Load document => update => save (possible concurrency issues)
2.Reorganize your documents structure like this(and update using one positional operator):
{
_id: id,
visits : [{
visitID: 12
}],
categories: [{
catagory_id: 1,
name: somename,
count: 11,
duration: 122,
visitID: 12
}]
}
}
3.Wait for multiple positional operators support (planning in 2.1 version of mongodb).
4.Reorganize your documents structure somehow else, in order to avoid multiple level arrays nesting.
Related
I need to divide my search result into two parts. 1 with those goods in which the number> 0 sort them by price and withdraw first. 2 products whose quantity = 0 sort by price and display at the end, after those products that are in stock. The main thing is that in the first group of goods (whose quantity> 0) there were no goods from the second group (whose quantity = 0) What unfortunately happens when I sort by two conditions
Use PHP 7.1
and Elastic Search 6.6.0
Small example, there is a table of goods
id | site_price | count
1 | 10 | 0
2 | 5 | 5
3 | 15 | 2
4 | 20 | 10
5 | 15 | 0
I need to sort first by quantity, and then by price (without losing the first sorting).
First sort: ('count'=>'desc').
Second sort: ('site_price'=>'asc').
Should get this result:
id | site_price | count
2 | 5 | 10
3 | 15 | 5
4 | 20 | 2
1 | 10 | 0
5 | 15 | 0
$this->params['body'] = array(
'from' => ($filters['page'] - 1) * 15,
'size' => 15,
'query' => array(
'bool' => array(
'must' => array(
"query_string" => array(
'query' => "*" . $filters['text'] . "*",
)
),
)
),
'sort' => array(
array("shops_count" => "desc"),
array("site_price" => "asc")
)
);
$result = $this->client->search($this->params);
It looks like that you want to achieve behavior similar to UNION in SQL, since you first want to split the result set into 2 groups, sort each group and then attach one group after another.
There are a few ways to do it.
1) By doing 2 queries
Like in this answer, it is suggested to do 2 queries:
POST /orders/_search
{
"query": {
"range": {
"count": {
"gt": 0
}
}
},
"sort" : [
{"site_price": "asc"},
]
}
POST /orders/_search
{
"query": {
"range": {
"count": {
"gte": 0,
"lte": 0
}
}
},
"sort" : [
{"site_price": "asc"},
]
}
And then joining them on the client side.
There is also a way to do it completely on the Elasticsearch side.
2) By using script sorting
We can use script based sorting and sort first on the availability (count > 0), then by price:
POST /orders/_search
{
"sort" : [
{
"_script" : {
"type" : "number",
"script" : {
"lang": "painless",
"source": "if (doc['count'].value > 0) { 1 } else { 0 } "
},
"order" : "desc"
}
},
{"site_price": "asc"}
]
}
However, scripting always has performance overhead. Solution #1 is more robust, although it performs 2 queries.
Here is another solution that uses single query and does not use expensive scripting.
3) Adding new field - for sorting
If we add a special field, "available", we will not need to use script sorting.
The documents might look like this:
doc1 = {
"id": 1,
"site_price": 10,
"count": 0,
"available": 0
}
doc2 = {
"id": 2,
"site_price": 5,
"count": 5,
"available": 1
}
Then the sorting will look like this:
POST /orders/_search
{
"sort" : [
{"available": "desc"},
{"site_price": "asc"}
]
}
This is a common pattern called denormalization which proves useful when tuning for best performance.
Hope that helps!
#Nikolay, thanks for the help.
Unfortunately, this did not help. I tried rewrote the query - but the result is the same. Here is an example: removed too much left only search and sorting
enter code here
$this->params['body'] = array(
'from' => ($filters['page'] - 1) * 15,
'size' => 15,
'query' => array(
'bool' => array(
'must' => array(
"query_string" => array(
'query' => "*" . $filters['text'] . "*",
)
),
)
),
'sort' => array(
array("shops_count" => "desc"),
array("site_price" => "asc")
)
);
$result = $this->client->search($this->params);
For example, we have a songs table, and a favorites (songs) table.
If I use Songs::with('favorites')->get(), it will return as follows:
"songs": [
{
"id": 43,
"name": "Song 1",
"favorites": [
{
"id": 52,
"user_id": 25,
"song_id": 43
}
]
},
{
"id": 44,
"name": "Song 2",
"favorites": []
},
What I want to do is, in case the song has been favorited, return 1, and if not, return 0, as follows:
{
"id": 43,
"name": "Song 1",
"favorites": true (or 1)
},
{
"id": 44,
"name": "Song 2",
"favorites": false (or 0)
},
Is there a way to do it without having to run through the returned collection array manually in PHP?
I think this is the simplest solution using Eloquent
Song::withCount([
'favorites' => function ($query) {
$query->select(DB::raw('IF(count(*) > 0, 1, 0)'));
}
])->orderBy('favorites_count', 'desc');
There are a bunch of different ways to do what you want, it all depends on what works best for you.
If you want the result returned in your result set, you can setup a scope:
class Song extends Model
{
public function scopeAddFavorite($query, $userId = null)
{
$andUser = !empty($userId) ? ' AND favorites.user_id = '.$userId : '';
return $query->addSelect(\DB::raw('(EXISTS (SELECT * FROM favorites WHERE favorites.song_id = songs.id'.$andUser.')) as is_favorite'));
}
}
Since this scope modifies the 'select' statement, you need to make sure to manually specify the other columns you want before adding in the scope.
$songs = Song::select('*')->addFavorite()->get();
I added in the ability to pass in the user id, in case you wanted to specify that the column only return true if the song has been favorited by a specific user.
$songs = Song::select('*')->addFavorite(25)->get();
Another option, you can add an accessor to your model that will handle the check for you. You can read about accessors here.
class Song extends Model
{
// only do this if you want to include is_favorite in your json output by default
protected $appends = ['is_favorite'];
public function getIsFavoriteAttribute()
{
// if you always want to hit the database:
return $this->favorites()->count() > 0;
// if you're okay using any pre-loaded relationship
// will load the relationship if it doesn't exist
return $this->favorites->count() > 0;
}
}
Usage:
$song = Song::find(1);
// access is_favorite like a normal attribute
var_dump($song->is_favorite);
// with $appends, will show is_favorite;
// without $appends, will not show is_favorite
var_dump($song);
Using a raw SQL approach
DB::select('SELECT s.*, (CASE WHEN f.id IS NULL THEN 0 ELSE 1 END) as favorites FROM songs s LEFT JOIN favorites f ON s.id = f.song_id GROUP BY s.id')
will return the following structure:
[
StdClass { 'id' => 1, 'name' => 'Song 1', 'favorites' => 1 },
StdClass { 'id' => 2, 'name' => 'Song 2', 'favorites' => 0 },
StdClass { 'id' => 3, 'name' => 'Song 3', 'favorites' => 1 },
]
Now, using Eloquent
Song::withCount('favorites')->get()
will return the an array of objects of the Song class
[
Song { 'id' => 1, 'name' => 'Song 1', 'favorites_count' => 1 },
Song { 'id' => 2, 'name' => 'Song 2', 'favorites_count' => 0 },
Song { 'id' => 3, 'name' => 'Song 3', 'favorites_count' => 3 }
]
The difference is the first will return an array of PHP standard objects while the second an array of Song objects and the first is faster than second.
You can't do something like this without somehow manipulating the original results object first.
You don't really specify how you need to use or iterate through the data, but maybe Query Scopes can help you? Either way you'll need to iterate over the data once to manipulate it. A high order function like map will help you do this.
suppose you have the $songs array, you can do this,
foreach($songs as $song)
$song["favorites"] = is_null($song["favorites"]) ? true : false;
In mongoDB i have two collection users and posts following this structure:
Posts
{
_id: ObjectId(""),
subject: "some post",
content: "here is the content",
user_id: "4351"
}
Users
{
user_id: "4351",
name: "John Marks",
picURL: "http://...",
aboutme: "historian of the future"
}
needing to get the posts in array with name.
db.posts.find().map(function(newPost){
newPost.name = db.users.findOne({user_id: newPost.user_id}).name;
return (newPost);
})
I wrote this code and it's work in mongoshell well returning this result:
{
_id: ObjectId(""),
subject: "some post",
content: "here is the content",
user_id: "4351",
name: "John Marks"
}
but i could not apply in php. You can't just simple get the output of the map function. It requires reduce function and output collection for the returning value.
Edit:
$map = new MongoCode('
function(newPost) {
newPost.username = db.users.findOne({user_id: newPost.user_id}).name;
return newPost;
}
');
post = $app->mongo->command(array(
"mapreduce" => "posts",
"map" => $map,
"reduce" => '',
"out" => array("inline" => 1)
));
var_dump($post);
This code must be work but accessing another collection in map function via 'db' is forbidden after mongo 2.4 release. That's why i changed my approach. Instead of using map/reduce, handled with php. Added posts user_ids to array and get the users information with following code.
$userInf = $app->mongo->selectCollection("users")->find(
array('user_id' => array('$in' => $user_ids)),
array("_id" => 0, "user_id" => 1, "name" => 1, "picURL" => 1)
);
I'm using PHP with MongoDB, How can apply below commend inside?
db.event.group({
keyf: function(doc) {
return {
year: doc.created.getFullYear(),
month: doc.created.getMonth() + 1,
day: doc.created.getDate()
}
},
reduce: function(curr, result){
result.count++;
},
initial: {count: 0}
});
I have tried below, but NOT working. Looks like not supprt keyf?
$keyf = 'function(doc){return {year: doc.created.getFullYear(), month: doc.created.getMonth()+1, day: doc.created.getDate()}}';
$initial = array('count' => 0);
$reduce = 'function(curr, result){result.count++;}';
$collection->group($keyf, $initial, $reduce);
It looks like you are basically counting the amount of documents under a date.
It should be noted that the group command has numerous flaws including:
Not officially supporting sharding (warning not to use it)
Is basically JavaScript
Is Basically a Map Reduce
Is extremely slow
that means it has since been "deprecated" in favour of the aggregation framework, which in PHP for you would be:
$db->collection->aggregate(array(
array('$group' => array(
'_id' => array(
'day' => array('$dayOfMonth' => '$created'),
'month' => array('$month' => '$created'),
'year' => array('$year' => '$created')
),
'count' => array('$sum' => 1)
))
));
To understand what operators I used etc you can look here:
http://docs.mongodb.org/manual/reference/operator/aggregation/dayOfMonth/
http://docs.mongodb.org/manual/reference/operator/aggregation/month/#exp._S_month
http://docs.mongodb.org/manual/reference/operator/aggregation/year/
http://docs.mongodb.org/manual/reference/operator/aggregation/sum/
The PHP driver does have the MongoCode class for constructing the JavaScript values that are required.
But you are actually better off using the .aggregate() command to this as it is "native* code and does not rely on the JavaScript engine. So it is much faster at producing results.
db.collection.aggregate([
{ "$group": {
"_id": {
"year": { "$year": "$created" },
"month": { "$month": "$created" },
"day": { "$dayOfMonth": "$created" }
},
"count": { "$sum": 1 }
}}
])
Data Problem
So the aggregate function works are expected, but you seem to have a problem with your test data. Here is cwhat you gave:
db.post.insert({'field':'b', 'created':new Date('2014, 1, 1')});
db.post.insert({'field':'c', 'created':new Date('2014, 1, 1 11:11:11')});
db.post.insert({'field':'d', 'created':new Date('2014, 1, 1 12:00:00')});
db.post.insert({'field':'a', 'created':new Date('2014, 1, 2')});
db.post.insert({'field':'b', 'created':new Date('2014, 1, 2')})
And this produces the data:
{ "field" : "a", "created" : ISODate("2013-12-31T13:00:00Z") }
{ "field" : "b", "created" : ISODate("2013-12-31T13:00:00Z") }
{ "field" : "c", "created" : ISODate("2014-01-01T00:11:11Z") }
{ "field" : "d", "created" : ISODate("2014-01-01T01:00:00Z") }
{ "field" : "a", "created" : ISODate("2014-01-01T13:00:00Z") }
{ "field" : "b", "created" : ISODate("2014-01-01T13:00:00Z") }
So it looks like you were trying to add "hours" in the same day to test the grouping. But the arguments to Date() are not correct. You wanted this:
db.post.insert({'field':'b', 'created':new Date('2014-01-01')});
db.post.insert({'field':'c', 'created':new Date('2014-01-01 11:11:11')});
So the whole date as a string and not the "comma" separated values
I have bunch of documents on mongodb, and this is a dummy insert
array (
'_id' => new MongoId("51a449866803fa680a000002"),
'a' => 'dweddwe',
'b' => 'asdasdad',
'c' =>
array (
'0' => 'car',
),
'u' => '1',
'x' =>
array (
'0' => '51a0c0356803fa890a000003',
'1' => '51a0c0356803fa890a000003',
),
'y' => 'merto',
)
I have more than 100 inserted documents in my mongo database, the problem is, when I use this code and index like this x_1__id_1 or in any other ways, I always get [scanAndOrder] => 1, I have no idea what might be the problem or solution, how can I sort it in an efficient way ? thank you :)
$m->cars->post->find(array("x" => array('$in' => $mendor["t"])))->limit(10)->sort(array("_id" => -1))->explain();
this is $mendor["t"],
't' =>
array (
'0' => '519f2de16803fabd0d000001',
'1' => '51a0bf996803fa890a000001',
'2' => '519f2db96803fad20d000001',
'3' => '519f1cc56803fa960d000001',
),
Compund indexes are not useable for reverse ordering, but have no worries about it will not help in this case probably if you have in $mendor["t"] a real list.
I made a test collection called t with simple documents like :
{ "_id" : ObjectId("51a4c2c75e0733e8428ab2c0"), "x" : [ 1, 2, 3, 4 ] }
{ "_id" : ObjectId("51a4c2c95e0733e8428ab2c1"), "x" : [ 1, 2, 3, 6 ] }
{ "_id" : ObjectId("51a4c2cd5e0733e8428ab2c2"), "x" : [ 1, 4, 3, 6 ] }
I created the index : x_1__id_1
For the query:
db.t.find({x:3}).hint("x_1__id_1").sort({_id:1}).explain()
{
"cursor" : "BtreeCursor x_1__id_1",
"isMultiKey" : true,
"n" : 14,
"nscannedObjects" : 14,
"nscanned" : 14,
"nscannedObjectsAllPlans" : 14,
"nscannedAllPlans" : 14,
"scanAndOrder" : false,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"x" : [
[
3,
3
]
],
"_id" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"server" : ""
}
So it works as you like your query to work but :
db.t.find({x:{$in:[3,4]}}).hint("x_1__id_1").sort({_id:1}).explain()
{
"cursor" : "BtreeCursor x_1__id_1 multi",
"isMultiKey" : true,
"n" : 16,
"nscannedObjects" : 28,
"nscanned" : 28,
"nscannedObjectsAllPlans" : 28,
"nscannedAllPlans" : 28,
"scanAndOrder" : true,
"indexOnly" : false,
"nYields" : 0,
"nChunkSkips" : 0,
"millis" : 0,
"indexBounds" : {
"x" : [
[
3,
3
],
[
4,
4
]
],
"_id" : [
[
{
"$minElement" : 1
},
{
"$maxElement" : 1
}
]
]
},
"server" : ""
}
This is sort of reasonable while the multikey index which is used, is store as separated index keys the values in the array. See documentation: http://docs.mongodb.org/manual/core/indexes/#multikey-indexes
So the different parts which where collected from the different keys will be merged by the engine and than sorted. If you use a criteria which is like my first query,so instead of an array just search for one value. Than it will scanAndOrder : false. Probably you have to make an x_1_id-1 index.
There is a workaround but it is ugly a bit. If you use an or clause for the query than every part of the or list will use separately an index usage. So instead of using the in:[] condition, use or:[] and define as many different queries as many values you have in your $mendor["t"] array. In mongoshell did not worked, but i am sure it should, maybe i missed something.
You have a compound index on x, _id. The ordering of the _id is not absolute but within the context of an x value. In your query you are selecting a set of documents for particular x values and then ordering by just _id. So to get you the order it has to actually sort the results based on _id values as opposed to just using index order. That is what you see scan and order to be true. Hope this helps.