I have the following structure within a mongoDB collection:
{
"_id" : ObjectId("5301d337fa46346a048b4567"),
"delivery_attempts" : {
"0" : {
"live_feed_id" : 107,
"remaining_attempts" : 2,
"delivered" : false,
"determined_status" : null,
"date" : 1392628536
}
}
}
// > db.lead.find({}, {delivery_attempts:1}).pretty();
I'm trying to select any data from that collection where remaining_attempts are greater than 0 and a live_feed_id is equal to 107. Note that the "delivery_attempts" field is of a type hash.
I've tried using an addAnd within an elemMatch (not sure if this is the correct way to achieve this).
$qb = $this->dm->createQueryBuilder($this->getDocumentName());
$qb->expr()->field('delivery_attempts')
->elemMatch(
$qb->expr()
->field('remaining_attempts')->gt(0)
->addAnd($qb->expr()->field('live_feed_id')->equals(107))
);
I do appear to be getting the record detailed above. However, changing the greater than
test to 3
->field('remaining_attempts')->gt(3)
still returns the record (which is incorrect). Is there a way to achieve this?
EDIT: I've updated the delivery_attempts field type from a "Hash" to a "Collection". This shows the data being stored as an array rather than an object:
"delivery_attempts" : [
{
"live_feed_id" : 107,
"remaining_attempts" : 2,
"delivered" : false,
"determined_status" : null,
"date" : 1392648433
}
]
However, the original issue still applies.
You can use a dot notation to reference elements within a collection.
$qb->field('delivery_attempts.remaining_attempts')->gt(0)
->field('delivery_attempts.live_feed_id')->equals(107);
It works fine for me if I run the query on mongo.
db.testQ.find({"delivery_attempts.remaining_attempts" : {"$gt" : 0}, "delivery_attempts.live_feed_id" : 107}).pretty()
so it seems something wrong with your PHP query, I suggest running profiler to see which query is actually run against mongo
db.setProfilingLevel(2)
This will log all operation since you enable profiling. Then you can query the log to see which the actual queries
db.system.profile.find().pretty()
This might help you to find the culprit.
It sounds like your solved your first problem, which was using the Hash type mapping (instead for storing BSON objects, or associative arrays in PHP) instead of the Collection mapping (intended for real arrays); however, the query criteria in the answer you submitted still seems incorrect.
$qb->field('delivery_attempts.remaining_attempts')->gt(0)
->field('delivery_attempts.live_feed_id')->equals(107);
You said in your original question:
I'm trying to select any data from that collection where remaining_attempts are greater than 0 and a live_feed_id is equal to 107.
I assume you'd like that criteria to be satisfied by a single element within the delivery_attempts array. If that's correct, the criteria you specified above may match more than you expect, since delivery_attempts.remaining_attempts can refer to any element in the array, as can the live_feed_id criteria. You'll want to use $elemMatch to restrict the field criteria to a single array element.
I see you were using elemMatch() in your original question, but the syntax looked a bit odd. There should be no need to use addAnd() (i.e. an $and operator) unless you were attempting to apply two query operators to the same field name. Simply add extra field() calls to the same query expression you're using for the elemMatch() method. One example of this from ODM's test suite is QueryTest::testElemMatch(). You can also use the debug() method on the query to see the raw MongoDB query object created by ODM's query builder.
Related
So in Laravel 5 there's the handy thing called JSON Where Clauses using MySQL's new ability to store and fetch JSON stored in a column:
User::where('meta->colors', 'red')->get()
would return all rows, where colors in the column meta would be set to red.
Now let's say colors is not a string, but an array containing multiple colors (colors => ['red', 'blue', 'green']).
What would be an efficient way to retrieve all rows, where colors contains e.g. the value red?
JSON_CONTAINS() does exactly what you're looking for:
JSON_CONTAINS(target, candidate[, path])
Indicates by returning 1 or 0 whether a given candidate JSON document is contained within a target JSON document, or—if a path argument was supplied—whether the candidate is found at a specific path within the target. — 12.16.3 Functions That Search JSON Values
Currently, Laravel's query builder does not provide a corresponding API. There's an open internals proposal for it though.
In the meantime, you can execute a raw query:
\DB::table('users')->whereRaw(
'JSON_CONTAINS(meta->"$.colors", \'["red"]\')'
)->get();
Which would return all users that have "red" in their meta->colors JSON field. Note that the -> operator requires MySQL 5.7.9+.
You can also call the whereRaw() directly on an Eloquent model.
Laravel 5.6
As of the 5.6 release, Laravel's query builder contains a new whereJsonContains method.
I think a way would be using the like operator:
User::where('meta->colors', 'like', '%"red"%')
However, this would only work if the values never contain the character " and the delimiters wouldn't change.
An update for this answer, according to MySQL or MariaDb, the correct syntax must be JSON_CONTAINS(#json, 'red', '$.colors'), and is necessary to use JSON_EXTRACT.
So them, the code inside Laravel (for version 5.5 or less).
Like say #Elwin, meta column must contains the following JSON: { "colors": ["red", "blue", "green"] }
User::whereRaw("JSON_CONTAINS(JSON_EXTRACT(meta, '$.colors'), '\"{$color}\"')")
Remember to use double quotes in value sentence.
JSON_CONTAINS(JSON_EXTRACT(meta, '$.colors'), '"red"')
The whereIn method verifies that a given column's value is contained within the given array.
Try this:
$colorArray = ['red', 'blue', 'green'];
$user = User::whereIn($meta->color, $colorArray)->get();
More about Laravel's whereIn.
I have a problem with UpdateAttributes, it seem to not work for me.
When I issue:
$ret = $sphinx->UpdateAttributes ( "products", array ("status"), array(506607786 => array(10)) );
it returns 1, but search still returns status as old value for this.
When I try
$ret = $sphinx->UpdateAttributes ( "products", array ("status", "image_id"), array(506607786 => array(10, 6666)) );
it returns 0 (false)
Does this function even work ?
Ok I have found (sphinx docs are ugly) that when issuing updateAtrributes() from PHP app then I will not see the results in search command line. However one problem still
exist - I'm not able to update 2 attributes in one updateAtrributes() - seperatly they are fine - any clues why ?
When UpdateAttributes returns 0 (not false) it does not mean it didn't work, what it means is it didn't find anything to update, basically no updates commited. A return of -1 actually means this function did not work.
Make sure that 506607786 is actually an id in your Sphinx index and that products is the name of your index.
To make the question more helpful you can provide an example row from your table, preferrably the one used in this function defined as 506607786. You cna also provide a full set of your code to make it easier.
As a side note: UpdateAttributes does not act like a realtime index. You will need to filter on these attributes specifically in your query in order for sphinx to take their new values into consideration.
I am trying to save some db action by compiling a looped bit of code with a single query, Before I was simply adding to the the like statements using a loop before firing off the query but i cant get the same idea going in Mongo, id appreciate any ideas....
I am basically trying to do a like, but with the value as an array
('app', replaces 'mongodb' down to my CI setup )
Here's how I was doing it pre mongofication:
foreach ($workids as $workid):
$this->ci->app->or_like('work',$workid) ;
endforeach;
$query = $this->ci->db->get("who_users");
$results = $query->result();
print_r($results);
and this is how I was hoping I could get it to work, but no joy here, that function is only designed to accept strings
$query = $this->ci->app->like('work',$workids,'.',TRUE,TRUE)->get("who_users");
print_r($query);
If anyone can think of a way any cunning methods I can get my returned array with a single call again it would be great I've not found any documentation on this sort of query, The only way i can think of is to loop over the query and push it into a new results array.... but that is really gonna hurt if my app scales up.
Are you using codeigniter-mongodb-library? Based on the existing or_like() documentation, it looks like CI wraps each match with % wildcards. The equivalent query in Mongo would be a series of regex matches in an $or clause:
db.who_users.find({
$or: [
{ work: /.*workIdA.*/ },
{ work: /.*workIdB.*/ },
...
]});
Unfortunately, this is going to be quite inefficient unless (1) the work field is indexed and (2) your regexes are anchored with some constant value (e.g. /^workId.*/). This is described in more detail in Mongo's regex documentation.
Based on your comments to the OP, it looks like you're storing multiple ID's in the work field as a comma-delimited string. To take advantage of Mongo's schema, you should model this as an array of strings. Thereafter, when you query on the work field, Mongo will consider all values in the array (documented discussed here).
db.who_users.find({
work: "workIdA"
});
This query would match a record whose work value was ["workIdA", "workIdB"]. And if we need to search for one of a set of ID's (taking this back to your OR query), we can extend this example with the $in operator:
db.who_users.find({
work: { $in: ["workIdA", "workIdB", ...] }
});
If that meets your needs, be sure to index the work field as well.
I am considering using MongoDB to store documents that include a list of key/value pairs. The safe but ugly and bloated way to store this is as
[ ['k1' : 'v1'] , ['k2' : 'v2'], ...]
But document elements are inherently ordered within the underlying BSON data structure, so in principle:
{k1 : 'v1',
k2 : 'v2', ...}
should be enough. However I expect most language bindings will interpret these as associative arrays, and thus potentially scramble the ordering. So what I need to know is:
Does MongoDB itself promise to preserve item ordering of the second form.
Do language bindings have some API which can extract it ordered form -- even if the usual "convenient" API returns an associative array.
I am mostly interested in Javascript and PHP here, but I would also like to know about other languages. Any help is appreciated, or just a link to some documentation where I can go RTM.
From Version 2.6 on, MongoDB preserves the order of fields where possible. However, the _id field always comes first an renaming fields can lead to re-ordering. However, I'd generally try not to rely on details like this. As the original question mentions, there are also additional layers to consider which each must provide some sort of guarantee for the stability of the order...
Original Answer:
No, MongoDB does not make guarantees about the ordering of fields:
"There is no guarantee that the field order will be consistent, or the same, after an update."
In particular, in-place updates that change the document size will usually change the ordering of fields. For example, if you $set a field whose old value was of type number and the new value is NumberLong, fields usually get re-ordered.
However, arrays preserve ordering correctly:
[ {'key1' : 'value1'}, {'key2' : 'value2'}, ... ]
I don't see why this is "ugly" and "bloated" at all. Storing a list of complex objects couldn't be easier. However, abusing objects as lists is definitely ugly: Objects have associative array semantics (i.e. there can only be one field of a given name), while lists/arrays don't:
// not ok:
db.foo2.insert({"foo" : "bar", "foo" : "lala" });
db.foo2.find();
{ "_id" : ObjectId("4ef09cd9b37bc3cdb0e7fb26"), "foo" : "lala" }
// a list can do that
db.foo2.insert({ 'array' : [ {'foo' : 'bar'}, { 'foo' : 'lala' } ]});
db.foo2.find();
{ "_id" : ObjectId("4ef09e01b37bc3cdb0e7fb27"), "array" :
[ { "foo" : "bar" }, { "foo" : "lala" } ] }
Keep in mind that MongoDB is an object database, not a key/value store.
As of Mongo 2.6.1, it DOES keep the order of your fields:
MongoDB preserves the order of the document fields following write operations except for the following cases:
The _id field is always the first field in the document.
Updates that
include renaming of field names may result in the reordering of
fields in the document.
http://docs.mongodb.org/manual/release-notes/2.6/#insert-and-update-improvements
One of the pain points of this is comparing documents to one another in the shell.
I've created a project that creates a custom mongorc.js which sorts the document keys by default for you when they are printed out so at least you can see what is going on clearly in the shell. It's called Mongo Hacker if you want to give it a whirl.
Though it's true that, as of Mongo 2.6.1, it does preserve order, one should still be careful with update operations.
mattwad makes the point that updates can reorder things, but there's at least one other concern I can think of.
For example $addToSet:
https://docs.mongodb.com/manual/reference/operator/update/addToSet/
$addToSet when used on embedded documents in an array is discussed / exemplified here:
https://stackoverflow.com/a/21578556/3643190
In the post, mnemosyn explains how $addToSet disregards the order when matching elements in its deep value by value comparison.
($addToSet only adds records when they're unique)
This is relevant if one decided to structure data like this:
[{key1: v1, key2: v2}, {key1: v3, key2: v4}]
With an update like this (notice the different order on the embedded doc):
db.collection.update({_id: "id"},{$addToSet: {field:
{key2: v2, key1: v1}
}});
Mongo will see this as a duplicate and NOT this object to the array.
I would like to know if there is some improvement on MongoCollection::findOne or if is just an "alias" or "shorcut" to MongoCollection::find with a limit of 1, for example.
Thank you
findOne() is an alias of find() with a limit(-1)
You can see this in the source code here. It does the equivalent to
find(...).limit(-1).getNext().
The -1 is actually relevant. Here's a snippet from the wire protocol docs:
If the number is negative, then the database will return that number
and close the cursor.
If you go to the shell and type > db.collection.findOne (no parens), you can see that the function is also just a helper in the shell.
So, "yes findOne() is just a helper".
From the mongo tutorials...
To show that the document we inserted in the previous step is there,
we can do a simple findOne() operation to get the first document in
the collection. This method returns a single document (rather than the
DBCursor that the find() operation returns), and it's useful for
things where there only is one document, or you are only interested in
the first. You don't have to deal with the cursor.
The MongoCollection::findOne method will directly return the result array and the MongoCollection::find one will return a MongoCursor instance even if it is a single valued result.
mongodb.org has an performance test report where they compared findOne and find. Based on the results it would seem that findOne is 35-45% faster.
Few data points from the report:
find_one (small, no index): 989 Ops/s
find (small, no index): 554 Ops/s
It is almost like an alias but instead of return you a list, it returns you an object.
It depends on your search query. E.g if you search by ID, since ID is unique it would not need to limit the results because only one result would be found. If more than one record is found then it would limit the results by 1. Another difference is that findOne returns an array, while find returns a mongoCursor.