I am trying to save some db action by compiling a looped bit of code with a single query, Before I was simply adding to the the like statements using a loop before firing off the query but i cant get the same idea going in Mongo, id appreciate any ideas....
I am basically trying to do a like, but with the value as an array
('app', replaces 'mongodb' down to my CI setup )
Here's how I was doing it pre mongofication:
foreach ($workids as $workid):
$this->ci->app->or_like('work',$workid) ;
endforeach;
$query = $this->ci->db->get("who_users");
$results = $query->result();
print_r($results);
and this is how I was hoping I could get it to work, but no joy here, that function is only designed to accept strings
$query = $this->ci->app->like('work',$workids,'.',TRUE,TRUE)->get("who_users");
print_r($query);
If anyone can think of a way any cunning methods I can get my returned array with a single call again it would be great I've not found any documentation on this sort of query, The only way i can think of is to loop over the query and push it into a new results array.... but that is really gonna hurt if my app scales up.
Are you using codeigniter-mongodb-library? Based on the existing or_like() documentation, it looks like CI wraps each match with % wildcards. The equivalent query in Mongo would be a series of regex matches in an $or clause:
db.who_users.find({
$or: [
{ work: /.*workIdA.*/ },
{ work: /.*workIdB.*/ },
...
]});
Unfortunately, this is going to be quite inefficient unless (1) the work field is indexed and (2) your regexes are anchored with some constant value (e.g. /^workId.*/). This is described in more detail in Mongo's regex documentation.
Based on your comments to the OP, it looks like you're storing multiple ID's in the work field as a comma-delimited string. To take advantage of Mongo's schema, you should model this as an array of strings. Thereafter, when you query on the work field, Mongo will consider all values in the array (documented discussed here).
db.who_users.find({
work: "workIdA"
});
This query would match a record whose work value was ["workIdA", "workIdB"]. And if we need to search for one of a set of ID's (taking this back to your OR query), we can extend this example with the $in operator:
db.who_users.find({
work: { $in: ["workIdA", "workIdB", ...] }
});
If that meets your needs, be sure to index the work field as well.
Related
I have a Laravel controller which takes a search parameter, looks through all the records in a certain time range, uses the laravel collection ->filter method, and then on the results of that filter method, returns the json ->json
$logs = RequestLog::orderBy('created_at', 'DESC')->whereBetween('created_at', [$start, $end])->get();
$logs = $logs->filter(function($log) { /* my own logic in here */ });
return response()->json($logs->toJson());
Now this seems to work just fine. It normally returns an array of the matching records, [{"id":1},{"id":2},{"id":3}]
But if the ->filter function leaves one result, let's say the 25th record in the array, the response now comes out like this: {25: {"id": 25}}. It ruins the normal array structure and returns a JSON object with a single key.
Is this expected? What's up with this? How do I deal with this?
Note: my filter function cannot be done via sql means, it has to be done in PHP.
[edit] it's actually returning an Object JSON anytime the filter results are anything other than the first items in the original Query results.
Normally after applying methods that modifies (especially removes an item from) a collection, the indexes may not come ordered. This is why you need ->values() on the collection.
So you'd have:
return response()->json($logs->values()->toJson());
values() Resets the keys on the underlying array.
I have the following structure within a mongoDB collection:
{
"_id" : ObjectId("5301d337fa46346a048b4567"),
"delivery_attempts" : {
"0" : {
"live_feed_id" : 107,
"remaining_attempts" : 2,
"delivered" : false,
"determined_status" : null,
"date" : 1392628536
}
}
}
// > db.lead.find({}, {delivery_attempts:1}).pretty();
I'm trying to select any data from that collection where remaining_attempts are greater than 0 and a live_feed_id is equal to 107. Note that the "delivery_attempts" field is of a type hash.
I've tried using an addAnd within an elemMatch (not sure if this is the correct way to achieve this).
$qb = $this->dm->createQueryBuilder($this->getDocumentName());
$qb->expr()->field('delivery_attempts')
->elemMatch(
$qb->expr()
->field('remaining_attempts')->gt(0)
->addAnd($qb->expr()->field('live_feed_id')->equals(107))
);
I do appear to be getting the record detailed above. However, changing the greater than
test to 3
->field('remaining_attempts')->gt(3)
still returns the record (which is incorrect). Is there a way to achieve this?
EDIT: I've updated the delivery_attempts field type from a "Hash" to a "Collection". This shows the data being stored as an array rather than an object:
"delivery_attempts" : [
{
"live_feed_id" : 107,
"remaining_attempts" : 2,
"delivered" : false,
"determined_status" : null,
"date" : 1392648433
}
]
However, the original issue still applies.
You can use a dot notation to reference elements within a collection.
$qb->field('delivery_attempts.remaining_attempts')->gt(0)
->field('delivery_attempts.live_feed_id')->equals(107);
It works fine for me if I run the query on mongo.
db.testQ.find({"delivery_attempts.remaining_attempts" : {"$gt" : 0}, "delivery_attempts.live_feed_id" : 107}).pretty()
so it seems something wrong with your PHP query, I suggest running profiler to see which query is actually run against mongo
db.setProfilingLevel(2)
This will log all operation since you enable profiling. Then you can query the log to see which the actual queries
db.system.profile.find().pretty()
This might help you to find the culprit.
It sounds like your solved your first problem, which was using the Hash type mapping (instead for storing BSON objects, or associative arrays in PHP) instead of the Collection mapping (intended for real arrays); however, the query criteria in the answer you submitted still seems incorrect.
$qb->field('delivery_attempts.remaining_attempts')->gt(0)
->field('delivery_attempts.live_feed_id')->equals(107);
You said in your original question:
I'm trying to select any data from that collection where remaining_attempts are greater than 0 and a live_feed_id is equal to 107.
I assume you'd like that criteria to be satisfied by a single element within the delivery_attempts array. If that's correct, the criteria you specified above may match more than you expect, since delivery_attempts.remaining_attempts can refer to any element in the array, as can the live_feed_id criteria. You'll want to use $elemMatch to restrict the field criteria to a single array element.
I see you were using elemMatch() in your original question, but the syntax looked a bit odd. There should be no need to use addAnd() (i.e. an $and operator) unless you were attempting to apply two query operators to the same field name. Simply add extra field() calls to the same query expression you're using for the elemMatch() method. One example of this from ODM's test suite is QueryTest::testElemMatch(). You can also use the debug() method on the query to see the raw MongoDB query object created by ODM's query builder.
I need to store some data that is essentially just an array of key-value pairs of date/ints, where the dates will always be unique.
I'd like to be able to store it like an associative array:
array(
"2012-02-26" => 5,
"2012-02-27" => 2,
"2012-02-28" => 17,
"2012-02-29" => 4
)
but I also need to be able to query the dates (ie. get everything where date > 2012-02-27), and so suspect that I'll need to use a schema more like:
array(
array("date"=>"2012-02-26", "value"=>5),
array("date"=>"2012-02-27", "value"=>2),
array("date"=>"2012-02-28", "value"=>17),
array("date"=>"2012-02-29", "value"=>4),
)
Obviously the former is much cleaner and more concise, but will I be able to query it in the way that I am wanting, and if not are there any other schemas that may be more suitable?
You've described two methods, let me break them down.
Method #1 - Associative Array
The key tool for querying by "associative array" is the $exists operator. Here are details on the operator.
So you can definitely run a query like the following:
db.coll.find( { $exists: { 'field.2012-02-27' } } );
Based on your description you are looking for range queries which does not match up well with the $exists operator. The "associative array" version is also difficult to index.
Method #2 - Array of objects
This definitely has better querying functionality:
db.coll.find( { 'field.date': { $gt: '2012-02-27' } } );
It can also be indexed
db.coll.ensureIndex( { 'field.date': 1 } );
However, there is a trade-off on updating. If you want to increment the value for a specific date you have to use this unwieldy $ positional operator. This works for an array of objects, but it fails for anything with further nesting.
Other issues
One issue with either of these methods is the long-term growth of data. As you expand the object size it will take more space on disk and in memory. If you have an object with two years worth of data that entire array of 700 items will need to be in memory for you to update data for today. This may not be an issue for your specific data, but it should be considered.
In the same vein, MongoDB queries always return the top-level object. Again, if you have an array of 700 items, you will get all of them for each document that matches. There are ways to filter out the fields that are returned, but they don't work for "arrays of objects".
I'm working on a project where all of the members and their info are stored in a JSON file. I'm in the process of creating a search form and I need help on how to iterate through the members and check to see if there's an exact match or a similar match.
The members are stored in a SESSION variable:
$_SESSION['members'] = json_decode($jsonFile);
but I'm uncertain how to use regex to check for matches that are similar (and not just exact). For example, if a member's name is "Jonathan", I'd like that result to be returned even if the user searches "Jon". Is regex the correct approach? Any help will be greatly appreciated - thank you!
-Manoj
I think I'd be using a database to store the data rather than JSON so that you can use the LIKE searches, e.g.
SELECT * FROM users WHERE name LIKE 'Jon%'
If you absolutely have to use JSON you could loop through all members and use a regexp like
preg_match('/^'.$term.'.*/i', $element, $matches);
to check them all.
If the $jsonFile contents is an array of some sort, you may find preg_grep() of use, though it doesn't work on multidimensional arrays. You might have have to loop over each individual member record and grep the relevant fields yourself, something like:
foreach ($_SESSION['members'] as $idx => $member) {
... match relevant fields...
}
someone wrote this code.
foreach ($node->taxonomy as $term) {
$tids[] = 't.tid = %d';
$args[] = $term->tid;
}
how he knows that in foreach "$node->taxonomy" is an array? and when i loop it,
foreach ($node->taxonomy as $term) {
}
the output that i get will be the $term's value. i don't know how it is change into the 't.tid = %d' and $term->tid. thank you.
In Drupal-related code, a $node is almost always an object produced by the node_load() function. Since every module has the opportunity to add its own properties to this object, it's very hard to find a central documentation of these properties.
By experience and by variable inspection, seasoned Drupal developers know that when set $node->taxonomy is always an array of term object (as returned by the taxonomy_get_term() function) indexed by their respective ids (named tids, for Term ID). This array is set by the taxonomy_nodeapi() function when $op == 'load' and is produced by the taxonomy_get_terms() function.
The question give little information but we can guess that the loop is meant to build two arrays used to generate a database query that filter on the tid column matching those of the $node object. Because the terms' data is already stored in the items of $node->taxonomy, let's hope that this query is not used to re-load the terms to display some of their name and/or description. Collecting 't.tid = %d' is probably a bad idea, the query would be better build with a single "tid in (". db_placeholder($args) .")" WHERE clause after collecting all the tids in $args.
The question is very unclear. All Items under the node object are arrays. You can check it yourself bu using:
print_r($node);
die;
Or using any PHP debugger.
for the foreach, It is very simple foreach... I don't understand what is the problem with that.
t.tid is simply an SQL query. %d is a placeholder for $args[], which consists of $term->tid. It's like this structure: PDO connections.