Mongodb find query with mutiple conditions in PHP - php

I have three working queries:
To find the rows with keyword in title field
$cursor = $collection->find(['title' => array('$regex'=>new MongoRegex($title_query))])->sort(array('timestamp'=>-1));
To find the rows with keyword in the author field
$cursor = $collection->find(['author' => array('$regex'=>new MongoRegex($author_query))])->sort(array('timestamp'=>-1));
To find the rows within a date range
$rangeQuery = array('timestamp' => array( '$gte' => $from_Id, '$lte' => $to_Id ));
$cursor = $collection->find($rangeQuery)->sort(array('timestamp'=>-1));
I want to combine the queries into 1&2, 1&3 and 2&3. However I am not able to write the correct query...
Here is my query for combining 1&2:
$cursor = $collection->find('title' => array('$regex'=>new MongoRegex($title_query)),
'author' => array('$regex'=>new MongoRegex($author_query)))->sort(array('timestamp'=>-1));
query for combining 1&3:
$rangeQuery = array('timestamp' => array( '$gte' => $from_Id, '$lte' => $to_Id ));
$cursor = $collection->find($rangeQuery, ['title' => array('$regex'=>new MongoRegex($title_query))])->sort(array('timestamp'=>-1));
Can anyone tell me how to write the correct query?

The $regex query operator should not be necessary if you are using the BSON regex type (i.e. MongoRegex in the PHP driver). Let's rewrite the original three queries:
Matching keyword in title, sorted by time descending:
$collection->find([
'title' => new MongoRegex($title_query),
])->sort(['timestamp' => -1]);
Matching keyword in author, sorted by time descending:
$collection->find([
'author' => new MongoRegex($author_query),
])->sort(['timestamp' => -1]);
Matching within a date range, sorted by time descending:
$collection->find([
'timestamp' => [
'$gte' => $from_Id,
'$lte' => $to_Id,
],
])->sort(['timestamp' => -1]);
There are several errors in the combined query examples you shared in the OP. For the "1&2" query, you were not passing an array as the first argument to find(), so that would have been a syntax error. For the "1&3" query, you're only passing the timestamp range as find() criteria, and the title regex is being incorrected passed as the second find() argument, which is reserved for specifying a project (i.e. which fields to return).
Combining the three queries is actually as easy as merging the criteria arrays. For example, we could combine all three like so:
$collection->find([
'title' => new MongoRegex($title_query),
'author' => new MongoRegex($author_query),
'timestamp' => [
'$gte' => $from_Id,
'$lte' => $to_Id,
],
])->sort(['timestamp' => -1]);
In some cases, it's not possible to merge criteria for the same field. For that reason, MongoDB has an $and query operator (see the examples for some use cases); however, in the examples above, the criteria is simple enough that you can simply combine the arrays.

Related

Dynamically add columns to query results via CakePHP 3 ORM queries

I'm trying to write a query using CakePHP 3.7 ORM where it needs to add a column to the result set. I know in MySQL this sort of thing is possible: MySQL: Dynamically add columns to query results
So far I've implemented 2 custom finders. The first is as follows:
// src/Model/Table/SubstancesTable.php
public function findDistinctSubstancesByOrganisation(Query $query, array $options)
{
$o_id = $options['o_id'];
$query = $this
->find()
->select('id')
->distinct('id')
->contain('TblOrganisationSubstances')
->where([
'TblOrganisationSubstances.o_id' => $o_id,
'TblOrganisationSubstances.app_id IS NOT' => null
])
->orderAsc('Substances.app_id')
->enableHydration(false);
return $query;
}
The second custom finder:
// src/Model/Table/RevisionSubstancesTable.php
public function findProductNotifications(Query $query, array $options)
{
$date_start = $options['date_start'];
$date_end = $options['date_end'];
$query = $this
->find()
->where([
'RevisionSubstances.date >= ' => $date_start,
'RevisionSubstances.date <= ' => $date_end
])
->contain('Substances')
->enableHydration(false);
return $query;
}
I'm using the finders inside a Controller to test it out:
$Substances = TableRegistry::getTableLocator()->get('Substances');
$RevisionSubstances = TableRegistry::getTableLocator()->get('RevisionSubstances');
$dates = // method to get an array which has keys 'date_start' and 'date_end' used later.
$org_substances = $Substances->find('distinctSubstancesByOrganisation', ['o_id' => 123);
if (!$org_substances->isEmpty()) {
$data = $RevisionSubstances
->find('productNotifications', [
'date_start' => $dates['date_start'],
'date_end' => $dates['date_end']
])
->where([
'RevisionSubstances.substance_id IN' => $org_substances
])
->orderDesc('RevisionSubstances.date');
debug($data->toArray());
}
The logic behind this is that I'm using the first custom finder to produce a Query Object which contains unique (DISTINCT in SQL) id fields from the substances table, based on a particular company (denoted by the o_id field). These are then fed into the second custom finder by implementing where(['RevisionSubstances.substance_id IN' ....
This works and gives me all the correct data. An example of the output from the debug() statement is as follows:
(int) 0 => [
'id' => (int) 281369,
'substance_id' => (int) 1,
'date' => object(Cake\I18n\FrozenDate) {
'time' => '2019-09-02T00:00:00+00:00',
'timezone' => 'UTC',
'fixedNowTime' => false
},
'comment' => 'foo',
'substance' => [
'id' => (int) 1,
'app_id' => 'ID000001',
'name' => 'bar',
'date' => object(Cake\I18n\FrozenDate) {
'time' => '2019-07-19T00:00:00+00:00',
'timezone' => 'UTC',
'fixedNowTime' => false
}
]
],
The problem I'm having is as follows: Each of the results returned contains a app_id field (['substance']['app_id'] in the array above). What I need to do is perform a count (COUNT() in MySQL) on another table based on this, and then add that to the result set.
I'm unsure how to do this for a couple of reasons. Firstly, my understanding is that custom finders return Query Objects, but the query is not executed at this point. Because I haven't executed the query - until calling $data->toArray() - I'm unsure how I would refer to the app_id in a way where it could be referenced per row?
The equivalent SQL that would give me the required results is this:
SELECT COUNT (myalias.app_id) FROM (
SELECT
DISTINCT (tbl_item.i_id),
tbl_item.i_name,
tbl_item.i_code,
tbl_organisation_substances.o_id,
tbl_organisation_substances.o_sub_id,
tbl_organisation_substances.app_id,
tbl_organisation_substances.os_name
FROM
tbl_organisation_substances
JOIN tbl_item_substances
ON tbl_organisation_substances.o_sub_id = tbl_item_substances.o_sub_id
JOIN tbl_item
ON tbl_item.i_id = tbl_item_substances.i_id
WHERE
tbl_item.o_id = 1
AND
tbl_item.date_valid_to IS NULL
AND
tbl_organisation_substances.app_id IS NOT NULL
ORDER BY
tbl_organisation_substances.app_id ASC
) AS myalias
WHERE myalias.app_id = 'ID000001'
This does a COUNT() where the app_id is ID000001.
So in the array I've given previously I need to add something to the array to hold this, e.g.
'substance' => [
// ...
],
'count_app_ids' => 5
(Assuming there were 5 rows returned by the query above).
I have Table classes for all of the tables referred to in the above query.
So my question is, how do you write this using the ORM, and add the result back to the result set before the query is executed?
Is this even possible? The only other solution I can think of is to write the data (from the query I have that works) to a temporary table and then perform successive queries which UPDATE with the count figure based on the app_id. But I'm really not keen on that solution because there are potentially huge performance problems of doing this. Furthermore I'd like to be able to paginate my query so ideally need everything confined to 1 SQL statement, even if it's done across multiple finders.
I've tagged this with MySQL as well as CakePHP because I'm not even sure if this is achievable from a MySQL perspective although it does look on the linked SO post like it can be done? This has the added complexity of having to write the equivalent query using Cake's ORM.

PHP - MongoDB Aggregation: GROUP BY on multi-type field

I have a field in my MongoDB Collection that is hosting two types of data. In some Documents that field has Integer value, e.g.
"campaign_code" : NumberLong(100097)
And in other Documents that field has Array value, e.g.
"campaign_code" : [NumberLong(100087), NumberLong(100136), NumberLong(100137), NumberLong(100138), NumberLong(100135)]
Now, previously I was grouping my result by "campaign_code", but at that time it had only Integer values. Now, the field is having two types of values. The question is is PHP MongoDB driver intelligent to perform the same functionality or do I need to change my code?
My previous PHP code:
$pipeline = array(
array('$match' => array('impression.affiliate_id' => $affiliate_id)),
array(
'$group' => array(
'_id' => array(
'impression.campaign_code' => '$impression.campaign_code'
),
'count' => array('$sum' => 1)
)
),
//sort
array('$sort' => array('count' => -1))
);
I did make some changes and added the following line of code:
array('$unwind' => '$impression.campaign_code')
But this throws an exception:
exception: Value at end of $unwind field path '$impression.campaign_code' must be an Array, but is a NumberLong64
Now the exception is quite valid because few documents have only Integer value in the field. Tell me how I can resolve this issue?

MONGO DB - PHP. Query in multiple nested data field

here expample of my data:
'_id' => new MongoId("54087e076c03943c3c8b456b"),
'fornitureFuture' =>
array (
'0' =>
array (
'data_start'▼ => new MongoDate(1412114400, 0),
'data_end' => new MongoDate(1414710000, 0),
'f1' => '65',
'f2' => new MongoInt32(0),
'f3' => '45',
'fornitore' => new MongoId("5346cb2ab9d6f0021e6b18a0"),
),
'1' =>
array (
'data_start' => new MongoDate(1420066800, 0),
'data_end' => new MongoDate(1427752800, 0),
'f1' => '63.75',
'f2' => new MongoInt32(0),
'f3' => '70.4',
'fornitore' => new MongoId("533406896c0394a62c8b4569"),
),
i need to find if exist a data in fornitureFuture with my MongoDate between data_start and data_end ...
first group data_start is 10/01/2014 and data_end 10/31/2014
Second group data_start is 01/01/2015 and data_end 03/31/2015
Something like:
//today is 09/19/2014
$dataTest = mktime(0,0,0,date('n')+2,14,date('Y')); //return 11/14/2014
$testMese = $this->db->getOne('MyCollection', array('_id'=> new \MongoId($thisPodPdr['_id']), 'fornitureFuture.data_start'=>array('$lte'=> new \MongoDate($dataTest)) , 'fornitureFuture.data_end'=>array('$gt'=> new \MongoDate($dataTest)) ) , array('fornitureFuture'=>1) );
I expect empty response but return the record..
2 example:
$dataTest = mktime(0,0,0,date('n')+7,14,date('Y')); //return 04/14/2015
$testMese = $this->db->getOne('MyCollection', array('_id'=> new \MongoId($thisPodPdr['_id']), 'fornitureFuture.data_start'=>array('$lte'=> new \MongoDate($dataTest)) , 'fornitureFuture.data_end'=>array('$gt'=> new \MongoDate($dataTest)) ) , array('fornitureFuture'=>1) );
return correctly empty!
I need to test in the same block ...something like
'fornitureFuture.$.data_start'=>array('$lte'=> new \MongoDate($dataTest)) , 'fornitureFuture.$.data_end'=>array('$gt'=> new \MongoDate($dataTest))
but dont work .
the \ is from namespace and this->db->getOne(collection,$query,$fields) is my function like $this->collection->findOne($query,$fields);
No syntax error.
sorry for my english and thanks for the help
Your first example, which returns 2 results when you expect none, runs the following query:
[
'_id' => new MongoId(...),
'fornitureFuture.data_start' => ['$lte' => new MongoDate(1418533200)],
'fornitureFuture.data_end' => ['$gt' => new MongoDate(1418533200)],
]
The example document you provided has two array elements, with the following date ranges:
1412114400 to 1414710000
1420066800 to 1427752800
This document matches because 1412114400 (of the first element) is less than 1418533200, and 1427752800 (of the second element) is greater than 1418533200. By simply referring to fornitureFuture.data_start and fornitureFuture.data_end, MongoDB's query matcher will be satisfied if any array element's sub-field meets the criteria.
You likely want to restrict start/end criteria to same array element, in which case $elemMatch is what you're looking for:
[
'_id' => new MongoId(...),
'fornitureFuture' => [
'$elemMatch' => [
'data_start' => ['$lte' => new MongoDate(1418533200)],
'data_end' => ['$gt' => new MongoDate(1418533200)],
],
],
]
This criteria should now match only when the start/end dates of the same element satisfy the range. On a related note, you may also be interested in the $ projection operator, to limit fornitureFuture to only the matched element(s).

Lithium & MongoDB: Finding documents by date range

I have numerous documents containing a field called "date" which is simply a unix timestamp.
whithin lithium, i want to find all documents in a given date range. i'm currently trying the following:
//$_stats contains two \DateTime objects which are properly initialized
$transactions = Transactions::all(
array('conditions' => array(
'tags' => array('$all' => array((string)$tag->_id)),
'date' => array('$gte' => array((int)$_stats['date_start']->getTimestamp()), '$lte' => array((int)$_stats['date_end']->getTimestamp()))
))
);
But this returns zero documents. When I remove the "date" condition, it works fine and I get all documents.
What am I missing?
Thanks, aenogym
There doesn't seem to be any need of giving an array of dates, so perhaps try:
$transactions = Transactions::all(
array('conditions' => array(
'tags' => array('$all' => array((string)$tag->_id)),
'date' => array('$gte' => (int)$_stats['date_start']->getTimestamp(), '$lte' => (int)$_stats['date_end']->getTimestamp())
))
);
Keep in mind that MongoDate stores dates as miliseconds while timestamp uses seconds. In other words MongoDate has higher precision.

MongoDB & PHP - How do I exclude results from a date range delimited query?

Long time peruser, first time question asker ...
Using PHP to query our MongoDB page visit log, I would like to get a set of records between two time periods, but exclude results that have a certain userAgent. I've figured out the time range but cannot find anywhere that explains the exclude.
Here's what I have for the query so far:
$dateRange = $collection->find(array("timeStamp" => array('$gt' => $start,
'$lt' => $end)));
Looking for code to complete the find function to exclude the records with a "userAgent" starting with "ELB"
What you're looking for is $ne or $nin, depending on whether the value you want to exclude is a single value or array of values. eg:
$dateRange = $collection->find(array("timeStamp" => array('$gt' => $start, '$lt' => end), 'userAgent' => array('$ne' => new MongoRegex('/^ELB/'))));
Documentation here:
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24ne
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24nin
You could append {$not: /^ELB/} to the mongo query.
Not really sure about the equivalent PHP but try something like this:
$dateRange = $collection->find(array(
'timeStamp' => array(
'$gt' => $start,
'$lt' => $end
),
'userAgent' => array(
'$not' => new MongoRegex('/^ELB/')
)
));

Categories