I wrote a query with join ( hasMany() ) relation.
I want to fetch distinct value and need count for the pagination.
But when set the distinct true, count becomes slow.
$query = Post::find();
$query->joinWith( [ 'tags' ] )->where( [ 'tags.tag_id' => 1 ] );
$query->distinct();
$pages = new Pagination( [ 'totalCount' => $query->count(), 'pageSize' => 25, 'page' => 1 ] );
Thanks in advance.
You do not need distinct()
I am reasonably sure that if you remove distinct everything will work ok. Each post will be there just 1 time. That is because of the way Active records handles things.
Related
I'm trying to write a query using CakePHP 3.7 ORM where it needs to add a column to the result set. I know in MySQL this sort of thing is possible: MySQL: Dynamically add columns to query results
So far I've implemented 2 custom finders. The first is as follows:
// src/Model/Table/SubstancesTable.php
public function findDistinctSubstancesByOrganisation(Query $query, array $options)
{
$o_id = $options['o_id'];
$query = $this
->find()
->select('id')
->distinct('id')
->contain('TblOrganisationSubstances')
->where([
'TblOrganisationSubstances.o_id' => $o_id,
'TblOrganisationSubstances.app_id IS NOT' => null
])
->orderAsc('Substances.app_id')
->enableHydration(false);
return $query;
}
The second custom finder:
// src/Model/Table/RevisionSubstancesTable.php
public function findProductNotifications(Query $query, array $options)
{
$date_start = $options['date_start'];
$date_end = $options['date_end'];
$query = $this
->find()
->where([
'RevisionSubstances.date >= ' => $date_start,
'RevisionSubstances.date <= ' => $date_end
])
->contain('Substances')
->enableHydration(false);
return $query;
}
I'm using the finders inside a Controller to test it out:
$Substances = TableRegistry::getTableLocator()->get('Substances');
$RevisionSubstances = TableRegistry::getTableLocator()->get('RevisionSubstances');
$dates = // method to get an array which has keys 'date_start' and 'date_end' used later.
$org_substances = $Substances->find('distinctSubstancesByOrganisation', ['o_id' => 123);
if (!$org_substances->isEmpty()) {
$data = $RevisionSubstances
->find('productNotifications', [
'date_start' => $dates['date_start'],
'date_end' => $dates['date_end']
])
->where([
'RevisionSubstances.substance_id IN' => $org_substances
])
->orderDesc('RevisionSubstances.date');
debug($data->toArray());
}
The logic behind this is that I'm using the first custom finder to produce a Query Object which contains unique (DISTINCT in SQL) id fields from the substances table, based on a particular company (denoted by the o_id field). These are then fed into the second custom finder by implementing where(['RevisionSubstances.substance_id IN' ....
This works and gives me all the correct data. An example of the output from the debug() statement is as follows:
(int) 0 => [
'id' => (int) 281369,
'substance_id' => (int) 1,
'date' => object(Cake\I18n\FrozenDate) {
'time' => '2019-09-02T00:00:00+00:00',
'timezone' => 'UTC',
'fixedNowTime' => false
},
'comment' => 'foo',
'substance' => [
'id' => (int) 1,
'app_id' => 'ID000001',
'name' => 'bar',
'date' => object(Cake\I18n\FrozenDate) {
'time' => '2019-07-19T00:00:00+00:00',
'timezone' => 'UTC',
'fixedNowTime' => false
}
]
],
The problem I'm having is as follows: Each of the results returned contains a app_id field (['substance']['app_id'] in the array above). What I need to do is perform a count (COUNT() in MySQL) on another table based on this, and then add that to the result set.
I'm unsure how to do this for a couple of reasons. Firstly, my understanding is that custom finders return Query Objects, but the query is not executed at this point. Because I haven't executed the query - until calling $data->toArray() - I'm unsure how I would refer to the app_id in a way where it could be referenced per row?
The equivalent SQL that would give me the required results is this:
SELECT COUNT (myalias.app_id) FROM (
SELECT
DISTINCT (tbl_item.i_id),
tbl_item.i_name,
tbl_item.i_code,
tbl_organisation_substances.o_id,
tbl_organisation_substances.o_sub_id,
tbl_organisation_substances.app_id,
tbl_organisation_substances.os_name
FROM
tbl_organisation_substances
JOIN tbl_item_substances
ON tbl_organisation_substances.o_sub_id = tbl_item_substances.o_sub_id
JOIN tbl_item
ON tbl_item.i_id = tbl_item_substances.i_id
WHERE
tbl_item.o_id = 1
AND
tbl_item.date_valid_to IS NULL
AND
tbl_organisation_substances.app_id IS NOT NULL
ORDER BY
tbl_organisation_substances.app_id ASC
) AS myalias
WHERE myalias.app_id = 'ID000001'
This does a COUNT() where the app_id is ID000001.
So in the array I've given previously I need to add something to the array to hold this, e.g.
'substance' => [
// ...
],
'count_app_ids' => 5
(Assuming there were 5 rows returned by the query above).
I have Table classes for all of the tables referred to in the above query.
So my question is, how do you write this using the ORM, and add the result back to the result set before the query is executed?
Is this even possible? The only other solution I can think of is to write the data (from the query I have that works) to a temporary table and then perform successive queries which UPDATE with the count figure based on the app_id. But I'm really not keen on that solution because there are potentially huge performance problems of doing this. Furthermore I'd like to be able to paginate my query so ideally need everything confined to 1 SQL statement, even if it's done across multiple finders.
I've tagged this with MySQL as well as CakePHP because I'm not even sure if this is achievable from a MySQL perspective although it does look on the linked SO post like it can be done? This has the added complexity of having to write the equivalent query using Cake's ORM.
Here is my correct sql query:
SELECT * FROM `messages` WHERE ( (sender_id=3 AND user_id=40) OR (sender_id=40 AND user_id=3)) AND offer_id=1
I want to use this in Cakephp syntax:
$this->Message->find('all',array('conditions'=>array(
'AND'=>array(
'OR'=>array(
'Message.offer_id'=>$offer_id,
'Message.sender_id'=>$sender_id,
'Message.user_id'=>$this->Auth->user('id'),
),
'OR'=>array(
'Message.offer_id'=>$offer_id,
'Message.user_id'=>$sender_id,
'Message.sender_id'=>$this->Auth->user('id')
)
)
),
'recursive'=>2
));
Is there anyone who can help me to figure out the issue. Basically I want to get all the messages whether I sent or received for an particular offer.
You should move $offer_id out of or conditions and move it to and conditions.
Why? Lets look at your first or array:
That conditions will return true if:
sender_id is 3
OR user_id is 40
OR offer_id is 1
So, that condition may return true event if offer_id != 1
That should be written this way (as precisely as possible according to original query):
$query = $this
->Messages
->find('all' , [
'conditions' => [
'or' => [
[
'sender_id' => $sender_id,
'user_id' => $this->Auth->user('id')
], [
'sender_id' => $this->Auth->user('id'),
'user_id' => $sender_id
]
],
'offer_id' => $offer_id,
]
]);
dump($query);
In dump we can see something like this:
"SELECT * FROM messages Messages WHERE (((sender_id = :c0 AND user_id = :c1) OR (sender_id = :c2 AND user_id = :c3)) AND offer_id = :c4)
asterisk in sql query dump for more readability
You have the AND and OR operators reversed.
'OR'=>array(
'AND'=>array(
'Message.offer_id'=>$offer_id,
'Message.sender_id'=>$sender_id,
'Message.user_id'=>$this->Auth->user('id'),
),
'AND'=>array(
'Message.offer_id'=>$offer_id,
'Message.user_id'=>$sender_id,
'Message.sender_id'=>$this->Auth->user('id')
)
)
I am using subquery for id field.
$db = $this->AccountRequest->getDataSource();
$subQuery = $db->buildStatement(
array(
'fields' => array('MAX(id)'),
'table' => $db->fullTableName($this->AccountRequest),
'alias' => 'MaxRecord',
'limit' => null,
'offset' => null,
'order' => null,
'group' => array("user_id")
),
$this->AccountRequest
);
$searching_parameters = array(
#"AccountRequest.id IN " => "(SELECT MAX( id ) FROM `account_requests` GROUP BY user_id)"
"AccountRequest.id IN " => "(".$subQuery.")"
);
$this->Paginator->settings = array(
#'fields' => array('AccountRequest.*'),
'conditions' => $searching_parameters,
'limit' => $limit,
'page' => $page_number,
#'group' => array("AccountRequest.user_id"),
'order' => array(
'AccountRequest.id' => 'DESC'
)
);
$data = $this->Paginator->paginate('AccountRequest');
This structure is producing a query is:
SELECT
`AccountRequest`.`id`,
`AccountRequest`.`user_id`,
`AccountRequest`.`email`,
`AccountRequest`.`emailchange`,
`AccountRequest`.`email_previously_changed`,
`AccountRequest`.`first_name`,
`AccountRequest`.`first_namechange`,
`AccountRequest`.`f_name_previously_changed`,
`AccountRequest`.`last_name`,
`AccountRequest`.`last_namechange`,
`AccountRequest`.`l_name_previously_changed`,
`AccountRequest`.`reason`,
`AccountRequest`.`status`,
`AccountRequest`.`created`,
`AccountRequest`.`modified`
FROM
`syonserv_meetauto`.`account_requests` AS `AccountRequest`
WHERE
`AccountRequest`.`id` IN '(SELECT MAX(id) FROM `syonserv_meetauto`.`account_requests` AS `MaxRecord` WHERE 1 = 1 GROUP BY user_id)'
ORDER BY
`AccountRequest`.`id` DESC
LIMIT 25
In the subquery, its add an extra single quote so it's producing an error.
So, How can I remove these single quotes from this subquery?
Thanks
What are you trying to achieve with the sub query?
The MAX(id) just means it will pull the id with the largest value AKA the most recent insert. The sub query is completely redundant when you can just ORDER BY id DESC.
using MAX() will return only one record, if this is what you want to achieve you can replicate by adding LIMIT 1
If the sub query is just an example and is meant to be from another table I would just run the query that gets the most recent id before running the main query. Getting the last inserted id in a separate query is very quick and I cant see much of a performance loss. I think it will result in cleaner code that`s easier to follow to.
edit 1: From the comments it sounds like all your trying to get is a particular users latest account_requests.
You dont need the sub query at all. My query below will get the most recent account record for the user id you choose.
$this->Paginator->settings = array(
'fields' => array('AccountRequest.*'),
'conditions' => array(
'AccountRequest.user_id' => $userID // you need to set the $userID
)
'page' => $page_number,
'order' => array(
'AccountRequest.id DESC' //shows most recent first
),
'limit' => 1 // set however many you want the maximum to be
);
The other thing you cold be meaning is to get multiple entries from multiple users and display them in order of user first and then the order of recent to old for that user. MYSQL lets you order by more than one field, in that case try:
$this->Paginator->settings = array(
'conditions' => array(
'AccountRequest.user_id' => $userID // you need to set the $userID
)
'page' => $page_number,
'order' => array(
'AccountRequest.user_id', //order by the users first
'AccountRequest.id DESC' //then order there requests by recent to old
)
);
If the example data you have added into the question is irrelevant and you are only concerned about how to do nested subqueries it has already been answered here
CakePHP nesting two select queries
However I still think based on the data in the question you can avoid using a nested query.
I'm new to cakephp2 and I wanted to ask you a little favor.I'm currently learning cakephp2 but I'm having some hard time trying to understand how to use where and groupby clause in cakephp2.
I am tring to convert this sql query below to cakphp2 but how will I do it using find() ?
select params,count(params) from pv_logs
where dt = '2014/9/25' and is_crawler = 0
group by params order by count(params) desc limit 100 ;
This is how I did it in cakephp2 format but do you see something wrong with this ?
$pvcount = $this->PvLog->find('all', array(
'fields' => array('dt','params','count(params)'),
'conditions'=>array('PvLog.dt'=>'2014/9/25','PvLog.is_crawler'=>0),
'group'=>array('PvLog.params'),
'order'=>array('PvLog.count(params)'),
'limit' => 100,
));
Try Below
$this->PvLog->virtualFields['params_count']='count(params)';
$pvcount = $this->PvLog->find('all', array(
'fields' => array('dt','params','params_count'),
'conditions'=>array('PvLog.dt'=>'2014-09-25','PvLog.is_crawler'=>0),
'group'=>array('PvLog.params'),
'order'=>array('params_count'=>'asc'),
'limit' => 100,
));
In my CakePHP model I'm trying to get some data from my table.
I tried using DISTINCT but it seems like using DISTINCT doesn't change the query results.
I can see many rows that has the same nick
with 'DISTINCT Mytable.nick'
$this->Mytable->find('all',
array(
'fields'=> array(
'DISTINCT Mytable.nick',
'Mytable.age', 'Mytable.location',
),
'conditions' => array('Mytable.id >=' => 1, 'Mytable.id <=' => 100),
'order' => array('Mytable.id DESC')
));
with 'group Mytable.nick'
$this->Mytable->find('all',
array(
'fields'=> array(
'Mytable.nick',
'Mytable.age', 'Mytable.location',
),
'conditions' => array('Mytable.id >=' => 1, 'Mytable.id <=' => 100),
'group' => 'Mytable.nick',
'order' => array('Mytable.id DESC')
));
with 'Mytable.nick'
$this->Mytable->find('all',
array(
'fields'=> array(
'Mytable.nick',
'Mytable.age', 'Mytable.location',
),
'conditions' => array('Mytable.id >=' => 1, 'Mytable.id <=' => 100),
'order' => array('Mytable.id DESC')
));
Edit: It seems like even CakePHP 2.1 can't use DISTINCT option. When I tried "GROUP BY" it solved my issue. But as you can see from my query I need to order results with Mytable.id descended. When I use GROUP BY, when Mysql finds relevant row, it doesn't take others. For example.
id=1, nick=mike, age=38, location=uk
id=2, nick=albert, age=60, location=usa
id=3, nick=ash, age=42, location=uk
id=4, nick=albert, age=60, location=new_zelland
When I use group Mytable.nick, I don't see 4th row in my results, I see 2nd row. Because when mysql saw "albert" second time, it doesn't put it into my results. But I need latest "albert" result. Is it not possible?
Edit2: It seems like order by/group by conflict is a common problem. I found some tips in this question. But it gives solution for native Mysql queries. I need a solution for CakePHP type queries.
Not clear on why you want to group by nick and order by id. Do you intend to use an aggregate function like COUNT() to see how many occurrences of the same nick there are? In short you overall goal still is not clear to me. Might be worth being aware of the HAVING MySQL keyword.
Updated: Ok, that makes more sense. So you need to use a sub select on the condition or perhaps express that as a join. I'll try and show an example using the sub select in the WHERE clause.
/* select last occurrence for each nick (if you need one for each location )*/
SELECT nick, age, location
FROM myTable t1
WHERE id =
(SELECT MAX(id)
FROM myTable t1
WHERE t1.nick = t2.nick);
Would think something like this would work:
$this->Mytable->find('all',
array(
'fields'=> array(
'Mytable.nick',
'Mytable.age', 'Mytable.location',
),
'conditions' => array('Mytable.id =' => '(SELECT MAX(id) FROM myTable t2 WHERE myTable.nick = t2.nick)', 'Mytable.id <=' => 100)
));