Database Query Execution Takes Too Long - php

Am currently having this issue where my database query takes too long to execute and sometimes ends up failing with an error: Maximum Execution Time Exceeded... Am trying to select data from different tables that aren't directly related to each other, i.e One table is related to a table which is related to another table and so on, Some of this tables have over a thousand records, I have a feeling the structure of the different tables contributes to this headache, but I haven't been able to figure out how to reduce this execution time. Am using a CodeIgniter model where i created a function with the following sql query:
/**
* Get payment history based on search term
* #param string $search
* #param int $limt => default is 10
* #return mixed
**/
public function get_search_history($usertin, $search, $limit, $offset){
$this->db->select("
rv.name AS 'Revenue Name',rv.code AS 'Rev. Code',
m.name AS 'MDA',m.sector AS 'Sector', b.code AS 'Bank Code',
b.name AS 'Bank',p.amount AS 'Amount',p.channel AS 'Payment Channel',
p.date AS 'Payment Date',p.platform AS 'Platform',p.ref AS 'Reference',
t.tin AS 'TIN',t.active AS 'Is Active?',r.ref AS 'Receipt Reference'
");
$this->db->from('revenue AS rv');
$this->db->join('mda AS m', 'm.id = rv.mda_id', 'inner');
$this->db->join('payment AS p', 'p.revenue_id = rv.id', 'inner');
$this->db->join('bank AS b', 'b.id = p.bank_id', 'inner');
$this->db->join('tin AS t', 't.id = p.tin_id', 'inner');
$this->db->join('receipt AS r', 'r.id = p.receipt', 'inner');
$this->db->where('t.tin', $usertin);
$this->db->like('rv.code', $search);
$this->db->or_like('rv.name', $search);
$this->db->or_like('m.name', $search);
$this->db->group_by('t.id');
$this->db->order_by('p.date', 'ASC');
$this->db->limit($limit, $offset);
$query = $this->db->get();
return ( ( $query->num_rows() > 0 ) ? $query->result_array() : 0 );
}
The query is supposed to retrieve a user's payment history based on a particular search term or keyword!
It's easier to see from the above query what I meant when I said "...One table is related to a table which is related to another table..." I Just can't figure out a better way of doing this. Can anyone help me out? It's given me serious headaches the past couple of days!
This is the output of the query:
SELECT rv.name AS 'Revenue Name', rv.code AS 'Rev. Code', m.name AS
'MDA', m.sector AS 'Sector', b.code AS 'Bank Code', b.name AS 'Bank',
p.amount AS 'Amount', p.channel AS 'Payment Channel', p.date AS 'Payment
Date', p.platform AS 'Platform', p.ref AS 'Reference', t.tin AS 'TIN',
t.active AS 'Is Active?', r.ref AS 'Receipt Reference'
FROM `revenue` AS `rv`
INNER JOIN `mda` AS `m` ON `m`.`id` = `rv`.`mda_id`
INNER JOIN `payment` AS `p` ON `p`.`revenue_id` = `rv`.`id`
INNER JOIN `bank` AS `b` ON `b`.`id` = `p`.`bank_id`
INNER JOIN `tin` AS `t` ON `t`.`id` = `p`.`tin_id`
INNER JOIN `receipt` AS `r` ON `r`.`id` = `p`.`receipt`
WHERE `t`.`tin` = '1904260529-0001'
AND `rv`.`code` LIKE '%With%' ESCAPE '!'
OR `rv`.`name` LIKE '%With%' ESCAPE '!'
OR `m`.`name` LIKE '%With%' ESCAPE '!'
GROUP BY `t`.`id`
ORDER BY `p`.`date` ASC
LIMIT 3, 20

If you have this much tables to join frequently, there are 2 possible ways I can suggest.
1. Use CodeIgniter's feature Query Caching
You just have to enable it from database.php and have to give it a folder path where all cached queries will lie.
Then you can create hook or something to delete and regenerate that cached queries on particular condition.
2. Create View in MySQL
You can create View in MySQL which will be much faster. It can be done easily right from PHPMyAdmin even.
Hope this helps.
p.s. Try indexing proper columns.
Simple thumb rule is As more conditions in WHERE clause, As lower query will be

Related

codeigniter join from 2nd table id with 3rd table id multi rows

I want to join multiple tables, as in my picture:
Here is my code:
$this->db->select('
pt2.turl as `p_img`,
p.title as `p_title`,
p.text as `p_text`,
p.create as `p_date`,
pt3.turl as `c_img`,
u.name as `c_name`,
c.text as `c_text`,
c.create as `c_date`
');
$this->db->from('posts as p, users as u, photos as pt2, photos as pt3');
$this->db->join('comments as c', 'p.id=c.pid AND u.id=c.uid');
$this->db->join('posts as p2', 'p2.pid=pt2.id', 'rihgt');
$this->db->join('users as u2', 'u2.photoid=pt3.id', 'right');
$this->db->order_by('c.id', 'DESC');
$this->db->limit('7');
$qry = $this->db->get();
return $qry->result();
If I understand you correctly, this is kind of what you're looking for. I haven't tried it out so it may not be exact, but you shouldn't need to make 2 table associations (pt2 and pt3) for the same table in the join. Just Include them in the select and join on the unique ID's
The "Left" is a join that is centered around you're left table so everything hangs off of that. Since you are joining the users table before the photo table, you should be able to join on its columns.
Hope this helps. Let me know if I missed something. :)
$select = array(
pt2.turl as `p_img`,
p.title as `p_title`,
p.text as `p_text`,
p.create as `p_date`,
pt2.turl as `c_img`,
u.name as `c_name`,
c.text as `c_text`,
c.create as `c_date`
);
//Set tables to variables. Just makes it easier for me
$postsTable = "posts as p"; //This will be your left table.
$userTable = "Users as u";
$commentsTable = "comments as c";
$photosTable = "photos as pt2";
$this
->db
->select($select)
->from($postsTable)
->join($userTable, "p.uid = u.id", "left")
->join($commentsTable, "p.cid = c.id", "left")
->join($photosTable, "u.photoid = pt2.id", "left")
->get();
I solved this problem myself
It would be like this:
$select= array (
//'*',
'pt.turl p_img',
'p.title p_title',
'p.text p_text',
'p.create p_date',
'pt2.turl c_img',
'c.text c_text',
'u.name c_name',
'c.create c_date'
);
$from = array (
'posts p'
);
$qry = $this
->db
->select($select)
->from($from)
->join('comments c', 'c.pid=p.id')
->join('photos pt', 'pt.id=p.pid')
->join('users u', 'u.id=c.uid')
->join('photos pt2', 'u.photoid=pt2.id')
->order_by('c.create', 'DESC')
->limit('7')
->get();
return $qry->result();

Convert from query to ModelSearch of Yii2

I'm new in Yii2, and I have a query with right result:
SELECT DISTINCT workloadTeam.project_id, wp.project_name, workloadTeam.user_id, workloadTeam.commit_time, wp.workload_type FROM
(SELECT p.id, p.project_name, w.user_id, w.commit_time, w.comment, w.workload_type
FROM workload as w, project as p
WHERE w.user_id = 23 AND p.id = w.project_id) wp
INNER JOIN workload as workloadTeam ON wp.id = workloadTeam.project_id
But in my ModelSearch.php, I wrote:
$user_id = Yii::$app->user->id;
$subquery = Workload::find()->select('p.id', 'p.project_name', 'w.user_id', 'w.commit_time', 'w.comment', 'w.workload_type')
->from(['project as p', 'workload as w'])
->where(['user_id' => $user_id, 'p.id' => 'w.project_id']);
$query = Workload::find()
->select(['workloadTeam.project_id', 'wp.project_name', 'workloadTeam.user_id', 'workloadTeam.from_date', 'workloadTeam.to_date', 'workloadTeam.workload_type', 'workloadTeam.comment'])
->where(['', '', $subquery]);
$query->join('INNER JOIN', 'workload as workloadTeam', 'wp.id = workloadTeam.project_id');
It happended error:
SELECT COUNT(*) FROM `workload` INNER JOIN `workload` `workloadTeam` ON wp.id = workloadTeam.project_id WHERE `` (SELECT p.project_name `p`.`id` FROM `project` `p`, `workload` `w` WHERE (`user_id`=20) AND (`p`.`id`='w.project_id'))
And I can't fix it with right query above.
You have any solution about this?
Is this error shown in the Yii-debug toolbar? Then your query (which you mentioned as error) is probably only the count from the query which is listed before.
You missed to add the sub-query in from clause like you shown in your working sql. Add this in your where clause were just the wrong place. Put sub-queries in whereconditions, if you have scalar results, because you have to use this result with operands like =, >=, in...
This could work:
$user_id = Yii::$app->user->id;
$subquery = Workload::find()->select([
'p.id as id',
'p.project_name as project_name',
'w.user_id as user_id',
'w.commit_time as commit_time',
'w.comment as comment',
'w.workload_type as workload_type'
])
->from([
'project as p',
'workload as w'
])
->where([
'user_id' => $user_id,
'p.id' => 'w.project_id'
]);
$query = Workload::find()
->select([
'workloadTeam.project_id',
'wp.project_name',
'workloadTeam.user_id',
'workloadTeam.from_date',
'workloadTeam.to_date',
'workloadTeam.workload_type',
'workloadTeam.comment'
])
->from([$subquery => 'wp']); //you were missing this line
$query->join('INNER JOIN', 'workload as workloadTeam', 'wp.id = workloadTeam.project_id');
But you don't use any selects from your workload table in your main-query $query...
Since I don't know what's your goal to achieve I can't help you at this topic...

Left join to get a single row in Laravel

I have been unsuccessfully trying to leftjoin and get the required data
Here is my code:
$album = Albums::->where('users_id',$user_id)
->leftJoin('photos',function($query){
$query->on('photos.albums_id','=','albums.id');
$query->where('photos.status','=',1);
//$query->limit(1);
//$query->min('photos.created_at');
})
->where('albums.status',1)->get();
The comments are some of my several trying...
I want to get only a single record from the photos table matching the foreign key album_id which was updated first and also with status 1
pls help...
I have used DB::raw() in order to achieve this
$album = Albums::select( 'albums.*',
DB::raw('(select photo from photos where albums_id = albums.id and status = 1 order by id asc limit 1) as photo') )
->where('users_id',$user_id)
->where('albums.status',1)->get();
#JarekTkaczyk 's coding was similar and displayed the same result as I needed, so a special thanks to him for his time and effort...
But comparing the execution time for the quires I stayed to mine as my above snippet
select `albums`.*, (select photo from photos where albums_id = albums.id and status = 1 order by id asc limit 1) as photo from `albums` where `users_id` = '1' and `albums`.`status` = '1'
took 520μs - 580μs
and #JarekTkaczyk 's
select `albums`.*, `p`.`photo` from `albums` left join `photos` as `p` on `p`.`albums_id` = `albums`.`id` and `p`.`created_at` = (select min(created_at) from photos where albums_id = p.albums_id) and `p`.`status` = '1' where `users_id` = '1' and `albums`.`status` = '1' group by `albums`.`id`
took 640μs - 750μs But both did the same...
You can achieve it using either leftJoin or rightJoin (but the latter would return Photo models, so probably you won't need that):
Albums::where('users_id', $user_id)
->leftJoin('photos as p', function ($q) {
$q->on('photos.albums_id', '=', 'albums.id')
->on('photos.updated_at', '=',
DB::raw('(select min(updated_at) from photos where albums_id = p.albums_id)'))
->where('photos.status', '=', 1);
})
->where('albums.status', 1)
->groupBy('albums.id')
->select('albums.*', fields from photos table that you need )
->get();
Are you trying to check for albums that have the status of '1'? If this is the case you are missing an equals sign from your final where.
Try:
->where('albums.status','=',1)->first();
Alternatively you may be able to achieve this with an 'on' instead of a 'where' inside the join function. You also don't need to split up query inside of the function and can do it as one line with the '->' :
$album = Albums::->where('users_id',$user_id)
->leftJoin('photos',function($query){
$query->on('photos.albums_id','=','albums.id')
->on('photos.status','=',1);
})
->where('albums.status','=',1)->first();
You need to make sure that you are using 'first', as it will return a single row of the first result. Get() will return an array.
As a straightforward answer which results in a single object I suggest the following query:
$album = DB::table('albums')->select('albums.*', 'photos.photo')
->join('photos', 'photos.id', '=', 'albums.id')
->where('users_id',$user_id)
->where('albums.status',1)
->first();

Creating a Laravel query from SQL with Join and alias

I have written a SQL query in Microsoft SQL Management Studio i want to write it in Laravel.
Here is my SQL query.
SELECT Ordre.OrdreId, Ordre.BestiltDato, Ordre.PlanlagtFerdigDato, Ordre.FerdigDato,Ordre.FakturaDato, OAA11.Fritekst, OAA09.Dato, OAA26.Dato
from Ordre
LEFT OUTER JOIN OrdreArbeidAvdeling OAA11 ON( OAA11.AvdelingsId = 11 AND OAA11.OrdreId = Ordre.OrdreId )
LEFT OUTER JOIN OrdreArbeidAvdeling OAA09 ON( OAA09.AvdelingsId = 9 AND OAA09.OrdreId = Ordre.OrdreId )
LEFT OUTER JOIN OrdreArbeidAvdeling OAA26 ON( OAA26.AvdelingsId = 26 AND OAA26.OrdreId = Ordre.OrdreId )
Where Ordre.BestiltDato > '01.01.2013' AND Ordre.FerdigDato IS NULL AND Ordre.FakturaDato IS NULL and Ordre.PlanlagtFerdigDato < '12.24.2014'
AND (lower(OAA11.Fritekst) not LIKE '%avven%')
AND (OAA09.Dato IS NULL OR (OAA09.Dato IS NOT NULL and OAA26.Dato IS NOT NULL))
AND (lower(OAA11.Fritekst) NOT LIKE '%2015%')
Using the documentation from Laravel i am able to do all but one join, the one where it uses different "AvdelingsId" ( 9 and 26) in one where statement.
AND (OAA09.Dato IS NULL OR (OAA09.Dato IS NOT NULL and OAA26.Dato IS NOT NULL))
my Laravel code at this point looks like this.
$orders = DB::table('Ordre')->join('OrdreArbeidAvdeling', function($OAA11)
{
$OAA11->on('OrdreArbeidAvdeling.OrdreId','=','Ordre.OrdreId')
->where('OrdreArbeidAvdeling.AvdelingsId', '=', '11')
->where('OrdreArbeidAvdeling.Fritekst', 'not like', '%avven%')
->where('OrdreArbeidAvdeling.Fritekst', 'not like', '%2015%');
},'left outer')
/*->join('OrdreArbeidAvdeling', function($OAA9and26){
$OAA9and26->on('OrdreArbeidAvdeling.OrdreId','=','Ordre.OrdreId')
#What am i suppose to do with AvdelingsId = 9 and AvdelingsId = 26
#
;
},'left outer')*/
->select('Ordre.OrdreId', 'Ordre.BestiltDato', 'Ordre.PlanlagtFerdigDato', 'Ordre.FerdigDato','Ordre.FakturaDato')
->where('BestiltDato', '>', '2013-01-01')
->where('PlanlagtFerdigDato', '<', '2014-12-24')
->whereNull('FerdigDato')
->whereNull('FakturaDato')
->orderBy('OrdreId')
->get();
If i where able to use alias name i might just be able to make this work. I have also limited knowledge of Laravel, what i know i have found in the documentation, any suggestions on how to do this would be very kind.
You should try to add alias in the join, as I said in the comments.
Something like this:
...->join('OrdreArbeidAvdeling as OAA11', ...)
I hope it works fine for you.

CodeIgniter Active Record use "USING" in query

I am trying to replicate the following query using Active Record in CodeIgniter but I can't find where in the documentation or searching how you use USING in a query.
SELECT *
FROM categories C
LEFT OUTER JOIN motorcycle_brands MB
USING ( CatID )
LEFT OUTER JOIN powersport_categories PS
USING ( CatID )
WHERE C.CatID = :uniqueID
This is what I have now:
$this->db->select('*');
$this->db->from('categories C');
$this->db->join('motorcycle_brands MB', 'CatID', 'left outer');
$this->db->join('powersport_categories PS', 'CatID', 'left outer');
$this->db->where('C.CatID =', $this->option_id);
$suboptions = $this->db->get();
I have tried changing
$this->db->join('motorcycle_brands MB', 'USING (CatID)', 'left outer');
$this->db->join('powersport_categories PS', 'USING (CatID)', 'left outer');
There is no built-in support for JOIN using USING in the active record class. The best that you can do is change the join() function in "system/database/DB_active_rec.php" this file.
Please refer this question for the complete reference.
codeigniter active records join with using?
The USING clause is used if several columns share the same name but you don’t want to join using all of these common columns. The columns listed in the USING clause can’t have any qualifiers in the statement, including the WHERE clause:
The ON clause is used to join tables where the column names don’t match in both tables. The join conditions are removed from the filter conditions in the WHERE clause:
So your code should be,
$this->db->select('*');
$this->db->from('categories C');
$this->db->join('motorcycle_brands MB', 'MB.CatID = C.Id', 'left outer');
$this->db->join('powersport_categories PS', 'C.Id = PS.CatID ', 'left outer');
$this->db->where('C.CatID =', $this->option_id);
$suboptions = $this->db->get();

Categories