I have two tables, one that looks like this:
ID, Datetime, User_ID, Location and Status // the rest is not relevant
And the other looks like this:
ID, Lastname // the rest is not relevant
Now I only want to get the entry of the first table with the highest Datetime per User_ID and ask the other table the lastname of the User_ID. Simple...
I tried it this way (whick looks like the most promising but is false nontheless):
SELECT w.Datetime, w.User_ID, w.Status, e.Lastname
FROM worktimes AS w
INNER JOIN employees AS e
ON w.User_ID=e.ID
RIGHT JOIN (SELECT max(Datetime) AS Datetime, User_ID
FROM worktimes
WHERE Datetime>1467583200 AND Location='16'
GROUP BY User_ID
ORDER BY Datetime DESC
) AS v
ON v.User_ID=w.User_ID
GROUP BY w.User_ID
ORDER BY e.Nachname;
Could someone give me a hint please? I'm really stuck at this for a while now and now i begin to get some knots in my brain... :(
You are very close, actually:
SELECT w.Datetime, w.User_ID, w.Status, e.Lastname
FROM worktimes w INNER JOIN
employees e
ON w.User_ID = e.ID LEFT JOIN
(SELECT max(Datetime) AS Datetime, User_ID
FROM worktimes
WHERE Datetime > 1467583200 AND Location = '16'
GROUP BY User_ID
) ww
ON ww.User_ID = w.User_ID AND w.DateTime = ww.DateTime
ORDER BY e.Nachname;
Notes:
You do need to join on the DateTime value.
The RIGHT JOIN is unnecessary. I replaced it with a LEFT JOIN, but I'm not sure that is what you want either. You might start with an INNER JOIN to see if that produces what you want.
Do not use ORDER BY in subqueries in most circumstances.
You do not need the GROUP BY in the outer query
What you are asking about is known as correlated subqueries. In standard SQL it can be implemented using APPLY and LATERAL constructs. Unfortunatelly, not all RDBMS support these elegant solutions. For example, MSSQL, recent versions of Oracle and Postgresql have these constructs, but MySQL does not. IMHO, it is a real pain for MySQL users, because in recent years MySQL started to lean towards standard, but in some strange manner - by default it switches off its non-standard hacks, but does not implement standard counterparts. For example, your own query presented in your question will not work by default in recent versions of MySQL, because sorting in subqueries is not supported any more and to make it work you have to use some nasty hack - add LIMIT some_really_big_number to the subquery.
Related
I am using yii2 data Provider to extract data from database. Raw query looks like this
SELECT `client_money_operation`.* FROM `client_money_operation`
LEFT JOIN `user` ON `client_money_operation`.`user_id` = `user`.`id`
LEFT JOIN `client` ON `client_money_operation`.`client_id` = `client`.`id`
LEFT JOIN `client_bonus_operation` ON `client_money_operation`.`id` = `client_bonus_operation`.`money_operation_id`
WHERE (`client_money_operation`.`status`=0) AND (`client_money_operation`.`created_at` BETWEEN 1 AND 1539723600)
GROUP BY `operation_code` ORDER BY `created_at` DESC LIMIT 10
this query takes 107 seconds to execute.
Table client_money operations contains 132000 rows. What do I need to do to optimise this query, or set up my database properly?
Try pagination. But if you must have to show large set of records in one go remove as many left joins as you can. You can duplicate some data in the client_money_operation table if it is certainly required to show in the one-go result set.
SELECT mo.*
FROM `client_money_operation` AS mo
LEFT JOIN `user` AS u ON mo.`user_id` = u.`id`
LEFT JOIN `client` AS c ON mo.`client_id` = c.`id`
LEFT JOIN `client_bonus_operation` AS bo ON mo.`id` = bo.`money_operation_id`
WHERE (mo.`status`=0)
AND (mo.`created_at` BETWEEN 1 AND 1539723600)
GROUP BY `operation_code`
ORDER BY `created_at` DESC
LIMIT 10
is a rather confusing use of GROUP BY. First, it is improper to group by one column while having lots of non-aggregated columns in the SELECT list. And the use of created_at in the ORDER BY does not make sense since it is unclear which date will be associated with each operation_code. Perhaps you want MIN(created_at)?
Optimization...
There will be a full scan of mo and (hopefully) PRIMARY KEY lookups into the other tables. Please provide EXPLAIN SELECT ... so we can check this.
The only useful index on mo is INDEX(status, created_at), and it may or may not be useful, depending on how big that date range is.
bo needs some index starting with money_operation_id.
What table(s) are operation_code and created_at in? It makes a big difference to the Optimizer.
But there is a pattern that can probably be used to greatly speed up the query. (I can't give you details without knowing what table those columns are in, nor whether it can be made to work.)
SELECT mo.*
FROM ( SELECT mo.id FROM .. WHERE .. GROUP BY .. ORDER BY .. LIMIT .. ) AS x
JOIN mo ON x.id = mo.id
ORDER BY .. -- yes, repeated
That is, first do (in a derived table) the minimal work to find ids for the 10 rows desired, then use JOIN(s) to fetch there other columns needed.
(If yii2 cannot be made to generate such, then it is in the way.)
The problem is that if there is 0 comment or 1 comment the count shows 1 while the rest is working well means that 2, 3, etc working fine.
$sql = "SELECT blog.*,count(blog.id) as Total FROM blog left JOIN comment on comment.id = blog.id GROUP BY date desc";
Your query should look like this:
SELECT b.date, count(c.id) as Total
FROM blog b LEFT JOIN
comment c
ON c.id = b.id
GROUP BY b.date DESC;
This assumes that date comes from blog (which should be the case if your current query is working). The difference is that you are counting from the second table, not the first.
This does not use * for columns from blog. That is usually a very, very bad idea when using GROUP BY. The best practice (enforced by almost all SQL engines) is to only include unaggregated columns in the SELECT when they are in the GROUP BY.
Note: It seems very awkward that the same column id is used for the JOIN between two very different entities (blogs and comments).
i just change to count(comment.id) from count(blog.id)
I have a query I need to run which is currently running extremely slow. The table I am working with has nearly 1 million records.
What I need to do is search for items with a LIKE name : Example "SELECT name, other, stuff FROM my_table WHERE name LIKE '%some_name%'"
In addition it has to be able to exclude certain terms with wildcards from the search results so it turns into something like this :
SELECT name, other, stuff
FROM my_table
WHERE name LIKE '%some_name%'
AND name NOT IN (SELECT name FROM my_table WHERE name LIKE '%term1%' AND name LIKE '%term2%' AND name LIKE '%term3%')
To top things off, I have two INNER JOINs and I have to execute it in PHP. I have indexes on the table for relevant columns. It is on a server that is fast for pretty much every other query I can throw at it.
The question is, is there a different method I could be using to do this type of query thats is quick?
UPDATES
This is my actual query after adding in FULLTEXT index to the chem table and trying the match function instead. NOTE : I don't have control over table/column names. I know they dont look nice.
CREATE FULLTEXT INDEX name_index ON chems (name)
SELECT f.name fname, f.state, f.city, f.id fid,
cl.alt_facilid_ida, cl.alt_facili_idb,
c.ave, c.name, c.id chem_id,
cc.first_name, cc.last_name, cc.id cid, cc.phone_work
FROM facilities f
INNER JOIN facilitlt_chemicals_c cl ON (f.id = cl.alt_facilid485ilities_ida)
INNER JOIN chems c ON (cl.alt_facili3998emicals_idb = c.id)
LEFT OUTER JOIN facilities_contacts_c con ON (f.id = con.alt_facili_ida)
LEFT OUTER JOIN contacts cc ON (con.alt_facili_idb = cc.id)
WHERE match(c.name) against ('+lead -gas' IN BOOLEAN MODE)
That is just an example of a simple query. The actual terms could be numerous.
You want to implement MySQL's full text capabilities on the columns you're searching. Read more about it here.
Moreover, you probably want to do a boolean search:
select
t1.name,
t1.stuff
from
my_table t1
where
match(t1.name) against ('+some_name -term1 -term2 -term3' in boolean mode)
I would suggest to use external Full-text engine like Lucene or Sphinx. Especially if you plan to fire search queries against relatively big datasets.
In Sphinx case you can use SELECT-like syntax called SphinxQL which will do exactly as required:
SELECT * FROM <sphinx_index_name> WHERE MATCH('some_name -term1 -term2 -term3');
as described in http://sphinxsearch.com/docs/current.html#extended-syntax (NOT operator in your case). Plus you could enable morphology support which might be helpful as well.
You can combine MySQL and Sphinx queries with http://sphinxsearch.com/docs/current.html#sphinxse
Can some one optimize this mysql query
SELECT submittedform.*, inspectors.first_name, inspectors.last_name
FROM (
SELECT `dinsp`,`departure`,`arrival`,'cabin' as type FROM cabinets
UNION
SELECT `dinsp`,`departure`,`arrival`,'cockpit' as type FROM cockpits
ORDER BY `date_of_inspection` ASC
) AS submittedform
INNER JOIN inspectors ON inspectors.id = submittedform.dinsp
I don't want to rely on nested query or is it fine in this case? Also suggest me a cakephp solution but the tables can't be related.
You can try:
SELECT sf.`dinsp`, sf.`departure`, sf.`arrival`, sf.`type`, i.`first_name`, i.`last_name`
FROM
`inspectors` AS i INNER JOIN (
SELECT `dinsp`, `departure`, `arrival`, `date_of_inspection`, 'cabin' AS `type`
FROM `cabinets`
UNION ALL
SELECT `dinsp`, `departure`, `arrival`, `date_of_inspection`, 'cockpit' AS `type`
FROM `cockpits`
) AS sf ON sf.`dinsp` = i.`id`
ORDER BY sf.`date_of_inspection`
UNION ALL will not check for duplicates. Always put the ORDER BY clause in the outer query to ensure proper ordering.
It would be better to avoid using UNION because it will not allow the query optimizer to use any index you may have on dinsp and date_of_inspection. But that would mean changing the schema.
An alternative to a UNION sub-query is to make the main query into two parts with a UNION between:
SELECT c.dinsp, c.departure, d.arrival, 'cabin' AS type, i.first_name, i.last_name
FROM cabinets AS c JOIN inspectors AS i ON i.id = c.dinsp
SELECT c.dinsp, c.departure, d.arrival, 'cockpit' AS type, i.first_name, i.last_name
FROM cockpits AS c JOIN inspectors AS i ON i.id = c.dinsp
It is not clear that this would give significantly different performance. If anything, it would be worse since it involves two scans of the Inspectors table, but that isn't likely to be very big so it may not matter very much. Your UNION sub-query minus the ORDER BY is likely to be as good as or slightly better than this. Your ORDER BY on a non-selected field is problematic in the inner query; and needs careful handling in the UNION I'm proposing (probably by selecting the extra column).
SELECT c.dinsp, c.date_of_inspection, c.departure, d.arrival, 'cabin' AS type,
i.first_name, i.last_name
FROM cabinets AS c JOIN inspectors AS i ON i.id = c.dinsp
SELECT c.dinsp, c.date_of_inspection, c.departure, d.arrival, 'cockpit' AS type,
i.first_name, i.last_name
FROM cockpits AS c JOIN inspectors AS i ON i.id = c.dinsp
ORDER BY date_of_inspection;
I just realized that I'm going to have to start aliasing my database calls due to repeating column names in my join tables. Is there a way to automatically tell SQL to alias all my column names so that they are returned with a prefix of the table name? Otherwise it appears to be quite confusing when only some of them are aliased. Just trying to be consistent without writing tons of extra code.
$sql = "SELECT contracts.po_number, contracts.start_date, contracts.end_date, contracts.description, contracts.taa_required, contracts.account_overdue, jobs.id AS jobs_id, jobs.job_number, companies.id AS companies_id, companies.name AS companies_name
FROM contracts
LEFT JOIN jobs ON contracts.job_id = jobs.id
LEFT JOIN companies ON contracts.company_id = companies.id
WHERE contracts.id = '$id'
ORDER BY contracts.end_date";
No, but you can make life a little easier by using table aliases:
SELECT c.po_number, c.start_date, c.end_date, c.description,
c.taa_required, c.account_overdue, j.id AS jobs_id, j.job_number,
cm.id AS companies_id, cm.name AS companies_name
FROM contracts c
LEFT JOIN jobs j ON c.job_id = j.id
LEFT JOIN companies cm ON c.company_id = cm.id
WHERE c.id = '$id'
ORDER BY c.end_date
you can use alias tables in your sql statements so you have to write less, but to actually access the columns from php there's no way around aliasing all of them, if you want to access them by name.
you can also access columns with indexes from php, but that's a maintenance nightmare
I would recommend to always alias table names. It makes it very hard to read later, if you skip alias.
For info, theres a gotcha in MySQL 5.6 (possibly others!)
SELECT *
FROM table1
LEFT JOIN table2 ON PKI = FKI
works as expected.. but recently I mispelt 'LEFT' as 'LEFY' in a query and it also worked but with a standard join! so therefore
SELECT *
FROM table1
LEFY JOIN table2 ON PKI = FKI
also works just fine as does any substitute for the word LEFY, so beware a typo changing your query !!