how to minimize the response time for this query in mysql - php

I have table having list of football matches. it has columns like match_id, team a score, team 2 score, rounds, match_date etc.
I need all rows from database with highest margin of that round
margin is difference of team a score and team b score.
my query is
SELECT *,( SELECT MAX(ABS((z.home_score - z.away_score)))
FROM tblform_matches z
WHERE YEAR(z.match_date) = YEAR(tblform_matches.match_date)
AND z.round = tblform_matches.round ) as highest_margin
from tblform_matches where some condtion
it is a simplified query where some condition is a large query string to select some specified matches according to filter.
currently there are around 5000 matches in database.
Due to sub-query my page is taking 4 more seconds to load.
there are 9 matches in each round and there are more than 20 rounds in every year
I am executing the above query for every team in php loop. I cant change this thing. as there are a lot of calculation for showing stats.
Sorry if my question is uncleared, I am here if I missed something as I am a new bee to stakoverflow
Thanks in advance.

This is your query:
SELECT m.*,
(SELECT MAX(ABS((m2.home_score - m2.away_score)))
FROM tblform_matches m2
WHERE YEAR(m2.match_date) = YEAR(m.match_date) AND
m2.round = m.round
) as highest_margin
from tblform_matches m
where some condition;
Presumably, the best way to optimize this is to focus on . Oh well. You will want the right indexes there.
Indexes are clearly the solution, but you have a problem because the of the year function. And easy solution is to use indequalities:
SELECT m.*,
(SELECT MAX(ABS((m2.home_score - m2.away_score)))
FROM tblform_matches m2
WHERE m2.round = m.round
(m2.match_date >= makedate(year(m.match_date), 1) and
m2.match_date < makedate(year(m.match_date) + 1, 1)
)
) as highest_margin
from tblform_matches m
where some condtion;
The best index for the subquery is tblform_matches(round, match_date, home_score, away_score). The first two columns are used for the where clause. The second two for the select.
Note: if you made two relatively minor changes to the data structure, this could work even better. Add a column for the year of the match date (redundant, but important for indexing). And, add a column for the absolute value of the difference between the scores. Then the query would be:
SELECT m.*,
(SELECT MAX(score_diff)
FROM tblform_matches m2
WHERE m2.round = m.round and m2.matchyear = m.matchyear
) as highest_margin
from tblform_matches m
where some condtion;
The index on this query would be: tblform_matches(round, matchyear, score_diff) and the lookup should be pretty fast.
EDIT:
You may get better performance with an explicit join:
SELECT m.*, m2.highest_margin
from tblform_matches m join
(select MAX(ABS((m2.home_score - m2.away_score))) as highest_margin
from tblform_matches m2
group by year(m2.match_date), m2.round
) m2
on year(m.match_date) = year(m2.match_date) and m2.round = m.round
where some condition;

Related

Select child records from tables mysql

I got the bellow piece of select statement that got level 2 child records, having problems to got deeper, can anyone help out?
SELECT
id_mobile AS ID_PROJETO,
UM.qtd_UC,
AM.qtd_AMBIENTE
FROM projetos_mobile AS PM
LEFT JOIN (
SELECT
COUNT(id) AS qtd_UC,
projeto,
data_hora_importacao,
id_uc_mobile
FROM ucs_mobile
WHERE data_hora_importacao = '2015-05-15 17:21:02'
GROUP BY projeto) AS UM
ON PM.id_mobile = UM.projeto
LEFT JOIN (
SELECT
COUNT(id_uc_mobile) AS qtd_AMBIENTE,
id_uc_mobile
FROM ucs_mobile
LEFT JOIN (
SELECT
uc
FROM ambientes_mobile AS s
WHERE data_hora_importacao = '2015-05-15 17:21:02') AS G
ON G.uc = ucs_mobile.id_uc_mobile
WHERE data_hora_importacao = '2015-05-15 17:21:02') AS AM
ON UM.id_uc_mobile = AM.id_uc_mobile
WHERE PM.data_hora_importacao = '2015-05-15 17:21:02'
http://sqlfiddle.com/#!9/2eecf
here is a sqlfiddle if anyone want to try a solution. I have the specific hierarchy: projeto>uc>ambiente>secao>medicoes
ucs_mobile.projeto refers to projetos_mobile.id_mobile
ambientes_mobile.uc refers to ucs_mobile.id_uc_mobile
secoes_iluminacao_mobile.ambiente refers to ambientes_mobile.id_ambiente_mobile
I need a count of each child for the parent I pass, I will have 5 functions that
return the count of each child for a given parent, for example, for a projeto parent I should have count(ucs),count(ambientes),count(secoes),count(medicoes)
So, hope you guys can help me. The database is terrible ugly but that's is what I got. Appreciate any help.
When you have really large queries like this, it can often be helpful to break them down individually, starting from the ground up and patching them together.
I started by just getting the count of each ucs_mobile row for each projetos_mobile value. You can do that by joining the two tables on the related row, and using COUNT(DISTINCT um.id) to get the number of rows. There are other ways to do it, but this particular method will scale better for the rest of your query:
SELECT pm.id, COALESCE(COUNT(DISTINCT um.id), 0) AS qty_uc
FROM projetos_mobile pm
LEFT JOIN ucs_mobile um ON um.data_hora_importacao = '2015-05-15 17:21:02' AND um.projeto = pm.id_mobile
GROUP BY pm.id;
The COALESCE function will be used to fill 0 counts. As long as you remember to use the DISTINCT keyword, and group by the proper id, you can just add in the child rows like so:
SELECT
pm.id,
COALESCE(COUNT(DISTINCT um.id), 0) AS qty_uc,
COALESCE(COUNT(DISTINCT am.id), 0) AS qty_am,
COALESCE(COUNT(DISTINCT sim.id), 0) AS qty_sim
FROM projetos_mobile pm
LEFT JOIN ucs_mobile um ON um.data_hora_importacao = '2015-05-15 17:21:02' AND um.projeto = pm.id_mobile
LEFT JOIN ambientes_mobile am ON am.data_hora_importacao = um.data_hora_importacao AND am.uc = um.id_uc_mobile
LEFT JOIN secoes_iluminacao_mobile sim ON sim.data_hora_importacao = am.data_hora_importacao AND sim.ambiente = am.id_ambiente_mobile
GROUP BY pm.id;
Here is an SQL Fiddle example. NOTE I changed your sample data slightly to ensure my query was working as expected.
Also, a side note. I noticed as you went along that you kept using the same date in your WHERE clauses, so I just joined each table on the date as well, and made sure that in my very first join I looked for the date specified, which in turn will carry its way over to the other tables.

Join a query into another query with column computation

I have three tables named issue_details, nature_payments, and rci_records. Now I have this query which joins this three tables.
SELECT issue_details.issue_date AS Date,
issue_details.check_no AS Check_No,
payees.payee_name AS Name_payee,
nature_payments.nature_payment AS Nature_of_Payment,
issue_details.issue_amount AS Checks_issued,
issue_details.nca_balance AS Nca_balance
FROM
issue_details
INNER JOIN
nature_payments ON
issue_details.nature_id = nature_payments.nature_id
INNER JOIN
payees ON
issue_details.payee_id = payees.payee_id
ORDER BY Date Asc, Check_no ASC
On my column in Nca_balance, this is a computed differences of every issuances of check. But you may not know what really the process of how I got the difference but to make it simple, let's say that I have another query
that dynamically get also the difference of this nca_balance column. Here is the query:
SELECT r.*,
(#tot := #tot - issue_amount) as bank_balance
FROM (SELECT #tot := SUM(nca_amount) as nca_total FROM nca
WHERE account_type = 'DBP-TRUST' AND
year(issue_date) = year('2015-01-11') AND
month(issue_date) = month('2015-01-11')
)
vars CROSS JOIN issue_details r
WHERE r.account_type = 'DBP-TRUST' AND
r.issue_date = '2015-01-11'
ORDER BY r.issue_date, r.check_no
I know it you may not get my point but I just want to replace the first query of the line
issue_details.nca_balance AS Nca_balance
with my own computation on my second query.
Please help me combine those two query into a single query. Thanks

How to calculate difference between values coming from the same row in mysql

I am trying to calculate the difference of values list coming from a database.
I would like to achieve it using php or mysql, but I do not know how to proceed.
I have a table named player_scores. One of its rows contains the goals scored.
Ex.
pl_date pl_scores
03/11/2014 18
02/11/2014 15
01/11/2014 10
I would like to echo the difference between the goals scored during the matches played in different dates.
Ex:
pl_date pl_scores diff
03/11/2014 18 +3
02/11/2014 15 +5
01/11/2014 10 no diff
How can I obtain the desired result?
You seem to want to compare a score against the score on a previous row.
Possibly simplest if done using a a sub query that gets the max pl_date that is less than the pl_date for the current row, then joining the results of that sub query back against the player_scores table to get the details for each date:-
SELECT ps1.pl_date, ps1.pl_scores, IF(ps2.pl_date IS NULL OR ps1.pl_scores = ps1.pl_scores, 'no diff', ps1.pl_scores - ps1.pl_scores) AS diff
FROM
(
SELECT ps1.pl_date, MAX(ps2.pl_date) prev_date
FROM player_scores ps1
LEFT OUTER JOIN player_scores ps2
ON ps1.pl_date > ps2.pl_date
GROUP BY ps1.pl_date
) sub0
INNER JOIN player_scores ps1
ON sub0.pl_date = ps1.pl_date
LEFT OUTER JOIN player_scores ps2
ON sub0.prev_date = ps2.pl_date
There are potentially other ways to do this (for example, using variables to work through the results of an ordered sub query, comparing each row with the value stored in the variable for the previous row)
SELECT score FROM TABLE WHERE DATE = TheDateYouWant
$score = $data['score'];
SELECT score FROM TABLE WHERE date = dateYouWant
$difference = $score - $data['score'];
Something like this?
You could use two queries, one to get the value to use in the comparison (in the example below is the smaller number of scores) and the second one to get the records with a dedicated column with the difference:
SELECT MIN(pl_scores);
SELECT pl_date, pl_scores, (pl_scores - minScore) as diff FROM player_scores;
Or, using a transaction (one query execution php side):
START TRANSACTION;
SELECT MIN(Importo) FROM Transazione INTO #min;
SELECT Importo, (Importo - #min) as diff FROM Transazione;
select *,
coalesce(
(SELECT concat(IF(t1.pl_scores>t2.pl_scores,'+',''),(t1.pl_scores-t2.pl_scores))
FROM tableX t2 WHERE t2.pl_date<t1.pl_date ORDER BY t2.pl_date DESC LIMIT 1)
, 'no data' ) as diff
FROM tableX t1
WHERE 1
order by t1.pl_date DESC

Get variance and standard deviation of two numbers in two different rows/columns with sqlite / PHP

I have a SQLite Database with the following structure:
rowid ID startTimestamp endTimestamp subject
1 00:50:c2:63:10:1a 1000 1090 entrance
2 00:50:c2:63:10:1a 1100 1270 entrance
3 00:50:c2:63:10:1a 1300 1310 door1
4 00:50:c2:63:10:1a 1370 1400 entrance
.
.
.
I have prepared a sqlfiddle here: http://sqlfiddle.com/#!2/fe8c6/2
With this SQL-Query i can get the average differences between the endTime and the startTime between one row and the following row, sorted by subject and ID:
SELECT
id,
( MAX(endtimestamp) - MIN(startTimestamp)
- SUM(endtimestamp-startTimestamp)
) / (COUNT(*)-1) AS averageDifference
FROM
table1
WHERE ID = '00:50:c2:63:10:1a'
AND subject = 'entrance'
GROUP BY id;
My problem: To calcute the average value is no problem, that does this query. But how can i
get the standard deviation and the variance of this values?
First finding the time differences of interest by joining the table to itself and grouping by ID, then finding the averages, variances as V(x) = E(x^2) - (E(x))^2 and standard deviation as sqrt(V)gives
SELECT ID, AVG(diff) AS average,
AVG(diff*diff) - AVG(diff)*AVG(diff) AS variance,
SQRT(AVG(diff*diff) - AVG(diff)*AVG(diff)) AS stdev
FROM
(SELECT t1.id, t1.endTimestamp,
min(t2.startTimeStamp) - t1.endTimestamp AS diff
FROM table1 t1
INNER JOIN table1 t2
ON t2.ID = t1.ID AND t2.subject = t1.subject
AND t2.startTimestamp > t1.startTimestamp -- consider only later startTimestamps
WHERE t1.subject = 'entrance'
GROUP BY t1.id, t1.endTimestamp) AS diffs
GROUP BY ID
For formulas that are more complex than simple summation, you have to compute the actual difference values for each record by lookin up the corresponding next start times, like this:
SELECT (SELECT MIN(startTimestamp)
FROM table1 AS next
WHERE next.startTimestamp > table1.startTimestamp
AND ID = '...'
) - endTimestamp AS timeDifference
FROM table1
WHERE nextStartTimestamp IS NOT NULL
AND ID = '...'
Then you can use all the difference values to do the calculations:
SELECT SUM(timeDifference) / COUNT(*) AS average,
AVG(timeDifference) AS moreEfficientAverage,
SUM(timeDifference * timeDifference) / COUNT(*) -
AVG(timeDifference) * AVG(timeDifference) AS variance
FROM (SELECT (SELECT MIN(startTimestamp)
FROM table1 AS next
WHERE next.startTimestamp > table1.startTimestamp
AND next.ID = '...'
) - endTimestamp AS timeDifference
FROM table1
WHERE nextStartTimestamp IS NOT NULL
AND ID = '...')
A number of points:
Your formula for the mean is wrong the correct formula is SUM(endtimestamp-starttimestamp)/COUNT(endtimestamp). I have no idea why you have the MIN/MAX terms. COUNT(*) will count NULL rows and will give the wrong result.
SQLlite has an avg function which finds the mean.
The formula for the variance is SUM((endtimestamp-starttimestamp)*(endtimestamp-starttimestamp)) - AVG(endtimestamp-starttimestamp)*AVG(endtimestamp-starttimestamp)
The standard deviation is the square root of the variance.
In response to the question authors comment, in order to compute the variance the start and end times must be paired with each other through a self join.
Becuase of the absence of a row_number function in SQL lite this is a little inelegant.
SELECT id,
AVG(startTimestamp-endTimestamp) as mean,
SUM((startTimestamp-endTimestamp)^2) - AVG(startTimestamp-endTimestamp)^2 as variance,
SQRT(SUM((startTimestamp-endTimestamp)^2) - AVG(startTimestamp-endTimestamp)^2) as stDev
FROM
(SELECT
t1.id,
t1.endTimestamp,
MIN(t2.startTimestamp) as starttimestamp
FROM table1 t1
INNER JOIN
table1 t2 ON t1.endTimestamp<=t2.startTimestamp
GROUP BY t1.id, t1.endTimestamp) t
GROUP BY id;
See SQL Fiddle

Subqueries to JOIN while keeping logic

I'm trying to optimize a slow query and I've come across the following (running consistently over 8 seconds).
SELECT entryID, entryID AS iE, 0 AS eE, 'clarus1' AS locationID, dateStamp, amount AS income, NULL AS expense, reconciled, leaseID AS vendorID, incomeID AS expenseID
FROM structu_income.iLedger
WHERE (dateStamp BETWEEN '2009-04-16' AND '2012-02-29') AND incomeID IS NOT NULL
AND (
leaseID IN (
SELECT lease.leaseID FROM structu_assets.lease WHERE lease.unitID IN (
SELECT unit.unitID FROM structu_assets.unit WHERE unit.locationID = 'clarus1'
)
)
OR locationID IN (SELECT locationID FROM structu_assets.deed WHERE ownerID = 'clarus')
)
Here's the EXPLAIN:
My thought was to refactor the subqueries to use JOIN. But keeping the logical OR is throwing me off.
In addition, the nested subqueries seem inevitable. Unless I predetermined the unitID in a separate query.
I'm not the original developer. But I'm charged with making it more performant without modifying the existing codebase or schema. So I'm attempting the pick off the slow queries.
As an aside, do cross database queries take a performance hit?
add an index to your dateStamp column.
use :
dateStamp > '2009-04-16' AND dateStamp < '2012-02-29'
instead of :
dateStamp BETWEEN '2009-04-16' AND '2012-02-29'
it's more efficient.
Finally, you can write a loop in PHP instead of subqueries.
It would be interesting to see if this answer a) works and b) is faster. Try this:
SELECT
il.entryID,
il.entryID AS iE,
0 AS eE,
'clarus1' AS locationID,
il.dateStamp,
il.amount AS income,
NULL AS expense,
il.reconciled,
il.leaseID AS vendorID,
il.incomeID AS expenseID
FROM
structu_income.iLedger il
INNER JOIN structu_assets.lease l ON il.leaseID = l.leaseID
INNER JOIN structu_assets.unit u ON l.unitID = u.unitID AND u.locationID = 'clarus1'
WHERE
il.dateStamp BETWEEN '2009-04-16' AND '2012-02-29'
AND il.incomeID IS NOT NULL
UNION
SELECT
il.entryID,
il.entryID AS iE,
0 AS eE,
'clarus1' AS locationID,
il.dateStamp,
il.amount AS income,
NULL AS expense,
il.reconciled,
il.leaseID AS vendorID,
il.incomeID AS expenseID
FROM
structu_income.iLedger il
INNER JOIN structu_assets.deed d ON il.locationID = d.locationID AND d.ownerID = 'clarus'
WHERE
il.dateStamp BETWEEN '2009-04-16' AND '2012-02-29'
AND il.incomeID IS NOT NULL
The first SELECT query takes care of the first half of your OR conditions, and the second SELECT query adds in the results for the second half. And you shouldn't get duplicate rows with UNION so I believe you should get the same results.

Categories