I try to use group_concat to create more quickly xml outpout.
The record number is different between traditional query. Indeed, when my query use group concat, I have less record.
SELECT GROUP_CONCAT(
CONCAT('\n<p>\n',
CONCAT('\n<id>',paIndex,'</id>\n'),
CONCAT('<prInitiales>',prInitiales,'</prInitiales>\n'),
CONCAT('<paNomPren>',paNomPrenom,'\n',ttT_Traitement_P,'</paNomPren>\n'),
CONCAT('<ttTStatutP>',ttTStatutP,' - ',DATE_FORMAT(ttDateStatut,'%d/%m/%Y'),'\n',ttUserImportant,'</ttTStatutP>\n'),
CONCAT('<paDossier1>',paDossier1,'\n',paDossier2,'</paDossier1>\n'),
CONCAT('<paNumTel1>',paNumTel1,'\n',paNumTel2,'</paNumTel1>\n'),
CONCAT('<paNaissanceS>',DATE_FORMAT(paNaissance,'%d/%m/%Y'),'</paNaissanceS>\n'),
'</p>') ORDER BY paNomPrenom DESC) AS xml
FROM 20Patients_1012
JOIN 30Traitemnt_201223 ON 20Patients_1012.paIndex = 30Traitemnt_201223.ttIndex
JOIN 12Praticien_02 ON 30Traitemnt_201223.ttPraticien = 12Praticien_02.prIndex
The traditional query:
SELECT 20Patients_1012.paIndex, 20Patients_1012.paNomPrenom, 20Patients_1012.paDossier1, 20Patients_1012.paDossier2, 20Patients_1012.paNaissance, 20Patients_1012.paNumTel1, 30Traitemnt_201223.ttTStatutP, 30Traitemnt_201223.ttDateStatut, 12Praticien_02.prInitiales
FROM 20Patients_1012
JOIN 30Traitemnt_201223 ON 20Patients_1012.paIndex = 30Traitemnt_201223.ttIndex
JOIN 12Praticien_02 ON 30Traitemnt_201223.ttPraticien = 12Praticien_02.prIndex ORDER BY 20Patients_1012.paNomPrenom ASC
Thanks for helping
As stated in the manual:
The result is truncated to the maximum length that is given by the group_concat_max_len system variable, which has a default value of 1024. The value can be set higher, although the effective maximum length of the return value is constrained by the value of max_allowed_packet. The syntax to change the value of group_concat_max_len at runtime is as follows, where val is an unsigned integer:
SET [GLOBAL | SESSION] group_concat_max_len = val;
You used LEFT JOIN in a query and simple JOIN in the other. This may lead to different results because LEFT JOIN also consider records that do not have a match in the other table.
I found the solution.
Indeed, if null value, record is ignored, so I use, COALESCE function to solve that!
Related
I have almost thousands of data to display for my reports and it makes my browser lags due to the heavy data. I think that my query is the real problem. How can I optimized my query? is there something that I should add in my query?
I am using Xampp which supports PHP7.
SELECT
`payroll_billed_units`.`allotment_code`,
`payroll_billed_units`.`category_name`,
`payroll_billed_units`.`ntp_number`,
`payroll_billed_units`.`activity`,
`payroll_billed_units`.`regular_labor`,
`payroll_sub`.`block_number`,
(SELECT
GROUP_CONCAT(DISTINCT `lot_number` SEPARATOR ', ')
FROM
`payroll_billed_units` `lot_numbers`
WHERE
`lot_numbers`.`allotment_code` = `payroll_billed_units`.`allotment_code`
AND `lot_numbers`.`category_name` = `payroll_billed_units`.`category_name`
AND `lot_numbers`.`ntp_number` = `payroll_billed_units`.`ntp_number`
AND `lot_numbers`.`activity` = `payroll_billed_units`.`activity`) AS `lot_numbers`,
(SELECT
COUNT(`billed`.`ntp_id`)
FROM
`regular_ntp` `billed`
WHERE
`billed`.`allotment_code` = `payroll_billed_units`.`allotment_code`
AND `billed`.`category_name` = `payroll_billed_units`.`category_name`
AND `billed`.`ntp_number` = `payroll_billed_units`.`ntp_number`
AND `billed`.`activity` = `payroll_billed_units`.`activity`) AS `billed`,
(SELECT
COUNT(`approved`.`id`)
FROM
`payroll_billed_units` `approved`
WHERE
`approved`.`allotment_code` = `payroll_billed_units`.`allotment_code`
AND `approved`.`category_name` = `payroll_billed_units`.`category_name`
AND `approved`.`ntp_number` = `payroll_billed_units`.`ntp_number`
AND `approved`.`activity` = `payroll_billed_units`.`activity`) AS `approved`
FROM
`payroll_billed_units`
JOIN payroll_transaction ON payroll_billed_units.billing_number =
payroll_transaction.billing_number
JOIN payroll_sub ON payroll_transaction.billing_number =
payroll_sub.billing_number
WHERE payroll_billed_units.billing_date = '2019-02-13'
AND payroll_transaction.contractor_name = 'Roy Codal' GROUP BY allotment_code, category_name, activity
I was expecting that it will load or display all my data.
The biggest problem are the dependendt sub-selects, they are responsible for a bad performance. A sub-select will be executed for EVERY ROW of the outer query. And if you cascade subs-selects, you'll quickly have a query run forever.
If any of the parts would yield only 5 resultsets, 3 sub-select would mean that the database has to run 625 queries (5^4)!
Use JOINs.
Several of your tables need this 'composite' index:
INDEX(allotment_code, category_name, ntp_number, activity) -- in any order
payroll_transaction needs INDEX(contractor_name), though it may not get used.
payroll_billed_units needs INDEX(billing_date), though it may not get used.
For further discussion, please provide SHOW CREATE TABLE for each table and EXPLAIN SELECT ...
Use simply COUNT(*) instead of COUNT(foo). The latter checks the column for being not-NULL before including it. This is usually not needed. The reader is confused by thinking that there might be NULLs.
Your GROUP BY is improper because it is missing ntp_number. Read about the sql_mode of ONLY_FULL_GROUP_BY. I bring this up because you can almost get rid of some of those subqueries.
Another issue... Because of the "inflate-deflate" nature of JOIN with GROUP BY, the numbers may be inflated. I recommend you manually check the values of the COUNTs.
I tried to create mysql function. For particular month it returning empty result. I need to set that empty result as '0'. How to do this? Here is my mysql function.
DELIMITER $$
CREATE FUNCTION `monthtargetbyuser`(`userId` BIGINT,month VARCHAR(11),year VARCHAR(11)) RETURNS int(11)
BEGIN
DECLARE target,da BIGINT;
set da = year-month;
SET target = (SELECT ifnull(user_target.monthly_target,0) as monthly_target from user_target left join users on users.id=user_target.user_id where date_format(users.doj,'%Y-%m') <= 'da' and user_target.year=year and year(users.doj)>0 and users.locked !=1 and users.id =userId );
RETURN target;
END
Thanks in Advance!!
set da = year-month; -> year and month are declared as varchars, however, in this context they would be interpreted as integers. If you wanted to create a string such as '2016-05', then use the concat() function: concat(year,'-',month). I have absolutely no idea why you declared da as bigint. It should be declared as varchar.
date_format(users.doj,'%Y-%m') <= 'da' -> this expression compares a string in year-month format with the string 'da'. Sql is not php, the variable names will not be expanded within a string. So, the expression should be: date_format(users.doj,'%Y-%m') <= da.
I also believe that you mixed up the order of the tables in the join. You should not really have a target for a user that does not exist.
If there is no such record that would satisfy your query, then the ifnull() function in the select statement will not even run, this is why you get an empty result in certain occasions. (Although how you tested your function is beyond me, given the issues above). Apparently, you expect to have a maximum of one record returned. In this case I suggest to use max() or min() function within the ifnull() because they guarantee to return a value: SELECT ifnull(max(user_target.monthly_target),0) as monthly_target ...
I found a solution... I made a mistake in concat of year and month. That makes a issue. This is my corrected code.
DELIMITER $$
CREATE FUNCTION `monthtargetbyuser`(`userId` BIGINT,month int(11),year int(11)) RETURNS BIGINT
BEGIN
DECLARE target,da varchar(50);
set da =concat(year,'-',month);
SET target = (SELECT ifnull(min(user_target.monthly_target),0) as monthly_target from user_target left join users on users.id=user_target.user_id where date_format(users.doj,'%Y-%m') <= da and user_target.year=year and year(users.doj)>0 and users.locked !=1 and users.id =userId );
RETURN target;
END
Thanks a lot. #Shadow
I'm having a performace problem with the execution of a select in PHP PDO.
Using a script available here at stackoverflow (Simplest way to profile a PHP script), I identified where the problem IS, but I have not found a solution.
My select that is the problem is:
SELECT REDACAO.ID_REDACAO AS ID_REDACAO,
DATE_FORMAT(REDACAO.DATA,'%d/%m/%Y') AS DATAE,
ALUNO.ID_ALUNO AS ID_ALUNO,
(SELECT IFNULL((DATEDIFF(DATE_ADD((SELECT MAX(DATA) FROM REDACAO WHERE ID_ALUNO = ALUNO.ID_ALUNO AND ID_REDACAO NOT IN (SELECT ID_REDACAO FROM CORRECAO)), INTERVAL 7 DAY), now())),NULL) as DATA FROM REDACAO LIMIT 1) AS ULTIMA,
ALUNO.NOME as ALUNO,
REDACAO.ID_TEMA AS ID_TEMA,
TEMA.TITULO as TEMA,
TEMA.MOTIVACIONAIS AS MOTIVACIONAIS,
REDACAO.TEXTO AS TEXTO,
REDACAO.ID_STATUS AS STATUS,
B.NOTA as NOTA,
B.RCORRIGIDA AS CORRIGIDA,
B.NOTA1,
B.COMENTARIO1,
B.NOTA2,
B.COMENTARIO2,
B.NOTA3,
B.COMENTARIO3,
B.NOTA4,
B.COMENTARIO4,
B.NOTA5,
B.COMENTARIO5,
B.COMENTARIO6,
C.COMENTARIO AS COMENTARIO
FROM REDACAO
LEFT OUTER JOIN (SELECT SUM(CORRECAO.C1+CORRECAO.C2+CORRECAO.C3+CORRECAO.C4+CORRECAO.C5) AS NOTA, RCORRIGIDA AS RCORRIGIDA, CORRECAO.C1 as NOTA1, CORRECAO.COM1 as COMENTARIO1, CORRECAO.C2 as NOTA2, CORRECAO.COM2 as COMENTARIO2, CORRECAO.C3 as NOTA3, CORRECAO.COM3 as COMENTARIO3, CORRECAO.C4 as NOTA4, CORRECAO.COM4 as COMENTARIO4, CORRECAO.C5 as NOTA5, CORRECAO.COM5 as COMENTARIO5, CORRECAO.COMGERAL AS COMENTARIO6, CORRECAO.ID_REDACAO FROM CORRECAO GROUP BY CORRECAO.ID_REDACAO) B
ON B.ID_REDACAO = REDACAO.ID_REDACAO
JOIN ALUNO ON ALUNO.ID_ALUNO = REDACAO.ID_ALUNO
JOIN TEMA ON TEMA.ID_TEMA = REDACAO.ID_TEMA
LEFT OUTER JOIN (SELECT (COUNT(COMENTARIO.ID_COMENTARIO)) AS COMENTARIO, COMENTARIO.ID_REDACAO FROM COMENTARIO GROUP BY COMENTARIO.ID_REDACAO) C
ON C.ID_REDACAO = REDACAO.ID_REDACAO
WHERE REDACAO.ID_PROFESSOR = $CodProfessor
and REDACAO.ID_STATUS != 6
ORDER BY (CASE WHEN REDACAO.ID_STATUS = 4 THEN 1 ELSE 0 END) DESC
I'm using (PDO :: FETCH_ASSOC) to get the data. Some columns respond in less than 1 second and others in more than 20 seconds.
Any idea what could be the problem and how to solve it?
Your query contains following that will slow it down:
many joins
many subselects
select without where
functions like COUNT, isnull, datediff, sum.(some of these may cancel an index)
case when
order by
group by
Depending on your indexes, on how the tables are joined, and on how big are the tables, this will eventually get very slower.
Try using 'explain' command, and simplify the query if possible.
explain output
a good video about explain
PDO is not at fault; the query is complex. And there may be missing indexes.
Turn this into a LEFT JOIN (because IN (SELECT...) optimizes poorly.
AND ID_REDACAO NOT IN ( SELECT ID_REDACAO FROM CORRECAO)
Upgrade to 5.6; it has some improvements.
You have two JOIN ( SELECT ... ). Before 5.6 that would be optimized terribly. Move one of them out into a temp table, to which you add a suitable index.
In one of the subqueries, GROUP BY CORRECAO.ID_REDACAO seems to be unnecessary.
These indexes (or PRIMARY KEYs) are needed:
CORRECAO: (ID_REDACAO)
REDACAO: (ID_REDACAO), (ID_PROFESSOR)
ALUNO: (ID_ALUNO)
TEMA: (ID_TEMA)
COMENTARIO: (ID_REDACAO, ID_COMENTARIO) ("compound index")
If those suggestions do not help enough, come back with SHOW CREATE TABLE for each table.
I have a MySQL Query which works fine except the field rnd_key is returned as null despite it having values. If I run a normal query without the join then the key is returned.
SELECT *, SUM(time_duration) as total_time
FROM rnd
LEFT JOIN time ON rnd.rnd_key = time.rnd_key
WHERE rnd.rnd_owner = $eng AND rnd.rnd_status != 90
AND (rnd.rnd_status != 80 OR rnd.rnd_status > '".$ninetyDate."')
GROUP BY rnd.rnd_key
Thanks
try two things:
First: try RIGHT JOIN instead of LEFT JOIN.
Second: try to give each of rnd_key columns diffrent alias names. duplicated names may cause such problem
I am not an expert in SQL therefore need your help.
I have this advanced query written by someone else:
SELECT cms_product_bid.*,
cms_products.*,
cms_products_images.pd_image,
cms_products_images.pd_image_label,
IF (cms_product_bid.pd_bid = 1,
IF(cms_product_bid.pd_bid_start_date <= CURDATE(),
IF(cms_product_bid.pd_bid_end_date >= CURDATE(),1,0)
,0)
,1) AS bid_product_check
FROM cms_products
LEFT JOIN cms_products_images
ON (cms_products.pd_id = cms_products_images.pd_id)
INNER JOIN cms_product_bid
ON (cms_product_bid.product_id = cms_products.pd_id)
INNER JOIN cms_home_product_slider
ON (cms_products.pd_id = cms_home_product_slider.pd_id)
WHERE cms_products.pd_status=1
AND cms_products.pd_visibility=1
AND cms_home_product_slider.cat_id='$featured_cat_id'
GROUP BY cms_products.pd_id
HAVING(bid_product_check =1)
ORDER BY cms_products.pd_sort ASC
All I am trying to do is to sort by cms_products.pd_sort but it isn't sorting by that column even though column is there in table with values such as 1, 2 and so on.
Can anyone point out the issue here ?
my guess is that your column is of type VARCHAR or string, you can simply CAST it
SELECT...
FROM...
WHERE...
ORDER BY CAST(cms_products.pd_sort AS SIGNED) ASC
it may be a bug... but try to list out the columns instead of using the .* shortcuts... then see if that column will be recognized and sorted correctly.
btw - using .* is not a best practice for many reasons.