mysql in select big data slowdown - php

$sql = "SELECT CC.*, CS.name as status_name, OS.name AS order_status_name, OS.id as order_status_id, O.payment_way, O.ship_country,O.bank_id,O.payment_text, O.reference_no, O.date_comp, O.rate as rate, O.currency as currency, IF(O.payment_way=4,'TEST',B.name) as bank_name, ICR.cancel_id, ICR.result, ICR.response
FROM cancels CC
INNER JOIN orders O ON O.id = CC.order_id
LEFT JOIN order_products OP ON OP.parent=O.id
LEFT JOIN product_variants PV ON PV.id=OP.product_variant_id
LEFT JOIN products P ON P.id=PV.product_id
LEFT JOIN banks B ON B.id=O.bank_id
LEFT JOIN order_status OS ON OS.id = O.status
LEFT JOIN mp_external_order AS meo ON meo.mdns_order_id = O.id
LEFT JOIN cancel_status AS CS ON CS.cname = CC.status
LEFT JOIN (SELECT IC.cancel_id, IC.result, IC.response, MAX(IC.id) AS cancels FROM internal_cancel_requests IC GROUP BY IC.cancel_id, IC.result, IC.response) AS ICR ON CC.id = ICR.cancel_id
WHERE CC.date_add BETWEEN '2021-03-10 00:00:00' AND '2021-08-10 23:59:59' AND CC.deleted = 0 AND CC.total <> 0 AND meo.id IS NULL AND CC.status NOT IN ('REF', 'M_REF') GROUP BY CC.id ORDER BY C.date_add ASC";
$query = selectQuery($sql);
in the code block above
LEFT JOIN (SELECT IC.cancel_id, IC.result, IC.response, MAX(IC.id) AS cancels FROM internal_cancel_requests IC GROUP BY IC.cancel_id, IC.result, IC.response) AS ICR ON CC.id = ICR.cancel_id
line causing slownes.
Without changing the date range and adding limits for uploaded big data. How should I change this line of code to improve performance.
internal_cancel_requests table
id
cancel_id
response
result
1
1250
xxxxx
1
2
1250
xyxyxy
0
3
1251
xx
0
in the above table For record number 1250 cancel_id, record number 2 must be brought

Related

LIMIT LEFT join to last updated row from multiple rows

This is my code i am trying to left join the latest team data, not every piece of data. i have tried just using limit 1 but doesnt return anything
ORDER BY updated DESC LIMIT 1
this doesnt work
Any ideas?
$sql = "SELECT
events.id, events.time,events.status, events.home_team,events.away_team,events.league,
ht.id as home_id,ht.name as home_name,at.name as away_name,
statistics.home_goals,statistics.away_goals,statistics.time as game_time,
leagues.id as league_id,leagues.name as league_name,leagues.type as league_type,
country.name as country_name,country.logo,
hts.home_scored, ats.away_scored,
hts.home_conceeded,ats.away_conceeded,
hts.home_win,ats.away_win,
hts.home_15,ats.away_15,
hts.home_25,ats.away_25,
hts.home_btts, ats.away_btts,
hts.home_fts, ats.away_fts,
hts.home_cs, ats.away_cs,
hts.home_corners_for, ats.away_corners_for,
hts.home_corners_against, ats.away_corners_against,
hts.home_cards, ats.away_cards
FROM events
LEFT JOIN teams ht
ON ht.id = events.home_team
LEFT JOIN teams at
ON at.id = events.away_team
LEFT JOIN leagues
ON leagues.id = events.league
LEFT JOIN country
ON country.id=leagues.country
LEFT JOIN ( SELECT team,home_scored,home_conceeded,home_win,home_15,home_25,home_btts,home_fts,home_cs,home_corners_for,home_corners_against,home_cards FROM team_quick_stats ORDER BY updated DESC) hts
ON ht.id=hts.team
LEFT JOIN ( SELECT team,away_scored,away_conceeded,away_win,away_15,away_25,away_btts,away_fts,away_cs,away_corners_for,away_corners_against,away_cards FROM team_quick_stats ORDER BY updated DESC) ats
ON at.id=ats.team
LEFT JOIN statistics
ON statistics.event_id=events.id
WHERE (events.time BETWEEN $start AND $end) ORDER BY country.list_order, leagues.country ASC , leagues.id ASC, events.time ASC, home_name ASC";
Here's one way. Replace LEFT JOIN (SELECT team... etc....) ats with...
LEFT
JOIN
( SELECT x.team
, x.etc...
FROM team_quick_stats x
JOIN
( SELECT team
, MAX(updated) updated
FROM team_quick_stats
GROUP
BY team
) y
ON y.team = x.team
AND y.updated = x.updated
) ats...

Combine 2 SQL queries into one - show gross & net sales

I have the following two queries - one showing gross sales and one showing paid sales (the only difference between the queries is an added condition - flag_payment = 1).
The output is two columns - Supplier Name and then Gross or Paid Sales in $ (gross - query #1, paid - query #2). If possible, I would like to combine the two queries so that the output will be Supplier Name, Gross Sales and Paid Sales (i.e. 3 columns).
Query 1: Gross Sales
SELECT C.value , SUM( SL.price )
FROM sales_listings SL
INNER JOIN sales S ON SL.sale_id = S.id
INNER JOIN custom_fields_data C ON SL.listing_id = C.owner_id
WHERE C.field_id =11
GROUP BY C.value
Query 2: Paid Sales
SELECT C.value , SUM( SL.price )
FROM sales_listings SL
INNER JOIN sales S ON SL.sale_id = S.id
INNER JOIN custom_fields_data C ON SL.listing_id = C.owner_id
WHERE S.flag_payment = 1 AND C.field_id =11
GROUP BY C.value
Is there a way to combine the following two queries? Thank you!
You can use conditional aggregation:
SELECT
c.value,
SUM(CASE WHEN s.flag_payment = 0 THEN sl.price ELSE 0 END) AS GrossSales,
SUM(CASE WHEN s.flag_payment = 1 THEN sl.price ELSE 0 END) AS PaidSales
FROM sales_listings sl
INNER JOIN sales s
ON sl.sale_id = s.id
INNER JOIN custom_fields_data c
ON sl.listing_id = c.owner_id
WHERE
c.field_id = 11
GROUP BY c.value
Since in your original query for the Gross Sales, there is no filter for s.flag_payment, you can omit the conditional SUM:
SELECT
c.value,
SUM(sl.price) AS GrossSales,
SUM(CASE WHEN s.flag_payment = 1 THEN sl.price ELSE 0 END) AS PaidSales
FROM sales_listings sl
INNER JOIN sales s
ON sl.sale_id = s.id
INNER JOIN custom_fields_data c
ON sl.listing_id = c.owner_id
WHERE
c.field_id = 11
GROUP BY c.value

Multitable query mysql

I am in process of creating multi variable & multi table search query for Job Portal. Got some success in creating following long query which gives results as required. I need to replace above highlighted red circle delimited values (which i created using GROUP_CONCAT function in following query) with follwoing table description values.
Note:- Delimited values are not in database field. I'am creating these using following query.
My other table structure is as follows:
+---------------------------------------------+
ID Description
+---------------------------------------------+
1 Delhi
2 Mumbai
3 Chennai
4 Kolkata
+----------------------------------------------+
Please help me to guide a way forward for one, rest i'll do it myself.
Select
a.user_id, g.username, a.mobile_no, a.location, a.DOB, a.current_CTC,
a.expected_CTC, a.work_exp, a.job_type, a.designation, a.Company_name,
a.keyskills, a.profile_headline, a.resume_path, a.Notice_period,
DATEDIFF(CURDATE(),Max(b.Login_date_time)) as Last_Login_days,
Max(b.Login_date_time) as Last_Available_on_site,
(Select GROUP_CONCAT(DISTINCT c.Education_ID ORDER BY c.Education_ID
SEPARATOR ', ') user_education from user_education c where a.user_id =
c.User_ID) as Education_ID,
(Select GROUP_CONCAT(DISTINCT d.FA_ID ORDER BY d.FA_ID SEPARATOR ', ') FA
from user_fa d where a.user_id = d.User_ID) as FA_ID,
(Select GROUP_CONCAT(DISTINCT e.Industry_ID ORDER BY e.Industry_ID
SEPARATOR ', ') Industry_ID from user_industry e where a.user_id =
e.User_ID) as Industry_ID,
(Select GROUP_CONCAT(DISTINCT f.Location_ID ORDER BY f.Location_ID
SEPARATOR ', ') Location_ID from user_preferred_locations f where
a.user_id = f.User_ID) as Preferred_Location_ID
from user_details a
left join login_history b on a.user_id = b.user_id
left join users g on a.user_id = g.id
left join user_education c on a.user_id = c.User_ID
left join user_fa d on a.user_id = d.User_ID
left join user_industry e on a.user_id = e.User_ID
left join user_preferred_locations f on a.user_id = f.User_ID
INNER JOIN locations h ON FIND_IN_SET(h.location_id, f.Location_ID)
WHERE MATCH(keyskills) AGAINST('MIS, Analysis' IN BOOLEAN MODE)
And YEAR(CURDATE()) - YEAR(DOB)>= 18 And YEAR(CURDATE()) - YEAR(DOB)<= 45
And work_exp >= 2 And work_exp <= 15
And a.current_CTC>=102 And a.current_CTC<=115
And a.industry IN (10001,10002,10004)
And a.functional_area IN(1001,1002,1003)
And a.location_preferred IN (1,2,3,4,6,7)
And Education_ID IN (1,5,7,8)
And a.job_type Like '%FTP%'
And Notice_period >= 15
And b.Login_date_time>=0
And a.mobile_no is not null
And resume_path is not null
group by a.user_id

mysql query become slow when data reached upto 100000

I build a query a month ago on a website. It was working fine. But after a month I was informed that the website become very slow to load the page.
When I search for the problem, I found that my query is executing very slow to fetch the data from mysql database. Then I check for the database and found that the 4 tables which I was using by joins, have around 216850, 167634, 64000, 931 rows respectively.
I have already have indexed that tables. So, where I'm lacking. Please help guys.
[Edit]
Table1: user_alert
Records: 216850
DB Type: InnoDB
Indexes: id(primary)
Table2: orders
Records: 167634
DB Type: InnoDB
Indexes: id(primary), order_id, customer_id
Table3: user_registration
Records: 64000 around
DB Type: InnoDB
Indexes: id(primary), email_address
Table4: cities
Records: 931
DB Type: InnoDB
Indexes: id(primary)
Query:
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address, o.order_id,
CASE WHEN ct.city_name IS NULL THEN uas.city_name ELSE ct.city_name END AS city_name
FROM `user_alert` uas
LEFT JOIN orders o ON o.customer_id = uas.user_id
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
GROUP BY uas.user_id
ORDER BY uas.create_date DESC
GROUP BY is used to aggregate values up. For example if you wanted the count of orders by a user you could use COUNT(o.order_id).....GROUP BY uas.user_id. There are multiple orders for each user, but the aggregate function is just counting them here. However if you just select o.order_id when you have a GROUP BY uas.user_id it doesn't know which of the possibly many order_id values to return for that user id.
In this case it possibly doesn't matter as it looks like the order table is the only one where there is multiple rows per use. If you want the latest one you could just use MAX(o.order_id) (assuming that the order_id is assigned is order). But if you wanted the order value it becomes more difficult.
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address, MAX(o.order_id) AS LatestOrderId,
IFNULL(ct.city_name, uas.city_name) AS city_name
FROM `user_alert` uas
LEFT JOIN orders o ON o.customer_id = uas.user_id
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
GROUP BY uas.user_id
ORDER BY uas.create_date DESC
If you wanted the (say) value of the latest order then it becomes more difficult.
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address, Sub1.MaxOrderId AS LatestOrderId, o.order_value
IFNULL(ct.city_name, uas.city_name) AS city_name
FROM `user_alert` uas
LEFT JOIN (SELECT customer_id, MAX(order_id) AS MaxOrderId FROM orders GROUP BY customer_id) Sub1 ON Sub1.customer_id = uas.user_id
LEFT OUTER JOIN orders o ON o.customer_id = Sub1.user_id AND o.order_id = Sub1.MaxOrderId
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
ORDER BY uas.create_date DESC
Or doing a bit of a fiddle based on GROUP_CONCAT
SELECT uas.alert_id, uas.user_id, uas.status, ur.first_name, ur.last_name, ur.email_address,
SUBSTRING_INDEX(GROUP_CONCAT(o.order_id ORDER BY o.order_id DESC), ',', 1) AS LatestOrderId,
SUBSTRING_INDEX(GROUP_CONCAT(o.order_value ORDER BY o.order_id DESC), ',', 1) AS LatestOrderValue,
IFNULL(ct.city_name, uas.city_name) AS city_name
FROM `user_alert` uas
LEFT OUTER JOIN orders o ON o.customer_id = uas.user_id AND o.order_id = Sub1.MaxOrderId
LEFT JOIN user_registration ur ON ur.id = uas.user_id
LEFT JOIN `cities` ct ON ct.city_id = uas.city_id
WHERE uas.status = '1'
GROUP BY uas.user_id
ORDER BY uas.create_date DESC

Optimizing a complex mysql query

The following query is taking 0.1313 seconds on phpmyadmin. Any way to optimize this to make things faster (say like to get it in 0.00XX seconds)? Index already added at columns that are doing the joinings.
SELECT m.id, m.civ, m.prenom, m.nom, m.sexe, m.depart, m.date_entree, m.date_sortie, m.login_userid, m.login_passwd, a.rank_id, r.rank_m, r.rank_f, d.user_id AS depID, c.nom AS cordo, z.rank
FROM `0_member` AS m
LEFT JOIN `0_area` AS a ON ( m.id = a.user_id
AND a.sec_id =2 )
LEFT JOIN `0_rank` AS r ON r.id = a.rank_id
LEFT JOIN `0_depart` AS d ON ( m.depart = d.depart
AND d.user_sec =2 )
LEFT JOIN `0_area` AS z ON ( d.user_id = z.user_id
AND z.sec_id =2 )
LEFT JOIN `0_member` AS c ON d.user_id = c.id
WHERE z.rank = 'mod'
ORDER BY nom
Your query has a final "WHERE" clause on the value being FOUND in the "Z" alias table with a rank of 'mod', yet your query is all LEFT JOINs indicating you want all members regardless of a possible match on the right side table you are joining to.
Additionally, you are joining downstream to the "z" table by depart and depart to a user ID, then re-joining directly to the '0_area' as A table directly on the user's ID which APPEARS it would be the same as found from the linking to the depart table to the 'z' table anyhow.
That said, and your member joins to depart and then to area...
My SUGGESTION (and I can rewrite the query as such) is to reverse the order of the query putting your Area table FIRST with an index on the "sec_id, rank" being available... I would have the key order based on whichever category had the smaller subset column first... so either SEC_ID, RANK or RANK, SEC_ID.
Then doing simple JOIN (not LEFT JOIN) to the other tables... At a minimum from:
SELECT STRAIGHT_JOIN
m.id,
m.civ,
m.prenom,
m.nom,
m.sexe,
m.depart,
m.date_entree,
m.date_sortie,
m.login_userid,
m.login_passwd,
a.rank_id,
r.rank_m,
r.rank_f,
d.user_id AS depID,
c.nom AS cordo,
z.rank
FROM
`0_area` AS z
JOIN `0_depart` AS d
on z.user_id = d.user_id
and d.user_sec = 2
JOIN `0_member` AS m
on d.depart = m.depart
AND z.user_id = m.id
LEFT JOIN `0_rank` AS r
on z.rank_id = .rid
WHERE
z.sec_id = 2
AND z.rank = 'mod'
ORDER BY
nom
In your original query, you had a join from
member
Links to Area (on member's user ID just to ensure the "sec_id = 2")
Since the new query is exclusively STARTING with the "area" table as "Z" alias, and THAT where clause is explicitly "sec_id = 2" value, you'll never need to backlink again...
Area (only SECID = 2 and rank = mod)
Links to Depart (on the User's ID)
Links to Members by (on the depart ID)
Try moving this statements " a.sec_id =2 ", "d.user_sec =2 ", " z.sec_id =2 " from ON sections to WHERE section like you already did with "z.rank = 'mod'". Like this:
SELECT m.id, m.civ, m.prenom, m.nom, m.sexe, m.depart, m.date_entree, m.date_sortie, m.login_userid, m.login_passwd, a.rank_id, r.rank_m, r.rank_f, d.user_id AS depID, c.nom AS cordo, z.rank
FROM `0_member` AS m
LEFT JOIN `0_area` AS a ON m.id = a.user_id
LEFT JOIN `0_rank` AS r ON r.id = a.rank_id
LEFT JOIN `0_depart` AS d ON m.depart = d.depart
LEFT JOIN `0_area` AS z ON d.user_id = z.user_id
LEFT JOIN `0_member` AS c ON d.user_id = c.id
WHERE z.rank = 'mod'
AND a.sec_id =2
AND d.user_sec =2
AND z.sec_id =2
ORDER BY nom

Categories