I have a proble with my query, i have to create report but there is no same data.
here is my database https://www.db-fiddle.com/f/2bA7StrBpz18tLFgAQh2QV/1
and this is my query example but the result is wrong :
SELECT
a.IdBukti,c.LineName,a.LineID,a.Tanggal,b.TypeProduksi AS partnamemonthly,a.PartID AS partmonthly,a.QtyPlanning AS qtymonthly,
d.partnamedaily,d.partiddaily,d.qtydaily
FROM
trans_ppicbdt_dt a
INNER JOIN ms_part b ON b.PartId = a.PartID
INNER JOIN ms_line c ON c.LineID = a.LineID
INNER JOIN(SELECT
c.LineName,
a.LineID,
a.Tanggal,
b.TypeProduksi AS partnamedaily,
a.PartID AS partiddaily,
a.QtyPlanning AS qtydaily
FROM
trans_ppich a
INNER JOIN ms_part b ON b.PartId = a.PartID
INNER JOIN ms_line c ON c.LineID = a.LineID
WHERE
a.Tanggal = '2018-04-11' AND a.DivisiId='DI070' AND a.IdLocation='1'
GROUP BY
a.LineID,
a.PartID) d on d.LineID=a.LineID AND d.Tanggal=a.Tanggal
WHERE
a.Tanggal = '2018-04-11' AND a.DivisiId='DI070' AND a.IdLocation='1'
GROUP BY
a.LineID,
a.PartID
So i have 2 data, first monthlyplan and second daily plan.
And i want the result like this
Can you help me to create the report in one single query
It's easier (for me) to figure out how to formulate a query if I can see the relations in a glance. Here's how Yours would look:
(monthly) (daily)
trans_ppicbdt_dt ms_line trans_ppich
---------------- ----------- --------------
IdUnik (primary) ----> LineID <---- IdBukti (FK)*
LineID ------------/ \--- LineID
There are some problems with your structure that need to be fixed at the earliest opportunity:
ms_line needs a primary key. lineID should work if it is unique. (no such luck, it is not unique)
trans_ppich needs a primary key. IdBukti is a foreign key to yet another table.
There needs to be an index on lineID in all tables.
Tables need to be normalized. You shouldn't have to repeat data over several lines.
Nevertheless, we can still get to where you want to go. Now that I can see the relation, the query is really rather simple in theory, but I don't have all the required tables and fields:
SELECT DISTINCT line.LineName,
monthly.{unknown field} as monthlyPartName,
monthly.PartID as monthlyPartID,
monthly.{unknown field} as monthlyProcess,
monthly.{unknown field} as monthlyQty,
daily.{unknown field} as dailyPartName,
daily.PartID as dailyPartID,
daily.{unknown field} as dailyQty,
{unknown table}.{unknown field} as remarks
FROM ms_line as line
LEFT JOIN trans_ppicbdt_dt AS monthly ON line.lineID=monthly.lineID
LEFT JOIN trans_ppich AS daily ON line.lineID=daily.lineID
WHERE
{your where clause to filter results}
The way this works is it iterates through the (distinct) line items, and looks for a match in the monthly table (left join monthly). If there is a match, it adds the values. If there isn't a match, it simply adds null for values. Then it does the same for the daily table: if there is a match, it adds the values, otherwise it simply gives null for all daily values.
This will fall apart, however, if lineID is duplicated in rows of the monthly and daily tables.
I don't know where the remarks come from, so that is left as an exercise for you. :)
Related
I have 3 tables in a MySQL database: courses, users and participants, which contains about 30mil, 30k and 3k entries respectively.
My goal is to (efficiently) figure out the number of users that have been assigned to courses that matches our criteria. The criteria is a little more complex, but for this example we only care about users where deleted_at is null and courses where deleted_at is null and active is 1.
Simplified these are the columns:
users:
id
deleted_at
1
null
2
2022-01-01
courses:
id
active
deleted_at
1
1
null
1
1
2020-01-01
2
0
2020-01-01
participants:
id
participant_id
course_id
1
1
1
2
1
2
3
2
2
Based on the data above, the number we would get would be 1 as only user 1 is not deleted and that user assigned to some course (id 1) that is active and not deleted.
Here is a list of what I've tried.
Joining all the tables and do simple where's.
Joining using subqueries.
Pulling the correct courses and users out to the application layer (PHP), and querying participants using WHERE IN.
Pulling everything out and doing the filtering in the application layer.
Calling using EXPLAIN to add better indexes - I, admittedly, do not do this often and may not have done this well enough.
A combination of all the above.
An example of a query would be:
SELECT COUNT(DISTINCT participant_id)
FROM `participants`
INNER JOIN
(SELECT `courses`.`id`
FROM `courses`
WHERE (`active` = '1')
AND `deleted_at` IS NULL) AS `tempCourses` ON `tempCourses`.`id` = `participants`.`course_id`
WHERE `participant_type` = 'Eloomi\\Models\\User'
AND `participant_id` in
(SELECT `users`.`id`
FROM `users`
WHERE `users`.`deleted_at` IS NULL)
From what I can gather doing this will create a massive table, which only then will start applying where's. In my mind it should be possible to short circuit a lot of that because once we get a match for a user, we can disregard that going forward. That would be how to handle it, in my mind, in the application layer.
We could do this on a per-user basis in the application layer, but the number of requests to the database would make this a bad solution.
I have tagged it as PHP as well as MySQL, not because it has to be PHP but because I do not mind offloading some parts to the application layer if that is required. It's my experience that joins do not always use indexes optimally
Edit:
To specify my question: Can someone help me provide a efficient way to pull out the number of non-deleted users that have been assigned to to active non-deleted courses?
I would write it this way:
SELECT COUNT(DISTINCT p.participant_id)
FROM courses AS c
INNER JOIN participants AS p
ON c.id = p.course_id
INNER JOIN users AS u
ON p.participant_id = u.id
WHERE u.deleted_at IS NULL
AND c.active = 1 AND c.deleted_at IS NULL
AND p.participant_type = 'Eloomi\\Models\\User';
MySQL may join the tables in another order, not the order you list the tables in the query.
I hope that courses is the first table MySQL accesses, because it's probably the smallest table. Especially after filtering by active and deleted_at. The following index will help to narrow down that filtering, so only matching rows are examined:
ALTER TABLE courses ADD KEY (active, deleted_at);
Every index implicitly has the table's primary key (e.g. id) appended as the last column. That column being part of the index, it is used in the join to participants. So you need an index in participants that the join uses to find the corresponding rows in that table. The order of columns in the index is important.
ALTER TABLE participants ADD KEY (course_id, participant_type, participant_id);
The participant_id is used to join to the users table. MySQL's optimizer will probably prefer to join to users by its primary key, but you also want to restrict that by deleted_at, so you might need this index:
ALTER TABLE users ADD KEY (id, deleted_at);
And you might need to use an index hint to coax the optimizer to prefer this secondary index over the primary key index.
SELECT COUNT(DISTINCT p.participant_id)
FROM courses AS c
INNER JOIN participants AS p
ON c.id = p.course_id
INNER JOIN users AS u USE INDEX(deleted_at)
ON p.participant_id = u.id
WHERE u.deleted_at IS NULL
AND c.active = 1 AND c.deleted_at IS NULL
AND p.participant_type = 'Eloomi\\Models\\User';
MySQL knows how to use compound indexes even if some conditions are in join clauses and other conditions are in the WHERE clause.
Caveat: I have not tested this. Choosing indexes may take several tries, and testing the EXPLAIN after each try.
I have the following tables
ea_users
id
first_name
last_name
email
password
id_roles
ea_user_cfields
id
c_id = custom field ID
u_id = user ID
data
ea_customfields
id
name = name of custom field
description
I want to get all users which have a certain role, but I also want to retrieve all the custom fields per user. This is for the backend of my software where all the ea_users and custom fields should be shown.
I tried the following, but for each custom field, it duplicates the same user
$this->db->join('(SELECT GROUP_CONCAT(data) AS custom_data, id AS dataid, u_id, c_id
FROM ea_user_cfields userc
GROUP BY id) AS tt', 'tt.u_id = ea.id','left');
$this->db->join('(SELECT GROUP_CONCAT(name) AS custom_name, id AS customid
FROM ea_customfields AS cf
GROUP BY id) AS te', 'tt.c_id = te.customid','left');
$this->db->where('id_roles', $customers_role_id);
return $this->db->get('ea_users ea')->result_array();
the problem that u did not understand properly how join works.
its ok, that u have duplicates in select when u have relation one to many.
in few words your case: engine tries to fetch data from table "A" (ea_users) then JOIN according to the conditions another table "B" (ea_customfields). If u have one to many relation between tables (it means that one record from table "A" (lets say that we have in this table A1 record) can contain few related rows in table "B", lets call them as B1.1, B1.2 and B1.3 and B1.4), in this case it will join this records and put join result in memory. So in memory u would see something like
| FromTable A | FromTableB |
| A1 | B1.1 |
| A1 | B1.2 |
| A1 | B1.3 |
| A1 | B1.4 |
if u have 10 records in table "B", which related to the table "A" it would put 10 times in memory copy of data from table "A" during fetching. And then will render it to u.
depending on join type rows, with missing related records, can be skipped at all (INNER JOIN), or can be filled up with NULLs (LEFT JOIN or RIGHT JOIN), etc.
When u think about JOINs, try to imagine yourself, when u try to join on the paper few big tables. U would always need to mark somehow which data come from which table in order to be able to operate with it later, so its quite logically to write row "A1" from table "A" as many times as u need to fill up empty spaces when u find appropriate record in table "B". Otherwise u would have on your paper something like:
| FromTable A | FromTableB |
| A1 | B1.1 |
| | B1.2 |
| | B1.3 |
| | B1.4 |
Yes, its looks ok even when column "FromTable A" contains empty data, when u have 5-10 records and u can easily operate with it (for example u can sort it in your head - u just need to imagine what should be instead of empty space, but for it, u need to remember all the time order how did u wrote the data on the paper). But lets assume that u have 100-1000 records. if u still can sort it easily, lets make things more complicated and tell, that values in table "A" can be empty, etc, etc.. Thats why for mysql engine simpler to repeat many times data from table..
Basically, I always stick to examples when u try to imagine how would u join huge tables on paper or will try to select something from this tables and then make sorting there or something, how would u look through the tables, etc.
GROUP_CONCAT, grouping
Then, next mistake, u did not understand how GROUP_CONCAT works:
The thing is that mysqlEngine fetch on the first step structure into memory using all where conditions, evaluating subqueries + appends all joins. When structure is loaded, it tried to perform GROUPing. It means that it will select from temporary table all rows related to the "A1". Then will try to apply aggregation function to selected data. GROUP_CONCAT function means that we want to apply concatenation on selected group, thus we would see something like "B1.1, B1.2, B1.3, B1.4". Its in few words, but I hope it will help a little to understand it.
I googled table structure so u can write some queries there.
http://www.mysqltutorial.org/tryit/query/mysql-left-join/#1
and here is example how GROUP_CONCAT works, try to execute there query:
SELECT
c.customerNumber, c.customerName, GROUP_CONCAT(orderNumber) AS allOrders
FROM customers c
LEFT JOIN orders o ON (c.customerNumber = o.customerNumber)
GROUP BY 1,2
;
can compare with results with previous one.
power of GROUP in aggregation functions which u can use with it. For example, u can use "COUNT()", "MAX()", "GROUP_CONCAT()" or many many others.
or example of fetching of count (try to execute it):
SELECT c.customerName, count(*) AS ordersCount
FROM customers AS c
LEFT JOIN orders AS o ON (c.customerNumber = o.customerNumber)
GROUP BY 1
;
so my opinion:
simpler and better to solve this issue on client side or on backend, after fetching. because in term of mysql engine response with duplication in column is absolutely correct. BUT of course, u can also solve it using grouping with concatenations for example. but I have a feeling that for your task its overcomplicating of logic
PS.
"GROUP BY 1" - means that I want to group using column 1, so after selecting data into memory mySql will try to group all data using first column, better not to use this format of writing on prod. Its the same as "GROUP BY c.customerNumber".
PPS. Also I read comments like "use DISTINCT", etc.
To use DISTINCT or order functions, u need to understand how does it work, because of incorrect usage it can remove some data from your selection, (same as GROUP or INNER JOINS, etc). On the first look, you code might work fine, but it can cause bugs in logic, which is the most complicated to find out later.
Moreover DISTINCT will not help u, when u have one-to-many relation(in your particular case). U can try to execute queries:
SELECT
c.customerName, orderNumber AS nr
FROM customers c
INNER JOIN orders o ON (c.customerNumber = o.customerNumber)
WHERE c.customerName='Alpha Cognac'
;
SELECT
DISTINCT(c.customerName), orderNumber AS nr
FROM customers c
INNER JOIN orders o ON (c.customerNumber = o.customerNumber)
WHERE c.customerName='Alpha Cognac'
;
the result should be the same. Duplication in customer name column and orders numbers.
and example how to loose data with incorrect query ;):
SELECT
c.customerName, orderNumber AS nr
FROM customers c
INNER JOIN orders o ON (c.customerNumber = o.customerNumber)
WHERE c.customerName='Alpha Cognac'
GROUP BY 1
;
I'm having tables like
product table: product_id | code
group table: id | fk-product_id | id_code
grade table id | fk-product_id | id_grade
person table id | fk-product_id | id_person
my sql query is:
"SELECT *
FROM product
JOIN group ON group.product_id = product_id
JOIN grade ON grade.product_id = product_id
JOIN person ON person.product_id = product_id
WHERE product.code='".$productCode."'");
I get the wright result, but there is too much of rows. I thing that I'm doing overkill.
All product are for sure in the table "product" but it's not necessary that the same "id_product" is in the table "group", "grade" or "person".
In my result are a lot of rows where my result is repeted. I there any way to avoid those duplication?
Is there better way to perform my query?
From your original query, you have listed the column in the group, grade and person table are
'fk-product_id' but your query is showing as just 'product_id'. So, I am implying your real column is just 'product_id' and the 'fk-' was just a reference that it was the foreign key to products table.
Now, that said, the equality comparison is just product_id. Since you are not qualifying it with alias.field, it is probably grabbing everything since each record in group will always have its own product_id = its own product_id.
In addition, you mention that not all tables will have a matching product ID, so you will need LEFT-JOINs for the other tables... Adjust to something like this
SELECT
p.*,
gp.id_code,
gd.id_grade,
per.id_person
FROM
product p
LEFT JOIN group gp
ON p.product_id = gp.product_id
LEFT JOIN grade gd
ON p.product_id = gd.product_id
LEFT JOIN person per
ON p.product_id = per.product_id
WHERE
p.code='".$productCode."'";
But I would head caution for sql-injection as you could get malicious values in your $productCode variable. Make sure you have it properly cleaned and escaped.
#5er, Left-Join says for each record on the left-side (first in this case is the Product Table), I want all records... and oh... by the way... I have the other tables (group, grade and persons). They MAY have a record too that has the same Product_ID value as the product table. If so, grab those pieces too, but don't exclude the original product record.
Now, why your query was failing, and I thought I described it well, but apparently not. You were getting a Cartesian result which means for every one record in the left-table (product), you were getting EVERY record in the RIGHT-side table... So, for a single product, and if you had 20 group, 10 grades and 100 people, you were basically getting 20,000 records.
So your JOIN
JOIN group ON group.product_id = product_id
WOULD have worked, but had less records IF you qualified with the PRODUCT reference
JOIN group ON group.product_id = PRODUCT.product_id
Otherwise, it was just comparing its own product_ID to itself and saying... Yup, these to product IDs match (even though it was the same record), it returned it. The engine can't guess for you which table you meant for the second part of the join, especially when there were a total of 4 tables referenced in the query, and EACH had a "Product_ID" column. So, I strongly suggest that for ALL your queries, qualify ALL fields as alias.field, including those of the select field list. Then, anyone else trying to help you in the future, or even take over where you left-off know where the fields are. Prevent ambiguity in your queries.
Your select does not match the table/column names above it. For example in table product you say you have column id_product, but in select you use product.id instead of product.id_product. That might be totally different column.
Your results are repeating because the JOIN is joining tables, but you are not filtering those cases where one JOIN matches, while the other isn't.
Try this:
SELECT *
FROM product
JOIN group ON group.product_id = product.id
JOIN grade ON grade.product_id = product.id
JOIN person ON person.product_id = product.id
WHERE product.code='".$productCode."'
GROUP BY product.id
So, I have a table named clients, another one known as orders and other two, orders_type_a and orders_type_b.
What I'm trying to do is create a query that returns the list of all clients, and for each client it must return the number of orders based on this client's id and the amount of money this customer already spent.
And... I have no idea how to do that. I know the logic behind this, but can't find out how to translate it into a MySQL query.
I have a basic-to-thinkimgoodbutimnot knowledge of MySQL, but to this situation I've got really confused.
Here is a image to illustrate better the process I'm trying to do:
Useful extra information:
Each orders row have only one type (which is A or B)
Each orders row can have multiple orders_type_X (where X is A or B)
orders relate with client through the column client_id
orders_type_X relate with orders through the column order_id
This process is being made today by doing a query to retrieve clients, and then from each entry returned the code do another query (with php) to retrieve the orders and yet another one to retrieve the values. So basically for each row returned from the first query there is two others inside it. Needless to say that this is a horrible approach, the performance sucks and I thats the reason why I want to change it.
UPDATE width tables columns:
clients:
id | name | phone
orders:
id | client_id | date
orders_type_a:
id | order_id | number_of_items | price_of_single_item
orders_type_b:
id | order_id | number_of_shoes_11 | number_of_shoes_12 | number_of_shoes_13 | price_of_single_shoe
For any extra info needed, just ask.
If I understand you correctly, you are looking for something like this?
select c.*, SUM(oa.value) + SUM(ob.value) as total
from clients c
inner join orders o on c.order_id = o.id
inner join orders_type_a oa on oa.id = o.order_type_id AND o.type = 'A'
inner join orders_type_b ob on ob.id = o.order_type_id AND o.type = 'B'
group by c.id
I do not know your actual field names, but this returns the information on each customer plus a single field 'total' that contains the sum of the values of all the orders of both type A and type B. You might have to tweak the various names to get it to work, but does this get you in the right direction?
Erik's answer is on the right track. However, since there could be multiple orders_type_a and orders_type_b records for each order, it is a little more complex:
SELECT c.id, c.name, c.phone, SUM(x.total) as total
FROM clients c
INNER JOIN orders o
ON o.client_id = c.id
INNER JOIN (
SELECT order_id, SUM(number_of_items * price_of_single_item) as total
FROM orders_type_a
UNION ALL
SELECT order_id, SUM((number_of_shoes_11 + number_of_shoes_12 + number_of_shoes_13) * price_of_single_shoe) as total
FROM orders_type_b
) x
ON x.order_id = o.id
GROUP BY c.id
;
I'm making a few assumptions about how to calculate the total based on the columns in the orders_type_x tables.
I have four tables I want to join and get data from. The tables look something like...
Employees (EmployeeID, GroupID[fk], EmployeeName, PhoneNum)
Positions (PositionID, PositionName)
EmployeePositions (EployeePositionID, EmployeeID[fk], PositionID[fk])
EmployeeGroup (GroupID, GroupName)
[fk] = foreign key
I want to create a query that will return all the information about an employee(given by EmployeeID). I want a query that will return the given employees Name, position(s), and group in one row.
I think it needs to involve joins, but I am not sure how to format the queries. MYSQL's manual is technical beyond my comprehension. I would be very grateful for any help.
It seems you have trouble with SQL, in general, rather than with mySQL in particular. The documentation of mySQL provides details about the various SQL expressions, but generally assume some familiarity with SQL. To get a quick start on SQL you may consider this W3schools.com primer.
The query you need is the following.
SELECT EmployeeName, PositionName, GroupName
FROM Employees E
LEFT JOIN EmployeePositions EP ON EP.EmployeeID = E.EmployeeID
LEFT JOIN Positions P ON P.PositionID = EP.PositionId
LEFT JOIN EmployeeGroup EG ON EG.GroupId = E.GroupId
WHERE E.EmployeeId = some_value
A few things to note:
The 'LEFT' in 'LEFT JOIN' will result in producing NULL in lieu of PositionName or GroupName when the corresponding tables do not have a value for the given FK. (Should only happen if the data is broken, say if for example some employees have GroupId 123 but somehow this groupid was deleted from the EmployeeGroup table.
The query returns one line per employee (1). You could use an alternative search criteria, for example WHERE EmployeeName = 'SMITH', and get a listing of all employees with that name. Indeed without a WHERE clause, you'd get a list of all employees found in Employees table.
(1) that is assuming that each employee can only have one position. If somehow some employees have more than one position (i.e. multiple rows in EmployeePositions for a given EmployeeID), you'd get several rows per employee, the Name and Group being repeated and a distinct PostionName.
Edit:
If a given employee can have multiple positions, you can use the query suggested by Tor Valamo, which uses a GROUP BY construct, with GROUP_CONCAT() to pivot all the possible positions in one single field value in the returned row.
SELECT e.EmployeeID, e.EmployeeName, e.PhoneNum,
g.GroupName, GROUP_CONCAT(p.PositionName) AS Positions
FROM Employees e
LEFT JOIN EmployeeGroup g ON g.GroupID = e.GroupID
LEFT JOIN EmployeePositions ep ON ep.EmployeeID = e.EmployeeID
LEFT JOIN Positions p ON p.PositionID = ep.PositionID
WHERE e.EmployeeID = 1
GROUP BY e.EmployeeID
Returns positions in a comma separated string on one row.