I'm creating an app for a trading card game called Magic: the Gathering and I have made a query that checks all user-submitted decks and gives you the percentage of cards you have in your inventory over the cards in the deck. But my problem is, it does this for all the decks on the database. What I want to do is only return the decks which I already have 50% of the cards for.
Here is the query:
SELECT
SUM(t.qty_inv) / t.deck_cards completed,
t.deck_id
FROM (
SELECT
CASE WHEN m.qty_inv IS NULL THEN 0 WHEN m.qty_inv > dc.card_qty THEN dc.card_qty ELSE m.qty_inv END qty_inv,
dc.deck_id,
d.deck_cards
FROM mtgb_test.decks d
INNER JOIN mtgb_test.decks_cards dc ON (d.deck_id = dc.deck_id)
LEFT OUTER JOIN (
SELECT
COUNT(*) qty_inv, item_print_id print_id
FROM mtgb_test.inventories_items
WHERE item_user_id = 1
GROUP BY item_print_id
) m ON (m.print_id = dc.card_print_id)
) t
GROUP BY deck_id
ORDER BY completed DESC;
The problem with this query is that I can't use the derived completed field in the where clause like so:
WHERE completed > 0.5
I don't know if variables can solve this problem, I tried a bit but it got mished mashed as I'm very new to user-defined variables.
Edit: Some good people answered below that I needed to have the HAVING syntax, and that is the correct and obvious answer. I'd just probably choose the best answer to the postscript question then.
Another thing, if I had 200,000 decks in my database and I had 2,000 cards in my inventory, I'm fine with this query looping through all those and finding which decks I already had half of the cards for. But my problem is with the LEFT OUTER JOIN. I don't know about how MySQL really works inside but maybe someone who does can point it out. When looping through all the 200,000 decks, would it do the left join for every deck there is or will it be smart enough to cache this so that it only queries my inventory items once?
Thanks in advance,
Ramon
You can use HAVING completed > 0.5 to further reduce the amount of rows returned. You can read more about HAVING clause at http://dev.mysql.com/doc/refman/5.5/en/select.html.
Related
I have been trying to get this code working for almost six hours an I have done almost six hours worth of research.
Originally this code was to be outputted from a database through PHP to a CSV file..
All I am trying to do is have three different tables and have them all joined together by their order_ID
Every time I keep trying to do this in PHP my admin either the code comes out weird in the output is not joining correctly or is giving some sort of strange error. Any help that anyone could figure out what is going on with this would be greatly appreciated
SELECT
oc_order.store_url,
oc_order.order_id,
oc_order.firstname,
oc_order.lastname,
oc_order.email,
oc_order_option.order_id,
oc_order_option.name,
oc_order_option.value,
oc_order_product.order_id,
oc_order_product.name,
oc_order_product.model,
oc_order_product.quantity
FROM oc_order, oc_order_option , oc_order_product
ORDER BY oc_order.order_id,oc_order_option.order_id,oc_order_product.order_id
As mentioned, you are missing your ON clauses. When you are using multiple tables in a SQL query, you have your LEFT table (first table in from clause) being JOINed to the second (right table). The "ON" clause tells how table 1 is related to table 2. And in your case, you have a table 3. Is this a relationship between table 1 to 3 or from table 2 to 3. I will update the query, you will need to fill in the blanks a bit since you have not shown structures. Also, using table aliases makes code more readable, especially if table names get long. You'll see in a moment.
SELECT
o.store_url,
o.order_id,
o.firstname,
o.lastname,
o.email,
oOp.order_id,
oOp.name,
oOp.value,
p.order_id,
p.name,
p.model,
p.quantity
FROM
oc_order o
JOIN oc_order_option oOp
on o.order_id = oOp.order_id
JOIN oc_order_product p
on o.order_id = p.order_id
ORDER BY
o.order_id
Notice the abbreviated aliases for the table names "o" for oc_orders, "oOp" for oc_order_option and "p" for oc_order_product
Again, YOU need to confirm the "ON" clauses on the left-table is associated with the right table. I have worked with manufacturing in the past and have seen that someone orders. Purchases a product, and that product has options, so Is there something on the order's detail line for the specific product that has an ID to the order OPTIONS table. That is where you probably need to focus. Again, other systems I have had, would have had a hierarchy something like
Order
Customer (who ordered)
OrderDetails (what of multiple products were ordered)
Product ( per orderDetail, what was the product)
ProductOption (color,size,etc)
See the hierarchical representation? In this scenario, you dont go just from Order table to the Product option. You have to get to the details to the product to the product's option. Anyway, HTH
Currently I'm developing a background cms for an online shop.
I split the tables as follow in my database:
-products
-productdetails (descrition...)
-productimages
-product variants (colors..)
-product cross selling
Now on the product edit page i need to fetch all data for a single product.
So my question is how i can get those details more efficient then make 3-5 database calls.
Or would the processing with php be less efficient then make those 3-5 calls ?
At the moment the query looks like that:
SELECT
pr.id, pr.categorieid, pr.itemnumber, pr.barcode, pr.price, pr.weight, pr.gender, pr.manufracture, pr.fsk18, pr.condition, pc.id AS pcid, pc.productcrossid, pc.sort, pd.productname,
pd.productdesc, pd.additional, pd.linktitle, pd.metatitle, pd.metadesc, pd.urlkeywords, pi.id AS piid, pi.wichimage, pi.variantid, pi.image, pi.imagealt, pv.id AS pvid, pv.variant,
pv.variantvalue, pv.sku, pv.price AS pvprice, pv.weight AS pvweight, pv.stock, pv.special
FROM
products pr
LEFT JOIN
productcross as pc
ON pr.id = pc.productid
LEFT JOIN
productdetails as pd
ON pr.id = pd.productid
LEFT JOIN
productimage as pi
ON pr.id = pi.productid AND pd.lang = pi.lang
LEFT JOIN
productvariants as pv
ON pr.id = pv.productid
WHERE
pr.id = :id
ORDER BY pd.lang ASC
As result i recieve many rows, because of the left join each value get joined with the rows i joined before.
The problem is there are dynamic many rows for cross selling, variants, images, so it can be random if variants or images are more (else i could group them atleast because each variant can get an own image, but there can be also more images then variants)
Products 1 row, productdetails according to how many languages are used, most likely 3.
Edit: According to Explain and the indexes i set, the performance of this single query is very good.
Edit:
According Paul Spiegel i tryed using GROUP_CONCAT
SELECT
pr.id, pr.categorieid, pr.itemnumber, pr.barcode, pr.price, pr.weight, pr.gender, pr.manufracture, pr.fsk18, pr.condition, pc.id AS pcid, pc.productcrossid, pc.sort, pd.productname,
pd.productdesc, pd.additional, pd.linktitle, pd.metatitle, pd.metadesc, pd.urlkeywords
FROM
products pr
LEFT JOIN
productsdetails as pd
ON pr.id = pd.productid
LEFT JOIN (
SELECT
GROUP_CONCAT(productcrossid) AS pcproductcrossid, GROUP_CONCAT(sort) AS pcsort, GROUP_CONCAT(id) AS pcid, productid
FROM productscross
WHERE productid = :id
) pc
ON pr.id = pc.productid
WHERE
pr.id = :id
ORDER BY pd.lang ASC
As result i recieve many rows, because of the left join each value get joined with the rows i joined before.
That's not what LEFT means.
X JOIN Y ON ... delivers rows that show up on both X and Y.
X LEFT JOIN Y ON ... delivers all the rows of X even if there is no matching row (or rows) in Y.
You might get "many rows" because the relationship is "1:many". Think of Classes JOIN Students With JOIN you get multiple rows per Class (one per student), except for any classes without any students. With LEFT JOIN, you additionally get a row for any Class with no students.
Your query with products will be a huge explosion of rows. All products, expanded by multiple details by multiple images, etc. It will be a mess.
In the EXPLAIN, multiply the numbers in the "Rows" column -- that will be a crude metric of how big the result set will be.
Use one query to get the images; another to get the colors; etc. Use JOIN (or LEFT JOIN only when needed.
GROUP_CONCAT() is handy sometimes. It might be useful to list the "colors". But for "images", you would then have to split it up so you can build multiple <img..> tags. That's rather easy to do, but it is extra work.
It is usually 'wrong' to have 1:1 mapping between tables. In such cases, why not have a single table?
Do not fear 3-5 queries. We are talking milliseconds. The rendering of the page is likely to take several times as long as the SELECTs. I often have several dozen queries to build a web page, yet I am satisfied with the performance. And, yes, I ascribe to the notion of putting all the info about one 'product' on the page at once (when practical). It's much better than having to click here to get the colors and click there to see the images, etc.
Rather than hitting so many query's you can refer to the concept which is known as flat tables in magento.
The logic behind using this concept is that what ever important data which is required to be show on the front end is stored in single table itself as well as the data is stored in there prescriptive tables.
So while querying you just need to pick the data from that flat table itself rather than querying to multiple tables and increasing the query execution time.
For reference Please check out the below link,Hope this helps.
Visit http://excellencemagentoblog.com/blog/2015/06/10/magento-flat-tables/
I do know the question is not regarding Magento but you can build your own logic to achieve this mechanism.
This is more of a theoretical query than anything else, but I have a complex join (resulting in upwards of 1900 records in the main table, combined with all the sub-result tables in the join -- join shown below), the resulting web page is taking 5-10 minutes on my local machine to process and complete building. I realize this could easily be many factors, but am hoping to get some hints. Basically I am loading an array of names from two tables (one is cross-references, so the array is used to sort the data on the names, with links and a field noting if it is a cross reference), then if a name is not a cross reference, I issue this join:
select
n.NameCode, n.AL_NameCode, n.Name, n.Name_HTML, n.Region, n.Local, n.Deceased,
n.ArmsLink, n.RollOfArms, n.Blazon, n.PreferredTitle, n.ShortBio,
n.HeadShotPhoto, n.HeadShotPhotographer, n.HeadShotContributor,
x.NameCode, x.NameAKA, x.AlternateName,
g.NameLink, g.`Group Name`,
p.NameLink, p.`Relationship Type`, p.`Related To Link`,
p2.Position_ID, p2.NameLink, p2.`Position Held`, p2.`Times Held`,
p2.`Date Started`, p2.`Date Ended`, p2.Hyperlink as pos_Hyperlink,
p2.`Screentip Text`,
a.`Name Link`, a.Description, a.EventDate, a.Hyperlink, a.`Screentip Text`,
a.ExternalLink
from who_names as n
left outer join who_crossref as x on n.NameCode=x.NameCode
left outer join who_groups as g on n.NameCode=g.NameLink
left outer join who_personal as p on n.NameCode=p.NameLink
left outer join who_positions as p2 on n.NameCode=p2.NameLink
left outer join who_arts as a on n.NameCode=a.`Name Link`
where n.NameCode = ?
order by n.Name desc, g.`Group Name`, p2.`Date Started`, a.EventDate;
In order to output the various parts of the data, I:
1) Start a table,
2) Output the name and some other info in the first row,
3) Then in order to process, say, the groups (sub-groups someone associates themselves with within the organization), I issue:
mysqli_data_seek( $result, 0 ); // to rewind to top of data so we're at first row
and see if there's anything to process for subgroups (not everyone has anything ...),
4) I repeat for personal relationships, and other sections, going back to the top of the data and looping back through if there's anything to process.
When done with that individual, I close off the table, and loop back in the array to the next name, and repeat ...
While this works, 5-10 minutes is way to long to load a web page.
I am pondering ideas to resolve this, but I am not sure if it is any specific aspect of my code. Is it the seeks back to the top of the rowset returned? Is it the tables in the browser? Is it a combination of both (very possibly)? The program is too big to post here in its entirety. I am feeling rather flummoxed at how to resolve this, and hoping someone has some pointers to help me speed the processing up, and I hope the details I've given are enough to give something to work with.
Based on comments and feedback below, in PHP Admin, I did the following:
explain select n.NameCode, n.AL_NameCode, n.Name, n.Name_HTML, n.Region, n.Local, n.Deceased,
n.ArmsLink, n.RollOfArms, n.Blazon, n.PreferredTitle, n.ShortBio, n.HeadShotPhoto,
n.HeadShotPhotographer, n.HeadShotContributor,
x.NameCode, x.NameAKA, x.AlternateName,
g.NameLink, g.`Group Name`,
p.NameLink, p.`Relationship Type`, p.`Related To Link`,
p2.Position_ID, p2.NameLink, p2.`Position Held`, p2.`Times Held`, p2.`Date Started`,
p2.`Date Ended`, p2.Hyperlink as pos_Hyperlink, p2.`Screentip Text`,
a.`Name Link`, a.Description, a.EventDate, a.Hyperlink, a.`Screentip Text`,
a.ExternalLink
from who_names as n
left outer join who_crossref as x on n.NameCode=x.NameCode
left outer join who_groups as g on n.NameCode=g.NameLink
left outer join who_personal as p on n.NameCode=p.NameLink
left outer join who_positions as p2 on n.NameCode=p2.NameLink
left outer join who_arts as a on n.NameCode=a.`Name Link`
where n.NameCode=638
order by n.Name desc, g.`Group Name`, p2.`Date Started`, a.EventDate
This returned:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE n const PRIMARY,ix1_names PRIMARY 4 const 1 Using temporary; Using filesort
1 SIMPLE x ref ix2_crossref ix2_crossref 4 const 1 NULL
1 SIMPLE g ref ix3_groups ix3_groups 4 const 3 NULL
1 SIMPLE p ref ix4_personal ix4_personal 4 const 1 NULL
1 SIMPLE p2 ref ix5_positions ix5_positions 4 const 13 NULL
1 SIMPLE a ref ix6_arts ix6_arts 4 const 28 NULL
Which appears to just be a list of the indexes, so it doesn't seem to be helping me.
Since you are using a SINGLE main table and the rest of the joins are all OUTER JOIN there's a single most important index that can make your query faster:
create index ix1_names on who_names (NameCode, Name);
Also, the Nested Loop Joins (NLJ) against the related tables will benefit of the following indexes. You may already have several of these so check if you have them first. If you don't, then create them:
create index ix2_crossref on who_crossref (NameCode);
create index ix3_groups on who_groups (NameLink);
create index ix4_personal on who_personal (NameLink);
create index ix5_positions on who_positions (NameLink);
create index ix6_arts on who_arts (`Name Link`);
But again, it's the first one the one I consider the most important one.
You'll need to test for real to see if the performance improves with it/them.
If the query is still slow, please retrieve the execution plan, as #memo suggested, by using:
explain select ...
First, try removing the "order by" clause and see if that improves anything. Sometimes it can happen that the query itself is fast, but the re-ordering is slow, requiring temporary files.
Second, feed the query to an EXPLAIN statement (e.g. EXPLAIN SELECT whathaveyou FROM table...). Check out the output for bottlenecks, missing indexes etc. (https://dev.mysql.com/doc/refman/8.0/en/using-explain.html)
After a lot of work, I found a few issues that I was able to resolve: I was (thinking it made sense at the time) opening some tables when they weren't necessary to get row counts; I dropped the big join and just opened the sub-tables as needed; cleaned up a few other places in the code; added a few more indexes on another set of tables that weren't in the original join. I was able to reduce the speed from 4 minutes to 45 seconds. While 45 seconds is a long time to load a page, I figure since this page was handling up to 1500 (sometimes more) primary records, and pulling data from up to 10 different tables, formatting (tables inside tables, etc.), that 45 seconds is probably doable, with a note at the top of the page and a progress bar that displays while loading the page. Thanks, all. The indexes did help, and the other explanations also helped a lot.
I have PHP system that runs a MYSQL query like below
select
order.id,
order.name,
order.date,
customer.name,
items.coupon_code,
from order
left join customer on order.custid = customer.id
left join items on items.coupon_code = order.coupon_code
where items.coupon_new_code is null
and order.status = 1000
AND order.promo_code in (1,2)
order table has 800K records and items table has 300k records. When I run this the query takes about 9 hours to finish!
If I comment the left join to the items table then the query runs in a few seconds! I am not very efficient with MySQL joins and would really really appreciate if someone can tell me how I can optimise this query to run in an acceptable time frame.
Try changing
LEFT JOIN to INNER JOIN (or just JOIN)
This will work to speed things up assuming that you only want to see orders that have both customers and items associated with them. Currently your query is trying to return all data from the order table but that's not needed. It's possible other changes to the database structure could improve things as well.
The top answer here provides a useful diagram that demonstrates the difference between these types of statements.
At the very least you need an index on coupon_code on both order and items tables. Consider also adding to a compound index, the other field you are joining on custid, as well as on your WHERE conditions items.coupon_new_code, order.status and order.promo_code. Knowing next to nothing about your data I can only speculate about what the dbms will use. Try various combinations in a compound key and run explain to see what's being used. It's really going to depend on the specificity of the data in your columns.
Posting the output of EXPLAIN along with the tables' schema will help us improve these answers.
Just getting to grips with some more advance sql queries and a bit stuck with one of them. I’ve been working on a small cinema project that allows user to book seats for upcoming productions.
In a table for chairs, I’ve created a query so that checkboxes are created on all of the chairs within a particular cinema area. However, I now need to add some sort of functionality to show chair availability i.e. whether one has already been booked by a previous order.
select * from `chairs` c
left outer join `booki1` b on b.`seatno`=c.`seatno`
left outer join `shows` s on s.`ddate`=b.`ddate`
left outer join `area` a on a.`name`=c.`area`
where (
c.`area` like '{$_SESSION['Area']}' and
s.`ddate` like '{$_SESSION['DDate']}' and
s.`ttime` like '{$_SESSION['TTime']}}'
);
Very hastily put together and totally untested, but you need to join the tables using suitable keys.