I have PHP system that runs a MYSQL query like below
select
order.id,
order.name,
order.date,
customer.name,
items.coupon_code,
from order
left join customer on order.custid = customer.id
left join items on items.coupon_code = order.coupon_code
where items.coupon_new_code is null
and order.status = 1000
AND order.promo_code in (1,2)
order table has 800K records and items table has 300k records. When I run this the query takes about 9 hours to finish!
If I comment the left join to the items table then the query runs in a few seconds! I am not very efficient with MySQL joins and would really really appreciate if someone can tell me how I can optimise this query to run in an acceptable time frame.
Try changing
LEFT JOIN to INNER JOIN (or just JOIN)
This will work to speed things up assuming that you only want to see orders that have both customers and items associated with them. Currently your query is trying to return all data from the order table but that's not needed. It's possible other changes to the database structure could improve things as well.
The top answer here provides a useful diagram that demonstrates the difference between these types of statements.
At the very least you need an index on coupon_code on both order and items tables. Consider also adding to a compound index, the other field you are joining on custid, as well as on your WHERE conditions items.coupon_new_code, order.status and order.promo_code. Knowing next to nothing about your data I can only speculate about what the dbms will use. Try various combinations in a compound key and run explain to see what's being used. It's really going to depend on the specificity of the data in your columns.
Posting the output of EXPLAIN along with the tables' schema will help us improve these answers.
Related
I have been trying to get this code working for almost six hours an I have done almost six hours worth of research.
Originally this code was to be outputted from a database through PHP to a CSV file..
All I am trying to do is have three different tables and have them all joined together by their order_ID
Every time I keep trying to do this in PHP my admin either the code comes out weird in the output is not joining correctly or is giving some sort of strange error. Any help that anyone could figure out what is going on with this would be greatly appreciated
SELECT
oc_order.store_url,
oc_order.order_id,
oc_order.firstname,
oc_order.lastname,
oc_order.email,
oc_order_option.order_id,
oc_order_option.name,
oc_order_option.value,
oc_order_product.order_id,
oc_order_product.name,
oc_order_product.model,
oc_order_product.quantity
FROM oc_order, oc_order_option , oc_order_product
ORDER BY oc_order.order_id,oc_order_option.order_id,oc_order_product.order_id
As mentioned, you are missing your ON clauses. When you are using multiple tables in a SQL query, you have your LEFT table (first table in from clause) being JOINed to the second (right table). The "ON" clause tells how table 1 is related to table 2. And in your case, you have a table 3. Is this a relationship between table 1 to 3 or from table 2 to 3. I will update the query, you will need to fill in the blanks a bit since you have not shown structures. Also, using table aliases makes code more readable, especially if table names get long. You'll see in a moment.
SELECT
o.store_url,
o.order_id,
o.firstname,
o.lastname,
o.email,
oOp.order_id,
oOp.name,
oOp.value,
p.order_id,
p.name,
p.model,
p.quantity
FROM
oc_order o
JOIN oc_order_option oOp
on o.order_id = oOp.order_id
JOIN oc_order_product p
on o.order_id = p.order_id
ORDER BY
o.order_id
Notice the abbreviated aliases for the table names "o" for oc_orders, "oOp" for oc_order_option and "p" for oc_order_product
Again, YOU need to confirm the "ON" clauses on the left-table is associated with the right table. I have worked with manufacturing in the past and have seen that someone orders. Purchases a product, and that product has options, so Is there something on the order's detail line for the specific product that has an ID to the order OPTIONS table. That is where you probably need to focus. Again, other systems I have had, would have had a hierarchy something like
Order
Customer (who ordered)
OrderDetails (what of multiple products were ordered)
Product ( per orderDetail, what was the product)
ProductOption (color,size,etc)
See the hierarchical representation? In this scenario, you dont go just from Order table to the Product option. You have to get to the details to the product to the product's option. Anyway, HTH
Currently I'm developing a background cms for an online shop.
I split the tables as follow in my database:
-products
-productdetails (descrition...)
-productimages
-product variants (colors..)
-product cross selling
Now on the product edit page i need to fetch all data for a single product.
So my question is how i can get those details more efficient then make 3-5 database calls.
Or would the processing with php be less efficient then make those 3-5 calls ?
At the moment the query looks like that:
SELECT
pr.id, pr.categorieid, pr.itemnumber, pr.barcode, pr.price, pr.weight, pr.gender, pr.manufracture, pr.fsk18, pr.condition, pc.id AS pcid, pc.productcrossid, pc.sort, pd.productname,
pd.productdesc, pd.additional, pd.linktitle, pd.metatitle, pd.metadesc, pd.urlkeywords, pi.id AS piid, pi.wichimage, pi.variantid, pi.image, pi.imagealt, pv.id AS pvid, pv.variant,
pv.variantvalue, pv.sku, pv.price AS pvprice, pv.weight AS pvweight, pv.stock, pv.special
FROM
products pr
LEFT JOIN
productcross as pc
ON pr.id = pc.productid
LEFT JOIN
productdetails as pd
ON pr.id = pd.productid
LEFT JOIN
productimage as pi
ON pr.id = pi.productid AND pd.lang = pi.lang
LEFT JOIN
productvariants as pv
ON pr.id = pv.productid
WHERE
pr.id = :id
ORDER BY pd.lang ASC
As result i recieve many rows, because of the left join each value get joined with the rows i joined before.
The problem is there are dynamic many rows for cross selling, variants, images, so it can be random if variants or images are more (else i could group them atleast because each variant can get an own image, but there can be also more images then variants)
Products 1 row, productdetails according to how many languages are used, most likely 3.
Edit: According to Explain and the indexes i set, the performance of this single query is very good.
Edit:
According Paul Spiegel i tryed using GROUP_CONCAT
SELECT
pr.id, pr.categorieid, pr.itemnumber, pr.barcode, pr.price, pr.weight, pr.gender, pr.manufracture, pr.fsk18, pr.condition, pc.id AS pcid, pc.productcrossid, pc.sort, pd.productname,
pd.productdesc, pd.additional, pd.linktitle, pd.metatitle, pd.metadesc, pd.urlkeywords
FROM
products pr
LEFT JOIN
productsdetails as pd
ON pr.id = pd.productid
LEFT JOIN (
SELECT
GROUP_CONCAT(productcrossid) AS pcproductcrossid, GROUP_CONCAT(sort) AS pcsort, GROUP_CONCAT(id) AS pcid, productid
FROM productscross
WHERE productid = :id
) pc
ON pr.id = pc.productid
WHERE
pr.id = :id
ORDER BY pd.lang ASC
As result i recieve many rows, because of the left join each value get joined with the rows i joined before.
That's not what LEFT means.
X JOIN Y ON ... delivers rows that show up on both X and Y.
X LEFT JOIN Y ON ... delivers all the rows of X even if there is no matching row (or rows) in Y.
You might get "many rows" because the relationship is "1:many". Think of Classes JOIN Students With JOIN you get multiple rows per Class (one per student), except for any classes without any students. With LEFT JOIN, you additionally get a row for any Class with no students.
Your query with products will be a huge explosion of rows. All products, expanded by multiple details by multiple images, etc. It will be a mess.
In the EXPLAIN, multiply the numbers in the "Rows" column -- that will be a crude metric of how big the result set will be.
Use one query to get the images; another to get the colors; etc. Use JOIN (or LEFT JOIN only when needed.
GROUP_CONCAT() is handy sometimes. It might be useful to list the "colors". But for "images", you would then have to split it up so you can build multiple <img..> tags. That's rather easy to do, but it is extra work.
It is usually 'wrong' to have 1:1 mapping between tables. In such cases, why not have a single table?
Do not fear 3-5 queries. We are talking milliseconds. The rendering of the page is likely to take several times as long as the SELECTs. I often have several dozen queries to build a web page, yet I am satisfied with the performance. And, yes, I ascribe to the notion of putting all the info about one 'product' on the page at once (when practical). It's much better than having to click here to get the colors and click there to see the images, etc.
Rather than hitting so many query's you can refer to the concept which is known as flat tables in magento.
The logic behind using this concept is that what ever important data which is required to be show on the front end is stored in single table itself as well as the data is stored in there prescriptive tables.
So while querying you just need to pick the data from that flat table itself rather than querying to multiple tables and increasing the query execution time.
For reference Please check out the below link,Hope this helps.
Visit http://excellencemagentoblog.com/blog/2015/06/10/magento-flat-tables/
I do know the question is not regarding Magento but you can build your own logic to achieve this mechanism.
I need to update several columns in one table, based on columns in another. To start with I am just updating one of them. I have tried 2 ways of doing this, which both work, but they are taking about 4 minutes using mySQL commands, and over 20 when run in php. Both tables are about 20,000 rows long.
My question is, is there a better or more efficient way of doing this?
Method 1:
UPDATE table_a,table_b
SET table_a.price = table_b.price
WHERE table_a.product_code=table_b.product_code
Method 2:
UPDATE table_a INNER JOIN table_b
ON table_a.product_code = table_b.product_code
SET table_a.price=table_b.price
I guess that these basically work in the same way, but I thought that the join would be more efficient. The product_code column is random text, albeit unique and every row matches one in the other table.
Anything else I can try?
Thanks
UPDATE: This was resolved by creating an index e.g.
CREATE UNIQUE INDEX index_code on table_a (product_code)
CREATE UNIQUE INDEX index_code on table_b (product_code)
If your queries are running slowly you'll have to examine the data that query is using.
Your query looks like this:
UPDATE table_a INNER JOIN table_b
ON table_a.product_code = table_b.product_code
SET table_a.price=table_b.price
In order to see where the delay is you can do
EXPLAIN SELECT a.price, b.price FROM table_b b
INNER JOIN table_a a ON (a.product_code = b.product_code)
This will tell you if indexes are being used, see the info on EXPLAIN and more info here.
In your case you don't have any indexes (possible keys = null) forcing MySQL to do a full table scan.
You should always do an explain select on your queries when slowness is an issue. You'll have to convert non-select queries to a select, but that's not difficult, just list all the changed fields in the select clause and copy join and where clauses over as is.
I have two tables in a MySQL database that I can't seem to work with very efficiently. The first is a payments table with around 25,000 rows and the following fields.
ID, Email, Type
The second is a users table which has 2,000 rows and the following:
ID, Email, AccessDate
Using the MySQL JOIN statement I have put this together, and it works perfectly:
UPDATE users INNER JOIN payments ON users.Email=payments.Email SET
AccessDate=NOW() WHERE payments.Type='success'
The problem is, it takes about 95 seconds to execute the query on a local machine. Any tips on how I might get around this speed issue?
Firstly you can use EXPLAIN to see what the optimiser is doing. You`ll have to reconstruct the code to be a SELECT:
EXPLAIN SELECT * FROM users INNER JOIN payments ON users.Email=payments.Email WHERE payments.Type='success'
Most likely is that the join has no index it can use. Does "ID" have an Index? Perhaps you can add that to the Join clauses? Or possibly add indexes for the "Email" column on both tables?
SELECT * FROM articles t LEFT OUTER JOIN category_type category ON (t.category_id=category.id)
WHERE (t.status = 6 AND t.publish_on <= '2014-02-14' AND t.id NOT IN (13112,9490,9386,6045,1581,1034,991,933,879,758) AND t.category_id IN (14)) ORDER BY t.id DESC LIMIT 7;
It take more then 1.5 second to execute this query.
Can you give me some idea ? How can I improve this query and minimum execution time ?
First thing => use where instead of inner join. Because where is faster than inner join query.
Second thing => use indexes for the frequently searched columns. As in your example you search on the basis of status, publish_on besides id as primary index.
If you are using mysql then you can try propose table structure option in the phpmyadmin which can help you to decide valid data types for your column names. This could help you to optimize your query processing.
query processing time depends on many things like: database server load, amount of data in the table and the data types used for the column names too.
why join it with category table?, the category table is not in the where clause nor in the select column clause, so why add it in the query?
oops, * was used, so it "is" in the category table
apologies