I have two tables:
One is called data and there is only one (unique) row per ID.
Second is called images and there are 3 rows per ID.
Every time the page loads i would like to fetch data and one image for exactly 5 different IDs.
My question now is: Two separate SELECT queries or one query where both are joined.
Queries:
...
$all = $row["iD"] // includes **5** last iDs - fetched from DB
$all = implode(',',$all);
SELECT Name, Address FROM data WHERE iD IN($all);
SELECT url FROM images WHERE iD IN ($all) LIMIT 1;
I already have 3 other select queries on page, so i would like to know what is best regarding performance, one bigger - joined or two small - faster queries.
If join, how would these two be joined?
You have three images per ID and desire one image per ID for the last inserted images (aka "recent content" )?
Then you could use one easy natural join combined with group by like this:
SELECT d.Name, d.Address, MAX(i.url)
FROM data d, images i
WHERE i.iD = d.iD
GROUP BY d.Name, d.Address
ORDER BY d.iD DESC
LIMIT 5
Most of the time it is better to combine selects to skip the programmitcally overhead (calling mysql_query() in an loop itself for example).
But sometimes it depends on the underlying data.
Since your queries go to completely separate tables, I recommend you stay with 2 separate queries: This keeps the result sets smaller and makes it more likely, that at least one stays in the query cache,
Concerning your 2nd query: Do you understand, that this is not guaranteed to fetch a special URL, but any? Mostly the first one by key, but not guaranteed so.
For an answer on performance issues, see JOIN queries vs multiple queries . What I can understand from there is that performance issues vary depending on the specific situation so you should test both.
For the join, you could do;
SELECT User.iD, User.Name, User.Address, Image.url
FROM images as Image
JOIN data as User
ON Image.iD = User.iD
WHERE Image.iD IN ($all)
LIMIT 1;
It is not tested yet, so you should take it with a grain of salt. It is at least a starting point.
Related
What I'm trying to do is: I'm trying to build a comments and replies system for my website. I already have this working, but the way I'm doing is probably not the best for performance.
I want to select 10 rows from a table that contains the comments, then I want to select 2 additional rows from another table that contains the replies for each of these comments. I do that by having a loop on PHP to select 2 replies from another table for each comment. Something more or less like this:
$comments = $MySQL->fetchRows("SELECT id, text FROM comments LIMIT 10");
foreach($comments as $i => $c) {
$comments[$i]["replies"] = $MySQL->fetchRows("SELECT id, text FROM replies WHERE comment_id = $c['id'] LIMIT 2");
}
Like I said, I'm sure this isn't the most optimal way of doing it, since it requires multiple calls to the database. Is there a better way of doing this in a single query using MySQL?
I often have 40 well-tuned queries on a single web page. It is not bad.
On the other hand, JOINs, UNIONs, Stored Procedures, etc can cut down the number of roundtrips to the server.
Notes:
A LIMIT without an ORDER BY does not make much sense.
The two queries you have can be combined using a JOIN and a "derived table".
SELECT c.text, r.text
FROM ( SELECT id, text FROM comments
WHERE ...
ORDER BY ...
LIMIT 10 ) AS c
JOIN replies AS r
ON r.id = c.id -- Really the same id??
That will find all the replies from some 10 "comments".
To have limits on both gets trickier.
Currently I'm developing a background cms for an online shop.
I split the tables as follow in my database:
-products
-productdetails (descrition...)
-productimages
-product variants (colors..)
-product cross selling
Now on the product edit page i need to fetch all data for a single product.
So my question is how i can get those details more efficient then make 3-5 database calls.
Or would the processing with php be less efficient then make those 3-5 calls ?
At the moment the query looks like that:
SELECT
pr.id, pr.categorieid, pr.itemnumber, pr.barcode, pr.price, pr.weight, pr.gender, pr.manufracture, pr.fsk18, pr.condition, pc.id AS pcid, pc.productcrossid, pc.sort, pd.productname,
pd.productdesc, pd.additional, pd.linktitle, pd.metatitle, pd.metadesc, pd.urlkeywords, pi.id AS piid, pi.wichimage, pi.variantid, pi.image, pi.imagealt, pv.id AS pvid, pv.variant,
pv.variantvalue, pv.sku, pv.price AS pvprice, pv.weight AS pvweight, pv.stock, pv.special
FROM
products pr
LEFT JOIN
productcross as pc
ON pr.id = pc.productid
LEFT JOIN
productdetails as pd
ON pr.id = pd.productid
LEFT JOIN
productimage as pi
ON pr.id = pi.productid AND pd.lang = pi.lang
LEFT JOIN
productvariants as pv
ON pr.id = pv.productid
WHERE
pr.id = :id
ORDER BY pd.lang ASC
As result i recieve many rows, because of the left join each value get joined with the rows i joined before.
The problem is there are dynamic many rows for cross selling, variants, images, so it can be random if variants or images are more (else i could group them atleast because each variant can get an own image, but there can be also more images then variants)
Products 1 row, productdetails according to how many languages are used, most likely 3.
Edit: According to Explain and the indexes i set, the performance of this single query is very good.
Edit:
According Paul Spiegel i tryed using GROUP_CONCAT
SELECT
pr.id, pr.categorieid, pr.itemnumber, pr.barcode, pr.price, pr.weight, pr.gender, pr.manufracture, pr.fsk18, pr.condition, pc.id AS pcid, pc.productcrossid, pc.sort, pd.productname,
pd.productdesc, pd.additional, pd.linktitle, pd.metatitle, pd.metadesc, pd.urlkeywords
FROM
products pr
LEFT JOIN
productsdetails as pd
ON pr.id = pd.productid
LEFT JOIN (
SELECT
GROUP_CONCAT(productcrossid) AS pcproductcrossid, GROUP_CONCAT(sort) AS pcsort, GROUP_CONCAT(id) AS pcid, productid
FROM productscross
WHERE productid = :id
) pc
ON pr.id = pc.productid
WHERE
pr.id = :id
ORDER BY pd.lang ASC
As result i recieve many rows, because of the left join each value get joined with the rows i joined before.
That's not what LEFT means.
X JOIN Y ON ... delivers rows that show up on both X and Y.
X LEFT JOIN Y ON ... delivers all the rows of X even if there is no matching row (or rows) in Y.
You might get "many rows" because the relationship is "1:many". Think of Classes JOIN Students With JOIN you get multiple rows per Class (one per student), except for any classes without any students. With LEFT JOIN, you additionally get a row for any Class with no students.
Your query with products will be a huge explosion of rows. All products, expanded by multiple details by multiple images, etc. It will be a mess.
In the EXPLAIN, multiply the numbers in the "Rows" column -- that will be a crude metric of how big the result set will be.
Use one query to get the images; another to get the colors; etc. Use JOIN (or LEFT JOIN only when needed.
GROUP_CONCAT() is handy sometimes. It might be useful to list the "colors". But for "images", you would then have to split it up so you can build multiple <img..> tags. That's rather easy to do, but it is extra work.
It is usually 'wrong' to have 1:1 mapping between tables. In such cases, why not have a single table?
Do not fear 3-5 queries. We are talking milliseconds. The rendering of the page is likely to take several times as long as the SELECTs. I often have several dozen queries to build a web page, yet I am satisfied with the performance. And, yes, I ascribe to the notion of putting all the info about one 'product' on the page at once (when practical). It's much better than having to click here to get the colors and click there to see the images, etc.
Rather than hitting so many query's you can refer to the concept which is known as flat tables in magento.
The logic behind using this concept is that what ever important data which is required to be show on the front end is stored in single table itself as well as the data is stored in there prescriptive tables.
So while querying you just need to pick the data from that flat table itself rather than querying to multiple tables and increasing the query execution time.
For reference Please check out the below link,Hope this helps.
Visit http://excellencemagentoblog.com/blog/2015/06/10/magento-flat-tables/
I do know the question is not regarding Magento but you can build your own logic to achieve this mechanism.
I have two tables tableOne = 90K data and tableTwo = 100k data, i will look for the duplicate numbers on both tables with the given conditions and the matching must be 1:1 if multiple match are on the other table only one will be tagged as match (given that the data on both tables has match data).
I have this select statement below, but when i run it on my local xampp and even on CMD the screen freezes after i press enter then it takes hours before it returns an error out of memory. Hope you can help me with this.
SELECT rNum,
cDate,
cTime,
aNumber,
bNumber,
duration,
tag,
aNumber2,
bNumber2,
'hasMatch',
concatDate,
timeMinutes
FROM tableOne a
LEFT JOIN
tableTwo b ON a.aNumber2 = b.aNumber2
AND a.bNumber2 = b.bNumber2
WHERE a.hasMatch = 'valid'
AND (a.duration - b.duration) <= 3
AND (a.duration - b.duration) >= -3
AND TIMEDIFF(a.concatDate,b.concatDate) <= 3
AND TIMEDIFF(a.concatDate,b.concatDate) >= -3
Thank you In advance.
If you're doing 1:1 relationship with two tables then I think you should probably go with INNER JOIN rather than LEFT JOIN
Secondly, your query doesn't seem to be indexed properly. So, better would be using EXPLAIN SELECT ... to see the profile of SQL and create INDEXES for Filters.
in your SELECT you have aNumber2 and based on your join rule both table a and table b have aNumber2 column. it's a problem. if two table have a column with the same name, on select you should specify the table.
for example like this
SELECT a.aNumber2 as a_number2,....
in your query the same problem exists for other columns like duration and concatDate
another thing is you should use INNER JOIN in your case instead of LEFT JOIN.
if you final result have many rows(thousands), take them step by step... add LIMIT to your example and take 100 result each time.
I'm trying to set up a high scores board for a game I'm making. On the board, I'm using foreign keys to two other boards, players and weapons. Each score stores the four weapons the player used on that run. The tables are set up like this:
Scores
id|playerid|score|weapon0id|weapon1id|weapon2id|weapon3id
Players
id|name
Weapons
id|name
I want to select multiple rows from the scores table with ids replaced by the appropriate names. I'm able to get the correct player name and one weapon using this statement:
SELECT scoreID, Players.playerName, scoreVal,
Weapons.weaponLabel, scoreW1, scoreW2, scoreW3
FROM Scores, Players, Weapons
WHERE Players.playerID = scorePlayer AND Weapons.weaponID = scoreW0
Everywhere I've looked shows that to be the best way to get a value from a row referred to by a foreign key. It works fine for the player name, but there seems to be no way to expand this to fill in multiple weapon names at once. Using an OR with the remaining weapons or using weaponID IN (w0,w1,w2,w3) seems to get one row for each weapon, not one row with each weapon in the appropriate spot.
Is there any way to get the correct weapon names just using the select statement? Or will I need to have extra code loop through and replace each weapon id with the correct name?
This design is questionable: weapon0..n will likely lead to nothing but difficult queries like this. The queries will also have to be de-normalized - e.g. one join per weapon0..n.
Anyway, the query is wrong and will return many more rows than desired because it uses the form FROM a,b which implies a CROSS JOIN between a and b and there is not appropriate selectors in the WHERE to make it an equi-join. Try to use a normal (INNER) JOIN and ON to make each join more apparent:
SELECT s.scoreID, p.playerName, s.scoreVal,
w0.weaponLabel as w0Label,
w1.weaponLabel as w1Label
-- etc
FROM Scores s
JOIN Players p ON p.id = s.playerID
JOIN Weapons w0 ON w0.weaponID = s.scoreW0
JOIN Weapons w1 ON w1.weaponID = s.scoreW1
-- etc, ick!!!
By now it should become apparent why the de-normalized data is icky!
Each column must be joined with a different relation (w0, w1, etc).
I usually have to create a looping procedure to get all the denormalized columns in one row per unique set, in your case player, weaponlabel.
I am trying to query 2 tables in a database, each query having nothing to do with each other, other then being on the same page.
Query 1 - The first query on the page will retrieve text and images that are found throughout the page from Table A.
Query 2 - The second query will retrieve several products with a image, description and title for each product from Table B.
I know that putting the second query inside the first query's while loop would work but of course is very inefficient.
How can I and what is the best way to retrieve all the data I need through 1 query?
Thanks,
Dane
So all you want to know is if its ok to have 2 queries on the same webpage? Its A-OK. Go right ahead. Its completelly normal. No one expects a join between table news and table products. Its normal to usetwo queries to fetch data from two unrelated tables.
Use LEFT or INNER JOIN (depends on whether you want to display records from TableA that have no correspondent records in TableB)
SELECT a.*, b.*
FROM TableA a
[LEFT or INNER] JOIN TableB b ON (b.a_id = a.id)
If there's no way to relate the two tables to each other, then you can't use a JOIN to grab records from both. You COULD use a UNION query, but that presumes that you can match up fields from each table, as a UNION requires you to select the same number/type of fields from each table.
SELECT 'pageinfo' AS sourcetable, page.id, page.images, page.this, page.that
WHERE page.id = $id
UNION
SELECT 'product' AS sourcetable, products.id, products.image, product.other, product.stuff
But this is highly ugly. You're still forcing the DB server to do two queries in the background plus the extra work of combining them into a single result set, and then you have to do extra work to dis-entangle in your code to boot.
It's MUCH easier, conceptually and maintenance-wise, to do two seperate queries instead.