I am looking for some input on querying multiple tables,
I currently have a list which contains the day, (each day the reports where made.)
$list = mysql_query("SELECT * FROM list ORDER BY id");
while ($row = mysql_fetch_assoc($list)){
$drop_list[] = $row['day'];
}
My end goal is to create a query which checks a unique row from each table,
I was thinking arround the lines of something like this.
foreach ($drop_list as $v) {
$daily = mysql_query("SELECT * FROM $v WHERE ID = 1");
while ($row = mysql_fetch_assoc($daily)){
$id = $row['id'];
$name = $row['name'];
$age = $row['age'];
$day = $row['day'];
}
echo "<tr><td>$id</td><td>$name</td><td>$age</td><td>$day</td></tr>";
}
Then put that into a function and echo it out in between the table tag.
I am sure the code works, (Have not tested yet Typing this from tablet) but was curious if using foreach item in array query the data from DB and echo it out to give me the daily results for the id in array?
Also curious if other have different method to accomplish this?
The short answer
Running several SQL queries inside a foreach loop will hurt performance. There is properly a better solution where everything is fetched in one query. Database queries are expensive and should be optimized as much as possible to reduce loading times of the webpage and save resources for other simultaneous requests. You can download the MySQL Workbench to help write queries as well as optimize them using the analyzer tool. There are plenty of tutorials on how to use this program around the web.
A possible solution
I assume you know the tables which you want to query and that the list of them stays the same for long periods of time. I would then fetch everything inside one query using multiple SELECT statements and the UNION keyword. This assumes the columns inside the different tables are the same. By looking at the code it seems they all declare the required columns.
SELECT * FROM table1 WHERE id = 1
UNION ALL
SELECT * FROM table2 WHERE id = 1
UNION ALL
SELECT * FROM table3 WHERE id = 1
This will fetch every single row from each of the listed tables where the id equals 1. By appending the ALL keyword to the union statement we assure duplicate rows from all the tables are also returned.
One big disadvantage of this solution is that we have no reference to from which table each row originates from. If this is required some more complex SQL queries are properly necessary, but I would still recommend combining the queries into one.
Important!
Please note that the mysql_* functions are deprecated. They are not supported anymore and security holes are not patched. I strongly recommend switching to the PDO or MySQLi extensions. They provide the better solutions in security and performance for PHP.
Side note
By looking at your code I really do not understand, why you have several tables all declaring the same columns? This seems redundant to me, but maybe I lack some more insight. It would be more effective to have only one table to maintain.
I hope this can help guide you, happy coding!
Related
For example, i have a table "tbl_book" with 100 records or more with multiple column like book_name, book_publisher,book_author,book_rate in mysql "db_bookshop". Now i would like to fetch them all by one query without iterate 100 times instead of one or two time looping. Is it possible? Is there any tricky way to do that. Generally we do what
$result = mysql_query("SELECT desire_column_name FROM table_name WHERE clause");
while( $row = mysql_fetch_array($result) ) {
$row['book_name'];
$row['book_publisher'];
$row['book_author'];
..........
$row['book_rate'];
}
// Or we may can use mysqli_query(); -mysqli_fetch_row(), mysqli_fetch_array(), mysqli_fetch_assoc();
My question is, is there any idea or any tricky way that we can be
avoided 1oo times iterate for fetching 1oo records? It's may be wired
to someone but one of the most experience programmer told me that it's
possible. But unfortunately i was not able to learn it from him. I
feel sorry for him because he is not anymore. Advance thanks for your idea sharing.
You should not use mysql_query the mysql extension is deprecated:
This extension is deprecated as of PHP 5.5.0, and has been removed as of PHP 7.0.0.
-- https://secure.php.net/manual/en/intro.mysql.php
When you use PDO you can fetch all items without looping over query like this
$connection = new PDO('mysql:host=localhost;dbname=testdb', 'dbuser', 'dbpass');
$statement = $connection->query('SELECT ...');
$rows = $statement->fetchAll();
The short answer - NO, it's impossible to fetch more than one record from a database without a loop.
But the the question here is that you don't want it.
There is no point in "just fetching" the data - you're always going to do something with it. With each row. Obviously, a loop is a natural way to do something with each row. Therefore, there is no point in trying to avoid a loop.
Which renders your question rather meaningless.
Regarding performance. The truth is that you experience not a single performance problem related to fetching just 100 records from a database. Which renters your problem an imaginary one.
The only plausible question I can think off your post is your performance as a programmer, as lack of education makes you write a lot of unnecessary code. If you manage to ask a certain question regarding that matter, you'll be shown a way to avoid the useless repetitive typing.
Have you tried using mysql_fetch_assoc?
$result = mysql_query("SELECT desire_column_name FROM table_name WHERE clause");
while ($row = mysql_fetch_assoc($result)) {
// do stuff here like..
if (!empty($row['some_field'])){
echo $row["some_field"];
}
}
It is possible to read all 100 records without loop by hardcoding the main column values, but that would involve 100 x number of columns to be listed, and there could be limitation on the number of columns you can display in MySQL.
eg,
select
case when book_name='abc' then book_name end Name,
case when book_name='abc' then book_publisher end as Publisher,
case when book_name='abc' then book_author end as Author,
case when book_name='xyz' then book_name end Name,
case when book_name='xyz' then book_publisher end as Publisher,
case when book_name='xyz' then book_author end as Author,
...
...
from
db_bookshop;
It's not practical but if you have less rows to query you might find it useful.
The time taken to ask the MySQL server for something is far greater than one iteration through a client-side WHILE loop. So, to improve performance, the goal is to have the SELECT go to the server in one round trip. Different API calls do this or don't do this; read their details.
I have written a lot of UIs with MySQL under the covers. I think nothing of fetching a few dozen rows at once, and then build a <table> (or something) with the results. I rarely fetch more than 100, not because of performance, but because 100 is (usually) too much for the user to take in on a single web page.
Also, I think nothing of issuing several, maybe dozens, of queries in support of a single web page. The delay is insignificant, especially when compared to the user's time for reading, digesting, and moving to the next page. So, I try to give the user a digestible amount of info without having to click to another page to get more. There are tradeoffs.
When it is practical to have SQL do the 'digesting', do so. It is faster for MySQL do do a SUM() and return just the total, rather than return dozens of rows for the client to add up. This is mostly a 'bandwidth' issue. Either way, MySQL will fetch (internally) all the needed rows.
In a site I maintain I have a need to query the same table (articles) twice, once for each category of article. AFAIT there are basically two ways of doing this (maybe someone can suggest a better, third way?):
Perform the db query twice, meaning the db server has to sort through the entire table twice. After each query, I iterate over the cursor to generate html for a list entry on the page.
Perform the query just once and pull out all the records, then sort them into two separate arrays. After this, I have to iterate over each array separately in order to generate the HTML.
So it's this:
$newsQuery = $mysqli->query("SELECT * FROM articles WHERE type='news' ");
while($newRow = $newsQuery->fetch_assoc()){
// generate article summary in html
}
// repeat for informational articles
vs this:
$query = $mysqli->query("SELECT * FROM articles ");
$news = Array();
$info = Array();
while($row = $query->fetch_assoc()){
if($row['type'] == "news"){
$news[] = $row;
}else{
$info[] = $row;
}
}
// iterate over each array separate to generate article summaries
The recordset is not very large, current <200 and will probably grow to 1000-2000. Is there a significant different in the times between the two approaches, and if so, which one is faster?
(I know this whole thing seems awfully inefficient, but it's a poorly coded site I inherited and have to take care of without a budget for refactoring the whole thing...)
I'm writing in PHP, no framework :( , on a MySql db.
Edit
I just realized I left out one major detail. On a given page in the site, we will display (and thus retrieve from the db) no more than 30 records at once - but here's the catch: 15 info articles, and 15 news articles. On each page we pull the next 15 of each kind.
You know you can sort in the DB right?
SELECT * FROM articles ORDER BY type
EDIT
Due to the change made to the question, I'm updating my answer to address the newly revealed requirement: 15 rows for 'news' and 15 rows for not-'news'.
The gist of the question is the same "which is faster... one query to two separate queries". The gist of the answer remains the same: each database roundtrip incurs overhead (extra time, especially over a network connection to a separate database server), so with all else being equal, reducing the number database roundtrips can improve performance.
The new requirement really doesn't impact that. What the newly revealed requirement really impacts is the actual query to return the specified resultset.
For example:
( SELECT n.*
FROM articles n
WHERE n.type='news'
LIMIT 15
)
UNION ALL
( SELECT o.*
FROM articles o
WHERE NOT (o.type<=>'news')
LIMIT 15
)
Running that statement as a single query is going to require fewer database resources, and be faster than running two separate statements, and retrieving two disparate resultsets.
We weren't provided any indication of what the other values for type can be, so the statement offered here simply addresses two general categories of rows: rows that have type='news', and all other rows that have some other value for type.
That query assumes that type allows for NULL values, and we want to return rows that have a NULL for type. If that's not the case, we can adjust the predicate to be just
WHERE o.type <> 'news'
Or, if there are specific values for type we're interested in, we can specify that in the predicate instead
WHERE o.type IN ('alert','info','weather')
If "paging" is a requirement... "next 15", the typical pattern we see applied, LIMIT 30,15 can be inefficient. But this question isn't asking about improving efficiency of "paging" queries, it's asking whether running a single statement or running two separate statements is faster.
And the answer to that question is still the same.
ORIGINAL ANSWER below
There's overhead for every database roundtrip. In terms of database performance, for small sets (like you describe) you're better off with a single database query.
The downside is that you're fetching all of those rows and materializing an array. (But, that looks like that's the approach you're using in either case.)
Given the choice between the two options you've shown, go with the single query. That's going to be faster.
As far as a different approach, it really depends on what you are doing with those arrays.
You could actually have the database return the rows in a specified sequence, using an ORDER BY clause.
To get all of the 'news' rows first, followed by everything that isn't 'news', you could
ORDER BY type<=>'news' DESC
That's MySQL short hand for the more ANSI standards compliant:
ORDER BY CASE WHEN t.type = 'news' THEN 1 ELSE 0 END DESC
Rather than fetch every single row and store it in an array, you could just fetch from the cursor as you output each row, e.g.
while($row = $query->fetch_assoc()) {
echo "<br>Title: " . htmlspecialchars($row['title']);
echo "<br>byline: " . htmlspecialchars($row['byline']);
echo "<hr>";
}
Best way of dealing with a situation like this is to test this for yourself. Doesn't matter how many records do you have at the moment. You can simulate whatever amount you'd like, that's never a problem. Also, 1000-2000 is really a small set of data.
I somewhat don't understand why you'd have to iterate over all the records twice. You should never retrieve all the records in a query either way, but only a small subset you need to be working with. In a typical site where you manage articles it's usually about 10 records per page MAX. No user will ever go through 2000 articles in a way you'd have to pull all the records at once. Utilize paging and smart querying.
// iterate over each array separate to generate article summaries
Not really what you mean by this, but something tells me this data should be stored in the database as well. I really hope you're not generating article excerpts on the fly for every page hit.
It all sounds to me more like a bad architecture design than anything else...
PS: I believe sorting/ordering/filtering of a database data should be done on the database server, not in the application itself. You may save some traffic by doing a single query, but it won't help much if you transfer too much data at once, that you won't be using anyway.
I have a database design here that looks this in simplified version:
Table building:
id
attribute1
attribute2
Data in there is like:
(1, 1, 1)
(2, 1, 2)
(3, 5, 4)
And the tables, attribute1_values and attribute2_values, structured as:
id
value
Which contains information like:
(1, "Textual description of option 1")
(2, "Textual description of option 2")
...
(6, "Textual description of option 6")
I am unsure whether this is the best setup or not, but it is done as such per requirements of my project manager. It definitely has some truth in it as you can modify the text easily now without messing op the id's.
However now I have come to a page where I need to list the attributes, so how do I go about there? I see two major options:
1) Make one big query which gathers all values from building and at the same time picks the correct textual representation from the attribute{x}_values table.
2) Make a small query that gathers all values from the building table. Then after that get the textual representation of each attribute one at a time.
What is the best option to pick? Is option 1 even faster as option 2 at all? If so, is it worth the extra trouble concerning maintenance?
Another suggestion would be to create a view on the server with only the data you need and query from that. That would keep the work on the server end, and you can pull just what you need each time.
If you have a small number of rows in attributes table, then I suggest to fetch them first, fetch all of them! store them into some array using id as index key in array.
Then you can proceed with building data, now you just have to use respective array to look for attribute value
I would recommend something in-between. Parse the result from the first table in php, and figure out how many attributes you need to select from each attribute[x]_values table.
You can then select attributes in bulk using one query per table, rather than one query per attribute, or one query per building.
Here is a PHP solution:
$query = "SELECT * FROM building";
$result = mysqli_query(connection,$query);
$query = "SELECT * FROM attribute1_values";
$result2 = mysqli_query(connection,$query);
$query = "SELECT * FROM attribute2_values";
$result3 = mysqli_query(connection,$query);
$n = mysqli_num_rows($result);
for($i = 1; $n <= $i; $i++) {
$row = mysqli_fetch_array($result);
mysqli_data_seek($result2,$row['attribute1']-1);
$row2 = mysqli_fetch_array($result2);
$row2['value'] //Use this as the value for attribute one of this object.
mysqli_data_seek($result3,$row['attribute2']-1);
$row3 = mysqli_fetch_array($result3);
$row3['value'] //Use this as the value for attribute one of this object.
}
Keep in mind that this solution requires that the tables attribute1_values and attribute2_values start at 1 and increase by 1 every single row.
Oracle / Postgres / MySql DBA here:
Running a query many times has quite a bit of overhead. There are multiple round trips to the db, and if it's on a remote server, this can add up. The DB will likely have to parse the same query multiple times in MySql which will be terribly inefficient if there are tons of rows. Now, one thing that your PHP method (multiple queries) has as an advantage is that it'll use less memory as it'll release the results as they're no longer needed (if you run the query as a nested loop that is, but if you query all the results up front, you'll have a lot of memory overhead, depending on the table sizes).
The optimal result would be to run it as 1 query, and fetch the results 1 at a time, displaying each one as needed and discarding it, which can reek havoc with MVC frameworks unless you're either comfortable running model code in your view, or run small view fragments.
Your question is very generic and i think that to get an answer you should give more hints to how this page will look like and how big the dataset is.
You will get all the buildings with theyr attributes or just one at time?
Cause your data structure look like very simple and anything more than a raspberrypi can handle it very good.
If you need one record at time you don't need any special technique, just JOIN the tables.
If you need to list all buildings and you want to save db time you have to measure your data.
If you have more attribute than buildings you have to choose one way, if you have 8 attributes and 2000 buildings you can think of caching attributes in an array with a select for each table and then just print them using the array. I don't think you will see any speed drop or improvement with so simple tables on a modern computer.
$att1[1]='description1'
$att1[2]='description2'
....
Never do one at a time queries, try to combine them into a single one.
MySQL will cache your query and it will run much faster. PhP loops are faster than doing many requests to the database.
The query cache stores the text of a SELECT statement together with the corresponding result that was sent to the client. If an identical statement is received later, the server retrieves the results from the query cache rather than parsing and executing the statement again.
http://dev.mysql.com/doc/refman/5.1/en/query-cache.html
If is posible, how can I fetch two results sets:
$sth=$dbh->prepare("SELECT * FROM tb1 WHERE cond1;
SELECT * from tb2 Where cond2");
$sth->execute();
$row=$sth->fetchAll(); print_r ($row);
These are two completely different tables (no fiels in common).
Yes PDO can fetch two (or more) rowsets, as long as the database you are using supports it. I think MS SQL Server and MySQL both support this functionality, but at the time of writing SQLite does not.
The function you want is PDOStatement::nextRowset
So in your example above, you might do something like;
$sth = $dbh->prepare("SELECT * FROM tb1 WHERE cond1;
SELECT * FROM tb2 WHERE cond2");
$sth->execute();
$rowset1 = $sth->fetchAll();
$sth->nextRowset();
$rowset2 = $sth->fetchAll();
print_r($rowset1);
print_r($rowset2);
It's perfectly reasonable for a single stored procedure to return more than one rowset.
$rowset[] = $sth->fetchAll(PDO::FETCH_OBJ);
WHILE($sth->nextRowset()) {
$rowset[] = $sth->fetchAll(PDO::FETCH_OBJ);
}
Now your $rowset will be an array. You may use count() to find out how many rowsets you have. And use foreach loop to get each rowset
Depending on the specific version of PHP you're using, you might not even be able to execute multiple statements per query / execute.
The short answer is that no, it is not better to return two separate tables in one query. Both tables would have to have the same amount of columns. There won't be a way of telling result #1 from result #2 in PHP. And if you ever want to change either of the tables, you'll have to ensure that the tables still remain compatible.
You might be able to write a benchmark which runs a couple of thousand queries, first as two separate ones, and then one as UNION ALL (as suggested above). I doubt you're going to find much of a performance difference between the two. If the SQL server is running on the same physical server, the query itself is near instantaneous and you may incur a performance hit for doing the UNION ALL. If it is over the wire, to another server on the network, or over the internet (for whatever reason) that will offset the benefit of the combined query.
If these are semantically separate pieces information, it just isn't a very good idea in the long run to tie the two queries together. This smacks of "premature optimization", and that is the root of all evil as Donald Knuth says.
$STH = $DBH->prepare('show create table `a`;
show create table `b`;
show create table `b`;
');
$STH->execute( );
do {
$row =$STH->fetchAll(\PDO::FETCH_NUM);
if($row && count($row)>0){
$sql.=$row[0][1] . ';';
}
} while ($STH->nextRowset());
A multi result set in one loop
To answer your question, no, it's not possible how you are doing it. You have two options:
Combine both queries into one
Execute both queries separately and combine their results
Since mysql queries tend to be expensive, I would combine the queries if at all possible. In your case you could use a simple UNION:
SELECT row1, row2, row3
FROM tb1
WHERE id < 10
UNION ALL
SELECT row1, row2, row3
FROM tb2
WHERE id < 10
However you should probably evaluate why you are using two separate tables. If they contain overlapping data, it may be better to combine into one table, if possible.
Perhaps you need JOIN if these tables are connected.
Or 2 separate calls if it's completely different and irrelevant tables.
It's impossible to tell more because of your extremely short and obscure question.
Anyway, there is no reason to combine queries only out of desire to run them all in one trip.
I'm trying to learn best practices with multiple SQL queries in PHP and mySQL. Not sure if I should have two separate queries or if I should join (or UNION?) my queries into one.
Can I do the following or is there another way to accomplish the same thing?
It's important to note that my first query is pulling a list of questions and answers for a quiz. The second query is pulling a list of score profiles that I'll be using to assign to users based on their quiz score. The only relationship between these two tables is that both use the same foreign key linked to an ID in my "quiz" table. I wouldn't need to JOIN these tables side by side, but I think I would probably need to UNION them so my second query results appear after the first.
Now if I should union my queries into one, I have no idea how I would loop through the query results to create my array. When I loop through each result, I would need some conditional logic that does something with my first query results and something different with my second query results. How would I write those conditions?
Here is what I'm doing now that seems to work fine as two separate queries...
...query the database for the first time to get my quiz questions and answers:
$result_quiz = mysql_query("
SELECT quiz_question.question_text, quiz_question.id, quiz_answer.answer_text, quiz_answer.points
FROM quiz
JOIN quiz_question ON (quiz.id = quiz_question.quiz_id)
JOIN quiz_answer ON (quiz_question.id = quiz_answer.quiz_question_id)
WHERE quiz.id = $id;
");
...then based on the above query, build my array of questions and answers:
$quiz = array();
while ($row = mysql_fetch_assoc($result_quiz)) {
if (!isset($quiz['questions']['q'.$row['id'].''])) {
$quiz['questions']['q'.$row['id'].''] = array(
'question' => $row['question_text'],
'answers' => array()
);
}
$quiz['questions']['q'.$row['id'].'']['answers'][] = array('answer' => $row['answer_text'],'points' => $row['points']);
}
...then query the database for the second time to get a list of score profiles:
$result_profile = mysql_query("
SELECT quiz_profile.name, quiz_profile.description, quiz_profile.min_points, quiz_profile.max_points
FROM quiz
JOIN quiz_profile ON (quiz.id = quiz_profile.quiz_id)
WHERE quiz.id = $id;
");
...then loop through my second query so I can append the score profiles to my array:
while ($row = mysql_fetch_assoc($result_profile)) {
$quiz['profiles'][] = array('name' => $row['name'],'description' => $row['description'],'min_points' => $row['min_points'],'max_points' => $row['max_points']);
}
Is this the best approach?
Or should I combine my queries into one using UNION?
If so, how can I write my conditions to know what results I'm looping through?
Thanks SO much!
I suggest sticking with the current approach, unless there is a serious problem with performance - combining the two queries might improve total throughput (slightly), but would significantly increase the complexity of the code required to display it.
I recommend to take a step back at the design of your app, not the code itself: If you go down the UNION road, you have to sacrifice all isolation between your questions-code and your scoreprofiles-code.
I say, this is not a good thing: What if one day the score profiles are in some way recalculated or trabsformed between db and user? Is it intuitive, that for that you have to touch the function getting the questions from the db?
In PHP, there is a rule of thumb, which (as all rules of thumb) is not 100% perfect, but quite a start: It should be possible to have all code relating to one field of functionality be located in one file, the includes and requires mirroring the dependencies.