Is it ok to a mysql query inside a while loop using the ID of each row passed to fetch results from another table? OR is there a better way to do it?
$q = $__FROG_CONN__->query("SELECT cms_page.id, cms_page.title, cms_page.slug, cms_page_part.* FROM cms_page LEFT JOIN cms_page_part ON cms_page_part.page_id=cms_page.id WHERE cms_page.parent_id='8'");
$r = $q->fetchAll(PDO::FETCH_ASSOC);
echo '<ul id="project-list">';
foreach ($r as $row) {
echo '<li>';
echo '<img src="<img src="phpThumb/phpThumb.php?src=public/images/'.$row[0].'/th.jpg&w=162" alt="" />';
echo '<div class="p-text">';
echo '<h4>'.$row["location"].'<span>'.$row["project_date"].'</span></h4>';
echo '<p>'.$row["body"].'</p>';
echo '</div>';
echo '</li>';
}
echo '</ul>';
I am trying to pull the project_date, body and location fields from another table where the sql query matches. The title and slug are held in another table. There should only be a maximum of eight or so results but im getting alot more.
The suggestions using IN are fine, but if you are getting the ids from another query, it might be better to combine these two queries into one query using a join.
Instead of:
SELECT id FROM users WHERE age <30
SELECT id, x FROM userinfo WHERE userid IN ($id1, $id2, ..., $idn)
do:
SELECT users.id, userinfo.x
FROM users
LEFT JOIN userinfo ON userinfo.userid = users.id
WHERE age < 30
To reduce the overhead of preforming a query, you may want to look at getting all the data in a single query. In which case you may want to take a look at IN(), e.g.
SELECT * WHERE x IN (1, 2);
There is also BETWEEN()
SELECT * WHERE x BETWEEN 1 AND 2;
See the mysql docs for more information
I would try to build the query in a way where I only need to pass it once. Something like WHERE ID=1 OR ID=2 OR ... Passing multiple queries and returning multiple recordsets is expensive in terms of processing and network traffic.
This will be very inefficient, what you want is to join the tables on the ID
SELECT * FROM table1 LEFT JOIN table2 ON (table1.ID = table2.ID) WHERE condition
Mysql join documentation
This will return one set of rows with all the information you need, returned from both tables.
In a small application / small result set, this might be okay, but it results in a lot of (small) calls to the database.
If you can find an alternative way (perhaps see Yacoby's suggestion?) in which you can do one call to the database, this is probably better.
EDIT
If you are only interested in the IDs from one table, in order to get the correct results out of another table, perhaps a JOIN is what you are looking for?
SELECT t1.fieldA, t1.fieldB
FROM table1 t1
JOIN table2 t2 ON t1.ID = t2.ID
WHERE t2.someField = 'someValue'
Is it ok to a mysql query inside a while loop using the ID of each row passed to fetch results from another table? OR is there a better way to do it?
You should reformulate your query in SQL. Say, put the ids into a memory table and use it in a JOIN:
SELECT *
FROM idtable
JOIN mytable
ON mytable.id = idtable.id
This way, MySQL will make the loops for you but will make them in (usually) more efficient way.
SQL is a language designed to work with sets.
So if you have a set of ids and a table (which is a set of tuples), your first thought should be "how do I apply the set-based operations to these sets using SQL"?
Of course it is possible to run a bunch of simple queries in a loop but this way you just do extra work which SQL engine developers most probably have already done for you (and usually have done it in more efficient way).
You may want to read this article in my blog:
Double-thinking in SQL
Related
I would like to count the number of rows in a statement returned by a query. The only solutions I found were:
sqlsrv_num_rows() This one seems a bit too complicated for such a simple task like this and I read that using this slows down the execution quite a bit
Executing a query with SELECT COUNT This method seems unnecessary, also it slows down the execution and if you already have a statement why bother with another query.
Counting the rows while generating a table As I have to generate a html table from the statemnt I could put a variable in the table generating loop and increment it by one, but this one only works when you already have to loop through the entire statement.
Am I missing some fundamental function and/or knowledge or is there no simpler way?
Any help or guidance is appreciated.
EDIT: The statement returned is only a small portion of the original table so it wouldn't be practical to execute another query for this purpose.
In sql server table rows information is stored in the catalog views and Dynamic Management Views you can use it to find the count
This method will only work for the physical tables. So you can store the records in one temp table and drop it later
SELECT Sum(p.rows)
FROM sys.partitions AS p
INNER JOIN sys.tables AS t
ON p.[object_id] = t.[object_id]
INNER JOIN sys.schemas AS s
ON t.[schema_id] = s.[schema_id]
WHERE p.index_id IN ( 0, 1 ) -- heap or clustered index
AND t.NAME = N'tablename'
AND s.NAME = N'dbo';
For more info check this article
If you don't want to execute another query then use select ##rowcount after the query. It will get the count of rows returned by previous select query
select * from query_you_want_to_find_count
select ##rowcount
I have a query as follows:
SELECT event
FROM log
WHERE user = (SELECT SUBSTRING(details,55) FROM log WHERE details LIKE 'ID: 308%')
I know I can use an inner join or php loop separate here but I have queries where I cannot use inner joins and a problem similar to this happens (which im about to explain).
The subquery for the where clause returns many email addresses, and I want to then bring up any log events relating to any of them. My problem is I then get an error message 'Subquery returns more than 1 row'.
If anyone could help that would be much appreciated!
I think this should work:
SELECT event FROM log
WHERE user IN
(SELECT SUBSTRING(details,55) FROM log WHERE details LIKE 'ID: 308%')
Use WHERE user IN <subquery> instead of WHERE user = <subquery>.
However, in my experience, MySQL's performance of IN <subquery> is usually very poor. This can always be rewritten as a JOIN, and that usually performs better.
Here's your example query rewritten as a JOIN:
SELECT event
FROM log l1
JOIN (SELECT DISTINCT SUBSTRING(details,55) loguser
FROM log
WHERE details LIKE 'ID: 308%') l2
ON l1.user = l2.loguser
In this particular case, I suspect performance will be similar. Where MySQL usually gets it wrong is when the subquery in the JOIN returns an indexed column, and you're joining with an indexed column in the main table. MySQL decides to use the index in the subquery instead of the main table, and that's usually wrong (in most cases of queries like this, the subquery returns a small subset of values).
Your other option is to use EXISTS
SELECT `event`
FROM log
WHERE EXISTS(SELECT *
FROM log AS log1
WHERE log1.details LIKE 'ID: 308%'
AND log.user = SUBSTRING(log1.details,55))
I have a problem about my code, this is my code on connecting to database, selecting some unique data and sum on these unique data from another table, nothing problem with my code, thats all okay, but it taking so long time on querying.
<?php
include "koneksi.php";
$no=1;
$arqury=odbc_exec($koneksi, "SELECT DISTINCT NIP_AR,NAMA_AR FROM USRBPS.MASTERFILE");
while($ar=odbc_fetch_array($arqury)){
$total=0;
$ambilqury=odbc_exec($koneksi, "SELECT NPWP FROM USRBPS.MASTERFILE WHERE NIP_AR='$ar[NIP_AR]'");
while($ambil=odbc_fetch_array($ambilqury)){
$testqury=mysql_query("SELECT SUM(jumlah_bayar) as PENERIMAAN FROM mpnruteng WHERE npwp='$ambil[NPWP]'");
$test=mysql_fetch_array($testqury);
$total += $test[PENERIMAAN];
}
if($ar[NIP_AR]==""){
echo "<tr><td>$no</td><td colspan=2>UNASSIGN</td><td>$total</td>";
}
else{
echo "<tr><td>$no</td><td>$ar[NAMA_AR]</td><td>$ar[NIP_AR]</td><td>$total</td></tr>";
}
$no++;
}
?>
on simply is being like this,
|Name |num_se
---------------
|andre |1111
|john |2222
|simon |3333
|andre |4444
|andre |5555
|simon |6666
|john |7777
|num_se |Total
---------------
|1111 |12
|2222 |15
|3333 |10
|4444 |8
|5555 |20
|6666 |18
|7777 |22
So, what i need is get the sum of 'Total', from each 'Name'. What i want to get is, list of the name "Uniquely" (in this example is Andre, John, and Simon) with each sum of "Total" that get from num_se.
sorry my english is bad, but i hope you're understand.
It would have been pertinent to explain you were working against two separate data sources up front.
It's slow because you're attempting to loop over your resultsets in PHP, calling individual queries to return single rows from another table, which is very inefficient.
Let the database handle the relationship between your two related tables by using a JOIN , and then let it handle to aggregation of your grand total, not just the individual totals.
The following query will get your total for each distinct pair of nip_ar and nama_ar:
SELECT
t1.NIP_AR,
t1.NAMA_AR,
SUM(t2.jumlah_bayar) as PENERIMAAN
FROM
USRBPS.MASTERFILE t1
INNER JOIN mpnruteng t2
ON t2.npwp = t1.npwp
GROUP BY
1, 2
But you're rolling this up into one grand $total anyhow, so this will get that for you:
SELECT
SUM(PENERIMAAN) as PENERIMAAN
FROM
(
SELECT
t1.NIP_AR,
t1.NAMA_AR,
SUM(t2.jumlah_bayar) as PENERIMAAN
FROM
USRBPS.MASTERFILE t1
INNER JOIN mpnruteng t2
ON t2.npwp = t1.npwp
GROUP BY
1, 2
)
You can configure a link to your mySql server from your Oracles server. Refer to Using Heterogenous Service Agents - chapter 4. Setting up access to non-Oracle systems. My understanding is that you cannot link to Oracle from mySql.
This will allow you to run the above queries, on your Oracle instance. Although you'll need to update your table names to fully qualify them with the datasource names.
If I understand correctly, this query may help
SELECT t1.Name, sum(t2.Total)
FROM (table1 t1 LEFT JOIN table2 t2 ON t1.num_se = t2.num_se)
GROUP BY t1.Name
Modify the table name and column name based on your case.
Update
I don't know why you use two separated data source. However, I think this approaches will improve the efficiency of your code.
First, get the list of all num_se belong to each name by this query:
"SELECT Name, GROUP_CONCAT(num_se) as nums FROM table1 GROUP BY Name"
Now your result array ('$ar') will have an element like this "1111,2222" with the key nums
Second, use this query to get the sum for each name
"SELECT SUM(Total) as total FROM Table2 WHERE num_se IN (" . $ar['nums'] . ")"
And you will get the total for each name without needing a second loop.
Remember to use escaping techniques to be sure your queries is safe.
If you're on Windows, I have noticed that using localhost instead of 127.0.0.1 slows down the connection to the database.
Before giving any solution I wanted to share the reason why your query is too slow. If you see your code carefully you are opening database connection for reach record in the first query result. The major cost is going in opening and running the query against the database . SO imagine if you have 1000 records in the first result , you will open the database connection 1000 times , that will make it very very slow. Use the Inner join or subquery as stated by Trung . Or if not possible using stored procedure and putting the logic inside the SP will also help you to gain the performance.Create a Stored procedure and pass the input parameter as comma separated Ids what you are getting from another database , Inside the SP use the comma separated Ids to either loop and do the select statement. You can use the temp table to do so. The advantage is that you will open the Database connection only once.
I am joining two tables: customers and queries.
I am getting the full_name from the customers table and the description from the queries table.
I am wondering if it is possible to have the results of an SQL join split into arrays that correspond with the table the data came from? For example:
$STH = $DBH->prepare("SELECT queries.description, customers.full_name FROM queries INNER JOIN customers ON queries.customer_id = customers.id");
$STH->execute();
$queries = $STH->fetchAll();
At the moment, I can access my data like this: $queries[0]['description'] and $queries[0]['full_name']
However, my question is whether there is an easy way to get the data like so: $job[0]['query']['description'] and $job[0]['customer']['full_name'].
Just as teresko mentioned, I can't understand why you'd need that.
I can only imagine you want to see on the PHP code what are the table that contained the information.
Maybe you could do something like SELECT queries.description as queries_description, then your php code would look like $queries['queries_description']. Would it be enough?
You can loop through the results in PHP and convert it to the data structure you want, but you cannot (as far as I know) automatically group the data into arrays based on the source table. A (somewhat messy) alternative, using SQL is to use a multi-query and create a temp table from your original results, then select the results on a per-table basis, like so:
CREATE TEMPORARY TABLE q AS
(SELECT queries.description, customers.full_name FROM queries
INNER JOIN customers
ON queries.customer_id = customers.id
);
SELECT q.description FROM q;
SELECT q.full_name FROM q;
So, in those SELECT statements, you'll have to list all the columns that you want for each result. Then in PHP, you'll have to iterate over each resultset and put the data into arrays (or objects/whatever) as needed. Errr. A fetchAll will still not get you what you want, but a fetchAll on the first non-empty resultset will get you all the rows from queries and the 2nd will get you all the rows from customers
If you had data in table 1 that you need to use to return data in table 2 for each row returned in table 1. What is more efficient to use a set of querys in PHP one inbedded in the while loop of the other or an SQL function within a query?
for example:
$qry = mysql_query(SELECT date FROM table1)
while($res = mysql_fetch_array($qry))
{
$qry = mysql_query("SELECT name FROM table2 WHERE date=$res['date']")
}
or to do this as a function that returns the Id from table1 within the query.
A (LEFT / RIGHT) JOIN?
Unless I've misunderstood the question...
I think you're looking for JOIN sql syntax. If you have 2 tables: messages and author and you want to return messages with authors. Then you can write following SQL statement:
SELECT m.body, a.name FROM message m
LEFT JOIN author a ON (a.id=m.author_id)
This will return message body with corresponding author name
Table author:
id - primary key
name - name of the author
Table message:
body - text of the message
author_id - id of the author
UPD1:
This will be faster then looping each message and select an author. Because JOIN allows you to return all data in single query (not N x queries when using for loop).
UPD2:
With your tables the query will look like:
SELECT t1.date, t2.name FROM table1 t1 LEFT JOIN table2 t2 ON (t2.date=t1.date)
It depends on if the data is easier to find during the while loop or in the query itself.
So if the DB has to base the sub-query on the result of each row in the main query, and there are 1000 total rows and 100 results in the main query, it has to check all of the rows 100 times, so that's 100,000 sub-queries it runs.
So think it terms of the number of results of the main query. If your while loop has to query the DB 100 times while the DB can do it much faster and efficiently in one query, that's better. But if you want a subset of answers that you can say 'query only based on the last set of results' the while loop is better.
What is more efficient to use
a set of querys in PHP one inbedded in the while loop of the other
or
an SQL function within a query
Seems you answered your question yourself, didn't you?
Every query you send to the dbms has to be sent over the network, parsed, analyzed then executed. You may want to minimize the number of queries sent to the db.
There may be exceptions, for example if the processing of the data requires operations which the dbms is not capable of.