I know you can execute 10 SQL queries inside a mysql_query() (or mysqli_...) but how is that different from executing 10 mysql_query()s with one SQL query in each one?
If they are different and the first solution was more efficient, how would I use mysql_fetch_assoc() function on one of the queries inside of it?
If the first solution allows me to limit the number of connections per page to 1 per mysql_query(), then I think I will have enough mysql connections to handle my traffic, but if it doesn't, what SQL technology (or other?) can I use that will allow me to connect to my database from PHP more efficiently (so I can handle more users)?
I am using Apache and PHP 5.4
You can only send a single query at a time ... see the docs
mysql_query() sends a unique query (multiple queries are not supported) to the currently active database on the server that's associated with the specified link_identifier.
You could use mysqli_mutli_query .. example from the docs :
$query = "SELECT CURRENT_USER();";
$query .= "SELECT Name FROM City ORDER BY ID LIMIT 20, 5";
/* execute multi query */
if ($mysqli->multi_query($query)) {
do {
/* store first result set */
if ($result = $mysqli->store_result()) {
while ($row = $result->fetch_row()) {
printf("%s\n", $row[0]);
}
$result->free();
}
/* print divider */
if ($mysqli->more_results()) {
printf("-----------------\n");
}
} while ($mysqli->next_result());
}
/* close connection */
$mysqli->close();
The difference between running 10 queries in one call to mysql_query() and running 10 mysql_query() would be that in the first case you can only get the result from the very last query. But as long as you don't do a new mysql_connect() it shouldn't reconnect between queries.
The alternative is to use mysqli_multi_query which will let you run several queries in parallel which will optimize it some, but not a whole lot.
From the sound of it you don't need a more effective way to connect to your database, you either need to optimize your queries, sort indices and the like, or maybe the database machine is simply incorrectly configured or under-dimensioned hardware wise. I'm guessing your actual problem is that your database questions are too slow?
Just running with mysql_query(), one query per call, mysql running on a separate machine from PHP and you can still do hundreds of MySQL queries per second without breaking a sweat.
Related
I'm trying to understand the benefits of using SQL's where clause over selecting through PHP (or vice-versa) What makes one more preferable than the other or does it simply not matter? This is regarding both performance, security and overall in general programming.
Example:
Why should I use:
$sql = SELECT * FROM database WHERE id='1'
over:
$sql = "SELECT * FROM database";
foreach ($db->query($sql) as $row) {
if ($row['id'] === $id) {
...
}
}
SELECT will perform the filtering in the database itself whereas to do the filtering in php the app has to transfer all the data over the network to php.
Imagine the CPU usage and network traffic if you need to filter out just one row from 1 million rows.
You are unnecessarily bringing all the data from DB Server to middletier and looping those data in middle tier, whereas in db querying it uses appropriate indexes and retrieve your required records faster
I have an SQL query which can return quite a lot results (something like 10k rows) but I cannot use the SQL LIMIT parameter, as I don't know the exact amount of needed rows (there's a special grouping done in PHP). So the plan was to stop fetching rows once I have enough.
Since PDO normally operates in buffered mode, which fetches the whole result set and passes it to PHP, I switched PDO to unbuffered mode with
$pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);
Now I expected that executing the query should take about the same time no matter what LIMIT I pass. So basically
$result = $pdo->query($query);
$count = 0;
while ($row = $result->fetch()) {
++$count;
if ($count > 10) break;
}
should execute in about the same time for
$query = 'SELECT * FROM myTable';
and
$query = 'SELECT * FROM myTable LIMIT 10';
However the first one takes 8 seconds whereas the second one executes instantly. So it seems like the unbuffered query also waits until the whole result set is fetched - which shouldn't be the case according to the documentation.
Is there any way to get the query result instantly in PHP with PDO and stop the query once I have enough results?
Database applications like "Sequel Pro SQL" can do this (I can hit cancel after 1 second and get the results that were already queried until that time) so it can't be a general problem with MySQL servers.
I can workaround the problem by choosing a very high LIMIT which always has enough valid results after my grouping. But since performance is an issue, I'd like to query only as many entries as really needed. Please don't suggest anything that involves grouping in MySQL, the terrible performance of that is the reason we have to change the behaviour.
Now I expected that executing the query should take about the same time no matter what LIMIT I pass. So basically
This might not be completely true. While you won't get the overhead of receiving all your results, they are all queried (without a limit)! You do get the advantage of keeping most of the results serverside until you need them, but your server actually does perform the whole query first as far as I know. I'm not sure how complicated your query is, but this could be the issue?
Say for instance you have a very slow join (not indexed), but only want the first 10 by id, your query will get 10 based on the index, and then only do the join for those 10. This'll be quick
But if you don't actually limit, but ask for the result in batches, your complete join will have to be done (slow!) and then your resultsset is released in parts.
A quicker method might be to repeat your limited query untill you have your result. I know, this will increase overhead, but it might be way quicker. Only way to know is to test.
as response to your comment: this is from the manual
Unbuffered MySQL queries execute the query and then return a resource while the data is still waiting on the MySQL server for being fetched.
So it executes the query. The complete query. So as I tried to explain above, it will not be as quick as the same query with a LIMIT 10, as it doesn't perform a partial query! The fact that a different DB engine does this does not mean MySQL can...
Have you tried using prepare/execute instead of query, and putting a $stmt->closeCursor(); call after the break?
$stmt = $dbh->prepare($query);
$stmt->execute();
$count = 0;
while ($row = $stmt->fetch()) {
++$count;
if ($count > 10) break;
}
$stmt->closeCursor();
The is one php file that governs rotating ads that is causing serious server performance issues and causing "too many connections" sql errors on the site. Here is the php script. Can anyone give me some insight into how to correct this as I am an novice at php.
<?
require("../../admin/lib/config.php");
// connect to database
mysql_pconnect(DB_HOST,DB_USER,DB_PASS);
mysql_select_db(DB_NAME);
$i = 1;
function grab()
{
$getBanner = mysql_query("SELECT * FROM sponsor WHERE active='Y' AND ID != 999 AND bannerRotation = '0' ORDER BY RAND() LIMIT 1");
$banner = mysql_fetch_array($getBanner);
if ($banner['ID'] == ''){
mysql_query("UPDATE sponsor SET bannerRotation = '0'");
}
if (file_exists(AD_PATH . $banner['ID'] . ".jpg")){
$hasAd = 1;
}
if (file_exists(BANNER_PATH . $banner['ID'] . ".jpg")){
return "$banner[ID],$hasAd";
} else {
return 0;
}
}
while ($i <= 3){
$banner = grab();
if ($banner != 0){
$banner = explode(",",$banner);
mysql_query("UPDATE sponsor SET bannerView = bannerView + 1 WHERE ID='$banner[0]'");
mysql_query("UPDATE sponsor SET bannerRotation = '1' WHERE ID = '$banner[0]'");
echo "banner$i=$banner[0]&hasAd$i=$banner[1]&";
$i++;
}
}
?>
I see not mysqli
The problem is that mysql_pconnect() opens a persistent connection to the database and is not closed at end of execution, and as you are not calling mysql_close() anywhere the connection never gets closed.
Its all in the manual: http://php.net/manual/en/function.mysql-pconnect.php
Well, the good news for your client is that the previous developer abandoned the project. He could only have done more damage if he had continued working on it.
This script is using ext/mysql, not ext/mysqli. It would be better to use mysqli or PDO_mysql, since ext/mysql is deprecated.
It's recommended to use the full PHP open tag syntax (<?php), not the short-tags syntax (<?). The reason is that not every PHP environment enables the use of short tags, and if you deploy code into such an environment, your code will be viewable by anyone browsing to the page.
This script does no error checking. You should always check for errors after attempting to connect to a database or submitting a query.
The method of using ORDER BY RAND() LIMIT 1 to choose a random row from a database is well known to be inefficient, and it cannot be optimized. As the table grows to have more than a trivial number of rows, this query is likely to be your bottleneck. See some of my past answers about optimizing ORDER BY RAND queries, or a great blog by Jan Kneschke on selecting random rows.
Even if you are stuck using ORDER BY RAND(), there's no need to call it three times to get three distinct random sponsors. Just use ORDER BY RAND() LIMIT 3. Then you don't need the complex and error-prone update against bannerRotation to ensure that you get sponsors that haven't been chosen before.
Using SELECT * fetches all the columns, even though they aren't needed for this function.
If a sponsor isn't eligible for random selection, i.e. if it has active!='Y' or if its ID=999, then I would move it to a different table. This will simplify your queries, and make the table of sponsors smaller and quicker to query.
The UPDATE in the grab() function has no WHERE clause, so it applies to all rows in the sponsor table. I don't believe this is intentional. I assume it should apply only to the single row WHERE ID=$banner['ID'].
This code has two consecutive UPDATE statements against the same row of the same table. Combine these into a single UPDATE statement that modifies two columns.
The grab() function appends values together separated by commas, and then explodes that string into an array as soon as it returns. As if the programmer doesn't know that a function can return an array.
Putting the $i++ inside a conditional block makes it possible for this code to run in an infinite loop. That means this script can run forever. Once a few dozen of these are running concurrently, you'll run out of connections.
This code uses no caching. Any script that serves ad banners must be quick, and doing multiple updates to the database is not going to be quick enough. You need to use some in-memory caching for reads and writes. A popular choice is memcached.
Why is this client coding their own ad-banner server so inexpertly? Just use Google DFP Small Business.
Yikes!
grab() is being called from within a loop, but is not parameterized. Nor does there seem to be any rationale for repeatedly calling it.
A 200% speedup is easily realizable.
I was informed by someone senior in our company today that the PHP code I have written for performing prepared statements on a MySQL database is "inefficient" and "too taxing on our server". Since then I find myself in the difficult position of trying to understand what he meant and then to fix it. I have no contact to said person for four days now so I am asking other developers what they think of my code and if there are any areas that might be causing bottlenecks or issues with server performance.
My code works and returns the results of my query in the variable $data, so technically it works. There is another question though as to whether it is efficient and written well. Any advice as to what that senior employee meant or was referring to? Here is the method I use to connect and query our databases.
(Please note, when I use the word method I do not mean a method inside a class. What I mean to say is this how I write/structure my code when I connect and query our databases.)
<?php
// Create database object and connect to database
$mysqli=new mysqli();
$mysqli->real_connect($hostname, $username, $password, $database);
// Create statement object
$stmt=$mysqli->stmt_init();
// Prepare the query and bind params
$stmt->prepare('SELECT `col` FROM `table` WHERE `col` > ?');
$stmt->bind_param('i', $var1);
// Execute the query
$stmt->execute();
// Store result
$stmt->store_result();
// Prepare for fetching result
$rslt=array();
$stmt->bind_result($rslt['col']);
// Fetch result and save to array
$data=array();
while($stmt->fetch()){
foreach($rslt as $key=>$value){
$row[$key]=$value;
}
$data[]=$row;
}
// Free result
$stmt->free_result();
// Close connections
$stmt->close();
$mysqli->close();
?>
Any advice or suggestions are useful, please do contribute and help out even if you are only guessing. Thanks in advance :)
There are two types of code that may be inefficient, the PHP code and the SQL code, or both.
For example, the SQL is a problem if the `col` column isn't indexed in the database. This puts lots of load on the database because the database has to scan very many rows to answer queries. If `col` isn't indexed in the given query, then all of the rows in the table would be scanned. Also, if the value passed in isn't very selective, then many rows will have to be examined, perhaps all of the rows, as MySQL will choose a table scan over an index scan when many rows will be examined. You will need to become familiar with the MySQL EXPLAIN plan feature to fix your queries, or add indexes to the database to support your queries.
The PHP would be a problem if you followed something like the pattern:
select invoice_id from invoices where customer_id = ?
for each invoice_id
select * from line_items where invoice_id = ?
That kind of pattern will lead to "over querying" the database, which puts extra load on it. Instead use a join:
select li.* from invoices i join line_items li using (invoice_id)
Ask your database administrator to turn on the slow query log and then process it with pt-query-digest
You can use pt-query-digest to report on queries that are expensive (take a long time to execute) and also to use it to report by frequency to detect over querying.
Currently I use in my projects ADODB library for integration with the database.
I want to migrate to PDO, but I have a question about the consultations.
Currently, with the ADODB I do a query and use the row with the set numerous times using the method MoveFirst().
Example:
//I consultation
$rs = $conn->execute('select * from mytable');
//Loop through the results
while(!$rs->EOF) {
echo $rs->fields('name');
$rs->MoveNext();
}
//I move the "pointer" to the beginning of the list
$rs->MoveFirst();
//I can go over the results without needing to re-select
while(!$rs->EOF) {
echo $rs->fields('name');
$rs->MoveNext();
}
I wonder if there is any way similar in PDO, so I do not need to run the query again.
The goal is to avoid unnecessary queries on the bench more often since they use the same query.
I'm not sure why you want to loop through the result set more than once over the database connection. Why pull the data over the network again when you can just retrieve and save it the first time? But what you are looking for is a scrollable cursor, which isn't supported by mysql. At least not through any PHP/MySQL driver. You may also want to look into buffered/unbuffered queries, which is supported by PDO.
PDO::CURSOR_SCROLL
http://www.php.net/manual/en/pdo.prepare.php