I'm trying to understand the benefits of using SQL's where clause over selecting through PHP (or vice-versa) What makes one more preferable than the other or does it simply not matter? This is regarding both performance, security and overall in general programming.
Example:
Why should I use:
$sql = SELECT * FROM database WHERE id='1'
over:
$sql = "SELECT * FROM database";
foreach ($db->query($sql) as $row) {
if ($row['id'] === $id) {
...
}
}
SELECT will perform the filtering in the database itself whereas to do the filtering in php the app has to transfer all the data over the network to php.
Imagine the CPU usage and network traffic if you need to filter out just one row from 1 million rows.
You are unnecessarily bringing all the data from DB Server to middletier and looping those data in middle tier, whereas in db querying it uses appropriate indexes and retrieve your required records faster
Related
Is it possible to SELECT specific rows and DELETE the selected result in ONE request?
$res = $Connection->query("SELECT * FROM tasks");
if($res->num_rows > 0){
while($row = $res->fetch_assoc()){ ...
The problem is that I have limited Number of queries to my SQL database and I want to minimize it as mush as possible.
You can't do that using the regular mysql-api in PHP. Just execute two queries. The second one will be so fast that it won't matter. This is a typical example of micro optimization. Don't worry about it. (So timing doesn't matter much)
For the record, since you are worried about the number of queries, it can be done using mysqli and the mysqli_multi_query-function.
P.S. - I haven't tried this, but since this mysql_multi_query is there in the documentation, it might help... :)
Both SELECT and DELETE need one request individually. And there is no SQL can do this work in the official documents.
I am trying to display the data from 'table' if a key inputted by the user is found in the database. Currently I have it set up so that the database checks if the key exists, like so:
//Select all from table if a key entry that matches the user specified key exists
$sql = 'SELECT * FROM `table` WHERE EXISTS(SELECT * FROM `keys` WHERE `key` = :key)';
//Prepare the SQL query
$query = $db->prepare($sql);
//Substitute the :key placeholder for the $key variable specified by the user
$query->execute(array(':key' => $key));
//While fetched data from the query exists. While $r is true
while($r = $query->fetch(PDO::FETCH_ASSOC)) {
//Debug: Display the data
echo $r['data'] . '<br>';
}
These aren't the only SQL statements in the program that are required. Later, an INSERT query along with possibly another SELECT query need to be made.
Now, to my understanding, using WHERE EXISTS isn't always efficient. However, would it be more efficient to split the query into two separate statements and just have PHP check if any rows are returned when looking for a matching key?
I took a look at a similar question, however it compares multiple statements on a much larger scale, as opposed to a single statement vs a single condition.
#MarkBaker Join doesn't have to be faster than exists statement. Query optymalizer is able to rewrite the query live if it sees better way to accomplish query. Exists statement is more readable than join.
Fetching all the data and making filtering directly in PHP is always bad idea. What if your table grow up to milions of records? MySQL is going to find the best execute plan for you. It will automaticaly cache the query if it is going to improve performance.
In other words, your made everything correctly as far as we can see your code now. For futher analyse show us all of your queries.
This is an optimisation question RE: 1st principles.. Imagine I am doing a big heavy lifting comparison.. 30k files vs 30k database entries.. is it most process efficient to do one big MySQL into an array then loop through physical files checking vs the array or is it better to loop through the files and then one at a time do one line MySQL calls..
Here is some pseudo code to help explain:
//is this faster?
foreach($recursiveFileList as $fullpath){
$Record = $db->queryrow("SELECT * FROM files WHERE fullpath='".$fullpath."'");
//do some $Record logic
}
//or is this faster
$BigList = array();
$db->query("SELECT * FROM files");
while($Record = $db->rows()){
$BigList[$Record['fullpath']] = $Record;
}
foreach($recursiveFileList as $fullpath){
if (isset($BigList[$fullpath])){
$Record = $BigList[$fullpath];
//do some $Record logic
}
}
Update: if you always know that your $recursiveFileList is 100% of the table, then doing one query per row would be needless overhead. In that case, just use SELECT * FROM files.
I wouldn't use either of the two styles you show.
The first style runs one separate SQL query for each individual fullpath. This causes some overhead of SQL parsing, optimization, etc. Keep in mind that MySQL does not have the capability of remembering the query optimization from one invocation of a similar query to the next; it analysis and performs query optimization every time. The overhead is relatively small, but it adds up.
The second style shows fetching all rows from the table, and sorting it out in the application layer. This has a lot of overhead, because typically your $recursiveFileList might match only 1% or 0.1% or an even smaller portion of the rows in the table. I have seen cases where transferring excessive amounts of data over the network literally exhausted a 1Gbps network switch, and this put a ceiling on the requests per second for the application.
Use query conditions and indexes wisely to let the RDBMS examine and return only the matching rows.
The two styles you show are not the only options. What I would suggest is to use a range query to match multiple file fullpath values in a single query.
$sql = "SELECT * FROM files WHERE fullpath IN ("
. array_fill(0, count($recursiveFileList), "?") . ")";
$stmt = $pdo->prepare($sql);
$stmt->execute($recursiveFileList);
while ($row = $stmt->fetch()) {
//do some $Record logic
}
Note I also use a prepared query with ? parameter placeholders, and then pass the array of fullpath values separately when I call execute(). PDO is nice for this, because you can just pass an array, and the array elements get matched up to the parameter placeholders.
This also solves the risk of SQL injection in this case.
The is one php file that governs rotating ads that is causing serious server performance issues and causing "too many connections" sql errors on the site. Here is the php script. Can anyone give me some insight into how to correct this as I am an novice at php.
<?
require("../../admin/lib/config.php");
// connect to database
mysql_pconnect(DB_HOST,DB_USER,DB_PASS);
mysql_select_db(DB_NAME);
$i = 1;
function grab()
{
$getBanner = mysql_query("SELECT * FROM sponsor WHERE active='Y' AND ID != 999 AND bannerRotation = '0' ORDER BY RAND() LIMIT 1");
$banner = mysql_fetch_array($getBanner);
if ($banner['ID'] == ''){
mysql_query("UPDATE sponsor SET bannerRotation = '0'");
}
if (file_exists(AD_PATH . $banner['ID'] . ".jpg")){
$hasAd = 1;
}
if (file_exists(BANNER_PATH . $banner['ID'] . ".jpg")){
return "$banner[ID],$hasAd";
} else {
return 0;
}
}
while ($i <= 3){
$banner = grab();
if ($banner != 0){
$banner = explode(",",$banner);
mysql_query("UPDATE sponsor SET bannerView = bannerView + 1 WHERE ID='$banner[0]'");
mysql_query("UPDATE sponsor SET bannerRotation = '1' WHERE ID = '$banner[0]'");
echo "banner$i=$banner[0]&hasAd$i=$banner[1]&";
$i++;
}
}
?>
I see not mysqli
The problem is that mysql_pconnect() opens a persistent connection to the database and is not closed at end of execution, and as you are not calling mysql_close() anywhere the connection never gets closed.
Its all in the manual: http://php.net/manual/en/function.mysql-pconnect.php
Well, the good news for your client is that the previous developer abandoned the project. He could only have done more damage if he had continued working on it.
This script is using ext/mysql, not ext/mysqli. It would be better to use mysqli or PDO_mysql, since ext/mysql is deprecated.
It's recommended to use the full PHP open tag syntax (<?php), not the short-tags syntax (<?). The reason is that not every PHP environment enables the use of short tags, and if you deploy code into such an environment, your code will be viewable by anyone browsing to the page.
This script does no error checking. You should always check for errors after attempting to connect to a database or submitting a query.
The method of using ORDER BY RAND() LIMIT 1 to choose a random row from a database is well known to be inefficient, and it cannot be optimized. As the table grows to have more than a trivial number of rows, this query is likely to be your bottleneck. See some of my past answers about optimizing ORDER BY RAND queries, or a great blog by Jan Kneschke on selecting random rows.
Even if you are stuck using ORDER BY RAND(), there's no need to call it three times to get three distinct random sponsors. Just use ORDER BY RAND() LIMIT 3. Then you don't need the complex and error-prone update against bannerRotation to ensure that you get sponsors that haven't been chosen before.
Using SELECT * fetches all the columns, even though they aren't needed for this function.
If a sponsor isn't eligible for random selection, i.e. if it has active!='Y' or if its ID=999, then I would move it to a different table. This will simplify your queries, and make the table of sponsors smaller and quicker to query.
The UPDATE in the grab() function has no WHERE clause, so it applies to all rows in the sponsor table. I don't believe this is intentional. I assume it should apply only to the single row WHERE ID=$banner['ID'].
This code has two consecutive UPDATE statements against the same row of the same table. Combine these into a single UPDATE statement that modifies two columns.
The grab() function appends values together separated by commas, and then explodes that string into an array as soon as it returns. As if the programmer doesn't know that a function can return an array.
Putting the $i++ inside a conditional block makes it possible for this code to run in an infinite loop. That means this script can run forever. Once a few dozen of these are running concurrently, you'll run out of connections.
This code uses no caching. Any script that serves ad banners must be quick, and doing multiple updates to the database is not going to be quick enough. You need to use some in-memory caching for reads and writes. A popular choice is memcached.
Why is this client coding their own ad-banner server so inexpertly? Just use Google DFP Small Business.
Yikes!
grab() is being called from within a loop, but is not parameterized. Nor does there seem to be any rationale for repeatedly calling it.
A 200% speedup is easily realizable.
Instead of hard coding sql queries like Select * from users where user_id =220202 can these be made dynamic like Select * from $users where $user_id = $input.
Reason i ask is when changes are needed to table/column names i can just update it in one place and don't have to ask developers to go line by line to find all references to update. It is very time consuming. And I do not like the idea of exposing database stuff in the code.
My major concern is load time. Like with dynamic pages, the database has to fetch the page content, same way if queries are dynamic first system has to lookup the references then execute the queries, so does it impact load times?
I am using codeignitor PHP.
If it is possible then the next question is where to store all the references? In the app, in a file, in the DB, and how?
---EDIT:
Even better: Can the SQL query itself be made dynamic? I can just reference $sqlA instead of the whole query? This way if I have to re-write the query I can just update 1 file.
Because you are using Codeigniter, I would reccomend utilizing the Active Record Class to accomplish what you are trying to do.
The active record class enables you to build queries dynamically in steps allowing you to build them logically. So to take your example using active record...
( this could be accomplished with less code, I'm just trying to illustrate Active Record )
$this->db->select('*');
$this->db->from($table);
$this->db->where($user_id, $input);
and so to show what I mean about building the query logically, you can build whatever logic you want INTO the query building process. Lets say you have a $limit variable that you set if you want to limit the number of results you get. BUT if it isn't set (or NULL) you don't want to set the limit clause.
if ( $isset($limit) ) {
$this->db->limit($limit);
}
and now to execute your query now that it has been built
$query = $this->db->get();
Then just deal with $query with your database class just like you would any other CodeIgniter query object.
Of course you can, if that's what you wish. I'd rather recommend you taking more time to design you database but changes in the schema are inevitable in the long run.
I don't think load time would be an issue with this because ussually the bottleneck in this applications is in the database.
Finally my recommendation is to save this in a file just by declaring the column names as php variables
It depends on the database driver(s) you are using. The old PHP database drivers did not support placeholders (PHP 3.x). The modern (PDO) ones do. You write the SQL with question marks:
SELECT * FROM Users WHERE User_ID = ?
You then provide the value of the user ID when you execute the query.
However, you cannot provide the column name like this - only values. But you could prepare a statement from a string such as:
SELECT * FROM Users WHERE $user_id = ?
Then you provide the value at execute time.
mysql_query() takes a string and it doesn't need to be a constant string, it can be a variable.
$SQL = "SELECT foo FROM bar b";
SQLSet = mysql_query($SQL);
Aa you can see, you can use ordinary string manipulation to build your whole SQL query.
$SQL="SELECT * FROM MyTable";
$BuzID = 5;
$Filter = "Buz=".$BuzID;
if (is_numeric($BuzID)) SQL .= " WHERE ".$Filter;
SQLSet = mysql_query($SQL);
This will expand to "SELECT * FROM MyTable WHERE Buz=5" if $BuzID is set to any number.
If not the statement will just be "SELECT * FROM MyTable"
As you can see, you can build very complex SQL statements on the fly without need of variable support in the SQL server.
IF you want constants such as database name, user login, you can but them in a separate include located outside the public directory.
SecretStuff.inc.php
$__DatabaseName = "localhost";
$__UserName = "DatabaseAccess";
$__Password = "E19A4F72B4AA091C6D2";
Or have the whole PHP database connection code in the same file.