I am trying to display the data from 'table' if a key inputted by the user is found in the database. Currently I have it set up so that the database checks if the key exists, like so:
//Select all from table if a key entry that matches the user specified key exists
$sql = 'SELECT * FROM `table` WHERE EXISTS(SELECT * FROM `keys` WHERE `key` = :key)';
//Prepare the SQL query
$query = $db->prepare($sql);
//Substitute the :key placeholder for the $key variable specified by the user
$query->execute(array(':key' => $key));
//While fetched data from the query exists. While $r is true
while($r = $query->fetch(PDO::FETCH_ASSOC)) {
//Debug: Display the data
echo $r['data'] . '<br>';
}
These aren't the only SQL statements in the program that are required. Later, an INSERT query along with possibly another SELECT query need to be made.
Now, to my understanding, using WHERE EXISTS isn't always efficient. However, would it be more efficient to split the query into two separate statements and just have PHP check if any rows are returned when looking for a matching key?
I took a look at a similar question, however it compares multiple statements on a much larger scale, as opposed to a single statement vs a single condition.
#MarkBaker Join doesn't have to be faster than exists statement. Query optymalizer is able to rewrite the query live if it sees better way to accomplish query. Exists statement is more readable than join.
Fetching all the data and making filtering directly in PHP is always bad idea. What if your table grow up to milions of records? MySQL is going to find the best execute plan for you. It will automaticaly cache the query if it is going to improve performance.
In other words, your made everything correctly as far as we can see your code now. For futher analyse show us all of your queries.
Related
Okay, basically I have a table that contains statements like:
incident.client_category = 1
incident.client_category = 8
incident.severity = 1
etc.
I would like to use the contents from this table to generate other tables that fulfill the conditions expressed in this one. So I would need to make it something like
SELECT * FROM incident WHERE incident.client_category = 1
But the last part of the where has to come from the first table. Right now what I'm trying to do is something like
SELECT * FROM incident WHERE (SELECT condition FROM condition WHERE id = 1)
id = 1 stands for the condition's id. Right now I only want to work with ONE condition for testing purposes. Is there a way to achieve this? Because if there isn't, I might have to just parse the first query's results through PHP into my incident query.
Table schemas:
Engineering Suggestion - Normalize the DB
Storing a WHERE clause, like id = 10, in a field in a MySQL table, is not a good idea. I recommend taking a look at MySQL Normalization. You shouldn't store id = 10 as a varchar, but rather, you should store something like OtherTableid. This allows you to use indices, to optimize your DB, and to get a ton of other features that you are deprived of by using fields as WHERE clauses.
But sometimes we need a solution asap, and we can't re-engineer everything! So let's take a look at making one...
Solution
Here is a solution that will work even on very old, v. 5.0 versions of MySQL. Set the variable using SET, prepare a statement using PREPARE, and execute it using EXECUTE. Let's set our query into a variable...
SET #query = CONCAT(
"SELECT * FROM incident WHERE ",
(SELECT condition FROM condition WHERE id = 1)
);
I know for a fact that this should work, because the following definitely works for me on my system (which doesn't require building any new tables or schema changes)...
SET #query = CONCAT("SELECT id FROM myTable WHERE id = ", (SELECT MAX(id) FROM myTable));
If I SELECT #query;, I get: SELECT id FROM myTable WHERE id = 1737901. Now, all we need to do is run this query!
PREPARE stmt1 FROM #query;
EXECUTE stmt1;
DEALLOCATE PREPARE stmt1;
Here we use a prepare to build the query, execute to execute it, and deallocate to be ready for the next prepared statement. On my own example above, which can be tested by anyone without DB schema changes, I got good, positive results: EXECUTE stmt1; gives me...
| id | 1737901 | .
here is one way to achieve your goal by using what is called dynamic sql, be ware that this works only select from condition table returns only one record.
declare #SQLSTRING varchar(4000)
, #condition VARCHAR(500) -- change the size to whatever condition column size is
SELECT #condition = condition
FROM
condition
WHERE
id = 1
SET #SQLSTRING= 'SELECT * FROM incident WHERE ' + #condition
exec sp_executesql(#SQLSTRING)
Since you have also tagged the question with PHP, I would suggest using that. Simply select the string from the condition table and use the result to build up a SQL query (as a string in PHP) including it. Then run the second query. Psudo-code (skipping over what library/framework you re using to call the db):
$query = "select condition from condition where id = :id";
$condition = callDbAndReturnString($query, $id);
$query = "select * from incident where " . $condition;
$result = callDb($query);
However, be very careful. Where and how are you populating the possible values in the condition table? Even how is your user choosing which one to use? You run the risk of opening yourself up to a secondary SQL injection attack if you allow the user to generate values and store them there. Since you are using the value from the condition table as a string, you cannot parametrise the query using it as you (hopefully!) normally would. Depending on the queries you run and the possible values there as conditions, there might also be risk even if you just let them pick from a pre-built list. I would seriously ask myself if this (saving parts of SQL queries as strings in another table) is the best approach. But, if you decide it is, this should work.
I need some suggestions and ideas.
Here's the scenario. The server receives a bunch of IDs from client via Ajax. Some of these IDs may already exist in database some may not. I need to save those that are not.
One way would be to set sql queries to select * whose ID is what I have. But this requires to a select statement for each id. Each time I receive something about 300 IDs which means 300 sql queries. This I think would slow the server. So what do you think is a better way to do this? Is there a way to extract the non-existing IDs with one SQL query?
P.S. The server is running on CakePHP.
I think what you need is SQL's IN keyword:
SELECT id FROM table WHERE id IN (?)
Where you would insert your IDs separated by comma, e.g.
$id_str = implode(',', $ids);
Make sure that $ids is an array of integers to prevent SQL injection
The outcome is a MySQL result containing all ids that exist. Build them into an array and use PHP's array_diff to get all IDs that do not exist. Full code:
$result = $connection->query('SELECT id FROM table WHERE id IN ('.
implode(',', $ids) . ')');
while($row = $result->fetch_row()) {
$existent[] = $row[0];
}
$not_existent = array_diff($ids, $existent);
If I understand you correctly, an insert ignore could do the trick
INSERT IGNORE INTO `table` (`id`,`col`,`col2`) VALUES ('id','val1','val2');
then any duplicate id's will be silently dropped, so long as id is unique or primary.
Also the keyword IN can be useful for finding rows with a value in a set. Eg
SELECT * FROM `table` WHERE `id` IN (2,4,6,7)
I often run into the situation where I want to determine if a value is in a table. Queries often happen often in a short time period and with similar values being searched therefore I want to do this the most efficient way. What I have now is
if($statment = mysqli_prepare($link, 'SELECT name FROM inventory WHERE name = ? LIMIT 1'))//name and inventory are arbitrarily chosen for this example
{
mysqli_stmt_bind_param($statement, 's', $_POST['check']);
mysqli_stmt_execute($statement);
mysqli_stmt_bind_result($statement, $result);
mysqli_stmt_store_result($statement);//needed for mysqli_stmt_num_rows
mysqli_stmt_fetch($statement);
}
if(mysqli_stmt_num_rows($statement) == 0)
//value in table
else
//value not in table
Is it necessary to call all the mysqli_stmt_* functions? As discussed in this question for mysqli_stmt_num_rows() to work the entire result set must be downloaded from the database server. I'm worried this is a waste and takes too long as I know there is 1 or 0 rows. Would it be more efficient to use the SQL count() function and not bother with the mysqli_stmt_store_result()? Any other ideas?
I noticed the prepared statement manual says "A prepared statement or a parametrized statement is used to execute the same statement repeatedly with high efficiency". What is highly efficient about it and what does it mean same statement? For example if two separate prepared statements evaluated to be the same would it still be more efficient?
By the way I'm using MySQL but didn't want to add the tag as a solution may be non-MySQL specific.
if($statment = mysqli_prepare($link, 'SELECT name FROM inventory WHERE name = ? LIMIT 1'))//name and inventory are arbitrarily chosen for this example
{
mysqli_stmt_bind_param($statement, 's', $_POST['check']);
mysqli_stmt_execute($statement);
mysqli_stmt_store_result($statement);
}
if(mysqli_stmt_num_rows($statement) == 0)
//value not in table
else
//value in table
I believe this would be sufficient. Note that I switched //value not in table
and //value in table.
It really depends of field type you are searching for. Make sure you have an index on that field and that index fits in memory. If it does, SELECT COUNT(*) FROM <your_table> WHERE <cond_which_use_index> LIMIT 1. The important part is LIMIT 1 which prevent for unnecessary lookup. You can run EXPLAIN SELECT ... to see which indexes used and probably make a hint or ban some of them, it's up to you. COUNT(*) works damn fast, it is optimized by design return result very quickly (MyISAM only, for InnoDB the whole stuff is a bit different due to ACID). The main difference between COUNT(*) and SELECT <some_field(s)> is that count doesn't perform any data reading and with (*) it doesn't care about whether some field is a NULL or not, just count rows by most suitable index (chosen internally). Actually I can suggest that even for InnoDB it's a fastest technique.
Also use case matters. If you want insert unique value make constrain on that field and use INSERT IGNORE, if you want to delete value which may not be in table run DELETE IGNORE and same for UPDATE IGNORE.
Query analyzer define by itself whether two queries are the same on or not and manage queries cache, you don't have to worry about it.
The different between prepared and regular query is that the first one contains rule and data separately, so analyzer can define which data is dynamic and better handle that, optimize and so. It can do the same for regular query but for prepared we say that we will reuse it later and give a hint which data is variable and which is fixed. I'm not very good in MySQL internal so you can ask such questions on more specific sites to understand details in a nutshell.
P.S.: Prepared statements in MySQL are session global, so after session they are defined in ends they are deallocated. Exact behavior and possible internal MySQL caching is a subject of additional investigation.
This is the kind of things in-memory caches are really good at. Something like this should work better than most microoptimization attempts (pseudocode!):
function check_if_value_is_in_table($value) {
if ($cache->contains_key($value)) {
return $cache->get($value);
}
// run the SQL query here, put result in $result
// note: I'd benchmark if using mysqli_prepare actually helps
// performance-wise
$cache->put($value, $result);
return $result;
}
Have a look at memcache or the various alternatives.
I was recently trying to do a project*, which caused me to ask this question. Although since then I've found an alternative solution, I am still curious if what I envisioned doing is, in any way, possible.
Essentially, I am wondering if there is anyway to perform a MySQL query on a MySQL query result in php. For example:
$result = mysql_query("SELECT * FROM foo WHERE bar=".$barValue);
AND THEN, be able to perform multiple queries on $result:
$newResult = mysql_query("SELECT * FROM $result WHERE otherBar=".$barValue);
OR
$otherNewResult = mysql_query("SELECT * FROM $result WHERE otherOtherBar=".$barValue." ORDER BY foobar ASC");
AND so on and so forth...
I realize that I could append the original query with my new WHERE statements and ORDER BYs, but that causes my to query the database unnecessarily and it prevents me from writing more objected oriented code (because I can't pass around a result to be queried, but rather have to requery the database in every function...)
Any advice, pieces of code, frameworks, or ramblings appreciated.
*BTW, my project was having to query a large database of people for people born in certain age groups and then query those age groups for different demographics.
Edit
No, writing a custom function to query the database is not worth the object-orientation (and modifiability) it would give me
You could do a nested query in the same SQL query and keep PHP out of it:
'SELECT * FROM (SELECT * FROM foo WHERE bar="something") AS q1 WHERE q1.bar2 = "something else"'
The question has already been answered. However following explanation will help someone who might be interested in knowing the details of it.
What are Nested query / subquery:
Subqueries are also known as nested queries. A subquery is a SELECT statement within another statement. MySQL supports all SQL standards and additionally provides MySQL specific features.
Why should I use Subquery:
Subquery is structured and it is possible to isolate each parts of statement
Subquery is more readable that complex joins and unions
Subquery provides alternative means to perform action which otherwise would require complex joins and unions
What Subquery returns:
A subquery can return a single value, a single row, a single column, or a table. These are called scalar, column, row, and table subqueries.
Reference: http://dev.mysql.com/doc/refman/5.7/en/subqueries.html
http://www.w3resource.com/sql/subqueries/nested-subqueries.php
Instead of hard coding sql queries like Select * from users where user_id =220202 can these be made dynamic like Select * from $users where $user_id = $input.
Reason i ask is when changes are needed to table/column names i can just update it in one place and don't have to ask developers to go line by line to find all references to update. It is very time consuming. And I do not like the idea of exposing database stuff in the code.
My major concern is load time. Like with dynamic pages, the database has to fetch the page content, same way if queries are dynamic first system has to lookup the references then execute the queries, so does it impact load times?
I am using codeignitor PHP.
If it is possible then the next question is where to store all the references? In the app, in a file, in the DB, and how?
---EDIT:
Even better: Can the SQL query itself be made dynamic? I can just reference $sqlA instead of the whole query? This way if I have to re-write the query I can just update 1 file.
Because you are using Codeigniter, I would reccomend utilizing the Active Record Class to accomplish what you are trying to do.
The active record class enables you to build queries dynamically in steps allowing you to build them logically. So to take your example using active record...
( this could be accomplished with less code, I'm just trying to illustrate Active Record )
$this->db->select('*');
$this->db->from($table);
$this->db->where($user_id, $input);
and so to show what I mean about building the query logically, you can build whatever logic you want INTO the query building process. Lets say you have a $limit variable that you set if you want to limit the number of results you get. BUT if it isn't set (or NULL) you don't want to set the limit clause.
if ( $isset($limit) ) {
$this->db->limit($limit);
}
and now to execute your query now that it has been built
$query = $this->db->get();
Then just deal with $query with your database class just like you would any other CodeIgniter query object.
Of course you can, if that's what you wish. I'd rather recommend you taking more time to design you database but changes in the schema are inevitable in the long run.
I don't think load time would be an issue with this because ussually the bottleneck in this applications is in the database.
Finally my recommendation is to save this in a file just by declaring the column names as php variables
It depends on the database driver(s) you are using. The old PHP database drivers did not support placeholders (PHP 3.x). The modern (PDO) ones do. You write the SQL with question marks:
SELECT * FROM Users WHERE User_ID = ?
You then provide the value of the user ID when you execute the query.
However, you cannot provide the column name like this - only values. But you could prepare a statement from a string such as:
SELECT * FROM Users WHERE $user_id = ?
Then you provide the value at execute time.
mysql_query() takes a string and it doesn't need to be a constant string, it can be a variable.
$SQL = "SELECT foo FROM bar b";
SQLSet = mysql_query($SQL);
Aa you can see, you can use ordinary string manipulation to build your whole SQL query.
$SQL="SELECT * FROM MyTable";
$BuzID = 5;
$Filter = "Buz=".$BuzID;
if (is_numeric($BuzID)) SQL .= " WHERE ".$Filter;
SQLSet = mysql_query($SQL);
This will expand to "SELECT * FROM MyTable WHERE Buz=5" if $BuzID is set to any number.
If not the statement will just be "SELECT * FROM MyTable"
As you can see, you can build very complex SQL statements on the fly without need of variable support in the SQL server.
IF you want constants such as database name, user login, you can but them in a separate include located outside the public directory.
SecretStuff.inc.php
$__DatabaseName = "localhost";
$__UserName = "DatabaseAccess";
$__Password = "E19A4F72B4AA091C6D2";
Or have the whole PHP database connection code in the same file.