Related
I ran into the following question while writing a PHP script. I need to store the first two integers from an array of variable lenght into a database table, remove them and repeat this until the array is empty. I could do it with a while loop, but I read that you should avoid writing SQL statements inside a loop because of the performance hit.
A simpliefied example:
while(count($array) > 0){
if ($sql = $db_connect->prepare("INSERT INTO table (number1, number2) VALUES (?,?)")){
$sql->bind_param('ii',$array[0],$array[1]);
$sql->execute();
$sql->close();
}
array_shift($array);
array_shift($array);
}
Is this the best way, and if not, what's a better approach?
You can do something like this, which is way faster aswell:
Psuedo code:
$stack = array();
while(count($array) > 0){
array_push($stack, "(" . $array[0] . ", " . $array[1] . ")");
array_shift($array);
array_shift($array);
}
if ($sql = $db_connect->prepare("INSERT INTO table (number1, number2)
VALUES " . implode(',', $stack))){
$sql->execute();
$sql->close();
}
The only issue here is that it's not a "MySQL Safe" insert, you will need to fix that!
This will generate and Array that holds the values. Within 1 query it will insert all values at once, where you need less MySQL time.
Whether you run them one by one or in an array, an INSERT statement is not going to make a noticeable performance hit, from my experience.
The database connection is only opened once, so it is not a huge issue. I guess if you are doing some insane amount of queries, it could be.
I think as long as your loop condition is safe ( will break in time ) and you got something from it .. it's ok
You would be better off writing a bulk insert statement, less hits on mysql
$sql = "INSERT INTO table(number1, number2) VALUES";
$params = array();
foreach( $array as $item ) {
$sql .= "(?,?),\n";
$params[] = $item;
}
$sql = rtrim( $sql, ",\n" ) . ';';
$sql = $db_connect->prepare( $sql );
foreach( $params as $param ) {
$sql->bind_param( 'ii', $param[ 0 ], $param[ 1 ] );
}
$sql->execute();
$sql->close();
In ColdFusion you can put your loop inside the query instead of the other way around. I'm not a php programmer but my general belief is that most things that can be done in language a can also be done in language b. This code shows the concept. You should be able to figure out a php version.
<cfquery>
insert into mytable
(field1, field2)
select null, null
from SomeSmallTable
where 1=2
<cfloop from="1' to="#arrayLen(myArray)#" index="i">
select <cfqueryparam value="myArray[i][1]
, <cfqueryparam value="myArray[i][]
from SomeSmallTable
</cfloop>
</cfquery>
When I've looked at this approach myself, I've found it to be faster than query inside loop with oracle and sql server. I found it to be slower with redbrick.
There is a limitation with this approach. Sql server has a maximum number of parameters it will accept and a maximum query length. Other db engines might as well, I've just not discovered them yet.
I have successfully gotten queries to execute and print in PDO, but I'm doing something wrong here. The important part of the code for this question is in the last couple blocks of code; I'm including the first portion just for clarity.
This code connects to an HTML form with multiple input fields. The PHP constructs a query by appending the data from each field with ANDs in the WHERE statement.
This is what throws me: I echo the $query variable, and I can see that the query is formed properly, but when I then try to print the query results, no results are printed.
I wrestled with using prepared statements here, and decided to try getting the code to work first without them after failing to construct a prepared statement with varying numbers of parameters. I did try, with the help of this post: LIKE query using multiple keywords from search field using PDO prepared statement
So, setting aside prepared statements for the moment, can anyone tell me what I'm doing wrong here? Any help would be greatly appreciated.
<?php
if(isset($_POST['submit'])) {
// define the list of fields
$fields = array('titleSearch', 'keywordSearch', 'fullSearch', 'fromYear', 'toYear',
'fromSeconds', 'toSeconds', 'withSound', 'withColor');
$conditions = array();
// loop through the defined fields
foreach($fields as $field){
// if the field is set and not empty
if(isset($_POST[$field]) && $_POST[$field] != '') {
// create a new condition, using a prepared statement
$conditions[] = "$field LIKE CONCAT ('%', $_POST[$field], '%')";
}
}
// build the query
$query = "SELECT keyframeurl, videoid, title, creationyear, sound, color,
duration, genre FROM openvideo ";
// if there are conditions defined, append them to the query
if(count($conditions) > 0) {
$query .= "WHERE " . implode(' AND ', $conditions);
}
//confirm that query formed correctly
echo $query;
//print query results
foreach ($dbh->query($query) as $row){
print $row['videoid'].' - '.$row['title'].'<br />';
}
}
?>
Instead of posting your query you have to run it.
That's the only way to fix the problem
a Stack Overflow passer-by do not have a database server in their head to run your query.
a Stack Overflow passer-by do not have your particular database server in their head to run your query.
So, you are the only one who can run your query against your database and ask it what's going wrong.
Turn on error reporting. Make sure sure you can see errors occurred. Try to add intentional error and see if it works.
Double-check your database data if it really contains desired values.
Double-check your input data, if it really match database values.
Run your assembled query against database in console or phpadmin.
Dig to some certain problem. Do not just sit and wait. Asking a question "I have a code it doesnt work" makes very little sense. Code have to be run, not stared into.
$conditions[] = "$field LIKE CONCAT ('%', $_POST[$field], '%')";
is the culprit: sending "something" for the title ends up in something like
WHERE titleSearch LIKE CONCAT('%', something, '%')
but you want
WHERE titleSearch LIKE CONCAT('%', 'something', '%')
with more quotes.
Be sure not to roll this out into production though, as you might end up with somebody posting "xxx') OR 1=1; --" just for the perofrmance fun, or even worse, depedning on their mood.
You've forgotten quotes around the $_POST values that you're directly inserting into your queries:
$conditions[] = "$field LIKE CONCAT ('%', '$_POST[$field]', '%')";
^-- ^--
so while this will fix your immediate problem, you'll still be wide open to sql injection attacks.
You don't even need the CONCAT built-in function, you can model the whole string as $conditions[] = "{$field} LIKE '%{$_POST[$field]}%'". But you should use prepared statements if you don't want to face serious SQL injection attacks in the short-term future.
Why don't you try something like this? (using PDO as an example):
if ($pdo = new \PDO("mysql:host=localhost;dbname=testdb;charset=utf8", "user", "password")) {
$fields = ["titleSearch","keywordSearch","fullSearch","fromYear","toYear","fromSeconds","toSeconds","withSound","withColor"];
$parameters = array_map(function ($input) { return filter_var($input, FILTER_SANITIZE_STRING); }, $fields)
$conditions = array_map(function ($input) { return (!empty($_POST[$input]) ? "{$input} LIKE ?" : null); }, $fields);
$query = "SELECT `keyframeurl`,`videoid`,`title`,`creationyear`,`sound`,`color`,`duration`,`genre` FROM `openvideo`" . (sizeof($conditions) > 0 ? " " . implode(" AND ", $conditions) : null);
if ($statement = $pdo->prepare($query, [\PDO::ATTR_CURSOR => \PDO::CURSOR_FWDONLY])) {
if ($statement->execute((!empty($parameters) ? $parameters : null))) {
$result = $statement->fetchAll(\PDO::FETCH_ASSOC);
}
}
}
Haven't tested it (just coming to my mind right now), but it should set up PDO, prepare a statement based on the conditions you seem to look for, add the parameters in the execute() method (pre-filtered, although there's FAR better filtering techniques) and return all results associated with your query.
Even if you decide not to use this, give it a thought at least... it's a good starting point on PDO and, of course, get a nice tutorial on GET/POST variable filtering (or use a 3rd-party tool like HTML Purifier, for that matter).
Hope that helps ;)
I was wondering if you think this is possible:
Ok so I have a database storing usernames and I would like to echo the admins which are inside a file called admins.php IF they match the usernames inside the database so far I have got:
admins.php;
$admins = array("username","username2","username3");
and
$users="SELECT username from usrsys";
$query_users=mysql_query($users);
while loop here.
The while loop should hopefully echo the users which matches the admins.php file. I assume I should use something like (inarray()), but I am really not sure.
You should definitely use IN clause in your SQL to do this. Selecting everything from the table in order to determine in PHP if it contains the user names you're looking for makes no sense and is very wasteful. Can you imagine what would happen if you had a table of 1 million users and you needed to see if two of them were on that list? You would be asking your DBMS to return 1 million rows to PHP so that you can search through each of those names and then determine whether or not any of them are the ones you're looking for. You're asking your DBMS to do a lot of work (send over all the rows in the table), and you're also asking PHP to do a lot of work (store all those rows in memory and compute a match), unnecessarily.
There is a much more efficient and faster solution depending on what you want.
First, if you only need to know that all of those users exist in the table then use SELECT COUNT(username) instead and your database will return a single row with a value for how many rows were found in the table. That way you have an all or nothing approach (if that's what you're looking for). Either there were 3 rows found in the table and 3 elements in the array or there weren't. This also utilizes your table indexes (which you should have properly indexed) and means faster results.
$admins = array("username","username2","username3");
// Make sure you properly escape your data before you put in your SQL
$list = array_map('mysql_real_escape_string', $admins);
// You're going to need to quote the strings as well before they work in your SQL
foreach ($list as $k => $v) $list[$k] = "'$v'";
$list = implode(',', $list);
$users = "SELECT COUNT(username) FROM usrsys WHERE username IN($list)";
$query_users = mysql_query($users);
if (!$query_users) {
echo "Huston we have a problem! " . mysql_error(); // Basic error handling (DEBUG ONLY)
exit;
}
if (false === $result = mysql_fetch_row($query_users)) {
echo "Huston we have a problme! " . mysql_error(); // Basic error handling (DEBUG ONLY)
}
if ($result[0] == count($admins)) {
echo "All admins found! We have {$result[0]} admins in the table... Mission complete. Returning to base, over...";
}
If you actually do want all the data then remove the COUNT from the SQL and you will simply get all the rows for those users (if any are found).
$admins = array("username","username2","username3");
// Make sure you properly escape your data before you put in your SQL
$list = array_map('mysql_real_escape_string', $admins);
// You're going to need to quote the strings as well before they work in your SQL
foreach ($list as $k => $v) $list[$k] = "'$v'";
$list = implode(',', $list);
$users = "SELECT username FROM usrsys WHERE username IN($list)";
$query_users = mysql_query($users);
if (!$query_users) {
echo "Huston we have a problem! " . mysql_error(); // Basic error handling (DEBUG ONLY)
exit;
}
// Loop over the result set
while ($result = mysql_fetch_assoc($query_users)) {
echo "User name found: {$result['username']}\n";
}
However, I really urge you to reconsider using the old ext/mysql API to interface with your MySQL database in PHP since it is deprecated and has been discouraged from use for quite some time. I would really urge you to start using the new alternative APIs such as PDO or MySQLi and see the guide in the manual for help with choosing an API.
In PDO, for example this process would be quite simple with prepared statements and parameterized queries as you don't have to worry about all this escaping.
There's an example in the PDOStatement::Execute page (Example #5) that shows you just how to do use the IN clause that way with prepared statements... You can then reuse this statement in other places in your code and it offers a performance benefit as well as making it harder for you to inadvertently expose yourself to SQL injection vulnerabilities.
// Connect to your database
$pdo = new PDO("mysql:dbname=mydb;host=127.0.0.1", $username, $password);
// List of admins we want to find in the table
$admins = array("username","username2","username3");
// Create the place holders for your paratmers
$place_holders = implode(',', array_fill(0, count($admins), '?'));
// Create the prepared statement
$sth = $dbh->prepare("SELECT username FROM usrsys WHERE username IN ($place_holders)");
// Execute the statement
$sth->execute($admins);
// Iterate over the result set
foreach ($sth->fetchAll(PDO::FETCH_ASSOC) as $row) {
echo "We found the user name: {$row['username']}!\n";
}
Your PHP code even looks so much better with PDO :)
Just include admins.php file and use the next construction in your loop:
while ($row = mysql_fetch_array($users)) {
if (in_array($users[0], $admins))
echo $users[0];
}
Try this:
<?php
# include admins.php file that holds the admins array
include "admins.php";
# join all values in the admins array using "," as a separator (to use them in the sql statement)
$admins = join(",", $admins);
# execute the query
$result = mysql_query("
SELECT username
FROM usrsys
WHERE username IN ($admins)
");
if ($result) {
while ($row = mysql_fetch_array($result)) {
echo $row["username"] . "<br>";
}
}
?>
If your looking for syntax to pull in only the users from your $admins array then you could use something like:
$users="SELECT username FROM usrsys WHERE username IN ('".join("','",$admins)."')";
Where the php function JOIN will print username,username2,username3. Your resulting MySQL statement will look like:
SELECT username FROM usrsys WHERE username IN ('username','username2','username3')
Alternatively, if your looking to iterate through your $query_vars array and separate your admins from non-admins then you could use something like:
<?php
while($row = mysql_fetch_assoc($query_users)){
if(in_array($row['username'],$admins)){
//do admin stuff here
}else{
//do NON-admin stuff here
}
}?>
The Gist
I want to perform an SQL query that depends on a variable number of parameters in my GET without being vulnerable to SQL injection.
The Parameters
My URL can be formed like this:
https://www.example.com/index.php?param1=blah1,param2=blah2,param3=a,b,c
or like this:
https://www.example.com/index.php?param1=blah1,param2=blah2,param3=a,b,c,d,e,f,g
In other words, param3 can have a variable number of comma-delimited parameters a,b,c,etc.
The White-list
I check to make sure that all parameters in a,b,c,etc. are in an approved white-list before I perform the query.
// $valid_params is an array of pre-approved parameters.
$arr = explode(',', clean($_GET['param3']));
$params = Array();
foreach($arr as $param){
if(in_array($param, $valid_params)){
array_push($params, $param);
}
}
The Query
I set up my database connection like this (with MySQL):
$db_connection = new PDO("mysql:host={$DB_HOST};dbname={$DB_NAME}",$DB_USER,$DB_PASS);
$db_connection->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$db_connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
And I want to perform a query like this (except safely):
$comma_separated_params = implode(',',$params);
$result = $db_connection->query("SELECT {$comma_separated_params} FROM some_table");
The Goal
Does anyone know how I could do this safely and efficiently?
Depending on your concern for overhead, you could just SELECT * and then filter the array in PHP - if the parameter is never sent to the database then there is no room for injection.
However it's not exactly the most elegant solution. Here's how I'd do it:
$comma_separated_params =
implode(
",",
array_map(
function($a) {return "`".$a."`";},
array_intersect(
explode(",",$_GET['param3']),
$valid_params
)
)
)
);
That one-line-wonder (with newlines added for clarity) will take the $_GET['param3'] variable, split it on commas, intersect it with your valid parameters (instead of your foreach loop), wraps each element in backticks (see note below) and finally glues them together with commas.
See, backticks allow you to use literally any string as a field name. Usually it's to allow keywords as names, but it can also allow for column names with spaces, and so on. The only character that has meaning within the backticks are backslashes and backticks - which it is safe to assume are not present since they'd have to be in your list of $valid_params to get this far.
Whitelisting is the way to go here. If you only allow things in that you've already specifically defined you should be okay. As for how efficient, this is all relative. The version you're using will perform well for relatively small lists, such as those with under 100 columns, so I wouldn't worry.
Bonus points for using PDO.
There's a chance your definition of 'allowed' columns and what's actually in the database might diverge. A more relaxed specification might be to grab the fields using SHOW FIELDS for the table in question and only allow those.
If you are only allowing a specific list of predefined values to be passed in param 3, and you are comparing the input values against them, I don;t think you have any injection exposure, as you then have full control over the value that ultimately go into your $comma_seperated_params variable.
This needs some work to finish, but with parameter binding it would look like this:
$binding = array();
$selects = array();
foreach ( $params as $value ) {
$binding[] = ':' . $value;
$selects = '?';
}
$select = implode(',', $select);
$result = $db_connection->prepare("SELECT $select FROM some_table");
foreach ( $binding as $key => $bind ) {
$result->bindParam($key, $bind, PDO::PARAM_STR);
}
$result->execute();
PDO::prepare will help you. This is exactly is recommended by experts. Never use mysql_real_escape_string (string). Always go for prepared statements.
In PHP, I want to insert into a database using data contained in a associative array of field/value pairs.
Example:
$_fields = array('field1'=>'value1','field2'=>'value2','field3'=>'value3');
The resulting SQL insert should look as follows:
INSERT INTO table (field1,field2,field3) VALUES ('value1','value2','value3');
I have come up with the following PHP one-liner:
mysql_query("INSERT INTO table (".implode(',',array_keys($_fields)).") VALUES (".implode(',',array_values($_fields)).")");
It separates the keys and values of the the associative array and implodes to generate a comma-separated string . The problem is that it does not escape or quote the values that were inserted into the database. To illustrate the danger, Imagine if $_fields contained the following:
$_fields = array('field1'=>"naustyvalue); drop table members; --");
The following SQL would be generated:
INSERT INTO table (field1) VALUES (naustyvalue); drop table members; --;
Luckily, multiple queries are not supported, nevertheless quoting and escaping are essential to prevent SQL injection vulnerabilities.
How do you write your PHP Mysql Inserts?
Note: PDO or mysqli prepared queries aren't currently an option for me because the codebase already uses mysql extensively - a change is planned but it'd take alot of resources to convert?
The only thing i would change would be to use sprintf for readability purposes
$sql = sprintf(
'INSERT INTO table (%s) VALUES ("%s")',
implode(',',array_keys($_fields)),
implode('","',array_values($_fields))
);
mysql_query($sql);
and make sure the values are escaped.
Nothing wrong with that. I do the same.
But make sure you mysql_escape() and quote the values you stick in the query, otherwise you're looking at SQL injection vulnerability.
Alternately, you could use parametrized queries, in which case you can practically pass the array in itself, instead of building a query string.
The best practice is either to use an ORM (Doctrine 2.0), an ActiveRecord implementation (Doctrine 1.0, RedBean), or a TableGateway pattern implementation (Zend_Db_Table, Propel). These tools will make your life a lot easier, and handle a lot of the heavy lifting for you, and can help protect you from SQL injections.
Other than that, there's nothing inherently wrong with what you're doing, you just might want to abstract it away into a class or a function, so that you can repeat the functionality in different places.
Using the sprintf trick mentioned by Galen in a previous answer, I have come up with the following code:
$escapedfieldValues = array_map(create_function('$e', 'return mysql_real_escape_string(((get_magic_quotes_gpc()) ? stripslashes($e) : $e));'), array_values($_fields));
$sql = sprintf('INSERT INTO table (%s) VALUES ("%s")', implode(',',array_keys($_fields)), implode('"," ',$escapedfieldValues));
mysql_query($sql);
It generates a escaped and quoted insert. It also copes independent of whether magic_quotes_gpc is on or off. The code could be nicer if I used new PHP v5.3.0 anonymous functions but I need it to run on older PHP installations.
This code is a bit longer that the original (and slower) but it is more secure.
I use this to retrieve the VALUES part of the INSERT.
But it might be an absurd way to do things. Comments/suggestions are welcome.
function arrayToSqlValues($array)
{
$sql = "";
foreach($array as $val)
{
//adding value
if($val === NULL)
$sql .= "NULL";
else
/*
useless piece of code see comments
if($val === FALSE)
$sql .= "FALSE";
else
*/
$sql .= "'" . addslashes($val) . "'";
$sql .= ", ";
};
return "VALUES(" . rtrim($sql, " ,") . ")";
}
There is a problem with NULL (in the accepted answer) values being converted to empty string "". So this is fix, NULL becomes NULL without quotes:
function implode_sql_values($vals)
{
$s = '';
foreach ($vals as $v)
$s .= ','.(($v===NULL)?'NULL':'"'.mysql_real_escape_string($v).'"');
return substr($s, 1);
}
Usage:
implode_sql_values(array_values( array('id'=>1, 'nick'=>'bla', 'fbid'=>NULL) ));
// =='"1","bla",NULL'
If you want to enhance your approach and add the possibility for input validation and sanitation, you might want to do this:
function insertarray($table, $arr){
foreach($arr as $k => $v){
$col[] = sanitize($k);
$val[] = "'".sanitize($v)."'";
}
query('INSERT INTO '.sanitize($table).' ('.implode(', ', $col).') VALUES ('.implode(', ', $val).')' );
}