I have a table with hundreds of columns. The table structure is out of my control (controlled by a third party). The table also has horrendous field names with spaces, single quotes, etc. and so do the table values. The table is updated once per hour via cron. The cron job truncates and rebuilds the table each time. I also keep an archive table of that table, that I use a REPLACE INTO statement to update or insert as required.
My challenge - I prefer not to have to explicitly define all 350 field names and values, and do so again in my REPLACE INTO statement as this will take a very long time and will require maintenance if the table changes. I would much rather use arrays. Here is what is not working but hopefully gives an idea of the goal (I realize this is deprecated MySQL but it is what it is for a variety of reasons):
$listings = mysql_query("SELECT * FROM current.table");
while ($listing = mysql_fetch_assoc($listings)){
//prepare variables
$fields = array_keys($listing);
$fields = implode('`, `', $fields);
$fields = "`$fields`";
$values = array_values($listing);
$values = implode("`, `", $values);
$values = "`$values`";
mysql_query('REPLACE INTO archive.table ($fields) VALUES ($values)');
}
Posting as a community wiki, no rep should come from this since it did solve the OP's question (as per suggested in comments).
"Aha! Erroneous single quotes on the mysql_query statement was the culprit. I also did mysql_real_escape_string on $values and used single quotes instead of ticks. Worked like a charm. Thank you! Final answer: – Tavish"
Use mysql_error() on the queries. What you posted seems legit code, however values needs to be quoted ' and escaped for possible injection, not ticked.
While using double quotes " for the second query's encapsulation.
mysql_query("REPLACE INTO archive.table ($fields) VALUES ($values)");
As well as other suggestions given.
OP's final code (taken from comments):
while ($listing = mysql_fetch_assoc($listings)){
$fields = array_keys($listing);
$fields = implode(', ', $fields);
$fields = "$fields";
$values = array_values($listing);
$values = implode(", ", $values);
$values = mysql_real_escape_string($values);
$values = str_replace("`","'",$values);
$values = "'$values'";
mysql_query("REPLACE INTO archive.table ($fields) VALUES ($values)");
}
Related
Is there something that may escape the sanitation in my script or is it safe from most SQL injections? The way I understand it, if you pass query as prepared argument, it does not matter how the query was build, right?
Edit2: I edited the code to reflect the suggestions of binding the $_POST values
$q = $pdo->prepare('SHOW COLUMNS FROM my_table');
$q->execute();
$data = $q->fetchAll(PDO::FETCH_ASSOC);
$key = array();
foreach ($data as $word){
array_push($key,$word['Field']);
}
$sqlSub= "INSERT INTO other_table(";
$n = 0;
foreach ($key as $index){
$sqlSub = $sqlSub.$index.", ";
$n = $n + 1;
}
$sqlSub = $sqlSub.") VALUES (";
for ($i=1; $i<$n;$i++){
$sqlSub = $sqlSub."?, ";
}
$sqlSub = $sqlSub.."?)";
$keyValues = array();
for($i=0;i<n;$i++){
array_push($keyValues,$_POST[$key[$i]]);
}
$q->$pdo->prepare($sqlSub);
q->execute($keyValues);
EDIT: This is how the final query looks like after suggested edits
INSERT INTO other_table($key[0],...,$key[n]) VALUES (?,...,nth-?);
No. The example code shown is not safe from most SQL Injections.
You understanding is entirely wrong.
What matters is the SQL text. If that's being dynamically generated using potentially unsafe values, then the SQL text is vulnerable.
The code is vulnerable in multiple places. Even the names of the columns are potentially unsafe.
CREATE TABLE foo
( `Robert'; DROP TABLE Students; --` VARCHAR(2)
, `O``Reilly` VARCHAR(2)
);
SHOW COLUMNS FROM foo
FIELD TYPE NULL
-------------------------------- ---------- ----
Robert'; DROP TABLE Students; -- varchar(2) YES
O`Reilly varchar(2) YES
You would need to enclose the column identifiers in backticks, after escaping any backtick within the column identifier with another backtick.
As others have noted, make sure your column names are safe.
SQL injection can occur from any external input, not just http request input. You can be at risk if you use content read from a file, or from a web service, or from a function argument from other code, or the return value of other code, or even from your own database... trust nothing! :-)
You could make sure the column names themselves are escaped. Unfortunately, there is no built-in function to do that in most APIs or frameworks. So you'll have to do it yourself with regular expressions.
I also recommend you learn about PHP's builtin array functions (http://php.net/manual/en/ref.array.php). A lot of your code could be quicker to develop the code, and it will probably better runtime performance too.
Here's an example:
function quoteId($id) {
return '`' . str_replace($id, '`', '``') . '`';
}
$q = $pdo->query("SHOW COLUMNS FROM my_table");
while ($field = $q->fetchColumn()) {
$fields[] = $field;
}
$params = array_intersect_key($_POST, array_flip($fields));
$fieldList = implode(",", array_map("quoteId", array_keys($params)));
$placeholderList = implode(",", array_fill(1, count($params), "?"));
$sqlSub = "INSERT INTO other_table ($fieldList) VALUES ($placeholderList)";
$q = $pdo->prepare($sqlSub);
$q->execute($params);
In this example, I intersect the columns from the table with the post request parameters. This way I use only those post parameters that are also in the set of columns. It may end up producing an INSERT statement in SQL with fewer than all the columns, but if the missing columns have defaults or allow NULL, that's okay.
There is exactly one way to prevent SQL injection: to make sure that the text of your query-string never includes user-supplied content, no matter how you may attempt to 'sanitize' it.
When you use "placeholders," as suggested, the text of the SQL string contains (probably ...) question marks ... VALUES (?, ?, ?) to indicate each place where a parameter is to be inserted. A corresponding list of parameter values is supplied separately, each time the query is executed.
Therefore, even if value supplied for last_name is "tables; DROP TABLE STUDENTS;", SQL will never see this as being "part of the SQL string." It will simply insert that "most-unusual last_name" into the database.
If you are doing bulk operations, the fact that you need prepare the statement only once can save a considerable amount of time. You can then execute the statement as many times as you want to, passing a different (or, the same) set of parameter-values to it each time.
I'm trying to copy a row from a structure I technically know nothing about.
This is what I have so far. This code does work but I'm pretty sure this isn't the most appropriate. Anyone have a better way or a right way of doing this? Any suggestions would be appreciated.
/*
$table is the table name
$id_field is the primary key
$id_value is the row I want to copy
*/
$selectEntity = $dbh->prepare("SELECT * FROM {$table} WHERE {$id_field} = :id_value");
$selectEntity->execute(array(':id_value' => $id_value));
$entity = $selectEntity->fetch(PDO::FETCH_ASSOC);
foreach ($entity as &$value){ if(is_null($value) == true) { $value = "NULL"; } else { $value = "'".htmlspecialchars($value, ENT_QUOTES)."'"; } }
//remove the primary key
$entity[$id_field] = "'"; // the ' was the only way I could get NULL to get in there
$insertCloned = $dbh->prepare("INSERT INTO {$table} (".implode(", ",array_keys($entity)).") VALUES ('".implode(", ",array_values($entity)).")");
$insertCloned->execute();
$lastInsertedID = $dbh->lastInsertId();
It's very messy.
Your quoting is not correct -- you have quotes around the entire VALUES list, they should be around each individual value. Also, you should use $dbh->escape($value) to escape the values; htmlspecialchars is for encoding HTML when you want to display it literally on a web page.
But it's better to use query parameters rather than substituting into the SQL. So try this:
$entity[$id_field] = null;
$params = implode(', ', array_fill(0, count($entity), '?'));
$insertCloned = $dbh->prepare("INSERT INTO {$table} VALUES ($params)");
$insertCloned->execute(array_values($entity));
You don't need to list the column names in the INSERT statement when the values are in the same order as the table schema. And since you used SELECT * to get the values in the first place, they will be.
So I have two arrays, $gDatabaseKeyNames (list of column names) and $gDatabaseKeyValues (a list of variable names containing values from a script). The variable names are generated dynamically so I keep track of them for an insert.
Rather than listing all of the columns and value variables, I tried this
$cols = implode(", ", array_values($gDatabaseKeyNames));
$vals = implode(", ", array_values($gDatabaseKeyValues));
$query = "INSERT INTO pings (survey_id, $cols) VALUES ('$surveyID', $vals)";
mysql_query ($query) or die(mysql_error());
But none of my actual values show up in the database (they are inserted as 0s - all my columns are numeric).
If I echo $query, I see this, which is the correct formatting:
INSERT INTO pings (survey_id, latitude, longitude, pingTimestamp) VALUES ('15', '$Dlatitude', '$Dlongitude', FROM_UNIXTIME('$DtimeStamp'))
However, if I change $query to
$query = INSERT INTO pings (survey_id, latitude, longitude, pingTimestamp) VALUES ('$surveyID', '$Dlatitude', '$Dlongitude', FROM_UNIXTIME('$DtimeStamp'));
It works perfectly!
Why isn't PHP interpreting the variables in the implode? How can I fix this?
I suspect that $cols has the literal value '$Dlatitude', '$Dlongitude', etc.... PHP does not "double-interpolate" strings. It'll replace $cols with its values, but will NOT replace $Dlatitude with that variable's value. In other words, you're literally inserting some strings-that-look-like-PHP-variable-names into your numeric fields.
What is in the $gDatabaseKeyValues/Names arrays?
It's probably caused because of string/number problems. You should consider changing the database engine (Consider PDO) and use prepared statements to bind the parameters to your query the right way.
I'm trying to insert form data into a MySQL 4.1 DB. The problem I'm having is form fields that include spaces get truncated before insertion. The POST variables are complete, spaces and all. Just being cut off somewhere. For instance, "South Lake Tahoe" is inserted simply as "South". Zip codes and telephone numbers with dashes are also fine. The site I'm working on is hosted by Yahoo Small Business, and they're still using MySQL 4.1. I don't know if that is the problem, but I do know I never had issues doing this with MySQL 5+. The user fills out a form to add a new member. Upon Submit, the form data is POSTED to another page for processing:
$k = array();
$v = array();
$first_name = $_POST['first_name'];
$last_name = $_POST['last_name'];
$result = mysql_query("SELECT * FROM members WHERE first_name='$first_name' AND last_name='$last_name'");
if(mysql_num_rows($result)>0){
mysql_free_result($result);
exit("Duplicate User in Database");
}
mysql_free_result($result);
array_pop($_POST);//Don't need the Submit value
foreach($_POST as $key=>$value){
array_push($k, "$key");
array_push($v, "$value");
}
$fields = implode(", ", $k);
$values = array();
foreach($v as $key=>$value){
array_push($values, '"'.$value.'"');
}
$values_string = implode(", ", $values);
$result = mysql_query("INSERT INTO members($fields) VALUES($values_string)");
I'm sure there are better ways of doing this, but I'm still on the way up the learning curve. Please point out any obvious flaws in my thinking.
Any suggestions are greatly appreciated.
EDIT: The field types in MySQL are correct and long enough. For example, the field for City is set as VARCHAR(30).
Thanks much,
Mark
This code is horrifically insecure - you're taking user-supplied values and plopping them directly into your SQL statements without any sanitization. You should call http://php.net/manual/en/function.mysql-real-escape-string.php on anything you insert into a query this way (parameterized queries with PDO are even better).
You also make some assumptions, such as $_POST always being ordered a certain way (is that guaranteed?) and that you have exactly as many elements in your form as you have fields in your table, and that they're named identically. The code as it's written is the kind of thing a lot of beginning programmers do - it feels efficient, right? But in the end it's a bad idea. Just be explicit and list out the fields - e.g.
$field1 = $_POST['field1'];
$field2 = $_POST['field2'];
$sql = "insert into mytable (field1, field2) values ('" . mysql_real_escape_string($field1) . "', '" . mysql_real_escape_string(field2) . "')";
mysql_query($sql);
I haven't touched on why stuff would cut off at the first space, as this would imply that your code as you have it presented is salvageable. It's not. I get the feeling that reworking it as I described above might make that problem go away.
<?php
// Remember to always escape user input before you use them in queries.
$first_name = mysql_real_escape_string($_POST['first_name']);
$last_name = mysql_real_escape_string($_POST['last_name']);
$result = mysql_query("SELECT * FROM members WHERE first_name='$first_name' AND last_name='$last_name'");
if (mysql_num_rows($result) > 0) {
mysql_free_result($result);
exit("Duplicate User in Database");
}
mysql_free_result($result);
// I removed your loop around $_POST as it was a security risk,
// and could also become non-working. (What would happen if the order
// of the $_POST keys were changed?)
// Also the code become clearer this way.
$result = mysql_query("INSERT INTO members(first_name, last_name) VALUES('$first_name', '$last_name')");
In PHP, I want to insert into a database using data contained in a associative array of field/value pairs.
Example:
$_fields = array('field1'=>'value1','field2'=>'value2','field3'=>'value3');
The resulting SQL insert should look as follows:
INSERT INTO table (field1,field2,field3) VALUES ('value1','value2','value3');
I have come up with the following PHP one-liner:
mysql_query("INSERT INTO table (".implode(',',array_keys($_fields)).") VALUES (".implode(',',array_values($_fields)).")");
It separates the keys and values of the the associative array and implodes to generate a comma-separated string . The problem is that it does not escape or quote the values that were inserted into the database. To illustrate the danger, Imagine if $_fields contained the following:
$_fields = array('field1'=>"naustyvalue); drop table members; --");
The following SQL would be generated:
INSERT INTO table (field1) VALUES (naustyvalue); drop table members; --;
Luckily, multiple queries are not supported, nevertheless quoting and escaping are essential to prevent SQL injection vulnerabilities.
How do you write your PHP Mysql Inserts?
Note: PDO or mysqli prepared queries aren't currently an option for me because the codebase already uses mysql extensively - a change is planned but it'd take alot of resources to convert?
The only thing i would change would be to use sprintf for readability purposes
$sql = sprintf(
'INSERT INTO table (%s) VALUES ("%s")',
implode(',',array_keys($_fields)),
implode('","',array_values($_fields))
);
mysql_query($sql);
and make sure the values are escaped.
Nothing wrong with that. I do the same.
But make sure you mysql_escape() and quote the values you stick in the query, otherwise you're looking at SQL injection vulnerability.
Alternately, you could use parametrized queries, in which case you can practically pass the array in itself, instead of building a query string.
The best practice is either to use an ORM (Doctrine 2.0), an ActiveRecord implementation (Doctrine 1.0, RedBean), or a TableGateway pattern implementation (Zend_Db_Table, Propel). These tools will make your life a lot easier, and handle a lot of the heavy lifting for you, and can help protect you from SQL injections.
Other than that, there's nothing inherently wrong with what you're doing, you just might want to abstract it away into a class or a function, so that you can repeat the functionality in different places.
Using the sprintf trick mentioned by Galen in a previous answer, I have come up with the following code:
$escapedfieldValues = array_map(create_function('$e', 'return mysql_real_escape_string(((get_magic_quotes_gpc()) ? stripslashes($e) : $e));'), array_values($_fields));
$sql = sprintf('INSERT INTO table (%s) VALUES ("%s")', implode(',',array_keys($_fields)), implode('"," ',$escapedfieldValues));
mysql_query($sql);
It generates a escaped and quoted insert. It also copes independent of whether magic_quotes_gpc is on or off. The code could be nicer if I used new PHP v5.3.0 anonymous functions but I need it to run on older PHP installations.
This code is a bit longer that the original (and slower) but it is more secure.
I use this to retrieve the VALUES part of the INSERT.
But it might be an absurd way to do things. Comments/suggestions are welcome.
function arrayToSqlValues($array)
{
$sql = "";
foreach($array as $val)
{
//adding value
if($val === NULL)
$sql .= "NULL";
else
/*
useless piece of code see comments
if($val === FALSE)
$sql .= "FALSE";
else
*/
$sql .= "'" . addslashes($val) . "'";
$sql .= ", ";
};
return "VALUES(" . rtrim($sql, " ,") . ")";
}
There is a problem with NULL (in the accepted answer) values being converted to empty string "". So this is fix, NULL becomes NULL without quotes:
function implode_sql_values($vals)
{
$s = '';
foreach ($vals as $v)
$s .= ','.(($v===NULL)?'NULL':'"'.mysql_real_escape_string($v).'"');
return substr($s, 1);
}
Usage:
implode_sql_values(array_values( array('id'=>1, 'nick'=>'bla', 'fbid'=>NULL) ));
// =='"1","bla",NULL'
If you want to enhance your approach and add the possibility for input validation and sanitation, you might want to do this:
function insertarray($table, $arr){
foreach($arr as $k => $v){
$col[] = sanitize($k);
$val[] = "'".sanitize($v)."'";
}
query('INSERT INTO '.sanitize($table).' ('.implode(', ', $col).') VALUES ('.implode(', ', $val).')' );
}