The following topic is close to what I'm trying to ask, but it's outdated considering the mysql functions are deprecated in php now and there are prepared statements for preventing sql injection. insert all $_POST data into mysql using PHP?
Basically, I have a huge number of columns in my database that all need to get filled up when I submit this form. The form matches each column with an input field of the same name (the name attribute on the input field is the same as the column name it belongs in. So $_POST['firstName'] goes in the firstName column, and so on).
Is there a way using mysqli or PDO that I could easily just take all my POST data and automatically insert it into the MySQL table without going through each field by hand? I could code them all out using prepared statements, but there are a ton of columns and I'd like to get them done all at once if possible.
This is the beginning of the long version I don't really want to have to complete.
$stmt = $connection->prepare("INSERT INTO area_retreat
(user,firstName,lastName,...etc)
VALUES
(?,?,?,...etc)
ON DUPLICATE KEY UPDATE
user=VALUES(user),
firstName=VALUES(firstName),
lastName=VALUES(lastName),
...etc
");
$stmt->bind_param("sss",
$username,
$_POST['firstName'],
$_POST['lastName']
);
$stmt->execute();
INSERT INTO area_retreat VALUES (?, ?, ...) -- however, you have to match ALL columns as shown in the database.
If you have an auto increment ID, you will need to provide NULL for that column in the proper column order.
To avoid errors you definitely need to store the list of variables one way or another. It could be as simple as an array:
$fields = array('firstName', etc.);
Then you can loop through your array to generate your sql statement dynamically and using named placeholders instead of question marks, you only need to bind them once. You can also store the values in an array and send that array as a parameter to execute():
// start of query
$values = array();
$query = '...';
foreach ($fields as $field)
{
if (isset($_POST[$field]))
{
// add to query
$query .= "...";
// add value to array so that you can feed the array to `execute`
$values[':' . $field] = $_POST[$field];
}
}
// add end of query
$query .= '...';
$stmt->execute($values);
If you want to use the same variables in an ON DUPLICATE KEY UPDATE section, you can do another loop or build an insert section that you can use twice after looping once.
Related
In PHP, I pull a large amount of JSON data from a URI, then serialize it into an associative PHP array via the built-in json_decode function.
Then, I create an array:
$inserts = array();
I loop through the JSON associative array, adding a new key/value pair to my $inserts array for each item in the JSON array:
foreach($JSON_data as $key => $value) {
$inserts[] = "(".mysql_real_escape_string($value["prop1"]).","
.mysql_real_escape_string($value["prop2"]).","
.mysql_real_escape_string($value["prop3"]).")";
}
Then, I perform a bulk insert simply by imploding the inserts I already prepared:
mysql_query("INSERT INTO `MyTable` (`col1`,`col2`,`col3`) VALUES ".implode(",",$inserts));
Anyways, I found that the mysql_* family is no longer suggested to be used. So I'm wondering how this type of pattern is suppose to be accomplished using prepared statements or w/e the new accepted constructs are? My concerns are to eliminate SQL injection, and also to update MySQL as quickly as possible with fewer than 10 concurrent, open connections (preferably 1). Also, to keep things as simple and quick as possible.
Or, if there's a new pattern or preferred method to perform such a bulk transaction.
If you use a prepared statement, you can loop over your $JSON_data array with a foreach loop and run the INSERT with that chunk of the data.
Using prepared statements will reduce the overhead of building the query, simply sending the new data to the database on each iteration of the loop.
$query = mysqli_prepare("INSERT INTO `MyTable` (`col1`,`col2`,`col3`)
VALUES(?,?,?)");
foreach($JSON_data as $key => $value) {
$query->bind_param('sss',$value["prop1"],$value["prop2"],$value["prop3"];
$query->execute();
}
Note that the first argument to bind_param() tells it how many values you will be binding, as well as the type for each value.
s corresponds to string data, i corresponds to integer data, d corresponds to double (floating point), and b corresponds to binary data.
One other word of caution, do NOT quote any string data, as the s datatype tells mysql to expect a string. If you quote the ? in the prepared statement, it will tell you the number of params is wrong. If you quote the strings, it will be quoted in mysql.
EDIT:
If you want to use the same paradigm (inserting multiple rows with one query), there are ways to do it. One way is to create a class that will aggregate the bind_param calls and do one bind_param when you execute the query. Code for that is here.
Use Mysqli or PDO
Here is how you would utilize prepared statements with Mysqli
<?php
//Basic layout to using parametized queries in PHP to prevent Bobby-tables
$VARIABLE = "Some Data";
$mysqli = new mysqli("SERVER","USER","PASSWORD","DATABASE");
$query = $mysqli->prepare("SELECT COLUMN_LIST FROM TABLE WHERE COLUMN = ?");
$query->bind_param('s',$VARIABLE); //'s' for string, use i for int d for double
$query->execute();
//Get results
$query->bind_result($VARIABLE_NAMES_MATCHING_COLUMN_NAMES_GO_HERE);
$query->fetch();
echo $VARIABLE_LIST_MATCHING_COLUMN_LIST;
?>
This question has kinda been asked already but I couldn't find my answer. I searched a while and found these related questions, but they didn't help me to understand or answer my problem.
SQL Insert Into with Inner Join
T-SQL INSERT INTO with LEFT JOIN
My question is how to insert data in 2 tables using joins. For example (with php) a user can enter his/her name and the foods he/she likes.
I store them in a variable and an array (the length of the array is not always 3 like below):
$name = "Niels"
$foodsHeLikes = array("apple", "pear", "banana");
This is how I want to store them:
USERS:
UserID name
1 Niels
FOODS:
FoodID userID name //userID is linked to UserID in users table
1 1 apple
2 1 pear
3 1 banana
The link to the first question I pasted above has an insert with a join but I don't see anywhere to put the values in like with a normal insert?
The query from that question:
INSERT INTO orders (userid, timestamp)
SELECT o.userid, o.timestamp FROM users u INNER JOIN orders o ON o.userid = u.id
Judging by what's been going on in the comment section, what you're asking is that you would like to have a more optimal query process. Right now you are using two different queries to populate your two tables, and you're wondering whether that could be done more optimally.
First things first, it's not possible to populate TWO different tables with ONE query.
However, what you could do, is use transactions.
The rest of this answer will follow the assumption that you are using PHP as your backend scripting language (as you tagged yourself).
Also, it is not inherently obvious whether you use prepared statements for your queries or not. In the case you don't, I would highly recommend using prepared statements. Otherwise, you're opening yourself up to SQL Injections (SQLI Attacks).
I will proceed by using mysqli prepared statements in this answer.
<?php
// Your input post variables
$name = $_POST['name'];
$foodArray = $_POST['foodArray'];
/*
I'm using a function to handle my queries,
simply because it makes large piles of code easier to read.
I now know that every time the function:
createUserAndFood($name, $foodArray);
is called, that it will populate my user and food table.
That way I don't have to worry about writing all the code multiple times.
*/
function createUserAndFood($name, $foodArray){
// food array values
$foodValues = array_values($foodArray);
// DB variables
$servername = "localhost";
$username = "username";
$password = "password";
$dbname = "myDB";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if($conn->connect_error){
die("Connection failed: " . $conn->connect_error);
}
/*
Stops the query from auto commiting,
I'll explain later, you can "maybe" disregard this.
*/
$conn->autocommit(FALSE);
// Declare the query
$sql = "INSERT INTO userTable(name) VALUES(?)";
// Prepare and bind
$stmt = $conn->prepare($sql);
$stmt->bind_param("s", $name);
// Execute the query
$stmt->execute();
// Fetch last inserted id
$lastID = $conn->insert_id;
$sql = "INSERT INTO foodTable(userId, food) VALUES(?, ?)";
$stmt = $conn->prepare($sql);
for($i = 0; $length = count($foodValues) > $i; $i++){
$stmt->bind_param("is", $lastID, $food);
$food = $foodValues[$i];
$stmt->execute();
}
// Commits the query / queries
$conn->commit();
// Close connection
$stmt->close();
$conn->close();
}
?>
Since you wanted to optimize your queries, the general idea that we are using here, is that we are making use of the MySQL function LAST_INSERT_ID(); via PHP and store it into a variable.
Now, this is mainly relevant if you are using auto incremented id's. If you are not, you can disregard this specific logic and use something else. But if you are, then keep reading.
The reason why we are storing the last id into a variable is because we need to use it multiple times (the new user might have more than one favorite food afterall). If you were not to store the last id into a variable, it would instead take the auto incremented value of the second table after the initial insert, which means upon your third insert statement and forward, you would be working with the wrong id.
Now, as I promised to explain, the reason I'm using $conn->autocommit(FALSE); and $conn->commit(); is because you might not want incomplete data sets in your database. Imagine that a user input is happening, but your database crashes in the middle of it all. You'll have incomplete data sets. If this is not really a concern of yours, then you can disregard that.
To simplify what's going on at the MySQL side of things, think of it like this:
BEGIN;
INSERT userTable SET name = '$name';
SET #lastID = LAST_INSERT_ID();
INSERT foodTable SET id = #lastID, food = '$food';
COMMIT;
I am creating an array from dynamic created input fields and then send the array over to php, in php I create a foreach loop and try to insert it into my database.
I have created a foreach loop for my array and insert the values like that in my database, however my problem is: You cannot bind a variable with an index number to your query string. I have already verified whether it's actually an array what I am sending and the answer is: Yes. it is an array.
$stmt = $conn->prepare('INSERT INTO vv(event_id, vvType, vvCosting) VALUES (?, ?, ?)');
foreach ($example as $index => $value) {
$stmt->bind_param('iss', $id, $example, $example_costs);
$id = $id;
$example = $value;
$example_costs = example_costs[$index]; //this DOES NOT work
$stmt->execute();
}
So what I want would be the following: Create one foreach loop for my $example variable and based on the index numbers it has also insert the data from the variable example_costs and insert that in the database as well. You shouldn't worry about whether the index value exists or not in my example_costs variable, since they're "pairs".
I know the problem is that I cannot bind my variable like this:
$example_costs[$index]
So now my question is: How can I bind my variable like the above? But in a way php DOES accept? Or do I need to create 2 foreach loops? --> Something I would rather NOT do.
You didn't specify what exactly you mean by "doesn't work" (i.e. you didn't describe what is going wrong or what behaviour you're seeing), but here's what I can observe just from the code:
1) You have (what appears to be) a typo: I expect example_costs[$index]; should really be $example_costs[$index];.
2) Assuming 1 is fixed, you're also overwriting the values of the variables $example and $example_costs, which are the ones you're trying to loop over, whilst the loop is still going on. Clearly this will destroy the original arrays and make them impossible to re-use next time it tries to loop.
In this situation, and as a general point of good coding practice, don't re-use variable names to represent two different things in the code - especially when you are still in the middle of using them for their original purpose!
In reality you don't really need these separate variables in the bind statement anyway. This should do the job:
$stmt = $conn->prepare('INSERT INTO vv(event_id, vvType, vvCosting) VALUES (?, ?, ?)');
foreach ($example as $index => $value) {
$stmt->bind_param('iss', $id, $value, $example_costs[$index]);
$stmt->execute();
}
The rest is either incorrect (over-writing the $example array, for instance) or redundant ($id = $id for instance - assigning a variable its own value is the very definition of pointless).
I have a user table with and id field and 10 other fields storing user details of various types that the user can change via various web forms. I want to have a PHP script that gets POSTed changed values for some subset of these fields, and UPDATEs only those fields that are received in the POST data. I'm finding this surprisingly difficult to do in a way that doesn't suck. We use mysqli for all database interaction in the rest of this application so mysqli-based solutions are strongly preferred.
The options I've considered and dismissed so far:
1) Run a separate UPDATE query for every field provided in the POST data - yuck, I don't want to hit the database up to 10 times for something that could be done in one query.
2) Have a dictionary mapping field names to the fields' data types, and iteratively construct the query by looping through the provided fields, checking whether they are text fields or not, calling mysqli_real_escape_string on the string fields and otherwise sanitizing the others (e.g. by type checking or sprintf with '%i' placeholders). - Yuck! I could probably safely do things this way if I was careful, but I don't want to make a habit of using this kind of approach because if I'm careless I'll leave myself open to SQL injection. Parameterized queries don't give me the potential to screw up dangerously, but this approach does. My ideal is to never concatenate any data into an SQL query manually and always rely upon parameterized queries; the database libraries of other languages, like Python, let me easily do this.
3) Use a parameterized query - this is my ideal for everything, since as long as I insert all externally-provided data into my query via the bind_param method of a mysqli statement object, I'm immune to SQL injection and don't have to worry about sanitization, but using parameterized queries seems to be impossible here. The trouble is that bind_param requires that the data be passed as variables, since all arguments after the first are passed by reference. I can reasonably elegantly iteratively construct a query with ? placeholders, and while I'm at it construct the string of types that gets passed as the first argument to bind_param ('ssiddsi' etc.), but then there's no way I can see to choose at runtime which of my 10 fields I pass to bind_params (unless I have a switch statement with 10^2 cases).
Is there some PHP language construct I'm missing (something similar to array unpacking) that will allow me to choose at runtime which variables to pass as arguments to bind_param? Or is there some other approach I haven't considered that will let me solve this simple problem cleanly and safely?
You can easily combine 2 and 3 by means of my SafeMySQL library.
The code will look like
$allowed = array('title','url','body','rating','term','type');
$data = $db->filterArray($_POST,$allowed);
$sql = "UPDATE table SET ?u WHERE id=?i";
$db->query($sql, $data, $_POST['id']);
note that $allowed array doesn't make all these fields necessarily updated - it just filters POST fields out. So, even $_POST with only id and url would be correctly updated.
Nevertheless, using prepared statements, although toilsome, also quite possible.
See the code below
public function update($data, $table, $where) {
$data_str = '' ;
foreach ($data as $column => $value) {
//append comma each time after first item
if (!empty($data_str)) $data_str .= ', ' ;
$data_str .= "$column = $value" ;
}
$sql = "UPDATE $table SET $data_str WHERE $where";
mysqli_query($sql) or die(mysqli_error());
return true;
}
$data is an array, in your case it's $_POST.
If you want to be more specific about the data to be saved from $_POST array, you can define an array of allowed columns. For example,
$allowed = array('id', 'username', 'email', 'password');
By doing this, you can filter your $_POST array and pass it to update() function.
Just wondering, to sanitize user input, I use mysql_real_escape_string() on data before it is inserted into a table. Therefore when a user enters something like this:
Hi I'm just testing this
It gets placed into the table just fine, exactly as above. Question is, if I were to pull that data and place it into a variable via PHP, say $string, what would happen if I then used that variable to insert data into a new row in the table? Such as:
<?php
$result = mysql_query( "SELECT data FROM table WHERE id='1'" ); //data = Hi I'm just testing this
$result_array = mysql_fetch_array( $result );
$string = $result_array['data']; //string = Hi I'm just testing this
$insert = mysql_query( "INSERT INTO table (data) VALUES ('$string')" ) or die(mysql_error());
?>
Would the single quote (') cause problems in this scenario? Should I be using $string = mysql_real_escape_string( $result_array['data'] ) in this case as well?
Thanks!
Once the data's pulled out of MySQL, it's just like any other piece of data that you want to use in a query: You have to do proper escaping/quoting, or use a prepared statement. There's no magical flag within PHP that says "this came from the database and shall return whence it came".
The alternative is to use the INSERT INTO ... SELECT FROM syntax to do the operation completely within the database, if you can meet the conditions.
mysql_real_escape_string() simply prepares it for insertion into the database, once you request that data again it will be in its original form, i.e. you will have to sanitize it again before trying to insert it like your example above.