I have to do a lot of insert in my DB, importing data from xml.
Now, if I open and close the connection in the loop, the script works but crashes for max connection number, and if I use the following code, it execute just one time the mysqli_multi_query.
I just need to know, how to maintain the connection to execute a new multi query in the loop.
$xml = simplexml_load_file('demo.xml');
$mysqli =new mysqli($servername, $username, $password, $dbname);
foreach($xml->datas as $data) {
$sql="INSERT IGNORE INTO table1 (hobby) values ('".$data->child74."');";
$sql.="INSERT IGNORE INTO table2 (pizza, spaghetti) values ('".$data->child55."', '".$data->child52."');";
// a lot more insert...
mysqli_multi_query($mysqli,$sql);
}
mysqli_close($mysqli);
For the multiple inserts you should be using prepared statements. This approach will solve all the problems you have at once. The only possible issue (related to possible non-optimal database settings) is solved by using a transaction.
The code below is using only single connection and as fast as it can be
$xml = simplexml_load_file('demo.xml');
$stmt1 = $mysqli->prepare("INSERT IGNORE INTO table1 (hobby) values (?)");
$stmt1->bind_param("s",$hobby);
$stmt2 = $mysqli->prepare("INSERT IGNORE INTO table2 (pizza, spaghetti) values (?,?)");
$stmt2->bind_param("ss", $pizza, $spaghetti);
$mysqli->autocommit(FALSE);
foreach($xml->order as $data) {
$hobby = $data->child74;
$pizza = $data->child55;
$spaghetti = $data->child52;
$stmt1->execute();
$stmt2->execute();
}
$mysqli->commit();
$mysqli->close();
Prepared statements with a transaction make it civilized, secure and efficient solution.
Related
I'm totally PHP beginner, and I'm trying to insert variables in a database in PHP and MySQL.
This is my code:
$link = mysql_connect('localhost','','','onlynews') or die('Cannot connect to the DB');
mysql_select_db('TEST',$link) or die('Cannot select the DB');
$strSQL = "INSERT INTO news(id, title,photo,url,source, at) VALUES('$x','$title','$url','$imgurl ','$source','$at')";
mysql_query($strSQL) or die(mysql_error());
The problem is it is doing: NOTHING! No Entries at all, Nothing changes in the database.
-How can I fix this?
-Do I have to write codes to prevent SQL Injection, even if the variables are coming from an API, not from users?
You have to execute your query using $conn->query($sql);.
However, to avoid SQL injections you should definitely use prepared statements or at least $conn->real_escape_string() to escape the values in your SQL statement.
For example, this is your code using prepared statements:
$servername = "localhost";
$username = "";
$password = "";
$dbname = "onlynews";
$tableName = "news";
$conn = new mysqli($servername, $username, $password, $dbname);
$stmt = $conn->prepare("INSERT INTO news (id, title, photo, url, source, at)
VALUES (?, ?, ?, ?, ?, ?)");
$stmt->bind_param('ssssss', $thetitle, $urlToImage, $theurl, $thesource, $thetime);
$stmt->execute();
$stmt->close();
You should also add some error checking, since $conn->prepare() and $stmt->execute() may fail (and return false). Of course, establishing the connection to the database during the construction of $conn could also fail, which can be checked using $conn->connect_error.
I've been spending a couple of hours trying to write mysqli queries to insert a new row in a database (with a primary key ID) and then select the ID of the new row. My code as it currently is:
<?php
include('connectionData.php');
$conn = mysqli_connect($server, $user, $pass, $dbname, $port)
or die('Connection error');
if(isset($_POST['submit'])) {
$pnum = $_POST['pnum'];
$phone_insert_text = "INSERT INTO `voterdatabase`.`phone` (`pnum`) VALUES (?)";
$phone_insert_query = $conn->prepare($phone_insert_text);
$phone_insert_query->bind_param('s', $pnum);
$phone_insert_query->execute();
$phone_select_text = "SELECT phone_id FROM voterdatabase.phone WHERE pnum=?";
$phone_select_query = $conn->prepare($phone_select_text);
$phone_select_query->bind_param('s', $pnum);
$phone_select_query->execute();
$phone_select_query->bind_result($phone_id);
echo $phone_id;
?>
$phone_insert_query executes without issue. But $phone_select_query doesn't appear to run at all, as echo $phone_id; has no effect. What might be going on here? I'm able to run the query directly in MySQLWorkbench.
Note that I previously tried doing this in one query using SELECT LAST_INSERT_ID();, but mysqli fails to execute any query containing that.
Please try this
$lastInsertID= mysqli_insert_id($conn);
Use insert_id property:
<?php
include('connectionData.php');
$conn = mysqli_connect($server, $user, $pass, $dbname, $port)
or die('Connection error');
if(isset($_POST['submit'])) {
$pnum = $_POST['pnum'];
$phone_insert_text = "INSERT INTO `voterdatabase`.`phone` (`pnum`) VALUES (?)";
$phone_insert_query = $conn->prepare($phone_insert_text);
$phone_insert_query->bind_param('s', $pnum);
$phone_insert_query->execute();
$phone_id = $conn->insert_id;
echo $phone_id;
?>
If you wish to be able to use the available functions to get the last inserted id, like mysqli_insert_id(), your table must have an AUTO_INCREMENT column. If not you will not get the id.
Also, even if you have the required columns, this will require two calls. To get around this, what you could do is something like create a stored procedure to do your insert for you and return the inserted id from the procedure.
I'm currently using deprecated code to get data from users, as follows:
/* retrieve */
$lastName = $_POST['lastName'];
$firstName = $_POST['firstName'];
$examLevel=$_POST['level'];
/* connect */
$dbc=mysql_connect("localhost", "user", "passw") or die('Error connecting to MySQL server');
mysql_select_db("db") or die('Error selecting database.');
/* sanitize */
$lastName=mysql_real_escape_string($lastName);
$firstName=mysql_real_escape_string($firstName);
$examLevel=mysql_real_escape_string($examLevel);
/* insert */
$query_personal = "INSERT INTO personal (LastName, FirstName) VALUES ('$lastName', '$firstName')";
$query_exam = "INSERT INTO exam (Level, Centre, BackupCentre, etc.) VALUES ('$examLevel', '$centre', '$backup', 'etc')";
This is working but I keep coming across warnings about security and lack of support. There's a small rewrite to connect with mysqli instead of mysql but what about mysqli_real_escape_string? I've seen it used in examples but I've also seen advice to use prepared statements instead which don't use mysqli_real_escape_string.
And how would I use prepared statements to INSERT my data? I'm a bit at sea with this bit so far. For example, is parameter binding only for INSERTs and result binding only for SELECTs?
Convert it to PDO
/* connect */
$dsn = "mysql:host=localhost;db=test;charset=utf8";
$opt = array(
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC
);
$pdo = new PDO($dsn,"user", "passw", $opt);
/* insert */
$query = "INSERT INTO personal (LastName, FirstName) VALUES (?, ?)";
$stmt = $pdo->prepare($query);
$stmt->execute(array($_POST['lastName'],$_POST['firstName']));
$query = "INSERT INTO exam (Level, Centre, BackupCentre, etc) VALUES (?, ?, ?, 'etc')";
$stmt = $pdo->prepare($query);
$stmt->execute(array($_POST['level'], $centre, $backup));
see this pages for converting mysql into mysqli
Converting_to_MySQLi
https://wikis.oracle.com/display/mysql/Converting+to+MySQLi
and see mysqli_real_escape_string manual that explain about mysqli_real_escape_string and Security problem and how to solve it.
php.net:
Security: the default character set
The character set must be set either at the server level, or with the API function mysqli_set_charset() for it to affect mysqli_real_escape_string(). See the concepts section on character sets for more information.
see this page for query for insert data
see this page for prepare data for inserting to mysql
and http://php.net/manual/de/mysqli.quickstart.prepared-statements.php
Im looking for efficient method for inserting CSV data into SQLite database on remote server.
I have to use PHP. Im using SQLite3, so i have to use PDO (sqlite_query will not work).
CSV is on server side and it has 100000+ rows (50MB+ filesize).
Question: is there any method for PHP faster than this?
for (/* each row, 100.000 rows */)
{
$a = 'something';
$b = 'something';
$query = $link->prepare("INSERT INTO user_info (a, b) VALUES (?, ?)");
$query->bindParam(1, $a);
$query->bindParam(2, $b);
$query->execute();
}
My SQL file is in the same directory.
I've read about .import command, but i don't know how to use it with PDO (shall i use prepare? how file path should look like?).
Im new to PDO and SQLite.
$link->beginTransaction();
$query = $link->prepare("INSERT INTO user_info (a, b) VALUES (?, ?)");
for (/* each row, 100.000 rows */)
{
//variable stuff + execute query
}
$link->commit();
This holds the writes until after the loop - much faster.
You'll need to create db object like this:
$link = new PDO('sqlite:your.db');
edit: Typo/paste correction
Well, I don't know if there is a bulk approach with SQLite that can be used via a PDO driver, but you are re-preparing the same statement on every loop.
This would probably be a touch more efficient:
$query = $link->prepare("INSERT INTO user_info (a, b) VALUES (?, ?)");
for (/* each row, 100.000 rows */)
{
$a = 'something';
$b = 'something';
$query->bindParam(1, $a);
$query->bindParam(2, $b);
$query->execute();
}
In an effort to make my scripts more secure, I have started using prepared statements to prevent mysql injection. This script inserts the data just fine, but getting the last inserted id (based on auto incrementation in the database) now returns 0 when it should return the correct id number.
This part inserts just fine.
$stmt = $conn->prepare("INSERT INTO users (userName) VALUES (?)");
$stmt->bind_param("s", $_SESSION['username']);
$stmt->execute();
This is where I am having problems. I am trying to get the last inserted ID of the user and it is returning 0.
// Get the last inserted ID for the users ID
$query = "SELECT LAST_INSERT_ID()";
$result = mysql_query($query);
if ($result) {
$nrows = mysql_num_rows($result);
$row = mysql_fetch_row($result);
$userId = $row[0];
}
This was working when I had my script before starting on Prepared Statements. The only thing I changed was adding the Prepared Statement to insert the data and my db connection is as follows:
$conn = new mysqli($server, $user, $password, $database) or die('Could not connect: ' . mysql_error());
I am a noob with php so any help would be greatly appreciated.
You may have a fundamental misunderstanding: When you switch to mysqli, your mysql_* functions will no longer use the same connection.
LAST_INSERT_ID() works on a per-connection basis.
You will have to make that query using the same library/connection as the main query, or as #zerkms points out, use $conn->insert_id.
mysqli_insert_id().