I am trying to figure out will MySQL be enough for my use case. I tried inserting 100 000 rows into my local mysql server, which went fine. I saw that DB started to get populated with the data.
Then I run same insert script agains the Google Cloud SQL. Everything seemed also fine, but for some reason DB stopped inserting entries after the 67667 entry even though the response from the DB was that the insertion was successful.
Does MySQL has some kind of limitation, or what may cause this kind of behavior?
My test script:
// Create connection
$conn = new mysqli($servername, $username, $password);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$conn->select_db($database);
$insertData = '';
for ($i = 0; $i < 100000; $i++) {
$insertData .= "INSERT INTO table (isin) VALUES ('".uniqid()."');";
}
if ($conn->multi_query($insertData) === true) {
echo "New records created successfully";
} else {
echo "Error: <br>" . $conn->error;
}
$conn->close();
My test table has only two columns, id and the isin number.
Try using mysql Batch insert for e.g.
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
Build this part i.e. (1,2,3),(4,5,6),(7,8,9) in your loop and then use only one INSERT query
Try this way :
$val = "";
for ($i = 0; $i < 100000; $i++) {
$val .= "('".uniqid()."'),";
}
$val = rtrim($val,",");
$insertData = "INSERT INTO table (isin) values $val";
$conn->query($insertData);
Related
I am trying to design a scoreboard for a project for my local youth club and would like to be able to click a button onto a wordpress page that will run a PHP script to update a value by 1 in a sql db table, then i can grab the value and display it else where.
Not too worried about passwords being used in scripts at this will only be used within the local network that nobody else has access to, have looked around and found a few bits of code but im not able to actually get it working, here's what i've got so far.
<?php
$servername = "localhost";
$username = "username";
$password = "password";
$dbname = "sot";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$sql = UPDATE wp_sotstats SET CaptainChest=CaptainChest+1 WHERE id=1
if ($conn->query($sql) === TRUE) {
echo "New record created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
$conn->close();
?>
The database name is 'sot' the table i want to update is called 'wp_sotstats' and the field within the table is 'CaptainChest' i only need this to really work with just the one entry which the id is '1'
Any thoughts?
$sql = UPDATE wp_sotstats SET CaptainChest=CaptainChest+1 WHERE id=1
if ($conn->query($sql) === TRUE) {
echo "New record created successfully";
Is not creating record, it updating it. For me, I will take current value from database as Select, then do ++ to it and do Update query.
$sql = UPDATE wp_sotstats SET CaptainChest=CaptainChest+1 WHERE id=1
Please wrap it in quotes and finish statement with semicolon
I'm trying to grab the id of the last inserted auto-increment row and cannot successfully grab it.
error_reporting(E_ALL);
ini_set('display_errors', 1);
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
$title = mysqli_real_escape_string($conxn,$_POST['blog_title']);
$entry = mysqli_real_escape_string($conxn,$_POST['blog_entry']);
$sourceName = mysqli_real_escape_string($conxn,$_POST['blog_source_name']);
$sourceLink = mysqli_real_escape_string($conxn,$_POST['blog_source_link']);
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
$sql="INSERT INTO blog (blog_title, blog_entry, blog_source, blog_link)
VALUES ('$title','$entry','$sourceName','$sourceLink')";
$lastID = $mysqli->insert_id;
if (!mysqli_query($conxn,$sql)) {
die('Error: ' . mysqli_error($conxn));
}
When I echo $lastID a "0" is returned after every submit.
You need to place the $mysqli->insert_id() after the actual mysqli_query(). See below.
if (!mysqli_query($conxn,$sql)) {
die('Error: ' . mysqli_error($conxn));
}
$lastID = $mysqli->insert_id;
That said, there are other issues with your code. First & foremost, you are mixing up the Object oriented style of calling mysqli_* with the procedural style. For example the OOP method of $mysqli->real_escape_string equates to the procedural method of mysqli_real_escape_string.
So this:
$lastID = $mysqli->insert_id;
Should be this:
$lastID = mysqli_insert_id($conxn);
So without seeing the rest of your code, unclear how to handle. Know the difference & experiment. But here are my suggestions in good faith based on the code you have presented.
For example, your references to $_POST values do not have single quotes, so I added that. Also, since you are using double quotes—which handle string substitution—you can condense your INSERT variable setting by getting rid of the . concatenation.
$title = mysqli_real_escape_string($conxn, $_POST['blog_title']);
$entry = mysqli_real_escape_string($conxn, $_POST['blog_entry']);
$sourceName = mysqli_real_escape_string($conxn, $_POST['blog_source_name']);
$sourceLink = mysqli_real_escape_string($conxn, $_POST['blog_source_link']);
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
$sql="INSERT INTO blog (blog_title, blog_entry, blog_source, blog_link)
VALUES ('$title','$entry','$sourceName','$sourceLink')";
if (!mysqli_query($conxn,$sql)) {
die('Error: ' . mysqli_error($conxn));
}
$lastID = mysqli_insert_id($conxn);
That done, this code chunklet can be cleaned up even more, and this is how I would handle it. I have made an array of the $_POST values you are grabbing so you don’t have to repeat code. Also added comments to make it clearer what is happening. And I have used the procedural format for all commands here. If OOP is what you want, then you need to change all of the commands to match OOP format.
// Set all of the `$_POST` values into an array.
$post_items = array('blog_title','blog_entry','blog_source_name', 'blog_source_link');
// Roll through those values with a `foreach` loop.
foreach ($post_items as $post_item) {
$$post_item = mysqli_real_escape_string($conxn, $_POST[$post_item]);
}
// MySQL connection error check.
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
// Set the SQL values.
$sql = "INSERT INTO blog (blog_title, blog_entry, blog_source, blog_link)
VALUES ('$blog_title','$blog_entry','$blog_source_name','$blog_source_link')";
// Run the query.
if (!$mysqli_query($conxn, $sql)) {
die('Error: ' . mysqli_error($conxn));
}
// Get the last insert ID via object oriented method.
// $lastID = $mysqli->insert_id;
// Get the last insert ID via procedural method.
$lastID = mysqli_insert_id($conxn);
I have been tasked to regenerate every class code, but there are over 4,000 and I'd rather not do that manually. I decided to make a test database with the same structure just to mess with it before I actually began messing with the live server.
I am having an issue trying to run the same query multiple times. I'd like to run it multiple times because I have a randomly generated string (that is also being generated every time this runs) that needs to replace other strings.
Any help, guys?
for ($i = 0; $i < 8; $i++) {
$password = str_rand();
$sql = "UPDATE test "
. "SET classCode = '$password'".
. "WHERE id = '$i'";
}
mysql_select_db('workshe3_worksheetwonder');
$retval = mysql_query( $sql, $conn );
if(! $retval ) {
die('Could not update data: ' . mysql_error());
}
echo "All codes regenerated successfully.";
mysql_close($conn);
}
Your code should look like this
mysql_select_db('workshe3_worksheetwonder');
for ($i = 0; $i < 8; $i++) {
$password = str_rand();
$sql = "
UPDATE test
SET classCode = '$password'
WHERE id = '$i'";
$retval = mysql_query($sql, $conn) or die('Could not update data: ' . mysql_error());
}
echo "All codes regenerated successfully.";
mysql_query() is the function that needs to go in the loop, as it's the thing that's actually doing the updating. Your $sql is just a string with the command in it. If you don't run the mysql_query() in your loop then your string just gets overwritten without being used.
mysql_ functions are deprecated, meaning you shouldn't really use them for new code. There are some examples of alternative protocols if you follow that link.
I want to import data stored in a .csv file into mysql via php. The LOAD DATA INFILE query does not seem to work and i managed to write a code that will import the records one by one. Here is the code:
<?php
$arr = array(array(),array());
$num = 0;
$row = 0;
$handle = fopen("./import.csv", "r");
while($data = fgetcsv($handle,1000,",")){
$num = count($data);
for ($c=0; $c < $num; $c++) {
$arr[$row][$c] = $data[$c];
}
$row++;
}
$con = mysql_connect('localhost','root','password22');
mysql_select_db("security",$con);
for($i=1; $i<$row; $i++){
$sql = "INSERT INTO sls VALUES ('".$arr[$i][0]."','".$arr[$i][1]."','".$arr[$i][2]."','".$arr[$i][3]."','".$arr[$i][4]."','".$arr[$i][5]."')";
mysql_query($sql,$con);
}
?>
The problem is that I have about 300 000 records to import and when the records get too much, no record gets imported into the database and I get an error message. Is there anyway I can import the data faster or are there any similar statements like LOAD DATA INFILE I can use in PHP?
This may help you out
<?php
require_once 'reader.php';
$data = new Spreadsheet_Excel_Reader();
$data->setOutputEncoding('CP1251');
$data->read('filename.xls');
$con=mysqli_connect("localhost","username","password","dbname");
// Check connection
if (mysqli_connect_errno())
{
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
// Create table
$sql="CREATE TABLE tablename(
quote_number VARCHAR(100),
line_no VARCHAR(100),
item_no VARCHAR(100),
name VARCHAR(100),
unit VARCHAR(100),
rm VARCHAR(100),
capex VARCHAR(100))";
// Execute query
if (mysqli_query($con,$sql))
{
echo "Table tablename created successfully";
}
else
{
echo "Error creating table: " . mysqli_error($con);
}
for($x = 2; $x <= count($data->sheets[0]["cells"]); $x++)
{
//$sno = $data->sheets[0]["cells"][$x][1];
$quote_number = $data->sheets[0]["cells"][$x][1];
$line_no = $data->sheets[0]["cells"][$x][2];
$item_no = $data->sheets[0]["cells"][$x][3];
$name = $data->sheets[0]["cells"][$x][4];
$unit = $data->sheets[0]["cells"][$x][5];
$rm = $data->sheets[0]["cells"][$x][6];
$capex = $data->sheets[0]["cells"][$x][7];
$res = mysqli_query($con,"INSERT INTO tablename
(quote_number, line_no, item_no,name,unit,rm,capex) VALUES ('$quote_number','$line_no','$item_no','$name','$unit','$rm','$capex')");
mysqli_close($res);
}
?>
you can download reader.php file from here
Since you have not shared the LOAD DATA INFILE I assume most probably the issue is with FILE Privileges
You have to GRANT FILE privileges to the person loading the file(Mysql user connecting through username/password from php). Sth like this:
GRANT FILE on *.* TO 'mysql_user'#'localhost' ;
Since FILE is global privilege you cannot localize that to a specific database e.g dbname.* , just to note. To run the above command itself you should login to mysql from command line as root or as a user who has GRANT privileges.
Once that is done you should able to load the file using LOAD DATA INFILE. IF you are trying this in shared-hosting environment, your chances are very little. In that case you have to use the above method you tried , read the file, split the each line ,validate the line , insert into db table.
I have a very long list in plain text which I need to insert into a table my database. Do I have to manually input each line of my plain text document or is there a way to insert long lists into independent rows in a table using a query?
I have a table with 2 columns, id and club_name, club_name is the list which is plain text in a notepad document.
You can use LOAD DATA, e.g.:
mysql> LOAD DATA LOCAL INFILE '/path/pet.txt' INTO TABLE pet
-> LINES TERMINATED BY '\r\n';
You can also use the multi-row insert syntax (if you are using InnoDB tables), like this:
INSERT INTO yourtable VALUES (1,2), (5,5), ...
You can load data into mysql using the LOAD DATA INFILE command.
Here is the documentation...
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
$file = file("content.txt"); //read file line by line
foreach ($file as $val) {
if (trim($val) != '') { //ignore empty lines
mysql_query("INSERT INTO xxx SET club='" . $val . "'");
}
}
Here's a quick PDO based example, but MySQL's LOAD DATA INFILE command may be a much better option if you don't need to manipulate the source data (trim, normalise etc).
<?php
$conn = new PDO($dsn, $username, $password);
$stmt = $conn->prepare('INSERT INTO table VALUES (NULL, ?)');
foreach(file('list.txt') as $club) {
$stmt->execute(array($club));
}
Anthony.
Here is how to do it with php, mysqli and a text file that has each entry on a single line:
<?php
$servername = "localhost";
$username = "user";
$password = "pass";
$dbname = "db";
// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
$file = file("list.txt"); //read file line by line
foreach ($file as $val) {
if (trim($val) != '') { //ignore empty lines
$sql = "INSERT INTO `db`.`table` (`column`) VALUES ('$val');";
if ($conn->query($sql) === TRUE) {
echo "New record created successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
}
}
?>
This is based off of DanFromGermany's answer but updated to use mysqli.
Insert the whole notepad content to a column of type "text"