Having Issue On MySQLi LOAD DATA INFILE to Load .CSV File - php

I am trying to load a 40 mg .CSV file into MYSQL database using MySQLi and PHP but I am getting only The user update failed: message (Witdout Error Message!) after loading the page
<?PHP
define ( 'DB_HOST', 'localhost' );
define ( 'DB_USER', 'root' );
define ( 'DB_PASS', '' );
define ( 'DB_NAME', 'map' );
$con = new mysqli(DB_HOST,DB_USER,DB_PASS,DB_NAME);
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
};
$sql = "LOAD DATA INFILE 'C:/wamp/www/UP/Modified_Single.csv'
INTO TABLE `single-tbl`
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES;";
$result = mysqli_query($con, $sql);
if (mysqli_affected_rows($con) == 1) {
$message = "The data was successfully added!";
} else {
$message = "The user update failed: ";
$message .= mysqli_error($con);
};
echo $message;
mysqli_close($con);
Can you please let me know why this is happening?

Your test for success is wrong. mysqli_affected_rows() returns the number of rows that were inserted into the table, which should be the same as the number of lines in the CSV file. I doubt a 40-meg file is only 1 line, so testing this for == 1 is wrong.
If you want to know if the query was successful, test $result.
if ($result) {
$message = "The data was successfully added!";
} else {
$message = "The user update failed: " . mysqli_error($con);
}

Related

What am I missing with PHP code or CSV file?

'LINES TERMINATED BY' PROBLEM - PHP, SQL database, CSV file'
Im learning how to update database with PHP, from local file.
It is working fine, except just the last entry from my CSV file is inserted into all fields in database table.
I've tried inspecting CSV with HEX, and also all thinkable versions of LINES TERMINATED BY (\r\n, \n etc.).
Here is my CSV file
This image shows the code was executed properly.
https://imgur.com/p13OAoj
Here is my PHP code:
$dbhost = 'hidden';
$dbuser = 'hidden';
$dbpass = 'hidden';
$dbname = 'hidden';
// connect to the database
$conn = mysqli_connect($dbhost, $dbuser, $dbpass, $dbname);
mysqli_options($conn, MYSQLI_OPT_LOCAL_INFILE, true);
// check connection
if(!$conn){
echo 'Connection error: '. mysqli_connect_error();
}
// create temporary table
$create = 'CREATE TABLE tmp
(
name_id varchar(255),
phone varchar(255),
INDEX (name_id)
)';
// check if creation was successful
if(mysqli_query($conn, $create)){
echo "Records were updated successfully CREATE.";
} else {
echo "ERROR: Could not able to execute $create. " . mysqli_error($conn);
} ?><br><br><?php
// load data from local file into temp table
$load = "LOAD DATA LOCAL INFILE 'try1.csv'
INTO TABLE tmp
FIELDS TERMINATED BY ';'
LINES TERMINATED BY '\r\n' IGNORE 1 LINES
(#name_id, #phone)" ;
// check if load was successful
if(mysqli_query($conn, $load)){
echo "Records were updated successfully LOAD.";
} else {
echo "ERROR: Could not able to execute $load. " . mysqli_error($conn);
} ?><br><br><?php
// set/update temp table with info from csv file
$set = "UPDATE tmp SET
name_id = #name_id,
phone = #phone "
;
// check if update was successful
if(mysqli_query($conn, $set)){
echo "Records were updated successfully SET.";
} else {
echo "ERROR: Could not able to execute $set. " . mysqli_error($conn);
}
mysqli_close($conn);
I would expect all 4 entries in CSV file to be inserted, instead I get the last rows values in all 4 entries in database.
This image shows the result in my database table.
https://i.imgur.com/2lKB2Ix.png
You need to modify two things in your code:
You don't need to update your data if you just have added the data.
The names of the columns in your load data are incorrect (you shoulkd remove the #), so you are inserting null lines. (If you only have those two columns in your table, you don't need to specify the names, so you can remove them too.)
Modify your code as:
$dbhost = 'hidden';
$dbuser = 'hidden';
$dbpass = 'hidden';
$dbname = 'hidden';
// connect to the database
$conn = mysqli_connect($dbhost, $dbuser, $dbpass, $dbname);
mysqli_options($conn, MYSQLI_OPT_LOCAL_INFILE, true);
// check connection
if(!$conn){
echo 'Connection error: '. mysqli_connect_error();
}
// create temporary table
$create = 'CREATE TABLE tmp
(
name_id varchar(255),
phone varchar(255),
INDEX (name_id)
)';
// check if creation was successful
if(mysqli_query($conn, $create)){
echo "Records were updated successfully CREATE.";
} else {
echo "ERROR: Could not able to execute $create. " . mysqli_error($conn);
} ?><br><br><?php
// load data from local file into temp table
$load = "LOAD DATA LOCAL INFILE 'try1.csv'
INTO TABLE tmp
FIELDS TERMINATED BY ';'
LINES TERMINATED BY '\r\n' IGNORE 1 LINES
(name_id, phone)" ;
// check if load was successful
if(mysqli_query($conn, $load)){
echo "Records were updated successfully LOAD.";
} else {
echo "ERROR: Could not able to execute $load. " . mysqli_error($conn);
} ?><br><br><?php
mysqli_close($conn);

Issue with loading large CSV file into MySql database with mysqli

I have a database that I am loading a large csv file into. Until recently the script below worked without any issues, and I have not made any changes to the script itself, however it has suddenly stopped working.
An example of the CSV file is
"Offer.ID","Offer.PhoneCost","Offer.TotalCost","Offer.MonthlyCost","Offer.FreeGift","Offer.FreeGiftImage","Offer.FreeGiftCategory","Offer.FullFreeGift","Offer.OfferCashback","Offer.AutoCashback","Offer.Clearance","Offer.OfferMins","Offer.OfferTxts","Offer.OfferRental","Offer.OfferLength","Offer.OfferText","Offer.Link"
"8676820","0.00","165.00","13.75","","","","15.00 Guaranteed Cashback","15.00","1","0","0","0","15.00","0","£15.00 Automatic Cashback","http://www.urmob.co.uk/t/a/psav89/URMOB-xmakex-xmodelx-xtariffx/track.php%253fid=8676820"
and the PHP script I'm using is...
<?php
require_once 'dbconnect.php';
$dbh = db_connect();
$log = fopen("database-log.txt", "w");
$time = date("Y-m-d H:i:s");
//start the log
$logstart = "------------------------------------------------\n" . $time . " log start\n------------------------------------------------\n\n";
fwrite($log, $logstart);
//clean the temp table
$query = "TRUNCATE TEMPDATA";
$trunc = mysqli_query($dbh, $query);
if ($trunc) {
fwrite($log, "TEMPDATA cleared\n");
} else {
fwrite($log, "TEMPDATA failed\n");
}
//load affiliate window csv
$query = "load data local infile '/home/data.csv' into table rim6jtvnox6vmwxk.TEMPDATA FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n' IGNORE 1 ROWS";
$update = mysqli_query($dbh, $query);
fwrite($log, $update);
if ($update) {
fwrite($log, "TEMPDATA Table updated with CSV\n");
} else {
fwrite($log, "Failed CSV update - " . mysqli_error($dbh) . "\n");
}
mysqli_close($dbh);
$endtime = date("Y-m-d H:i:s");
$logend = "\n\n------------------------------------------------\n" . $endtime . " log end\n------------------------------------------------\n";
//end the log
fwrite($log, $logend);
fclose($log);
?>
I know that the connection to the database is working without any issues, and can see the database being cleared. But the load script doesn't seem to execute and doesn't give any error from mysqli_error either.

TXT to SQL that automatically removes that line of txt once complete

I have a wordpress plugin that exports form entries to a txt file. So I need to write a php script to add them to a sql database as I want the submissions added to a database on a different domain (otherwise I’d just get the plugin to do it for me). I’m fine about how I get it to connect to the database, it’s just how I code it to interpret the data as the column names are always next to the field as shown.
{"Entry_ID":"235","Name":"matt","Email":"matt#gmail.com","Date":"03/10/2017"}{"Entry_ID":"236","Name":"matt","Email":"matt#btinternet.com","Date":"10/10/2017"}
Is there a way to get it to ignore the column name and only interpret the data within the “” after the : ?
Once these have been added to the sql database I would then need to get the lines removed from the txt
So far I have this but it isn't working...
$file= fopen('http://mpcreations.staging.wpengine.com/wp-content/themes/red-seal-resources/test.txt', 'r');
while (($data = fgetcsv($file)) !== FALSE) {
$object = json_encode($data[0]);
$servername = "";
$username = "";
$password = "";
$dbname = "";
// Create connection
$conn = mysqli_connect($servername, $username, $password, $dbname);
// Check connection
if (!$conn) {
die("Connection failed: " . mysqli_connect_error());
}
$query = "INSERT INTO 'wp_forms' LINES TERMINATED BY '\n';
if (mysqli_multi_query($conn, $query)) {
echo "New records created successfully";
} else {
echo "Error: " . $query . "<br>" . mysqli_error($conn);
}
mysqli_close($conn);
}
Any help would be greatly appreciated.
Thank you
Each line in the txt file has JSON data? Process the txt file, parse the data and INSERT it into the database table.
$file= fopen('file.txt', 'r');
while (($data = fgetcsv($file)) !== FALSE) {
$object = json_encode($data[0]);
// Prepare INSERT query here...
}

LOAD DATA LOCAL INFILE only enters 5 line

I have a csv file and I am using LOAD DATA LOCAL INFILE command to load it inside mysql, but I've got no idea why it only reads 5 lines of it.
Here is my php code:
<?php
include "../config.php";
$conn = new mysqli( $servername, $username, $password, $dbname );
if ( $conn->connect_error ) {
die( "Connection failed: " . $conn->connect_error );
} //$conn->connect_error
$sql= "LOAD DATA LOCAL INFILE 'uploads/cell.csv'
INTO TABLE table3
CHARACTER SET UTF8
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'";
$result = $conn->query( $sql );
echo("Error description: " . mysqli_error($conn));
?>
and the csv file is attached in link below.
cell.csv
Mysql after executing the php code looks like this:
I finally figured it out by myself, the problem was /r/n, somr of lines finish with /r and some finish with /r/n, i chenged the code to:
<?php
include "../config.php";
$conn = new mysqli( $servername, $username, $password, $dbname );
if ( $conn->connect_error ) {
die( "Connection failed: " . $conn->connect_error );
} //$conn->connect_error
$sql= "LOAD DATA LOCAL INFILE 'uploads/cell.csv'
INTO TABLE table3
CHARACTER SET UTF8
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r'";
$result = $conn->query( $sql );
echo("Error description: " . mysqli_error($conn));
?>
and it fixed the problem.

How can I create CSV files with a secure/unique separation of fields?

I create a CSV file:
$newFileName = "myfile.csv";
$fileHandle = fopen($newFileName,"w");
foreach ($data as $line){
fputcsv($fileHandle, array($line['id'], $line['name'],$line['path']));
}
fclose($fileHandle);
if (!empty($fileHandle)) {
echo "CSV was successfully created";
} else {
echo "Error creating CSV";
}
and store it into the mySQL database:
$sql = "LOAD DATA LOCAL INFILE 'myfile.csv'
INTO TABLE test
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(id,name,path)";
$con=mysqli_connect("localhost","dbuser","dbpassword","dbname");
if (mysqli_connect_errno()) {
echo "Failed to connect to MySQL: " . mysqli_connect_error();
};
$result = mysqli_query($con, $sql);
if (mysqli_affected_rows($con) == 1) {
$message = "The data was successfully added!";
} else {
$message = "The user update failed: ";
$message .= mysqli_error($con);
};
echo $message;
mysqli_close($con);
To me the separation with "comma" doesn't seem to be very secure. I am afraid, if in the path for example, there will be a "comma" somewhere the data will be stored incorrect into the database. Can you suggest me a unique or more secure separation method of fields for the cvs file?

Categories