This question already has answers here:
Import CSV file directly into MySQL
(6 answers)
Closed 9 years ago.
I have a CSV file that I'm wanting to import into a database, and then export the same data with omitted columns. How would I go about doing this? Any suggestions?
<?php
$databasehost = "localhost";
$databasename = "test";
$databasetable = "sample";
$databaseusername="test";
$databasepassword = "";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "filename.csv";
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator));
echo "Loaded a total of $affectedRows records from this csv file.\n";
?>
Related
I am using the below php script and can't figure out why it is not working. I do get the echo that it processed with no errors in terminal, but the table remains empty. Is there something incorrect with the code causing this. I confirmed it is locating the file in the directory as well. Hopefully something simple is missing.
<?php
try {
$inputfile1 = '////MainDirectory/SubDiretory/data.csv';
$table1 = 'database.testTable1';
$time1 = microtime(true);
require_once("configFile.php"); //this changes the below as now this is the reference point
$mysql_host = DB_HOST;
$mysql_database = DB_NAME;
$mysql_username = DB_USER;
$mysql_password = DB_PASS;
$db = new PDO("mysql:host=$mysql_host; dbname=$mysql_database", $mysql_username, $mysql_password);
// set the PDO error mode to exception
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$db->exec("SET CHARACTER SET utf8"); // Sets encoding UTF-8
} catch (Exception $e) {
die("Unable to connect: " . $e->getMessage());
}
try {
// Return errors
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
// Begin transaction
$db->beginTransaction();
if(file_exists($inputfile1)) {
$Found = 'found';
// Query D2 Load Temp table
$tempA2 = $db->prepare("LOAD DATA INFILE :inputfile1
INTO TABLE $table1
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(column1, column2, column3);");
$tempA2->bindParam(':inputfile1', $inputfile1);
$tempA2 -> execute();
}
} catch (Exception $e) {
// If transaction fail, use checkpoint and rollback
$db->rollBack();
echo "Conversion failed: " . $e->getMessage().'<br />';
file_put_contents('PDOErrors.txt', $e->getMessage(), FILE_APPEND);
}
//time of script
//sleep(1);
$timeC = microtime(true) - $_SERVER["REQUEST_TIME_FLOAT"];
echo "Full Process Time: {$timeC} $Found";
?>
CSV format:
"HeaderData1","HeaderData2","HeaderData3","HeaderData4","HeaderData5"
"DataA","DataB","DataC"
"Data2","Data2","Data2"
MySQL table format
id INT PK with AI and NN
column1 Varchar
column2 Varchar
column3 Varchar
Thank you for any help
This question already has answers here:
PHP: How to check if image file exists?
(22 answers)
Closed 5 years ago.
Can someone please help me to make this script check if the file exists, before it truncate the table.
If filename not exists, I want to stop the import.
<?php
//set the connection variables
$hostname = "host";
$username = "username";
$password = "pass";
$database = "database";
$filename = "filename.csv";
//connect to mysql database
$connection = mysqli_connect($hostname, $username, $password, $database) or die("Error " . mysqli_error($connection));
mysqli_query($connection, "TRUNCATE TABLE `my_tablename`");
// open the csv file
$fp = fopen($filename,"r");
//parse the csv file row by row
while(($row = fgetcsv($fp,"500",",")) != FALSE)
{
//insert csv data into mysql table
$sql = "INSERT INTO Pristabell (Produkt, Pris, Rabattkr, Rabattprosent, Lagerstatus, Butikk, TAGS) VALUES('" . implode("','",$row) . "')";
if(!mysqli_query($connection, $sql))
{
die('Error : ' . mysqli_error($conection));
}
}
fclose($fp);
//close the db connection
mysqli_close($connection);
?>
Thanks :-)
http://php.net/manual/en/function.file-exists.php
if(file_exists($pathtofile)){
//do import
}else{
//stop
}
One simple solution is use
file_exist;
in an if() chunk.
Before the while(), if true continue else exit or trow an exception.
I am using the below code to import a huge csv file.(almost 42452 lines):
<?php
$databasehost = "localhost";
$databasename = "framework_db";
$databasetable = "Main_CSV_Table";
$databaseusername="root";
$databasepassword = "";
$fieldseparator = ",";
$lineseparator = "";
$csvfile = "main.csv";
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)."
INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator)."
IGNORE 1 LINES
(Code,Name,Trx,DocType,Date,Period,Ref,Description,Category,Dr,Cr)
;");
echo "Loaded a total of $affectedRows records from this csv file.\n";
?>
Table Structre:
But when I import this csv to mysql, sometimes the data is inserted in wrong columns like below:
Also there is an another issue. When I import the file of 42452 lines, it only imports 38777 lines.(upload_max_filesize = 500M is in php.ini)
I would be grateful if someone could help me on this problem?
Thanks in advance!
UPDATE
CSV File is give below:
I'm trying to Upload a CSV-File into a MySQL Database via the PHP PDO function. The CSV is uploaded by a html form and saved on the server. After this I do some validation (I have exclude this from the code below). However I was pretty sure it was functional and I uploaded several times successful a CSV-Files, but after a week it stopped working with the following error code:
Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 7890 Can't find file
This is the code im using:
$csv_upload_status = "none";
$sec_pass = "4220334";
$databasehost = "127.0.0.3";
$databasename = "database_name";
$databasetable = "table_name";
$databaseusername="database_user";
$databasepassword = "XXXXXXXXXXXX";
$fieldseparator = ",";
$lineseparator = "\r";
if(isset($_POST["submit"])) {
$valid = 1;
$target_dir = "schedule-uploads/";
$target_file = $target_dir . basename($_FILES["csv"]["name"]);
$FileType = pathinfo($target_file,PATHINFO_EXTENSION);
if ($valid == 1) {
move_uploaded_file($_FILES["csv"]["tmp_name"], $target_file);
$csvfile = basename( $_FILES["csv"]["name"]);
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator)."
(#date, show_time, show_title, show_plot)
SET show_date = STR_TO_DATE(#date, '%m/%d/%Y');");
$csv_upload_status = "success";
}
}
I have a php file that imports a CSV file into my database. Since my hosting provider upgraded to PHP 5.4, I get an error on the line 'LINES TERMINATED BY ', but I am not sure why. Here is my code:
<?php
require('phpsqlajax_dbinfo.php');
$databasehost = $db_host;
$databasename = $db_name;
$databasetable = "tbl_csvImport";
$databaseusername=$db_user;
$databasepassword = $db_pass;
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "doecsv.csv";
echo $lineseparator;
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$pdo->exec("DELETE FROM `$databasetable`");
try {
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".preg_replace('/"[^"]+"/','',$pdo->quote($csvfile))." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
OPTIONALLY ENCLOSED BY ".$pdo->quote('"')."
LINES TERMINATED BY ".$pdo->quote($lineseparator)."IGNORE 1 LINES");
} catch (Exception $e) {
die ("CSV Parse Failed: ".$e->getMessage()." | Error on line: ".$e->getLine());
}
echo" Success. <br/>";
?>
Error Code: SQLSTATE[42000]: Syntax error or access violation: 1148 The used command is not allowed with this MySQL version | Error on line: 35
I found a workaround that works for me:
http://crewow.com/CSV-Importer-in-MySQL.php