I have been trying to create a database by using the first row of a CSV file, however I keep getting an error about my PDO syntax. Apparently something is going wrong with varchar(250) in the $columns variable:
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'varchar(250))' at line 1
The code is below:
<?
/********************************************************************************/
// Parameters: filename.csv table_name
$argv = 'seo_domains.csv seo_domains';
if($argv[1]) { $file = $argv[1]; }
else {
echo "Please provide a file name\n"; exit;
}
if($argv[2]) { $table = $argv[2]; }
else {
$table = pathinfo($file);
$table = 'seo_domains';
}
/********************************************************************************/
// Get the first row to create the column headings
$fp = fopen('seo_domains.csv', 'r');
$frow = fgetcsv($fp);
$ccount = 0;
foreach($frow as $column) {
$ccount++;
if($columns) $columns .= ', ';
$columns .= "$column varchar(250)";
}
$qry = $dbcon->prepare("CREATE TABLE if not exists $table ($columns);");
$qry->execute();
/********************************************************************************/
// Import the data into the newly created table.
$file = $file;
$qry = $dbcon->prepare("load data infile '$file' into table $table fields terminated by ',' ignore 1 lines");
$qry->execute();
?>
I changed the $fp fopen to a static value instead of using $file because apparently the guy who coded the code mentioned above, did not mention how to format the $argv variable. Yes, I already tried formatting according to his "Parameters" comment on the first line but still, no avail. I also statically changed the $table variable to 'seo_domains' since the $argv variable is not being split properly. Instead of re-coding the above code, I am wondering if anyone has any thoughts as to why my database would be resulting in the described error. Any help is appreciated. Just trying to create a table based on first row of the CSV file provided. Upon creation I would like to continue to insert all the row values below row 1 in the CSV, per usual.
try:
CREATE TABLE if not exists seo_domains (Domain varchar(250), Server varchar(250), IP varchar(250), Username varchar(250), Password varchar(250), Nameserver varchar(250), NameCheap varchar(250), GA varchar(250), WMT varchar(250), SB varchar(250), GAW varchar(250), notes varchar(250), BlankField varchar(250), id INT KEY NOT NULL AUTO_INCREMENT varchar(250));
key is a reserved word in SQL. See image below
As you may find that some of the column names in CSV could be reserved words you will need to escape them. In MySQL backticks are used. ie `key`
Related
Sorry if this has been asked before, but I couldnt find anything that would relate to my case here on SE.
I am trying to import a CSV file into my Mysql database table with both the table the CSV having the exact same amount and order of columns, except that the table's column ID is not missing in the CSV file.
What I want to achieve is to import the CSV into the table while generating an ID number that automatically increases with each record. This does not seem possible as the CSV always seem to want to insert its data into the first colum in the table, but in my case I would need it to be the 2nd column.
How do I approach this and is there any reference code I can study? I currently am working off this PDO approach but am having the above mentioned difficulties.
PHP
<?php
$databasehost = "localhost";
$databasename = "test";
$databasetable = "sample";
$databaseusername="test";
$databasepassword = "";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "filename.csv";
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator));
echo "Loaded a total of $affectedRows records from this csv file.\n";
?>
Thank you
You can have MySQL set values for certain columns during import. If your id field is set to auto increment, you can set it to null during import and MySQL will then assign incrementing values to it.
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator))."
SET id=null;
EDIT - In case the ID column is not present in CSV
The col1, col2, col3,... are names of actual columns in the DB table (without id column)
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator))."
(col1, col2, col3,...)
SET id=null;
The AUTO_INCREMENT attribute can be used to generate a unique identity for new rows. Most version of mysql and engin support this. You need not worry about the ID and can use cron job to insert the needed field and AUTO_INCREMENT will take care of the id itself.
No value was specified for the AUTO_INCREMENT column, so MySQL assigned sequence numbers automatically. You can also explicitly assign 0 to the column to generate sequence numbers, unless the NO_AUTO_VALUE_ON_ZERO SQL mode is enabled. If the column is declared NOT NULL, it is also possible to assign NULL to the column to generate sequence numbers. When you insert any other value into an AUTO_INCREMENT column, the column is set to that value and the sequence is reset so that the next automatically generated value follows sequentially from the largest column value.
You can retrieve the most recent automatically generated AUTO_INCREMENT value with the LAST_INSERT_ID() SQL function or the mysql_insert_id() C API function. These functions are connection-specific, so their return values are not affected by another connection which is also performing inserts.
See example from official link :
[https://dev.mysql.com/doc/refman/5.7/en/example-auto-increment.html]
As you want to recreate the table over and over and want to manipulate the Data from the CSV, try this:
// You have to create the TABLE if not exists
$pdo->exec("TRUNCATE TABLE sample"); // No need to drop the table if columns don't change.
$csvContent = file_get_contents($csvfile); // Raw Data from file
$lines = explode("
", $csvContent); // The standard line separator is an ENTER
// Now you have each line separated
for($i = 0; $i < coount($lines); $i++) {
$col = explode(";", $lines[$i]); // Would be a comma
// Now you have each column separated
$pdo->exec("INSERT INTO sample (id, col1, col2, col3 ... coln) VALUES (NULL, '".$col[0]."', '".$col[1]."', '".$col[2]."' ... '".$col[n]."')");
}
This way you can dig into your Data and, besides setting an AUTO_INCREMENT ID, you can validate what is coming from the CSV and can correct/prevent importation errors.
I have an array containing column names of a table i want to create which i got from an excel sheet.
I tried something like this
$sql = "CREATE TABLE IF NOT EXISTS ".$month."-".date('Y')."(
".foreach($tableColumnNames as $columnName){
echo $columnName." VARCHAR(200) NULL,";
}
."
)";
It returns error saying Parse error: syntax error, unexpected 'foreach' same thing with while loops
How can i create Table with column names i have in an array. And How can i set each column data type unique (INT,VARCHAR,..)
you want to build the sql string like so:
$sql = "CREATE TABLE IF NOT EXISTS ".$month."-".date('Y')."(";
foreach($tableColumnNames as $columnName){
$sql .= $columnName." VARCHAR(200) NULL,";
}
$sql=rtrim($sql,',');//remove last comma
$sql .=")";
Am wondering if someone can validate my code,
I seem to have a bit of a issue when running this code on a Linux server.
If I run this local host (Xammp) This runs through without any errors, no timing out.
Script created database and populated from TXT or CSV.
When I move the scrip to the web server to run
I end up with the following error:
Error creating table: Table 'Sales' already exists
Error: LOAD DATA INFILE 'Salest.CSV' REPLACE INTO TABLE `Sales` FIELDS TERMINATED BY ',' IGNORE 1 LINES
Access denied for user 'USERNAME'#'%' (using password: YES)
I get a access denied? but if I delete the table:
and re-run the code:
Table Sales created successfully
Error: LOAD DATA INFILE 'Salest.CSV' REPLACE INTO TABLE `Sales` FIELDS TERMINATED BY ',' IGNORE 1 LINES
Access denied for user 'USERNAME'#'%' (using password: YES)
Of course the username and password are correct, as it has created the table within the database, Why am I experiencing an issue when populating it?
What am I missing?
<?php
ini_set("memory_limit",-1); // Unlimited Memory handle this properly just quick internal testing
// Lets start by connecting to the Database
$databasehost = "localhost"; // Connection Local? Remote?
$databasename = "DBNAME"; // Database Name
$databasetable = "DBTABLE"; // Database Table
$databaseusername ="USERNAME"; // Database Username
$databasepassword = ""; // Database Password
$filename = "Sales.txt"; // File we will import (This will be picked up from FTP
$fieldseparator = "\t"; // Field Deliminator / separator
$lineseparator = "\n"; // New Line Deliminator / separator
/********************************/
$addauto = 0; // add an empty field at the beginning of these records 1 = yes / 0 = no
/********************************/
/********************************/
$save = 0; // Convert the file to a SQL file 0 = Import to MYSQL or 1 = Save to SQL File
$outputfile = "output.sql"; // The name of the SQL file you are going to save
/********************************/
// As the above is configurable, we should use a config file for the above, and use the code beloew in static file
// below code can be one one file, and we can use a FTP Collect routine and call this after.
if (!file_exists($filename)) { // Does the file exist?
echo "File not found. Make sure you specified the correct path.\n"; // Print message if it does not
exit; // Exit out of it
}
$file = fopen($filename,"r"); // open the CSV or TXT file (Test if this works on TXT may need re-formatting)
if (!$file) { // statement if problem opening file
echo "Error opening data file.\n"; // Print the error message
exit; // exit out
} // end of statement
$size = filesize($filename); //check file exists but has a valid size (not 0kb)
if (!$size) { // If the size is 0kb i.e. not valid then error
echo "File is empty.\n"; // rpint the error message
exit; // exit out
} // end of statement
$filecontent = fread($file,$size); //Declare varable CSV file - Read in the file and size
fclose($file); // close the file
// Now lets connect to the database and select the database we will work from
$conn = new mysqli($databasehost, $databaseusername, $databasepassword, $databasename);
// Check connection
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
// Now we are connected to the Database, Lets create the table we will start to import data too.
// List of colums we will be using in the Sales Table:
//uniquekey, type, itemno, styleno, styledesc, variatdesc, date, hour, day, month, year, locn, dept2, dept1, dept0, depta, season, reason, supplier, brand, fullprice, soldat, cost, margin, qty, cashier, custkey, stockkey
$sqlcreate = "CREATE TABLE $databasetable (
uniquekey Int(11) PRIMARY KEY,
type VARCHAR(20) NULL,
itemno Int(11) NULL,
styleno Int(11) NULL,
styledesc VARCHAR(40) NULL,
variatdesc VARCHAR(20) NULL,
date DATE,
hour Int(11) NULL,
day Int(11) NULL,
month Int(11) NULL,
year Int(11) NULL,
locn VARCHAR(20) NULL,
dept2 VARCHAR(20) NULL,
dept1 VARCHAR(20) NULL,
dept0 VARCHAR(20) NULL,
depta VARCHAR(20) NULL,
season VARCHAR(20) NULL,
reason VARCHAR(20) NULL,
supplier VARCHAR(20) NULL,
brand VARCHAR(20) NULL,
fullprice FLOAT(9,3) NULL,
soldat FLOAT(9,3) NULL,
cost FLOAT(9,3) NULL,
margin FLOAT(9,3) NULL,
qty Int(11) NULL,
cashier VARCHAR(40) NULL,
custkey Int(11) NULL,
stockkey VARCHAR(40) NULL
)";
if ($conn->query($sqlcreate) === TRUE) {
echo "Table <b>" . $databasetable ."</b> created successfully";
} else {
echo "Error creating table: " . $conn->error;
}
// exit ; // Exit was used to end the script early, Write the code, and test the code in 2 sections.
// Now the table is created, How about grabbing a text file for an example and seeing if we can put it into the SQL
// So Another SQL Query Instead of using INTO Table ****(Tablename)**** We will use REPLACE INTO TABLE
// So if the same data is re-sent up, it will replace its original, Need to check this method does not have issues
// down the line with Indexs, If so, then the replace statement could be met with When log_time is > than original
$sqlimport = "
LOAD DATA INFILE '$filename'
REPLACE INTO TABLE `$databasetable`
FIELDS TERMINATED BY '$fieldseparator'
IGNORE 1 LINES
" ;
if ($conn->query($sqlimport) === TRUE) {
echo "<br>Data Imported from " .$filename." OK!";
} else {
echo "<br>Error: " . $sqlimport . "<br>" . $conn->error;
}
?>
So after alot of trial and error, i was able to resolve the issue.
$sqlimport = "
LOAD DATA INFILE '$filename'
replaced with:
$sqlimport = "
LOAD DATA LOCAL INFILE '$filename'
Give me a new error Along the lines of Local command not allow with MySQL Version.
SQL Statement: SHOW VARIABLES LIKE 'local_infile'
Give me the result of: local_infile OFF
Bingo!!! Changed to On, and rebooted and script works as expected.
Thanks for your help and suggestions.
So I have a CSV file that I'm trying to make into a table.
I gave up on the import GUI after too many errors, and am trying to accomplish the import through a php file.
//create table with KNOWN values
mysql_query("CREATE TABLE uri_faculty
(
id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY(id),
id INT(15),
lname VARCHAR(50),
fname VARCHAR(50),
mi VARCHAR(3),
Spc_Title VARCHAR(50),
title VARCHAR(50),
deptid INT(5),
dept VARCHAR(50),
degree1 VARCHAR(10),
earned1 INT(4),
school1 VARCHAR(75),
degree2 VARCHAR(10),
earned2 INT(4),
school2 VARCHAR(75),
degree3 VARCHAR(10),
earned3 INT(4),
school3 VARCHAR(75),
degree4 VARCHAR(10),
earned4 INT(4),
school4 VARCHAR(75),
degree5 VARCHAR(10),
earned5 INT(4),
school5 VARCHAR(75),
degree6 VARCHAR(10),
earned6 INT(4),
school6 VARCHAR(75)
)");
//Get CSV file
$getfile = 'faculty_delim2.csv';
$csvfopen = fopen($getfile, "r");
//loop to fill csvget with arrays
for($i=0;!feof($csvfopen);$i++){
$array = fgetcsv($csvfopen);
$insert = implode("','", $array);
//to exclude the first 2 lines (titles of document)
if($i>=1){
//values to be inserted into SQL are displayed
//echo var_dump($array[$i])." <br> ";
$sqlval = $insert;
// var_dump($sqlval);
//the while loop will constantly place values into the database until the file is finished
mysql_query("INSERT INTO uri_faculty (id,lname,fname,mi,Spc_Title,title,deptid,dept,
degree1,earned1,school1,
degree2,earned2,school2,
degree3,earned3,school3,
degree4,earned4,school4,
degree5,earned5,school5,
degree6,earned6,school6,) VALUES ('$sqlval')
");
}
}
fclose($csvfopen);
echo "complete";
?>
I keep getting an error saying that implode is receiving incorrect parameters, yet every bit of documentation I've found says that I am correct.
I changed the permissions of the file, and it is in the right place.
Instead of this:
for($i=0;!feof($csvfopen);$i++){
$array = fgetcsv($csvfopen);
I would write this:
while ($array = fgetcsv($csvfopen)) {
The loop will automatically finish when there are no more rows to read. Your error is probably an edge case, where the file still thinks it is not at feof but in spite of that, there are no more rows, and fgetcsv returns false.
By the way, two other issues:
You will run into another error soon:
. . . degree6,earned6,school6,) VALUES . . .
You must not put a comma after the last column. Write this instead:
. . . degree6,earned6,school6) VALUES . .
You are wide open to SQL injection issues. What happens when one of your CSV fields contains an apostrophe? You should learn how to use PDO with query parameters. Or failing that, use escaping:
$insert = implode("','", array_map('mysql_real_escape_string', $array));
Finally, you should consider skipping fgetcsv and use LOAD DATA INFILE. Then all your CSV issues, and escaping issues just go away.
mysql_query("LOAD DATA LOCAL INFILE 'faculty_delim2.csv' INTO TABLE uri_faculty IGNORE 2 LINES");
So, a snippet of my code which is resulting in an error is :
$con = mysqli_connect('localhost', 'root', '', 'notesDB');
if(isset($_POST['tableName'])) {
$tName = htmlentities($_POST['tableName']);
$firstQuery = mysqli_query($con,"INSERT into notes(Title) VALUES( '$tName'); CREATE TABLE $tName(id int NOT NULL AUTO_INCREMENT, Title varchar(20) NOT NULL, Description varchar(100), PRIMARY KEY(id));");
if($firstQuery){
header("Location: create2.php");
}
else
echo mysqli_error($con);
}
The output of this is :
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CREATE TABLE test1(id int NOT NULL AUTO_INCREMENT, Title varchar(20) NOT NULL, D' at line 1
Well, the funny thing is that the exact code (except the variable - I just removed the $ sign) executed perfectly in phpMyAdmin.
Also, to prove that there is nothing really wrong with the php, the query executed without any error when it was only the INSERT query (and not the CREATE query).
mysqli_query can only perform one query at a time.
Try mysqli_multi_query instead.
As an aside creating tables on the fly is usually a sign of larger design issues. Schema should be relatively static while data should be dynamic.
You are trying to run two separate queries at a time in the code, which you can't run like that. You have to run them separately like below:
$con = mysqli_connect('localhost', 'root', '', 'notesDB');
if(isset($_POST['tableName'])) {
$tName = htmlentities($_POST['tableName']);
$firstQuery = mysqli_query($con,"INSERT into notes(Title) VALUES( '$tName')");
$secondQuery = mysqli_query("CREATE TABLE '$tName' (id int NOT NULL AUTO_INCREMENT, Title varchar(20) NOT NULL, Description varchar(100), PRIMARY KEY(id));");
if($firstQuery || $secondQuery){
header("Location: create2.php");
}
else
echo mysqli_error($con);
}
Your database architecture is wrong.
You shouldn't create tables on the fly. So, you have only register whatever new entity with simple regular INSERT query. And then use this entity's id to link records from another [already existing] table.
if(isset($_POST['tableName'])) {
$stm = mysqli_prepare($con,"INSERT into notes(Title) VALUES(?)");
$stm->bind_param("s",$_POST['tableName']);
$stm->execute();
}