i am importing csv data into mysql database using codegniter framework. my database has two table promotion and promotion_product.promotion table is 3 fields name as promotion_id,name,start_dateand end_date.promotion_id is the primary key of promotion table.promotion_id is the auto incremment.promotion_product table is 5 fields id,promotion_id,sku,price,map. promotion_id is the foreign key of this table. i am importing csv file this table.csv field is sku,price and map.i am not getting how to insert promotion_id of promotion_porduct table.i am using last_insert_id.my code is below:
my model is:
public function save_csv($data,$data1) {
$this->db->insert('promotion', $data1);
$insert_id = $this->db->insert_id();
print_r($insert_id);
$infile = $data['upload_data']['full_path'];
//print_r($infile);
$sql = "LOAD DATA INFILE '" . $infile . "'
INTO TABLE promotion_product
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\r'
IGNORE 1 LINES
(promotion_id,sku, price, map)";
print_r($sql);
$query = $this->db->query($sql);
//var_dump($query);
return $query;
}
Yo have to do manually.
First insert into database then you will get last_insert_id and then read CSV file and create array of CSV and last_insert_id dump that array into a database.
Related
Sorry if this has been asked before, but I couldnt find anything that would relate to my case here on SE.
I am trying to import a CSV file into my Mysql database table with both the table the CSV having the exact same amount and order of columns, except that the table's column ID is not missing in the CSV file.
What I want to achieve is to import the CSV into the table while generating an ID number that automatically increases with each record. This does not seem possible as the CSV always seem to want to insert its data into the first colum in the table, but in my case I would need it to be the 2nd column.
How do I approach this and is there any reference code I can study? I currently am working off this PDO approach but am having the above mentioned difficulties.
PHP
<?php
$databasehost = "localhost";
$databasename = "test";
$databasetable = "sample";
$databaseusername="test";
$databasepassword = "";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "filename.csv";
if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}
try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename",
$databaseusername, $databasepassword,
array(
PDO::MYSQL_ATTR_LOCAL_INFILE => true,
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
)
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}
$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator));
echo "Loaded a total of $affectedRows records from this csv file.\n";
?>
Thank you
You can have MySQL set values for certain columns during import. If your id field is set to auto increment, you can set it to null during import and MySQL will then assign incrementing values to it.
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator))."
SET id=null;
EDIT - In case the ID column is not present in CSV
The col1, col2, col3,... are names of actual columns in the DB table (without id column)
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
LINES TERMINATED BY ".$pdo->quote($lineseparator))."
(col1, col2, col3,...)
SET id=null;
The AUTO_INCREMENT attribute can be used to generate a unique identity for new rows. Most version of mysql and engin support this. You need not worry about the ID and can use cron job to insert the needed field and AUTO_INCREMENT will take care of the id itself.
No value was specified for the AUTO_INCREMENT column, so MySQL assigned sequence numbers automatically. You can also explicitly assign 0 to the column to generate sequence numbers, unless the NO_AUTO_VALUE_ON_ZERO SQL mode is enabled. If the column is declared NOT NULL, it is also possible to assign NULL to the column to generate sequence numbers. When you insert any other value into an AUTO_INCREMENT column, the column is set to that value and the sequence is reset so that the next automatically generated value follows sequentially from the largest column value.
You can retrieve the most recent automatically generated AUTO_INCREMENT value with the LAST_INSERT_ID() SQL function or the mysql_insert_id() C API function. These functions are connection-specific, so their return values are not affected by another connection which is also performing inserts.
See example from official link :
[https://dev.mysql.com/doc/refman/5.7/en/example-auto-increment.html]
As you want to recreate the table over and over and want to manipulate the Data from the CSV, try this:
// You have to create the TABLE if not exists
$pdo->exec("TRUNCATE TABLE sample"); // No need to drop the table if columns don't change.
$csvContent = file_get_contents($csvfile); // Raw Data from file
$lines = explode("
", $csvContent); // The standard line separator is an ENTER
// Now you have each line separated
for($i = 0; $i < coount($lines); $i++) {
$col = explode(";", $lines[$i]); // Would be a comma
// Now you have each column separated
$pdo->exec("INSERT INTO sample (id, col1, col2, col3 ... coln) VALUES (NULL, '".$col[0]."', '".$col[1]."', '".$col[2]."' ... '".$col[n]."')");
}
This way you can dig into your Data and, besides setting an AUTO_INCREMENT ID, you can validate what is coming from the CSV and can correct/prevent importation errors.
As a workaround, I am trying to use this syntax to import my csv file into my database table instead of fgetcsv because files uploaded may be quite big. So I made this:
$connect = mysql_connect("localhost","root","");
mysql_select_db("dbtest",$connect);
$filename = $file_path;
$sql = "LOAD DATA INFILE '$filename'
INTO TABLE tbltest
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n'
(#rcode, #pcode, #mcode, #bcode, #ecode, #filetype, #rec_count, #ges_count, #ck_count, #ic_count, #ran_count);" ;
//echo $sql;
mysql_query ($sql,$connect) ;
Now, the problem is nothing is inserted in my table. When I do echo $sql, the result is this:
LOAD DATA INFILE 'uploads/testfile.csv' INTO TABLE tbltest FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\n' (#rcode, #pcode, #mcode, #bcode, #ecode, #filetype, #rec_count,
#ges_count, #ck_count, #ic_count, #ran_count);
The testfile.csv looks like this:
01|01|01|01|0000|FORM1|5|5|5|||||||||
01|01|01|01|0000|FORM2|238|238||||||||||
01|01|01|01|0000|FORM4|0|||||||||||
01|01|01|01|0000|FORM5|1|||||||||||
01|01|01|01|0000|FORM3|3|3||||||||||
I use the "load data infile" statement to load multiple .csv files in one table with a foreach loop. This works, only I have one problem. The columns in de .csv's are not the same. For example I have this:
CSV1:
id,name,ean,description
CSV2:
id,name,image,ean,description
And my MySQL table is:
id,name,ean,description
So because I want to import multiple csv's through a loop, I have a problem with CSV2, because of the image column. If possible, I want to match the name of the csv column with the name of the table column. So in this example, [image] is not imported. I can use a #ignore variable, but because every .csv is different, this doesn't work. I have this:
$sql = "
LOAD DATA LOCAL INFILE '$value[path]'
INTO TABLE db1.shoptest
FIELDS TERMINATED BY '$value[seperator]'
OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
(id, name, ean, description)
;";
Is it possible to match the csv column names with the table column names and skip all other columns in the .csv?
Thanks so much for helping me out!
Kind regards,
Mark
I think I understood the question. You wan't to get the column headers from the first row of the CSV, then build a string to use as the col_name_or_user_var argument of LOAD DATA and use #ignore variable assignments as needed.
id, name, state, ean, favorite_color, description
becomes...
(id, name, #bunk, ean, #bunk, description)
$whiteList = ['id', 'name', 'ean', 'description'];
$columnHeader = 'id, name, state, ean, favorite_color, description';
$columnHeader = explode(',', $s);
foreach ($columnHeader as $k => $v) {
if (! in_array(trim($v), $whiteList)) $columnHeader[$k] = '#bunk';
}
echo implode(',', $columnHeaders);
I'm using MySQL LOAD DATA INFILE to move CSV file data into a database table. I'm looking for a way to reference the original file line number (row) in the imported record.
So that a table like this:
CREATE TABLE tableName (
uniqueId INT UNSIGNED NOT NULL PRIMARY_KEY,
import_file VARCHAR(100),
import_line INT,
import_date = DATETIME,
fieldA VARCHAR(100),
fieldB VARCHAR(100),
fieldC VARCHAR(100)
);
Where import_file, import_line and import_date are meta data relevant to the specific file import. fieldA, fieldB and fieldC represent the actual data in the file.
Would be updated by a query like this:
LOAD DATA INFILE '$file'
REPLACE
INTO TABLE '$tableName'
FIELDS TERMINATED BY ',' ENCLOSED BY '\"'
LINES TERMINATES BY '\n'
IGNORE 1 LINES # first row is column headers
(fieldA,fieldB,fieldC)
SET import_date = now(),
import_file = '" . addslashes($file) . "',
import_line = '???';
Is there a variable I can set 'import_line' to?
Thanks,
-M
You can do that by setting a user variable first, and increment this variable in your SET clause, i.e.
SET #a:=0; -- initialize the line count
LOAD DATA INFILE 'c:/tools/import.csv' -- my test import
REPLACE
INTO TABLE tableName
FIELDS TERMINATED BY ',' ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES # first row is column headers
(fieldA,fieldB,fieldC)
SET import_date = now(),
import_file = 'import.csv',
import_line = #a:=#a+1; -- save the incremented line count
I have a mysql table (my_table) which have 3 columns (Name, Age, Place). I have a new text file (data.txt tab separated data file) which have have 2 columns (Name and Place). I want to insert information from file "data.txt" into my_table with the condition if Name is same (in mysql table and in text file "data.txt") update row (insert Place value in 3rd column) otherwise create new row (in which Age will be NULL).
if NAME_my_tabe== Name_data.txt --> update this row (do Place_my_tabe = Place_data.txt)
if Name_data.txt not present in my_tabe --> create new row (do Name_my_table=Name_data.txt, Age_my_table=NULL, Place_my_tabe = Place_data.txt)
How to do this in mysql ??? (I want to insert values from text file)
First of all, make sure your table contains the following:
UNIQUE on Name
Age is NULLABLE
The following query will do what you need:
INSERT INTO my_table SET name = 'Name', place = 'Place'
ON DUPLICATE KEY UPDATE place = 'New Place'
With the code to read your file, you could use:
$file = new SplFileObject('data.txt');
while($file->eof() === true) {
$line = $file->fgets();
$data = explode("\t", $line);
// execute query:
$query = sprintf(
"INSERT INTO my_table SET name = '%1$s', place = '%2$s' ON DUPLICATE KEY UPDATE place = '%2$s'",
$data[0], // escape this
$data[1] // escape this
);
}
Note that you will need to execute that query using your mysql client library.