I am trying to import a csv file into the data base without defining any of the rows as it will be automatic when the page loads -
$file = '../csv/file.csv';
$table = 'table_name';
// get structure from csv and insert db
ini_set('auto_detect_line_endings',TRUE);
$handle = fopen($file,'r');
// first row, structure
if ( ($data = fgetcsv($handle) ) === FALSE ) {
echo "Cannot read from csv $file";die();
}
$fields = array();
$field_count = 0;
for($i=0;$i<count($data); $i++) {
$f = strtolower(trim($data[$i]));
if ($f) {
// normalize the field name, strip to 20 chars if too long
$f = substr(preg_replace ('/[^0-9a-z]/', '_', $f), 0, 255);
$field_count++;
$fields[] = $f.' VARCHAR(255)';
}
}
$sql = "CREATE TABLE $table (" . implode(', ', $fields) . ')';
$conn->query($sql);
while ( ($data = fgetcsv($handle) ) !== FALSE ) {
$fields = array();
for($i=0;$i<$field_count; $i++) {
$fields[] = '\''.addslashes($data[$i]).'\'';
}
$sql = "Insert into $table values(" . implode(', ', $fields) . ')';
$conn->query($sql);
}
fclose($handle);
ini_set('auto_detect_line_endings',FALSE);
It loads the data in the page if I echo it and it creates the table correctly just doesn't load the data into the table and I cant find out why..
Thanks!
UPDATED
Here is the first row I am getting when I echo $sql -
Insert into 1001_inventory values('New', '581613', '88888888888888888', '2016', 'Toyota')
So I found that the varchar(255) was to small for some of the fields so I expanded it and the problem is solved, Thanks for all the help!
Related
In my previous question, I asked how to create table using backticks for my fields. Now our team decided to have computations to some of our .csv files when importing it to mysql. However, our default datatype is VARCHAR(500). Our table has an Amount column and we like that datatype to be INT. Can you give us hints on how to do it? Thank you.
Here's my code:
for($i=0;$i<count($data); $i++) {
$f = strtolower(trim($data[$i]));
if ($f) {
// normalize the field name, strip to 20 chars if too long
$f = substr(preg_replace ('/[^0-9a-z]/', '_', $f), 0, 100);
$field_count++;
$fields[] = $f.' VARCHAR(500)';
}
}
$sqlcreate = "CREATE TABLE $table (" . implode(', ', $fields) . ')';
echo $sqlcreate;
If you need to use a different column type based on the name of the column, you could use either an if/then or switch/case statement:
for($i=0;$i<count($data); $i++) {
$f = strtolower(trim($data[$i]));
if ($f) {
// normalize the field name, strip to 20 chars if too long
$f = substr(preg_replace ('/[^0-9a-z]/', '_', $f), 0, 100);
$field_count++;
if($f == 'Amount') {
$fields[] = $f.' INT(10)';
} else {
$fields[] = $f.' VARCHAR(500)';
}
}
}
$sqlcreate = "CREATE TABLE $table (" . implode(', ', $fields) . ')';
echo $sqlcreate;
I have the following code to insert records into a database via a csv file
$get_columns = $db_website->prepare("SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'mytable' AND TABLE_NAME = 'products'");
$get_columns->execute();
while ($row = $get_columns->fetch(PDO::FETCH_ASSOC)) {
$want[] = $row['COLUMN_NAME'];
}
$file = fopen($_FILES['filename']['tmp_name'], "r");
$counter = 0;
while (!feof($file)) {
if ($counter === 1)
break;
$have = fgetcsv ($file, 5000);
++$counter;
}
fclose ($file);
$map = array_intersect($have, $want);
$num_feilds = implode($map);
$fields = "`".implode("`,`",$map)."`";
if ($num_feilds != '') {
$file = fopen($_FILES['filename']['tmp_name'], "r");
$line = fgetcsv($file, 1000, ",");
while (($line = fgetcsv($file)) !== FALSE) {
$data = array_intersect_key($line, $map);
$implode = str_replace("'", ''', $data);
$implode = str_replace("£", '£', $implode);
$implode = "'".implode("','",$implode)."'";
$query = $db_website->prepare("SELECT p.stock_id
FROM products AS p
WHERE p.stock_id = :data");
$query->bindValue(':data', $data[0], PDO::PARAM_INT);
$query->execute();
$product_exists = $query->rowCount();
if ($product_exists == 0) {
$product_import = "INSERT INTO products ($fields, token, date_created) VALUES ($implode, :token, :date_created)";
$product_import = $db_website->prepare($product_import);
$product_import->execute(array(':token'=>$token, ':date_created'=>$todays_date_time));
$update_slug = "UPDATE products SET slug = LOWER(title),
slug = replace(slug, char(128), '')
WHERE token = :token";
$update_slug = $db_website->prepare($update_slug);
$update_slug->execute(array(':token'=>$token));
} else {
while ($row = $query->fetch(PDO::FETCH_ASSOC)) {
$stock_id = $row['stock_id'];
$product_import = "UPDATE products SET $this_is_the_variable_i_need_to_create_from_the_implode, token = :token, date_updated = :date_updated
WHERE stock_id = :stock_id";
$product_import = $db_website->prepare($product_import);
$product_import->execute(array(':stock_id'=>$stock_id, ':token'=>$token, ':date_updated'=>$todays_date_time));
}
$update_slug = "UPDATE products SET slug = LOWER(title),
slug = replace(slug, char(128), '')
WHERE token = :token";
$update_slug = $db_website->prepare($update_slug);
$update_slug->execute(array(':token'=>$token));
}
}
fclose($file);
}
My problems lies in that I want it to update existing products as well as create new ones.
In the code above I have begun by doing a query to check whether the stock id exists and if it doesn't insert the record with an else to say update if it does.
The part I am struggling on is how do I make it implode the COLUMN_NAME and the data that is sent in the csv file.
Any tip in the right direction would be greatly appreciated.
Thank you
Dan
If I'm understanding you correctly, you need to create a series of set clauses based on what's in the $data array (which is an array containing the values from a single line of your CSV). Excluding any kind of validation (either of the columns in your import file, or the data in your import file) you could do something like this:
$sets = array();
$update_values = array();
foreach( $data as $index => $val )
{
if(empty($have[ $index ]))
continue;
$field_name = $have[ $index ];
$update_values[] = $val;
$sets[] = "{$field_name} = ':val{$index}'";
}
if( $sets )
{
$update_values[] = $stock_id;
$set_clause = implode(',',$sets);
$product_import = $db_website->prepare("UPDATE products SET {$set_clause} WHERE stock_id = :stock_id");
$product_import->execute( $update_values );
}
Again, you're going to want validate your input, but this should give you the idea.
Thank you oliakaoil,
This is the code I used in the end for anybody else who may need it in the future
$sets = array();
$update_values = array();
foreach ($data as $index => $val) {
if (empty($have[$index]))
continue;
$field_name = $have[$index];
$update_values[] = $val;
$sets[] = "{$field_name} = '{$val}'";
}
if ($sets) {
$update_values[] = $stock_id;
$set_clause = implode(',',$sets);
$product_import = "UPDATE products SET {$set_clause}, token = :token
WHERE stock_id = :stock_id";
$product_import = $db_website->prepare($product_import);
$product_import->execute(array(':stock_id'=>$update_values[0], ':token'=>$token));
}
i want to insert data from a csv file into my mysql database with php. But i dont know what i doing wrong.
This is my php code
if ($_FILES[csv][size] > 0){
$csv_file = $_FILES[csv][tmp_name]; // Name of your CSV file
$csvfile = fopen($csv_file, 'r');
$theData = fgets($csvfile);
$i = 0;
while (!feof($csvfile)) {
$csv_data[] = fgets($csvfile, 1024);
$csv_array = explode(",", $csv_data[$i]);
$insert_csv = array();
$insert_csv['id'] = $csv_array[0];
$insert_csv['name'] = $csv_array[1];
$insert_csv['email'] = $csv_array[2];
if(!empty($insert_csv['email'])){
$query = "INSERT INTO contacts(id,name,email)
VALUES('','".$insert_csv['name']."','".$insert_csv['email']."')";
$n=mysqli_query($database->connection,$query);
}
$i++;
}
fclose($csvfile);
}
This is my csv looks like.
id---- name ------- email
1 ---- user1--------bla#hotmail.com
2 ---- user2 --------blah
3------ user 3 ------ blah
When i run this code my mysql results are
in email table = ##0.00 "TL")$# en in my name table= also ##0.00
"TL")$#;
What do i wrong?
You might want to use MySQL to do the whole loading process with the LOAD DATA INFILE statement.
if($_FILES['csv']['error'] === UPLOAD_ERR_OK && $_FILES['csv']['size'] > 0) {
$query = "LOAD DATA INFILE '" . $_FILES['csv']['tmp_name']
. "' INTO TABLE contacts FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\n' (id, name, email);";
if(!mysqli_query($query)){
die('Oops! Something went wrong!');
}
}
If required you can tweak the loading parameters (FIELDS TERMINATED BY, ENCLOSED BY, LINES TERMINATED BY).
Do take note that if you use this approach your temporary file needs to be stored in a place where its accessible by the MySQL server (like /tmp).
To start with, I think you should remove the first
$data = fgetcsv($getfile, 1000, ",");
line, outside of the while loop...
Please try like this as a example , it should work for you as you want
I think you missed qoutes in "
$query = "INSERT INTO contacts(id,name,email)
VALUES('".$col1."','".$col2."','".$col3."')";
"
<?php
$csv_file = 'C:\wamp\www\stockmarket\test.csv'; // Name of your CSV file with path
if (($getfile = fopen($csv_file, "r")) !== FALSE) {
$data = fgetcsv($getfile, 1000, ",");
while (($data = fgetcsv($getfile, 1000, ",")) !== FALSE) {
$num = count($data);
for ($c=0; $c < $num; $c++) {
$result = $data;
$str = implode(",", $result);
$slice = explode(",", $str);
$col1 = $slice[0];
$col2 = $slice[1];
$col3 = $slice[2];
// SQL Query to insert data into DataBase
$query = "INSERT INTO contacts(id,name,email)
VALUES('".$col1."','".$col2."','".$col3."')";
$s=mysql_query($query, $connect );
}
}
}
?>
I here have a codes that inputs excel into my table Biometrics. Its working but i cant find a way to import it to other table which has only 3 fields. in my CSV I have 5 columns. I only want to get in my CSV 1 column and put it in my User_dummy table can you help me with
`
mysql_select_db("hris_db",$link) or die ("Cannot select the database!");
// Set your CSV feed
$feed = 'excel/SampleLogs.csv';
//$uploadid = 2;
// Arrays we'll use later
$keys = array();
$newArray = array();
// Function to convert CSV into associative array
function csvToArray($file, $delimiter) {
if (($handle = fopen($file, 'r')) !== FALSE) {
$i = 0;
while (($lineArray = fgetcsv($handle, 0, $delimiter, '"')) !== FALSE) {
for ($j = 0; $j < count($lineArray); $j++) {
$arr[$i][$j] = $lineArray[$j];
}
$i++;
}
fclose($handle);
}
return $arr;
}
// Do it
$data = csvToArray($feed, ',');
// Set number of elements (minus 1 because we shift off the first row)
$count = count($data);
//Use first row for names
$labels = array('empno','date_created','time_created','status','device');
foreach ($labels as $label) {
$keys[] = trim($label);
}
// Bring it all together
for ($j = 0; $j < $count; $j++) {
$d = array_combine($keys, $data[$j]);
$newArray[$j] = array_map('trim',$d);
}
//count number of rows in database
$q = "SELECT * FROM biometrics";
$res = mysql_query($q);
$numrows = mysql_num_rows($res);
$slicearray = array_slice($newArray,$numrows);
echo $numrows;
var_dump($slicearray);
//$reverse = array_reverse($newArray,true);
//var_dump($reverse);
//$uniqueid = uniqid();
foreach($slicearray as $key=>$value){
$implodearray = "'" . implode($value, "','") . "'";
$keysString = implode(",", array_keys($value));
$keylower = strtolower(str_replace(str_split(" '-/"),'_',$keysString));
$sql = "INSERT INTO biometrics ($keylower)
SELECT * FROM (SELECT '".$value['empno']."','".$value['date_created']."','".$value['time_created']."','".$value['status']."' as status,'".$value['device']."' as device) As tmp
WHERE NOT EXISTS (SELECT $keylower FROM biometrics WHERE empno = '".$value['empno']."' AND date_created = '".$value['date_created']."' AND time_created = '".$value['time_created']."' AND status = '".$value['status']."' AND device = '".$value['device']."')";
mysql_query($sql) or die(mysql_error());
//echo $sql;
//var_dump($keylower);
//var_dump($value);
}
?>`
yes this is very simple
http://imgur.com/pZTzVbj
use this link image and you get idea how to import csv in database
or if you want to this by query then use it
LOAD DATA LOCAL INFILE '/your_file.csv'
INTO TABLE your_table
FIELDS TERMINATED BY ','
LINES TERMINATED BY ','
(field1, filed2, field3);
Is it possible to remove extra rows when uploading in .csv file in PHP or removing specific words?
Please take a look at my screen shot. I want to exclude those encircled when I process uploading in MySQL.
Full Size
Here's my code found over the net.
<?php
$message = null;
$allowed_extensions = array('csv');
$upload_path = '';
if (!empty($_FILES['file'])) {
if ($_FILES['file']['error'] == 0) {
// check extension
$file = explode(".", $_FILES['file']['name']);
$extension = array_pop($file);
if (in_array($extension, $allowed_extensions)) {
if (move_uploaded_file($_FILES['file']['tmp_name'], $upload_path.'/'.$_FILES['file']['name'])) {
if (($handle = fopen($upload_path.'/'.$_FILES['file']['name'], "r")) !== false) {
$keys = array();
$out = array();
$insert = array();
$line = 1;
while (($row = fgetcsv($handle, 0, ',', '"')) !== FALSE) {
foreach($row as $key => $value) {
if ($line === 1) {
$keys[$key] = $value;
} else {
$out[$line][$key] = $value;
}
}
$line++;
}
fclose($handle);
if (!empty($keys) && !empty($out)) {
$db = new PDO('mysql:host=localhost;dbname=attendance', 'root', 'root');
$db->exec("SET CHARACTER SET utf8");
foreach($out as $key => $value) {
$sql = "INSERT INTO `report` (`";
$sql .= implode("`, `", $keys);
$sql .= "`) VALUES (";
$sql .= implode(", ", array_fill(0, count($keys), "?"));
$sql .= ")";
$statement = $db->prepare($sql);
$statement->execute($value);
}
$message = '<span class="green">File has been uploaded successfully</span>';
}
}
}
} else {
$message = '<span class="red">Only .csv file format is allowed</span>';
}
} else {
$message = '<span class="red">There was a problem with your file</span>';
}
}
?>
Well, without more info - it looks like the rows you want to exclude have less rows than the others (I'm assuming you don't want the blank rows either).
You can use the function
$arr= str_getcsv();
to read a line of CSV data into an array the do
if(sizeof($arr)!=$validCount)
on the resulting array you get to exclude lines not of a certain length ($validCount).
Obviously we could get much more in depth for exclusion, but I'm guessing this will get your started.
Note - this requires PHP5.3 or greater.
HTH
R