Separate CSV import fields in a query - php

I would to load a csv into MySQL and manage the single field. the original code uploaded the content with automatic VALUES('','$linemysql');. How can I separate the fields? I would a query like:
$sql = "INSERT IGNORE INTO `database`.`table` (`field1`,field2`, `field3`, `field4`, `field5`) VALUES(`csvfield1`,csvfield2`, `csvfield3`, `csvfield4`, `csvfield5`);";
This because I can manage which row will be included
$lines = 0;
$queries = "";
$linearray = array();
$allLines = split($lineseparator,$csvcontent);
array_shift($allLines); // removes the 1st element
foreach($allLines as $line) {
$lines++;
$line = trim($line," \t");
$line = str_replace("\r","",$line);
/************************************
This line escapes the special character. remove it if entries are already escaped in the csv file
************************************/
$line = str_replace("'","\'",$line);
/*************************************/
$linearray = explode($fieldseparator,$line);
$linemysql = implode("','",$linearray);
if($addauto)
/* Change This query VALUES('','$linemysql'); */
$query = "INSERT IGNORE INTO `database`.`table` (`field1`,field2`, `field3`, `field4`, `field5`) VALUES('','$linemysql');";
else
$query = "insert into $databasetable values('$linemysql');";
$queries .= $query . "\n";

Use PHP's built in CSV functionality including fgetcsv(). From the docs:
<?php
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
}
fclose($handle);
}
?>

You can modify your query like this
$query = "INSERT IGNORE INTO `database`.`table` (`field1`,field2`, `field3`, `field4`, `field5`) VALUES("'.$linemysql[0].'","'.$linemysql[1].'","'.$linemysql[2].'","'.$linemysql[3].'","'.$linemysql[4].'");";
Thanks

Related

PHP - Uploading CSV, inserting to multiple tables and displaying elsewhere

I'm trying to connect the dots for a few processes. I have a project that requires a CSV to be uploaded via form on webpage, which exists already, on a daily basis. Every time the CSV is loaded in the 110 columns need to be split into about 5 different MySQL tables. The CSV will always be in the same order, but I need to know exactly which variables/columns from the CSV to put into the specific tables and fields. Once this is done, I'll need to display the data on other pages, by query only (only some things will be displayed depending on the page). I have the following code to test putting the CSV into an array:
if(isset($_POST['submit']))
{
$array = $fields = array(); $i=0;
$file = $_FILES['file']['tmp_name'];
$handle = fopen($file, "r");
if($handle){
while (($row = fgetcsv($handle, 4096)) !== false) {
if (empty($fields)) {
$fields = $row;
continue;
}
foreach ($row as $k=>$value) {
$array[$i][$fields[$k]] = $value;
}
$i++;
}
echo "Success!";
var_dump($array);
if (!feof($handle)) {
echo "Error: unexpected fgets() fail\n";
}
fclose($handle);
}}?>
This seems to work fine with the Var_dump. Then, separately, I've used the following code on a small CSV with only ten fields:
while(($filesop = fgetcsv($handle, 4096, ",")) !== false)
{
$one = $filesop[0];
$two = $filesop[1];
$three = $filesop[2];
$four = $filesop[3];
$five = $filesop[4];
$six = $filesop[5];
$seven = $filesop[6];
$eight = $filesop[7];
$nine = $filesop[8];
$ten = $filesop[9];
$sql = "INSERT INTO staging (One, Two, Three, Four, Five, Six, Seven, Eight, Nine, Ten) VALUES ('$one','$two', '$three','$four','$five','$six','$seven','$eight','$nine','$ten')";
}
if ($connect->query($sql) === TRUE) {
echo "You database has imported successfully";
} else {
echo "Error: " . $sql . "<br>" . $conn->error;
}
I'm still confused on how I can map the 110 fields of these daily uploaded CSV to their correct tables (10 fields into one table, 25 into another, 30 into another, etc. ) so that I can use the variables to display certain fields on other pages.
UPDATE:
New code for arrays:
$table_cols[5] ="workOrderNum,lowSideMIUNum,highSideMIUNum,accountNum,custName,address,locID,utility,serialNumber,serviceName";
$tablenames = array("staging","clients","meters","tests","costs","workorders");
for($tableno = 0;$tableno < sizeof($tablenames);$tableno++){
$q = "";
$q .= "INSERT INTO ".$tablenames[$tableno]." (".$table_cols[$tableno].") VALUES (";
$cols = explode("," ,$table_cols);
$data = array();
foreach($col as $key => $fldname) {
$data[] = "'".$coldata[$fldname]."'";
}
$q .= implode(",",$data).");";
}
echo'File submitted';
See Update Below This part of the answer left for context
You could create an array with indexes of the columns to be saved in each table. Then run one insert per table, getting the column values based on the index in your array for that table.
$tbl1_idxs = array(1,4,10,20,15,22);
$tbl2_idxs = array(2,9,1,7,32,44);
$tbl3_idxs = array(27,15,65,110,12);
$q1 = "";
$q1 .= "INSERT INTO `Table1` (`fld1`,`fld2`,`fld3`,`fld4`,`fld5`,`fld6`) ";
$q1 .= "VALUES ('";
$q1 .= $filesop[tbl1_idxs[0]]."','";
$q1 .= $filesop[tbl1_idxs[1]]."','";
$q1 .= $filesop[tbl1_idxs[2]]."','";
$q1 .= $filesop[tbl1_idxs[3]]."','";
$q1 .= $filesop[tbl1_idxs[4]]."','";
$q1 .= $filesop[tbl1_idxs[5]]."');";
// Execute query #1....
$q2 = "";
$q2 .= "INSERT INTO `Table2` (`fld1`,`fld2`,`fld3`,`fld4`,`fld5`,`fld6`) ";
$q2 .= "VALUES ('";
$q2 .= $filesop[tbl2_idxs[0]]."','";
$q2 .= $filesop[tbl2_idxs[1]]."','";
$q2 .= $filesop[tbl2_idxs[2]]."','";
$q2 .= $filesop[tbl2_idxs[3]]."','";
$q2 .= $filesop[tbl2_idxs[4]]."','";
$q2 .= $filesop[tbl2_idxs[5]]."');";
// Execute query #2....
$q3 = "";
$q3 .= "INSERT INTO `Table3` (`fld1`,`fld2`,`fld3`,`fld4`,`fld5`,`fld6`) ";
$q3 .= "VALUES ('";
$q3 .= $filesop[tbl3_idxs[0]]."','";
$q3 .= $filesop[tbl3_idxs[1]]."','";
$q3 .= $filesop[tbl3_idxs[2]]."','";
$q3 .= $filesop[tbl3_idxs[3]]."','";
$q3 .= $filesop[tbl3_idxs[4]]."','";
$q3 .= $filesop[tbl3_idxs[5]]."');";
// Execute query #3....
UPDATED ANSWER
With that many columns, I would map by column name instead. There are several ways this could be made manageable.
In any case, instead of hard variable names for each column of the CSV, I would put them in an associative array, with the keys matching the column names of the tables. You could then have a comma separated list of columns for each table.
You would still have the 203 lines of code to map the CSV fields to names, but maintaining which column goes into which table would at least be readable. And changing the existing code that assigned each CSV column to a variable should be a simple regex search and replace.
$coldata = array();
$coldata['name'] = $filesop[0];
$coldata['addr1'] = $filesop[1];
$coldata['addr2'] = $filesop[2];
$coldata['city'] = $filesop[3];
$coldata['state'] = $filesop[4];
$coldata['zip'] = $filesop[5];
$coldata['country'] = $filesop[6];
$coldata['gender'] = $filesop[7];
$coldata['age'] = $filesop[8];
$coldata['birthdate'] = $filesop[9];
$table_cols = array();
$table_cols[0] = "name,gender,age";
$table_cols[1] = "name,addr1,addr2,city,state,zip";
$table_cols[2] = "name,age,birthdate";
$tablenames = array("staging","info","other");
for($tableno = 0;$tableno < sizeof($tablenames);$tableno++) {
$q = "";
$q .= "INSERT INTO ".$tablenames[$tableno]." (".$table_cols[$tableno].") VALUES (";
$cols = explode(",",$table_cols[$tableno]);
$data = array();
foreach($col as $key => $fldname) {
$data[] = "'".$coldata[$fldname]."'";
}
$q .= implode(",",$data).");";
// Execute query in $q
}

Import particular Colums of csv in database in php/sql

I have a CSV which contains 15 columns.
But I want to import only 2 columns from the CSV in database.
whats should I do?
I tried importing but it's importing all the columns.
<?php
//database connection details
$connect = mysql_connect('localhost','root','123456');
if (!$connect) {
die('Could not connect to MySQL: ' . mysql_error());
}
//your database name
$cid =mysql_select_db('test',$connect);
// path where your CSV file is located
define('CSV_PATH','C:/wamp/www/');
// Name of your CSV file
$csv_file = CSV_PATH . "test.csv";
if (($handle = fopen($csv_file, "r")) !== FALSE) {
fgetcsv($handle);
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
for ($c=0; $c < $num; $c++) {
$col[$c] = $data[$c];
}
$col1 = $col[0];
$col2 = $col[1];
$col3 = $col[2];
// SQL Query to insert data into DataBase
$query = "INSERT INTO csvtbl(ID,name,city) VALUES('".$col1."','".$col2."','".$col3."')";
$s = mysql_query($query, $connect );
}
fclose($handle);
}
echo "File data successfully imported to database!!";
mysql_close($connect);
?>

Insert an array in the database

I want to insert an array in the database. The array can be changed all the time. I want different rows in the database.
My code:
$var = file_get_contents("test2.txt");
$test = preg_replace('/\\\\/', '', $var);
$poep = explode(" ", $test);
Yeah, there is no database connection, because I want to know how to 'split' the array to insert it in the database.
I have tried this:
foreach($poep as $row) {
$row = $mysqli->real_escape_string($row);
if($mysqli->query("insert into data('array') VALUES ($row)") == false){
echo 'Doesnt works!';
}
It returns 'Doesnt works', so I think there is a problem with query?
#NadirDev Hi. Assuming that you are using Core PHP programming. After exploding the string by the space, run foreach loop and then insert individual rows. Look at this rough code to get idea:
foreach($poep as $row) {
// $row now contains one word. Add that in database.
$row = mysql_real_escape_string($row);
$query = mysql_query("insert into tableName('fieldName') VALUES ($row)");
}
here's some code I wrote. It processes a CSV file and stores separate rows into a db table (difference is just that you have a TXT file). It does the mysql insertion in batches of 250 rows. Hope it can help you!
// read all input rows into an array
echo "Processing input..<br /><br />";
$row = 0;
$input = array();
if (($handle = fopen($file['tmp_name'], "r")) !== FALSE) {
while (($data = fgetcsv($handle, 0, ",")) !== FALSE) {
$num = count($data);
for ($c=0; $c < $num; $c++) {
$input[$row][] = addslashes($data[$c]);
}
$row++;
}
fclose($handle);
}
$count = 0;
$q = "INSERT INTO `inputs` (`keyword`, `percent`, `link`, `added_on`) VALUES ";
foreach ($input as $inp) {
$q .= "('" . addslashes($inp[0]) . "', '" . addslashes($inp[1]) . "', '" . addslashes($inp[2]) . "', '" . date('Y-m-d H:i:s') . "'), ";
$count++;
if ($count >= 250) {
$q = substr($q, 0, -2);
$q = mysqli_query($con, $q);
$q = "INSERT INTO `inputs` (`keyword`, `percent`, `link`, `added_on`) VALUES ";
$count = 0;
}
}
if ($count > 0) {
$q = substr($q, 0, -2);
$q = mysqli_query($con, $q);
}
echo "Successfully added " . count($input) . " rows to the input list.";

Convert a csv to mysql through php starting from a particular line?

I have the following code below which works, but I want to insert values to my database from the second row of my csv file. Please any pointers will be great.
$con = #mysql_connect($databasehost,$databaseusername,$databasepassword) or die(mysql_error());
#mysql_select_db($databasename) or die(mysql_error());
$row = 1;
if (($handle = fopen($csvfile, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br /></p>\n";
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br />\n";
}
$query = "INSERT into $databasetable(email,emailSource,espBanner,firstName,lastName,Address,City,
State,zip,home,mobile,card,ethnicity,language,gender,filename,is_uploaded)
values('$data[0]','$data[1]','$data[2]','$data[3]','$data[4]','$data[5]','$data[6]'
,'$data[7]','$data[8]','$data[9]','$data[10]','$data[11]','$data[12]','$data[13]',
'$data[14]','$data[15]','$data[16]')";
mysql_query($query) or die(mysql_error());
}
fclose($handle);
}
The result will insert every row of the csv.
$con = #mysql_connect($databasehost,$databaseusername,$databasepassword) or die(mysql_error());
#mysql_select_db($databasename) or die(mysql_error());
$start_from_row = 2;
$row = 0;
if (($handle = fopen($csvfile, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$num = count($data);
echo "<p> $num fields in line $row: <br></p>\n";
$row++;
if($row < $start_from_row)
continue;
for ($c=0; $c < $num; $c++) {
echo $data[$c] . "<br>\n";
}
$query = "INSERT into $databasetable(email,emailSource,espBanner,firstName,lastName,Address,City,
State,zip,home,mobile,card,ethnicity,language,gender,filename,is_uploaded)
values('$data[0]','$data[1]','$data[2]','$data[3]','$data[4]','$data[5]','$data[6]'
,'$data[7]','$data[8]','$data[9]','$data[10]','$data[11]','$data[12]','$data[13]',
'$data[14]','$data[15]','$data[16]')";
mysql_query($query) or die(mysql_error());
}
fclose($handle);
}

How to convert the date while importing Excel sheet to the database?

I'm having an Excel sheet with the format 01-Mar-10. When I import that csv file into my database it is storing as 0000-00-00. How to convert the dateformat while importing, or is there any other way?
This is the code im using to import to database
$databasetable = "sample";
$fieldseparator = ",";
$lineseparator = "\n";
$csvfile = "../uploads/" . basename( $_FILES['file']['name']);
$addauto = 0;
$save = 1;
$outputfile = "../temp/output.sql";
if(!file_exists($csvfile)) {
#echo $csvfile;
echo "File not found. Make sure you specified the correct path.\n";
return 0;
exit;
}
$file = fopen($csvfile,"r");
if(!$file) {
#echo "Error opening data file.\n";
return 0;
exit;
}
$size = filesize($csvfile);
if(!$size) {
echo "File is empty.\n";
return 0;
exit;
}
$csvcontent = fread($file,$size);
fclose($file);
$lines = 0;
$queries = "";
$linearray = array();
foreach(split($lineseparator,$csvcontent) as $line) {
$lines++;
$line = trim($line," \t");
$line = str_replace("\r","",$line);
$line = str_replace("'","\'",$line);
$linearray = explode($fieldseparator,$line);
$linemysql = implode("','",$linearray);
if($addauto)
$query = "insert into $databasetable values('','$linemysql');";
else
$query = "insert into $databasetable values('$linemysql');";
$queries .= $query . "\n";
#mysql_query($query);
why not use 'LOAD DATA INFILE' with mysql's date formatting?
http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_date-format
http://dev.mysql.com/doc/refman/5.0/en/load-data.html
$d = "01-Mar-10";
$in_db = date('Y-m-d', strtotime($d));
echo $in_db;
Output:
2010-03-01
Use date() conversion to get valid form of date for storing in database. Use strtotime() to convert excel date format to timestamp.
After exploding data to $linearray use
$linearray[ID]=date('Y-m-d', strtotime($linearray[ID]));
to change it's value to proper format. Replace ID with array index of element with date field.

Categories