I'm trying to parse a small CSV file and insert the records into MySQL.
The problem I'm having is that the CSV has 150 rows, but only 2 seem to be inserted:
$file = new SplFileObject($uploadedFile);
$separator = ',';
$rowCounter = 1;
$errors = array();
$fh = fopen($uploadedFile, 'r');
if(!$fh) die('File no good!');
// Get headings
$headings = fgetcsv($fh, 0, ',');
$num_cols = count($headings);
$num_rows = 1;
// Read the file as csv
while ($row = $file->fgetcsv($separator)) {
//missed product if num columns in this row not the same as num headings
if (count($row) != $num_cols) {
$row = array_pad($row, $num_cols, NULL);
}
for ($i=0; $i<count($headings); $i++) {
$raw_prod[$headings[$i]] = $row[$i];
}
$item = new Item();
$item->name = $raw_prod['NAME'];
$item->age = $raw_prod['AGE'];
$item->location = $raw_prod['LOCATION'];
$item->save();
}
If I var_dump($item) I get all 150 records, but only 2 ever get inserted.
I just don't understand why this is happening
Thanks
This line
while ($row = $file->fgetcsv($separator)) {
should be
while (($row = $file->fgetcsv($separator)) !== false) {
Related
First of all I load PHPExcel.php
Secondly, I am using this code:
$location = '/path/file.csv';
$inputFileType = 'CSV';
$objReader = PHPExcel_IOFactory::createReader($inputFileType);
$objPHPExcel = $objReader->load($location);
$worksheet = $objPHPExcel->getActiveSheet();
$list = array();
foreach ($worksheet->getRowIterator() as $row)
{
$rowIndex = $row->getRowIndex();
$cellValue = $worksheet->getCell('A'.$rowIndex)->getValue();
array_push($list, $cellValue);
}
$count = count($list);
for ($rowIndex = $count; $rowIndex != 1; $rowIndex--)
{
$cellValue = $worksheet->getCell('A'.$rowIndex)->getValue();
for ($i = $rowIndex - 2; $i != 0; $i--)
{
if ($list[$i] == $cellValue)
{
$worksheet->removeRow($rowIndex);
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'CSV');
$objWriter->save($location);
break;
}
}
}
So, I am trying to remove the rows when there are duplicate values in the first column. The code does not work. When I first run it in putty, I have to wait for ages. I interrupt the process and then I run it again. Then it runs, but in my csv file I have wrong results (duplicates are 300 but I am getting -600 rows).
In order to read a CSV file you dont have to use PHPExcel. Instead you can use a native php code like this one:
<?php
// Array which will hold all analyzed lines
$uniqueEntries = array();
$dublicatedEntries = array();
$delimiter = ',';
$file = 'test.csv';
//Open the file
if (($handle = fopen($file, "r")) !== false) {
// read each line into an array
while (($data = fgetcsv($handle, 8192, $delimiter)) !== false) {
// build a "line" from the parsed data
$line = join($delimiter, $data);
//If the line content has ben discovered before - save to duplicated and skip the rest..
if (isset($uniqueEntries[$line])){
dublicatedEntries[] = $line;
continue;
}
// save the line
$uniqueEntries[$line] = true;
}
fclose($handle);
}
// build the new content-data
$contents = '';
foreach ($uniqueEntries as $line => $bool) $contents .= $line . "\r\n";
// save it to a new file
file_put_contents("test_unique.csv", $contents);
?>
This code is untested but should work.
This will give you a .csv file with all unique entries.
I have the following code to insert records into a database via a csv file
$get_columns = $db_website->prepare("SELECT COLUMN_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_SCHEMA = 'mytable' AND TABLE_NAME = 'products'");
$get_columns->execute();
while ($row = $get_columns->fetch(PDO::FETCH_ASSOC)) {
$want[] = $row['COLUMN_NAME'];
}
$file = fopen($_FILES['filename']['tmp_name'], "r");
$counter = 0;
while (!feof($file)) {
if ($counter === 1)
break;
$have = fgetcsv ($file, 5000);
++$counter;
}
fclose ($file);
$map = array_intersect($have, $want);
$num_feilds = implode($map);
$fields = "`".implode("`,`",$map)."`";
if ($num_feilds != '') {
$file = fopen($_FILES['filename']['tmp_name'], "r");
$line = fgetcsv($file, 1000, ",");
while (($line = fgetcsv($file)) !== FALSE) {
$data = array_intersect_key($line, $map);
$implode = str_replace("'", ''', $data);
$implode = str_replace("£", '£', $implode);
$implode = "'".implode("','",$implode)."'";
$query = $db_website->prepare("SELECT p.stock_id
FROM products AS p
WHERE p.stock_id = :data");
$query->bindValue(':data', $data[0], PDO::PARAM_INT);
$query->execute();
$product_exists = $query->rowCount();
if ($product_exists == 0) {
$product_import = "INSERT INTO products ($fields, token, date_created) VALUES ($implode, :token, :date_created)";
$product_import = $db_website->prepare($product_import);
$product_import->execute(array(':token'=>$token, ':date_created'=>$todays_date_time));
$update_slug = "UPDATE products SET slug = LOWER(title),
slug = replace(slug, char(128), '')
WHERE token = :token";
$update_slug = $db_website->prepare($update_slug);
$update_slug->execute(array(':token'=>$token));
} else {
while ($row = $query->fetch(PDO::FETCH_ASSOC)) {
$stock_id = $row['stock_id'];
$product_import = "UPDATE products SET $this_is_the_variable_i_need_to_create_from_the_implode, token = :token, date_updated = :date_updated
WHERE stock_id = :stock_id";
$product_import = $db_website->prepare($product_import);
$product_import->execute(array(':stock_id'=>$stock_id, ':token'=>$token, ':date_updated'=>$todays_date_time));
}
$update_slug = "UPDATE products SET slug = LOWER(title),
slug = replace(slug, char(128), '')
WHERE token = :token";
$update_slug = $db_website->prepare($update_slug);
$update_slug->execute(array(':token'=>$token));
}
}
fclose($file);
}
My problems lies in that I want it to update existing products as well as create new ones.
In the code above I have begun by doing a query to check whether the stock id exists and if it doesn't insert the record with an else to say update if it does.
The part I am struggling on is how do I make it implode the COLUMN_NAME and the data that is sent in the csv file.
Any tip in the right direction would be greatly appreciated.
Thank you
Dan
If I'm understanding you correctly, you need to create a series of set clauses based on what's in the $data array (which is an array containing the values from a single line of your CSV). Excluding any kind of validation (either of the columns in your import file, or the data in your import file) you could do something like this:
$sets = array();
$update_values = array();
foreach( $data as $index => $val )
{
if(empty($have[ $index ]))
continue;
$field_name = $have[ $index ];
$update_values[] = $val;
$sets[] = "{$field_name} = ':val{$index}'";
}
if( $sets )
{
$update_values[] = $stock_id;
$set_clause = implode(',',$sets);
$product_import = $db_website->prepare("UPDATE products SET {$set_clause} WHERE stock_id = :stock_id");
$product_import->execute( $update_values );
}
Again, you're going to want validate your input, but this should give you the idea.
Thank you oliakaoil,
This is the code I used in the end for anybody else who may need it in the future
$sets = array();
$update_values = array();
foreach ($data as $index => $val) {
if (empty($have[$index]))
continue;
$field_name = $have[$index];
$update_values[] = $val;
$sets[] = "{$field_name} = '{$val}'";
}
if ($sets) {
$update_values[] = $stock_id;
$set_clause = implode(',',$sets);
$product_import = "UPDATE products SET {$set_clause}, token = :token
WHERE stock_id = :stock_id";
$product_import = $db_website->prepare($product_import);
$product_import->execute(array(':stock_id'=>$update_values[0], ':token'=>$token));
}
I have a CSV file with lots of data in it, from which I create many different tables using
PHP, specifying certain rows and columns.
However, I have a requirement for only showing two cells, A7 and A8. The code I currently use requires fields for table head and table data, which on this occasion I don't need.
What is the most effect way of doing this?
<?php
function get_cell($cell, $row)
{
$file = 'stuff.csv';
$row = $row -1;
// Reset the $row if it's negative.
if($row < 0)
{
$row = 0;
}
$handle = fopen ($file, "r");
$counter = 0;
while (($csvline = fgetcsv($handle, 450,";")) != false)
{
if($counter == $row)
{
$new_array = array();
$begin = 'a';
foreach($csvline as $cellnr)
{
$new_array[$begin] = $cellnr;
$begin++;
}
echo $new_array[strtolower($cell)];
}
$counter++;
}
}
get_cell('a', 1);
echo '<br>';
get_cell('b', 1);
?>
i have a while loop which is fetching data from csv.Inside while loop there is a condition(if condition) if the condition is true foreach loop will get executed where we want to insert one row at a time into the database.This should continue for every row of the csv. if I have 10 rows in csv,it should insert all 10 rows in the database.But mycode is inserting the first row 10 times.
$handle = fopen($_FILES['upcsv']['tmp_name'], "r");
$count = count(file($_FILES['upcsv']['tmp_name']));
fgetcsv($handle, 1000, ",");
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
$clt = mysql_query("select MCLNTLKNOFLG,MCLNTDKTRNGFRM,MCLNTDKTRNGTO from mclientmst where MCLNTCD ='".$data[1]."'");
if(mysql_num_rows($clt)>0)
{
$clts = mysql_fetch_array($clt);
if($clts['MCLNTLKNOFLG']==1)
{
$i=1 ;
foreach(range ($clts['MCLNTDKTRNGFRM'], $clts['MCLNTDKTRNGTO']) as $num)
{
$dkt = mysql_query("select XCMPCD from xdockethdr where XDKTNO ='$num'");
$ndkt = mysql_query("select XCMPCD from xtempdockethdr where XDKTNO ='$num'");
if(mysql_num_rows($dkt)==0 && mysql_num_rows($ndkt)==0)
{
$date = explode('/',$data[3]);
$dt = $date[2].'-'.$date[1].'-'.$date[0];
$dktid = mysql_query("select MAX(XDKTID) as maxid from xtempdockethdr");
$maxid = mysql_fetch_array($dktid);
$max = $maxid['maxid'] +1;
$query = mysql_query("insert into xtempdockethdr (XCMPCD,XCLNTCD,XDKTNO,XCNSGCD,XDKTPUDATE,XDKTPUTIME,XDKTNOPKGS,XDKTMODLV,XDKTHTOCONCD,XDKTDCTVAL,XDKTDIMWT,XDKTACTWT,XUNIQUEID,XDKTID) VALUES ('".$data[0]."','".$data[1]."','".$num."','".$data[2]."','".$dt."','".$data[4]."','".$data[5]."','".$data[6]."','".$data[7]."','".$data[8]."','".$data[9]."','".$data[10]."','".$data[11]."','".$max."')");
$i++;
}
}
}
}
}
fclose($handle);
header('Location:upload_docketentry.php');
You need to set a break; after your code.
It will close first loop (foreach) and goes down.
Add continue 2; after sql inserts.
this will end this loop and goes to the begining of while loop and continue work.
$handle = fopen($_FILES['upcsv']['tmp_name'], "r");
$count = count(file($_FILES['upcsv']['tmp_name']));
fgetcsv($handle, 1000, ",");
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) // **#1 point**
{
$clt = mysql_query("select MCLNTLKNOFLG,MCLNTDKTRNGFRM,MCLNTDKTRNGTO from mclientmst where MCLNTCD ='".$data[1]."'");
if(mysql_num_rows($clt)>0)
{
$clts = mysql_fetch_array($clt);
if($clts['MCLNTLKNOFLG']==1)
{
$i=1 ;
foreach(range ($clts['MCLNTDKTRNGFRM'], $clts['MCLNTDKTRNGTO']) as $num)
{
$dkt = mysql_query("select XCMPCD from xdockethdr where XDKTNO ='$num'");
$ndkt = mysql_query("select XCMPCD from xtempdockethdr where XDKTNO ='$num'");
if(mysql_num_rows($dkt)==0 && mysql_num_rows($ndkt)==0)
{
$date = explode('/',$data[3]);
$dt = $date[2].'-'.$date[1].'-'.$date[0];
$dktid = mysql_query("select MAX(XDKTID) as maxid from xtempdockethdr");
$maxid = mysql_fetch_array($dktid);
$max = $maxid['maxid'] +1;
$query = mysql_query("insert into xtempdockethdr (XCMPCD,XCLNTCD,XDKTNO,XCNSGCD,XDKTPUDATE,XDKTPUTIME,XDKTNOPKGS,XDKTMODLV,XDKTHTOCONCD,XDKTDCTVAL,XDKTDIMWT,XDKTACTWT,XUNIQUEID,XDKTID) VALUES ('".$data[0]."','".$data[1]."','".$num."','".$data[2]."','".$dt."','".$data[4]."','".$data[5]."','".$data[6]."','".$data[7]."','".$data[8]."','".$data[9]."','".$data[10]."','".$data[11]."','".$max."')");
$i++;
// continue 2; // Goes to #1
// break; // Goes to #2
}
} // #2 point
var_dump(__LINE__); // This will executed if you place break operator
}
}
}
fclose($handle);
header('Location:upload_docketentry.php');
I here have a codes that inputs excel into my table Biometrics. Its working but i cant find a way to import it to other table which has only 3 fields. in my CSV I have 5 columns. I only want to get in my CSV 1 column and put it in my User_dummy table can you help me with
`
mysql_select_db("hris_db",$link) or die ("Cannot select the database!");
// Set your CSV feed
$feed = 'excel/SampleLogs.csv';
//$uploadid = 2;
// Arrays we'll use later
$keys = array();
$newArray = array();
// Function to convert CSV into associative array
function csvToArray($file, $delimiter) {
if (($handle = fopen($file, 'r')) !== FALSE) {
$i = 0;
while (($lineArray = fgetcsv($handle, 0, $delimiter, '"')) !== FALSE) {
for ($j = 0; $j < count($lineArray); $j++) {
$arr[$i][$j] = $lineArray[$j];
}
$i++;
}
fclose($handle);
}
return $arr;
}
// Do it
$data = csvToArray($feed, ',');
// Set number of elements (minus 1 because we shift off the first row)
$count = count($data);
//Use first row for names
$labels = array('empno','date_created','time_created','status','device');
foreach ($labels as $label) {
$keys[] = trim($label);
}
// Bring it all together
for ($j = 0; $j < $count; $j++) {
$d = array_combine($keys, $data[$j]);
$newArray[$j] = array_map('trim',$d);
}
//count number of rows in database
$q = "SELECT * FROM biometrics";
$res = mysql_query($q);
$numrows = mysql_num_rows($res);
$slicearray = array_slice($newArray,$numrows);
echo $numrows;
var_dump($slicearray);
//$reverse = array_reverse($newArray,true);
//var_dump($reverse);
//$uniqueid = uniqid();
foreach($slicearray as $key=>$value){
$implodearray = "'" . implode($value, "','") . "'";
$keysString = implode(",", array_keys($value));
$keylower = strtolower(str_replace(str_split(" '-/"),'_',$keysString));
$sql = "INSERT INTO biometrics ($keylower)
SELECT * FROM (SELECT '".$value['empno']."','".$value['date_created']."','".$value['time_created']."','".$value['status']."' as status,'".$value['device']."' as device) As tmp
WHERE NOT EXISTS (SELECT $keylower FROM biometrics WHERE empno = '".$value['empno']."' AND date_created = '".$value['date_created']."' AND time_created = '".$value['time_created']."' AND status = '".$value['status']."' AND device = '".$value['device']."')";
mysql_query($sql) or die(mysql_error());
//echo $sql;
//var_dump($keylower);
//var_dump($value);
}
?>`
yes this is very simple
http://imgur.com/pZTzVbj
use this link image and you get idea how to import csv in database
or if you want to this by query then use it
LOAD DATA LOCAL INFILE '/your_file.csv'
INTO TABLE your_table
FIELDS TERMINATED BY ','
LINES TERMINATED BY ','
(field1, filed2, field3);