I'm importing data from CSV file to database by using php
$i = 0;
while(($data = fgetcsv($handle, 1000, ",")) !== FALSE)
{
if($i!=0) {
if(trim($data[0])!=' ') {
$branddata['brand'] = $data[0];
$branddata['create_date'] = date('Y-m-d H:i:s');
print_r($branddata); die;
$res = $this->mdl_brands->insertIntoBrand($branddata);
}
}
$i++;
}
and My csv file has following data
After the Brand Name in the 2 row I have added more than one blank space which is inserting entry into the database.
However I have used the trim function to remove the blank spaces but it not working.
Please guide how to avoid this.
As you said it:
I have used the trim function to remove the blank spaces
You removed them! So you don't have to check for 1 space, just do:
if(trim($data[0]) != '') {
//^^ See here
Related
Is it possible to detect in the following code example if the current row is the last row? Here is my code:
$file = \fopen('/path/to/file.csv', 'rb');
while (($row = \fgetcsv($file, 0, ';')) !== false) {
// Detect if it is the last row
$isLastRow = ?;
// Do some stuff for all rows and pass if it is the last row
$myService->handleRow($row, $isLastRow);
}
I already tried it with $isLastRow = feof($file);, but that was not working. Does anyone have any ideas on how to do this?
I think I have found a solution after all, contrary to the comments. This seems to work:
$fileEndPosition = \filesize('/path/to/file.csv');
$file = \fopen('/path/to/file.csv', 'rb');
while (($row = \fgetcsv($file, 0, ';')) !== false) {
// Detect current file pointer position
$currentPosition = ftell($file);
// Detect if it is the last row
$isLastRow = $currentPosition >= $fileEndPosition;
// Do some stuff for all rows and pass if it is the last row
$myService->handleRow($row, $isLastRow);
}
I have a CSV file that contains around 8500 lines but I'm getting a really weird "bug".
I'm validating the data inside the CSV to make sure the data is cool to import into the database. I currently just log the data errors to a log file, but when I open it I see error reports for rows upto 8800 (give or take).
I did some basic debugging to see what's what and did this to begin with:
foreach ($csv as $key => $row)
{
if ($key > 8500) {
echo '<pre>';
print_r($row);
echo '</pre>';
}
}
and that only returned about 50/60 more which is fine as the total rows is around that number.
I then tried doing this to get the end array result:
$last = end($csv);
print_r($last);
and that showed an array with data as expected. However when I do this:
var_dump(array_keys($csv));
then it shows 8800 (give or take) values. Doing count($csv) returns the same number.
I've tried going into the actual CSV and highlighting everything below the last row and hitting clear but it still has the same affect..
Here's how I build my $csv array:
$skus = $csv = [];
if (($handle = fopen($fileTmp, 'r')) !== false) {
set_time_limit(0);
$i = 0;
while (($csvData = fgetcsv($handle, 1000, ',')) !== false)
{
$colCount = count($csvData);
$csv[$i]['sku'] = $csvData[0];
$csv[$i]['desc'] = $csvData[1];
$csv[$i]['ean'] = $csvData[2];
$csv[$i]['rrp_less_vat'] = $csvData[3];
$csv[$i]['rrp_inc_vat'] = $csvData[4];
$csv[$i]['stock'] = $csvData[5];
$csv[$i]['est_delivery'] = $csvData[6];
$csv[$i]['img_name'] = $csvData[7];
$csv[$i]['vatable'] = $csvData[8];
$csv[$i]['obsolete'] = $csvData[9];
$csv[$i]['dead'] = $csvData[10];
$csv[$i]['replacement_product'] = $csvData[11];
$csv[$i]['brand'] = $csvData[12];
$csv[$i]['ext_desc'] = $csvData[13];
$i++;
}
fclose($handle);
}
Am I doing something wrong that I can't see in building the array or is this unexpected behaviour?
PHP version: 7.1
OS: Linux Mint
You have lines that are longer than the $length argument you are passing to fgetcsv(). From the documentation, emphasis mine:
Must be greater than the longest line (in characters) to be found in the CSV file (allowing for trailing line-end characters). Otherwise the line is split in chunks of length characters, unless the split would occur inside an enclosure.
The easiest fix is to stop limiting the length of the line to 1000:
while (($csvData = fgetcsv($handle)) !== false)
Read from a csv file and pick certain columns and write again to another csv file. Everything fine except that the last column is always get wrap around with a pair of double quotes and the last double quotes is in a new line, which is strange. Below is my code and the output
if( file_exists($file_path) && filesize($file_path) > 0){
$counter = 0;
if(false !== ($read_file = fopen($file_path,'r')) ){
$output_file = fopen($file_output,'w');
while(false !== ($data = fgetcsv($read_file))){
$outputData = array($data[1], $data[6], $data[19]);
fputcsv($output_file, $outputData,',');
}
}
fclose($read_file);
fclose($output_file);
unset($outputData);
}
Output: data1,data6,"data9
"
Yes, so the last double quotes is in the newline and the last column is wrapped with a double quotes regards of what the data type is.
Thank you very much
Hello everyone and I immediately apologize, as
I have seen various threads on the site, but unfortunately my knowledge is still insufficient to complete my project.
I have a text file and I have to do the sum of each column (just need the total):
1003|name1|1208.00|2.00 |96.00 |0.00|0.00|0.00|0.00|98.00 |90.95 |7.05 |8516.40
1011|name2|1450.00|2.00 |49.00 |0.00|0.00|0.00|0.00|51.00 |44.62 |6.38 |9243.7
1004|name3|1450.00|25.00|170.00|0.00|0.00|0.00|0.00|195.00|175.75|19.25|27912.5 <br>
1002|name4|765.00 |1.00 |17.00 |0.00|0.00|0.00|0.00|18.00 |15.13 |2.87 |2193.26
I need to get this(I have this file on linux then we can use Bash, PHP, Mysql... ):
1003|name1|1208.00|2.00 |96.00 |0.00|0.00|0.00|0.00|98.00 |90.95 |7.05 |8516.40
1011|name2|1450.00|2.00 |49.00 |0.00|0.00|0.00|0.00|51.00 |44.62 |6.38 |9243.7
1004|name3|1450.00|25.00|170.00|0.00|0.00|0.00|0.00|195.00|175.75|19.25|27912.5 <br>
1002|name4|765.00 |1.00 |17.00 |0.00|0.00|0.00|0.00|18.00 |15.13 |2.87 |2193.26 <br>
xxxx|Total |4873.00|30.00|332.00|0.00|0.00|0.00|0.00|362.00 |326.45|35.55|47865.86
Where xxxx is the Id number (No sum here).
I've been trying to do this in PHP and MySQL -- No luck so far.
try something like:
$file = '/path/to/your_file.txt';
if ( ($file = fopen($file, "r")) !== FALSE) {
$total = 0;
$row_1 = 0;
while (($line = fgetcsv($file, 1000, "|")) !== FALSE) {
// brutal dirt sanitization
foreach ( $line as $k => $v ) {
$line[$k] = (float) preg_replace('#[^0-9\.]#','', $v);
}
$total = $total + array_sum(array_slice($line, 2));
$row_1 = $row_1 + array_sum(array_slice($line, 2, 1));
//...
}
echo $total.' | '.$row_1; //...
}
else echo 'error ...';
also, you can sanitize each row by replacing array_sum() by array_map() wih a callback function
Psuedocode:
open source file for reading
open destination file for writing
initialise totaling array to zero values
while not EOF
read in line from file
explode line into working array
for x=2 ; x<14; x++
add totalling array with floatval( working array )
write line out to destination file
close read file
write out totals array to destination file
close destingation file
Try to get the text file data into an excel spreadsheet and then add up the columns.
You can use VB to get the text into excel and then continue adding up the values of each column.
1) replace all | chars with , using str_replace
2) Use str_getcsv to create array out of the above resulting csv string
3) use foreach and loop through each row and calculate total
some PHP code
$str = file_get_contents('myfile.txt');
$str = str_replace('|', ',', $str);
$csv = str_getcsv($str);
$totals = array(0,0,0,0);
foreach ($csv as $row) {
$totals[0] += trim($row[0]);
$totals[1] += trim($row[2]);
$totals[2] += trim($row[3]);
$totals[3] += trim($row[4]);
}
the $totals array contains all totals!
How can I trim these 7 columns down to 5 columns, by running some script of sorts.
I recall you can do this using regex / php but buggered if I can recall how we did it.
Example code ( of a GEO IP db ( 115,000 lines )
"3231296768","3231297023","ripencc","702518400","EU","EU","European Union"
"3231297024","3231297279","ripencc","441763200","EU","EU","European Union"
"3231297280","3231297535","ripencc","702518400","EU","EU","European Union"
"3231297536","3231297791","ripencc","702518400","EU","EU","European Union"
"3231297792","3231298047","ripencc","702518400","EU","EU","European Union"
"3231298048","3231298303","ripencc","702518400","EU","EU","European Union"
"3231298304","3231298559","ripencc","702518400","EU","EU","European Union"
I need to remove columns 3 and 4 from every line.
Any help appreciated.
While jimw's answer is the best answer in general, if you want a pure PHP solution I would suggest the following:
$input = 'input.txt';
$output = 'output.txt';
if (false !== ($ih = fopen($input, 'r'))) {
$oh = fopen($output, 'w');
while (false !== ($data = fgetcsv($ih))) {
// this is where you build your new row
$outputData = array($data[0], $data[1], $data[4], $data[5], $data[6]);
fputcsv($oh, $outputData);
}
fclose($ih);
fclose($oh);
}
From 'a script of sorts' and 'regex/PHP', I infer that you just want this done, and don't care what language is used. If you're on *nix:
cut -d, -f1,2,5,6,7 file.csv
'cut' is a standard unix command-line utility, found on everything from OS X to AIX. The arguments I've used are:
-d, # this sets the 'delimiter' to a comma, for CSV
-f1,2,5,6,7 # this selects which fields to print
So together, it takes a file where each line consists of fields separated by commas and prints out fields one to five of it.
The same effect can be achieved in any programming language. I don't know PHP very well, so I won't attempt to produce it in PHP.
Edit: From the PHP docs, adapted slightly:
function apply_quotes($string) {
return '"'.$string.'"';
}
$row = 1;
if (($handle = fopen("test.csv", "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) {
$data = array_map("apply_quotes", $data);
echo join(",", array($data[0], $data[1], $data[4], $data[5], $data[6]))."\n";
}
fclose($handle);
}