from csv to array with end of line chars - php

To parse CSV files in php im using this function:
private function _csvToArray($url, $delimiter=',')
{
$csvData = file_get_contents($url);
$lines = explode(PHP_EOL, $csvData);
$array = array();
foreach ($lines as $line) {
$array[] = str_getcsv($line, $delimiter);
}
return $array;
}
The problem here is Im using EOL to determine where a line ends, if the CSV file have any field with any end of line chars im getting errors.
Example:
Product_Name, "Description"
Product_Name, "Description"
Product_Name, "Description"
Product_Name, "Description"
This works ok, but if I have something like this:
Product_Name, "Description_line_1
Description_line_2"
Product_Name, "Description_line_1
Description_line_2"
Product_Name, "Description_line_1
Description_line_2"
The script will fail, is there any way I can improve the script in order to consider this or is better to use a regular expression to fix first the CSV before calling the sript?

If you want to save writing to a temporary file yourself you can use the memory stream.
private function _csvToArray($url, $delimiter=',')
{
$fp = fopen('php://memory', 'r+');
fwrite($fp, file_get_contents($url));
fseek($fp, 0);
$array = array();
while ($row = fgetcsv($fp, 0, $delimiter)) {
$array[] = $row;
}
fclose($fp);
return $array;
}

fgetcsv can handle EOL in fields if the field data is between enclosure characters.
private function _csvToArray($url, $delimiter=',', $enclosure='"')
{
$handle = fopen($url, 'r');
$array = array();
while($row = fgetcsv($handle, 0, $delimiter, $enclosure))
{
$array[] = $row;
}
fclose($handle);
return $array;
}

Something like this should work (havent properly tested the code):
$csv = array_map('str_getcsv', file($url), ',', '"');
I had an old code lying around which fixed this once for me... But remember... it's from way way back;
$url = 'file.csv';
$csv = array();
$csvContents = file_get_contents($url);
$lines = explode('"'."\n", trim($csvContents));
foreach($lines as $lineNumber => $line) {
$csv[$lineNumber] = array();
$fields = explode(',', $line);
foreach($fields as $field) {
$csv[$lineNumber][] = ltrim(rtrim($field, '"'), '"');
}
}

Related

skip columns while converting tab delimited text file to csv php

I am trying to convert a tab delimited file to csv. The problem is its a huge file. 100000 plus records. And i want only specific columns from that file. The file is not generated by me but by amazon so i cant really control the format.
The code i made works fine. But i need to ignore/remove some columns or rather i want only few columns from that. How do i do that without effecting the performance of conversion from txt to csv.
$file = fopen($file_name.'.txt','w+');
fwrite($file,$report);
fclose($file);
$handle = fopen($file_name.".txt", "r");
$lines = [];
$row_count=0;
$array_count = 0;
$uid = array($user_id);
if (($handle = fopen($file_name.".txt", "r")) !== FALSE)
{
while (($data = fgetcsv($handle, 100000, "\t")) !== FALSE)
{
if($row_count>0)
{
$lines[] = str_replace(",","<c>",$data);
array_push($lines[$array_count],$user_id);
$array_count++;
}
$row_count++;
}
fclose($handle);
}
$fp = fopen($file_name.'.csv', 'w');
foreach ($lines as $line)
{
fputcsv($fp, $line);
}
fclose($fp);
I am using unset to remove any column. But is there a better way ? for multiple columns.
I would do that by checking keys. For example:
// columns keys you don't wanna skip
$keys = array(0, 1, 3, 4, 7, 9);
$lines = file($file_name);
$result_lines = array();
foreach ($lines as $line) {
$tmp = array();
$tabs = explode("\t", $line);
foreach($tabs as $key => $value){
if(in_array($key, $keys)){
$tmp[] = $value;
}
}
$result_lines[] = implode(",", $tmp);
}
$finalString = implode("\n", $result_lines);
// Then write string to file
Hope it helps.
Cheers,
SiniĊĦa
In its simplest form i.e. without worrying about removing columns from the output this will do a simple read line and write line, therefore no need to maintain any memory hungry arrays.
$file_name = 'tst';
if ( ($f_in = fopen($file_name.".txt", "r")) === FALSE) {
echo 'Cannot find inpout file';
exit;
}
if ( ($f_out = fopen($file_name.'.csv', 'w')) === FALSE ) {
echo 'Cannot open output file';
exit;
}
while ($data = fgetcsv($f_in, 8000, "\t")) {
fputcsv($f_out, $data, ',', '"');
}
fclose($f_in);
fclose($f_out);
This is one way of removing the unwanted columns
$file_name = 'tst';
if ( ($f_in = fopen("tst.txt", "r")) === FALSE) {
echo 'Cannot find inpout file';
exit;
}
if ( ($f_out = fopen($file_name.'.csv', 'w')) === FALSE ) {
echo 'Cannot open output file';
exit;
}
$unwanted = [26,27]; //index of unwanted columns
while ($data = fgetcsv($f_in, 8000, "\t")) {
// remove unwanted columns
foreach($unwanted as $i) {
unset($data[$i]);
}
fputcsv($f_out, $data, ',', '"');
}
fclose($f_in);
fclose($f_out);

Add 2 new column header and contents in csv file using php

I've an existing csv file with following values
column1 column2
Fr-fc Fr-sc
Sr-fc Sr-sc
I want to add 2 new columns in it and achieve the following format
column1 column2 column3 column4
Fr-fc Fr-sc 1 2
Sr-fc Sr-sc 1 2
If I use following code it inserts same column header value in column data for the newly created columns
$a = file('amit.csv');// get array of lines
$new = '';
foreach($a as $line){
$line = trim($line);// remove end of line
$line .=";column3";// append new column
$line .=";column4";// append new column
$new .= $line.PHP_EOL;//append end of line
}
file_put_contents('amit2.csv', $new);// overwrite the same file with new data
How I can achieve the above?
Instead of reinventing the wheel, you can use php's inbuilt csv functions fgetcsv and fputcsv respectively to ease your work. First read in each row with fgetcsv and store the data in a multidimensional array:
$delimiter = "\t"; //your column separator
$csv_data = array();
$row = 1;
if (($handle = fopen('test.csv', 'r')) !== FALSE) {
while (($data = fgetcsv($handle, 1000, $delimiter)) !== FALSE) {
$csv_data[] = $data;
$row++;
}
fclose($handle);
}
Next edit the rows to add the extra columns using array_merge:
$extra_columns = array('column3' => 1, 'column4' => 2);
foreach ($csv_data as $i => $data) {
if ($i == 0) {
$csv_data[$i] = array_merge($data, array_keys($extra_columns));
} else {
$csv_data[$i] = $data = array_merge($data, $extra_columns);
}
}
Finally use fputcsv to enter each row into the csv.
if (($handle = fopen('test.csv', 'w')) !== FALSE) {
foreach ($csv_data as $data) {
fputcsv($handle, $data, $delimiter);
}
fclose($handle);
}
You can combine these steps to make your code more efficient by reducing the number of loops.
This approach is less code
<?php
$inFile = fopen('test.csv','r');
$outFile = fopen('output.csv','w');
$line = fgetcsv($inFile);
while ($line !== false) {
$line[] = 'third column';
$line[] = 'fourth column';
fputcsv($outFile, $line);
$line = fgetcsv($inFile);
}
fclose($inFile);
fclose($outFile);

PHP Processing CSV file separated by semicolon

I've a csv file with the following structure:
a; b; c,c c; d
When I try to process it, it says offset 2 and 3 are undefined. Took me a while to realize it is caused by the , and have no idea how to solve this. If I remove the , everything runs fine.
Here's my processing function:
function process_csv($file) {
$file = fopen($file, "r");
$data = array();
while (!feof($file)) {
$csvdata = fgetcsv($file);
$data[] = explode(';', $csvdata[0]);
}
fclose($file);
return $data;
}
Tried fgetcsv($file); as fgetcsv($file, '"'); but didn't help.
Your problem is, that fgetcsv uses , as delimiter by default. If you change it to ; you don't need to explode.
function process_csv($file) {
$file = fopen($file, "r");
$data = array();
while (!feof($file)) {
$data[] = fgetcsv($file, null, ';');
}
fclose($file);
return $data;
}

Use PHP to convert text file into array

I have a text file here which I need to be able to convert into rows to extract the second, third, fourth, and fifth values from.
The first 7 values of each row are tab delimited, then there is a newline, then the final three values are tab delimited.
I removed the interrupting newlines so that each row is fully tab delimited.
<?php
$file="140724.txt";
$fopen = fopen($file, "r");
$fread = fread($fopen,filesize("$file"));
fclose($fopen);
$remove = "\n";
split = explode($remove, $fread);
foreach ($split as $string)
{
echo "$string<br><br>";
}
?>
Which produces this.
I'm not sure where to progress from this point. I'm teaching myself PHP and am still quite new to it, so I don't even know if where I've started from is a good place. My instinct is to write the previous output to a new textfile, then create another block of code similar to the first but exploding based on tabs, this time.
Help?
You can process this file in one go like this:
<?php
$file="140724.txt";
$fopen = fopen($file, 'r');
$fread = fread($fopen,filesize($file));
fclose($fopen);
$remove = "\n";
$split = explode($remove, $fread);
$array[] = null;
$tab = "\t";
foreach ($split as $string)
{
$row = explode($tab, $string);
array_push($array,$row);
}
echo "<pre>";
print_r($array);
echo "</pre>";
?>
The result will be a jagged array:
You will need to clean up the 1st and the last element.
That is structured data, delimited by tabs. You can use fgetcsv() to read that data into an array. For an example see the PHP documentation.
<?php
$myfile = fopen("test.txt", "r") or die("Unable to open file!");
// Output one line until end-of-file
while(!feof($myfile)) {
$text[] = fgets($myfile);
}
fclose($myfile);
print_r($text);
?>
There is another answer here which converts file/raw strings into an associative array. It is really very handy in such cases.
function tab_to_array($src='', $delimiter=',', $is_file = true)
{
if($is_file && (!file_exists($src) || !is_readable($src)))
return FALSE;
$header = NULL;
$data = array();
if($is_file){
if (($handle = fopen($src, 'r')) !== FALSE)
{
while (($row = fgetcsv($handle, 1000, $delimiter)) !== FALSE)
{
if(!$header)
$header = $row;
else
$data[] = array_combine($header, $row);
}
fclose($handle);
}
}
else{
$strArr = explode("\n",$src);
foreach($strArr as $dataRow){
if($row = explode($delimiter,$dataRow))
{
if(!$header)
$header = $row;
else
$data[] = array_combine($header, $row);
}
}
}
return $data;
}
/**
* Example for file
*/
print_r(tab_to_array('example.csv'));
/**
* Example for raw string
*/
$str = "name number
Lorem 11
ipsum 22";
print_r(tab_to_array($str, "\t", false));

Convert csv in php and get unique value

I would like to convert a csv file that has duplicate contents and i would like to sum the quantity and extract the price without sum it.
file.csv :
code,qty,price
001,2,199
001,1,199
002,2,159
002,2,159
Actual php that sum the quantiy and get a result with unique value and total qty.
<?php
$tsvFile = new SplFileObject('file.csv');
$tsvFile->setFlags(SplFileObject::READ_CSV);
$tsvFile->setCsvControl("\t");
$file = fopen('file.csv', 'w');
$header = array('sku', 'qty');
fputcsv($file, $header, ',', '"');
foreach ($tsvFile as $line => $row) {
if ($line > 0) {
if (isset($newData[$row[0]])) {
$newData[$row[0]]+= $row[1];
} else {
$newData[$row[0]] = $row[1];
}
}
}
foreach ($newData as $key => $value) {
fputcsv($file, array($key, $value), ',', '"');
}
fclose($file);
?>
the result for this is:
code,qty
001,3
002,4
and i would like to add price, but without sum it.
The result i need is:
code,qty,price
001,3,199
002,4,159
I haven't tested this yet, but I think this is what you are looking for:
<?php
$tsvFile = new SplFileObject('file.csv');
$tsvFile->setFlags(SplFileObject::READ_CSV);
$tsvFile->setCsvControl("\t");
$file = fopen('file.csv', 'w');
$header = array('sku', 'qty');
fputcsv($file, $header, ',', '"');
foreach ($tsvFile as $line => $row) {
if ($line > 0) {
if(!isset($newData[$row[0]])) {
$newData[$row[0]] = array('qty'=>0, 'price'=>$row[2]);
}
$newData[$row[0]]['qty'] += $row[1];
}
}
foreach ($newData as $key => $arr) {
fputcsv($file, array($key, $arr['qty'], $arr['price']), ',', '"');
}
fclose($file);
?>
To start with, there's a nice function on the PHP page str_getcsv which will help you end up with a more legible array to work with:
function csv_to_array($filename='', $delimiter=',') {
if(!file_exists($filename) || !is_readable($filename))
return FALSE;
$header = NULL;
$data = array();
if (($handle = fopen($filename, 'r')) !== FALSE) {
while (($row = fgetcsv($handle, 1000, $delimiter)) !== FALSE) {
if(!$header)
$header = $row;
else
$data[] = array_combine($header, $row);
}
fclose($handle);
}
return $data;
}
This is purely for legibility sake but now comes the code which would allow you to work over the array.
$aryInput = csv_to_array('file.csv', ',');
$aryTemp = array();
foreach($aryInput as $aryRow) {
if(isset($aryTemp[$aryRow['code'])) {
$aryTemp[$aryRow['code']['qty'] += $aryRow['qty'];
} else {
$aryTemp[$aryRow['code']] = $aryRow;
}
}
In the above code, it simply:
Loops through the input
Checks whether the key exists in a temporary array
If it does, it just adds the new quantity
If it doesn't, it adds the entire row
Now you can write out your expectant csv file :)

Categories