str_getcsv to separate lines removes enclosing characters in lines - php

Question

str_getcsv () has some odd behaviour. It removes all characters that match the enclosing character, instead of just the enclosing ones. I'm trying to parse a CSV string (contents of an uploaded file) in two steps:
split the CSV string into an array of lines
split each line into an array of fields
with this code:
$whole_file_string = file_get_contents($file);
$array_of_lines = str_getcsv ($whole_file_string, "\n", "\""); // step 1. split csv into lines
foreach ($array_of_lines as $one_line_string) {
$splitted_line = str_getcsv ($one_line_string, ",", "\""); // step 2. split line into fields
};
In the code example nothing is done with $splitted_line for clarity of example
Then I feed this script a file with the following contents: "text,with,delimiter",secondfield.
When step 1 is performed the first (and only) element of $array_of_lines is text,with,delimiter,secondfield. So when step 2 is performed, it splits the line into 4 fields, but that needs to be 2.
I can't use fgetcsv() because some string conversion is done (checking BOM, converting encoding accordingly and stuff like that) after reading the file and before splitting it into lines in step 1.
I'm at the point of writing my own string parser (which isn't that complicated for CSV format), but before I do so I want to make sure that that's the best approach. I'm a bit disappointed that the PHP functions are letting me down on this simple (and I guess quite common) use case: processing an uploaded csv file with varying encoding.
Any tips?

Answer 1

You should only be calling str_getcsv() on one line at a time, not the whole file.
$array_of_lines = file($file, FILE_IGNORE_NEW_LINES); // split CSV into lines
foreach($array_of_lines as $one_line_string) {
$splitted_line = str_getcsv($one_line_string, ",", "\""); // split line into fields
}

Answer 2

Here's my own CSV parser, fully compliant with IETF rfc 4180.
I'm curious to know if this can be done in regular expressions, those are not my forte.
/**
* Parse a string according to CSV format (https://tools.ietf.org/html/rfc4180), with variabele delimiter (default ,).
* #param string $string String to be parsed as csv
* #param string $delimiter character to be used as field delimiter
* #return array Array with for each line an array with csv field values
*/
function csv_parse ($string, $delimiter = ",", $line_mode = true) {
// This function parses on line-level first ($line_mode = true) and calls itself recursively to parse each line on field-level ($line_mode = false).
// when in line mode, the delimiter is eol (\n, \r\n and \n\r).
// when in field mode, the delimiter is the passed $delimiter.
$delimiter = substr ($delimiter,0,1); // delimiter is one character
$length = strlen ($string);
$parsed_array = array();
$end_of_line_state = false;
$enclosed_state = false;
$i = 0;
$field = "";
do {
switch (true) {
case (!$enclosed_state && $end_of_line_state && ($string[$i] == "\r" && $string[$i-1] == "\n")) :
case (!$enclosed_state && $end_of_line_state && ($string[$i] == "\n" && $string[$i-1] == "\r")) :
// ...found second character of eol (\r\n of \n\r). Ignore
$end_of_line_state = false;
break;
case (!$enclosed_state && !$end_of_line_state && ($string[$i] == "\n" || $string[$i] == "\r")) :
// ... found first character of eol \n, \r\n of \n\r
$end_of_line_state = true; // eol can be two characters, so prepare for the second
if ($field != "") { // ignore empty lines. Prohibited in csv
$parsed_array [] = csv_parse ($field, $delimiter, false); // recursive call to parse on field-level. Flush result
};
$field = ""; // prepare for next one
break;
case (!$enclosed_state && $string[$i] == $delimiter && !$line_mode) :
// ...delimiter found
$parsed_array [] = $field; // flush field as new array element
$field = ""; // prepare for next one
break;
case ($string[$i] == "\"") :
// ...encloser found
if ($enclosed_state) {
if ($i < $length && $string[$i+1] == "\"") {
// ... escaped " found
if (!$line_mode) {
$field .= "\""; // when parsing fieldlevel, only " is part of the line
} else {
$field .= "\"\""; // when parsing line level, the escaping " is also part of the line
};
$i++;
} else {
// ...closing encloser found
$enclosed_state = false;
if ($line_mode) {
$field .= $string[$i]; // when parsing line level, the enclosing " are part of the line
};
};
} else {
// ... opening encloser found
$enclosed_state = true;
if ($line_mode) {
$field .= $string[$i]; // when parsing line level, the enclosing " are part of the line
};
};
break;
default:
// ...regular character found
$field .= $string[$i];
};
$i++;
if ($i >= $length) { // end of string
if ($line_mode) {
$parsed_array [] = csv_parse ($field, $delimiter, false); // recursive call to parse on field-level. Flush result.
} else {
$parsed_array [] = $field; // flush last field
};
};
} while ($i < $length);
return $parsed_array;
};

Answer 3

Hm. Strange. I'm fiddling around a bit and I discovered something interesting:
$a = str_getcsv("\"#ne piece, #f text\"", "\n");
$b = str_getcsv("\"#ne piece, #f text\"", "\n","\"");
$c = str_getcsv("\"#ne piece, #f text\"", "\n","#");
echo $a; // #ne piece, #f text
echo $b; // #ne piece, #f text
echo $c; // "#ne piece, #f text"
So Passing an # as enclosing character instead of " (or leaving it out and using the default, which is " as well) is semantically bullocks, but it does the job. It leaves the enclosing " around the field, so if you then str_getcsv($c, ",") it results in one value as it should. And if you have a field enclosed in # it leaves it untouched. I tested that.
It is clear that str_getcsv has its flaws. On splitting to lines it shouldn't strip quotes, just like it doesn't strip # when that's the enclosing parameter. But sadly it does and therefore isn't CSV compliant.