PHP has a function for converting CSV strings to PHP arrays, but not a function for vice-versa, so I wrote one:
function echocsv(array $arr, $quo = '"', $sep = ',') {
$escape = $quo . $quo;
foreach($arr as &$val) {
if(strpos($val, $quo) !== false) {
$val = $quo . str_replace($quo, $escape, $val) . $quo;
}
}
echo implode($sep, $arr) . PHP_EOL;
}
Is there anything I'm overlooking? From wikipedia it basically says that quotes should be escaped with another quote, and that's pretty much all there is to it. The .CSV file will need to be openable in MS Excel.
My primitive tests seem to suggest it's working.
(I'm echoing it rather than returning a string because I'm going to stream it right to the browser)
$stdout = fopen('php://output','w'); // 'stdout' is for CLI, 'output' is for Browser
fputcsv($stdout, array('val,ue1','val"ue2','value3','etc'));
fflush($stdout); // flush for fun :)
fclose($stdout);
^ this outputs CSV to the Browser.
PHP does indeed contain the function you need: fputcsv()
To stream to the browser, use stdout as your "file":
$stdout = fopen('php://stdout','w');
fputcsv($stdout, array('val,ue1','val"ue2','value3','etc'));
fclose($stdout);
You also need to check if $val contains $sep (i.e. quote the string if it contains a comma):
if (strpos($val, $quo) !== false || strpos($val, $sep) !== false) {
...
}
Otherwise, fputcsv() will do the job (but only to a file/stream).
Related
I'm creating an update mechanism for my first program written in c++.
Theory is:
program sends it's version to the server php as a http header
server checks if later version exists
if it does, server sends the new binary to the client.
Most of it works however the binary received is malformed. When I compare the malformed exe with the working exe I have differences at places where I have \r\ns in the compiled exe. Seems like the \r is doubled.
My c++ code for downloading:
void checkForUpdates () {
SOCKET sock = createHttpSocket (); // creates the socket, nothing wrong here, other requests work
char* msg = (char*)"GET /u/2 HTTP/1.1\r\nHost: imgup.hu\r\nUser-Agent: imgup uploader app\r\nVersion: 1\r\n\r\n";
if (send(sock, msg, strlen(msg), 0) == SOCKET_ERROR) {
error("send failed with error\n");
}
shutdown(sock, SD_SEND);
FILE *fp = fopen("update.exe", "w");
char answ[1024] = {};
int iResult;
bool first = false;
do {
if ((iResult = recv(sock, answ, 1024, 0)) < 0) {
error("recv failed with error\n");
}
if (first) {
info (answ); // debug purposes
first = false;
} else {
fwrite(answ, 1, iResult, fp);
fflush(fp);
}
} while (iResult > 0);
shutdown(sock, SD_RECEIVE);
if (closesocket(sock) == SOCKET_ERROR) {
error("closesocket failed with error\n");
}
fclose(fp);
delete[] answ;
}
and my php to process the request
<?php
if (!function_exists('getallheaders')) {
function getallheaders() {
$headers = '';
foreach ($_SERVER as $name => $value) {
if (substr($name, 0, 5) == 'HTTP_') {
$headers[str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', substr($name, 5)))))] = $value;
}
}
return $headers;
}
}
$version = '0';
foreach (getallheaders() as $name => $value) {
if (strtolower ($name) == 'version') {
$version = $value;
break;
}
}
if ($version == '0') {
exit('error');
}
if ($handle = opendir('.')) {
while (false !== ($entry = readdir($handle))) {
if ($entry != '.' && $entry != '..' && $entry != 'u.php') {
if (intval ($entry) > intval($version)) {
header('Content-Version: ' . $entry);
header('Content-Length: ' . filesize($entry));
header('Content-Type: application/octet-stream');
echo "\r\n";
ob_clean();
flush();
readfile($entry);
exit();
}
}
}
closedir($handle);
}
echo 'error2';
?>
notice the way I flush content after I send the headers ob_clean(); flush(); so I don't have to parse them in c++. The first bytes written to the file are fine, so I doubt there is any problem here.
Also, example comparison of the binaries http://i.imgup.hu/meC16C.png
Question: Does http escape \r\n in binary file transfers? If not, what is causing this behavior and how do I solve this problem?
fopen opens a File in the mode you specified, first read/write/both, then Append, then a binary identifier.
r/w should be clear to you, append is also quite obvious. The Trick & Trouble in your case is the binary-mode.
If a file is threated as a Text-File (without the "b") then, depending on the environment where the application runs, some special character conversion may occur in input/output operations in text mode to adapt them to a system-specific text file format. On Windows this would be \r\n, on a linux machine you have \n and on some architectures exist, where it is \r.
In your case, the input file is read as a text file. This means, all your line-endings get converted when reading the file from the HTTP-Data.
Opening the File as a binary file (wich indeed it is!) avoids trouble that your file is not binary identically anymore.
The problem is that the output file isn't being opened in binary mode. To do that, change the mode to "wb" versus just "w" like this:
FILE *fp = fopen("update.exe", "wb");
In text mode on Windows the ctrl+z character specifies the end of the file when seeking/reading, and the linefeed character \n is translated to \r\n when writing and \r\n pairs are translated to \n on reading. In binary mode, the file data is not interpreted or translated in any way.
On other platforms the translations may not apply, but it is still good practice to show the intent of the code by specifying the explicit mode even when not strictly necessary. This is especially true for code meant to be portable.
I seem to be in a catch-22 with a small app I'm developing in PHP on Google App Engine using Quercus;
I have a remote csv-file which I can download & store in a string
To parse that string I'd ideally use str_getcsv, but Quercus doesn't have that function yet
Quercus does seem to know fgetcsv, but that function expects a file handle which I don't have (and I can't make a new one as GAE doesn't allow files to be created)
Anyone got an idea of how to solve this without having to dismiss the built-in PHP csv-parser functions and write my own parser instead?
I think the simplest solution really is to write your own parser . it's a piece of cake anyway and will get you to learn more regex- it makes no sense that there is no csv string to array parser in PHP so it's totally justified to write your own. Just make sure it's not too slow ;)
You might be able to create a new stream wrapper using stream_wrapper_register.
Here's an example from the manual which reads global variables: http://www.php.net/manual/en/stream.streamwrapper.example-1.php
You could then use it like a normal file handle:
$csvStr = '...';
$fp = fopen('var://csvStr', 'r+');
while ($row = fgetcsv($fp)) {
// ...
}
fclose($fp);
this shows a simple manual parser i wrote with example input with qualifed, non-qualified, escape feature. it can be used for the header and data rows and included an assoc array function to make your data into a kvp style array.
//example data
$fields = strparser('"first","second","third","fourth","fifth","sixth","seventh"');
print_r(makeAssocArray($fields, strparser('"asdf","bla\"1","bl,ah2","bl,ah\"3",123,34.234,"k;jsdfj ;alsjf;"')));
//do something like this
$fields = strparser(<csvfirstline>);
foreach ($lines as $line)
$data = makeAssocArray($fields, strparser($line));
function strparser($string, $div = ",", $qual = "\"", $esc = "\\") {
$buff = "";
$data = array();
$isQual = false; //the result will be a qualifier
$inQual = false; //currently parseing inside qualifier
//itereate through string each byte
for ($i = 0; $i < strlen($string); $i++) {
switch ($string[$i]) {
case $esc:
//add next byte to buffer and skip it
$buff .= $string[$i+1];
$i++;
break;
case $qual:
//see if this is escaped qualifier
if (!$inQual) {
$isQual = true;
$inQual = true;
break;
} else {
$inQual = false; //done parseing qualifier
break;
}
case $div:
if (!$inQual) {
$data[] = $buff; //add value to data
$buff = ""; //reset buffer
break;
}
default:
$buff .= $string[$i];
}
}
//get last item as it doesnt have a divider
$data[] = $buff;
return $data;
}
function makeAssocArray($fields, $data) {
foreach ($fields as $key => $field)
$array[$field] = $data[$key];
return $array;
}
if it can be dirty and quick. I would just use the
http://php.net/manual/en/function.exec.php
to pass it in and use sed and awk (http://shop.oreilly.com/product/9781565922259.do) to parse it. I know you wanted to use the php parser. I've tried before and failed simply because its not vocal about its errors.
Hope this helps.
Good luck.
You might be able to use fopen with php://temp or php://memory (php.net) to get it to work. What you would do is open either php://temp or php://memory, write to it, then rewind it (php.net), and then pass it to fgetcsv. I didn't test this, but it might work.
I'm using fputcsv in PHP to output a comma-delimited file of a database query. When opening the file in gedit in Ubuntu, it looks correct - each record has a line break (no visible line break characters, but you can tell each record is separated,and opening it in OpenOffice spreadsheet allows me to view the file correctly.)
However, we're sending these files on to a client on Windows, and on their systems, the file comes in as one big, long line. Opening it in Excel, it doesn't recognize multiple lines at all.
I've read several questions on here that are pretty similar, including this one, which includes a link to the really informative Great Newline Schism explanation.
Unfortunately, we can't just tell our clients to open the files in a "smarter" editor. They need to be able to open them in Excel. Is there any programmatic way to ensure that the correct newline characters are added so the file can be opened in a spreadsheet program on any OS?
I'm already using a custom function to force quotes around all values, since fputcsv is selective about it. I've tried doing something like this:
function my_fputcsv($handle, $fieldsarray, $delimiter = "~", $enclosure ='"'){
$glue = $enclosure . $delimiter . $enclosure;
return fwrite($handle, $enclosure . implode($glue,$fieldsarray) . $enclosure."\r\n");
}
But when the file is opened in a Windows text editor, it still shows up as a single long line.
// Writes an array to an open CSV file with a custom end of line.
//
// $fp: a seekable file pointer. Most file pointers are seekable,
// but some are not. example: fopen('php://output', 'w') is not seekable.
// $eol: probably one of "\r\n", "\n", or for super old macs: "\r"
function fputcsv_eol($fp, $array, $eol) {
fputcsv($fp, $array);
if("\n" != $eol && 0 === fseek($fp, -1, SEEK_CUR)) {
fwrite($fp, $eol);
}
}
This is an improved version of #John Douthat's great answer, preserving the possibility of using custom delimiters and enclosures and returning fputcsv's original output:
function fputcsv_eol($handle, $array, $delimiter = ',', $enclosure = '"', $eol = "\n") {
$return = fputcsv($handle, $array, $delimiter, $enclosure);
if($return !== FALSE && "\n" != $eol && 0 === fseek($handle, -1, SEEK_CUR)) {
fwrite($handle, $eol);
}
return $return;
}
Using the php function fputcsv writes only \n and cannot be customized. This makes the function worthless for microsoft environment although some packages will detect the linux newline also.
Still the benefits of fputcsv kept me digging into a solution to replace the newline character just before sending to the file. This can be done by streaming the fputcsv to the build in php temp stream first. Then adapt the newline character(s) to whatever you want and then save to file. Like this:
function getcsvline($list, $seperator, $enclosure, $newline = "" ){
$fp = fopen('php://temp', 'r+');
fputcsv($fp, $list, $seperator, $enclosure );
rewind($fp);
$line = fgets($fp);
if( $newline and $newline != "\n" ) {
if( $line[strlen($line)-2] != "\r" and $line[strlen($line)-1] == "\n") {
$line = substr_replace($line,"",-1) . $newline;
} else {
// return the line as is (literal string)
//die( 'original csv line is already \r\n style' );
}
}
return $line;
}
/* to call the function with the array $row and save to file with filehandle $fp */
$line = getcsvline( $row, ",", "\"", "\r\n" );
fwrite( $fp, $line);
As webbiedave pointed out (thx!) probably the cleanest way is to use a stream filter.
It is a bit more complex than other solutions, but even works on streams that are not editable after writing to them (like a download using $handle = fopen('php://output', 'w'); )
Here is my approach:
class StreamFilterNewlines extends php_user_filter {
function filter($in, $out, &$consumed, $closing) {
while ( $bucket = stream_bucket_make_writeable($in) ) {
$bucket->data = preg_replace('/([^\r])\n/', "$1\r\n", $bucket->data);
$consumed += $bucket->datalen;
stream_bucket_append($out, $bucket);
}
return PSFS_PASS_ON;
}
}
stream_filter_register("newlines", "StreamFilterNewlines");
stream_filter_append($handle, "newlines");
fputcsv($handle, $list, $seperator, $enclosure);
...
alternatively, you can output in native unix format (\n only) then run unix2dos on the resulting file to convert to \r\n in the appropriate places. Just be careful that your data contains no \n's . Also, I see you are using a default separator of ~ . try a default separator of \t .
I've been dealing with a similiar situation. Here's a solution I've found that outputs CSV files with windows friendly line-endings.
http://www.php.net/manual/en/function.fputcsv.php#90883
I wasn't able to use the since I'm trying to stream a file to the client and can't use the fseeks.
windows needs \r\n as the linebreak/carriage return combo in order to show separate lines.
I did eventually get an answer over at experts-exchange; here's what worked:
function my_fputcsv($handle, $fieldsarray, $delimiter = "~", $enclosure ='"'){
$glue = $enclosure . $delimiter . $enclosure;
return fwrite($handle, $enclosure . implode($glue,$fieldsarray) . $enclosure.PHP_EOL);
}
to be used in place of standard fputcsv.
I have an array of strings and want a way to create a CSV line from them. Something like:
$CSV_line = implode(',',$pieces);
Will not work as the pieces may contain comma and double quotes.
Is there a PHP built in function that takes the pieces and return a well formatted CSV line?
Thanks,
Roge
If you want to write that line to a file, you can use fputcsv
Using the streams functionnality of PHP, it should be possible to write to a variable -- indeed, there's one guy in the comments of str_getcsv who posted his implementation of str_putcsv :
<?php
function str_putcsv($input, $delimiter = ',', $enclosure = '"') {
// Open a memory "file" for read/write...
$fp = fopen('php://temp', 'r+');
// ... write the $input array to the "file" using fputcsv()...
fputcsv($fp, $input, $delimiter, $enclosure);
// ... rewind the "file" so we can read what we just wrote...
rewind($fp);
// ... read the entire line into a variable...
$data = fread($fp, 1048576); // [changed]
// ... close the "file"...
fclose($fp);
// ... and return the $data to the caller, with the trailing newline from fgets() removed.
return rtrim( $data, "\n" );
}
?>
Note : this code is not mine -- it's a copy-paste of the one posted by Ulf on php.net.
You could even write a Class to do this
class CSV_ARRAY {
static public function arr_to_csv_line($arr) {
$line = array();
foreach ($arr as $v) {
$line[] = is_array($v) ? self::arr_to_csv_line($v) : '"' . str_replace('"', '""', $v) . '"';
}
return implode(",", $line);
}
static public function arr_to_csv($arr) {
$lines = array();
foreach ($arr as $v) {
$lines[] = self::arr_to_csv_line($v);
}
return implode("\n", $lines);
}
}
you could useit like this
$csvlines = CSV_ARRAY::arr_to_csv_line($myarray);
file_put_contents($csvlines,"mycsv.csv");
thats it ;-)
As far as I know there is no such built in function. The one I'm aware of is fputcsv that does exactly what you want
but it does not return the CSV line, but writes it to a file.
<?php
$pieces = array (
'a,b,c,d', // contains comma.
'"1","2"' // contains double quotes.
);
$fp = fopen('file.csv', 'w');
fputcsv($fp, $pieces); // "a,b,c,d","""1"",""2""" written
fclose($fp);
?>
Most of the times you want create the line of CSV and write it to the file. So the above function should suffice.
I'd written one such function for PHP4 as it does not support fputcsv. I'll share it with you:
// If a value contains a comma, a quote, a space, a tab, a newline, or a linefeed,
// then surround it with quotes and replace any quotes inside it with two quotes
function make_csv_line($values)
{
// iterate through the array ele by ele.
foreach($values as $key => $value)
{
// check for presence of special char.
if ( (strpos($value, ',') !== false) || (strpos($value, '"') !== false) ||
(strpos($value, ' ') !== false) || (strpos($value, "\t") !== false) ||
(strpos($value, "\n") !== false) || (strpos($value, "\r") !== false))
{
// if present, surround the ele in quotes..also replace
// already existing " with ""
$values[$key] = '"' . str_replace('"', '""', $value) . '"';
}
}
// now create the CSV line by joining with a comma, also put a \n at the end.
return implode(',', $values) . "\n";
}
Here is a simpler fputcsv implementation:
$stdout = fopen('php://output','w'); // 'stdout' is for CLI, 'output' is for Browser
fputcsv($stdout, array('val,ue1','val"ue2','value3','etc'));
fflush($stdout); // flush for fun :)
fclose($stdout);
I snarfed this elswhere on SO and posted here for faster reference, please bump/credit the original posting:
Is my array to csv function correct?
Well, the thing is, there is no well formatted CSV. For instance, Open Office uses the semicolon (;) as default delimiter, while Excel expects it to be a comma (,). But fact is, no standard exists.
Assuming the Array looks something like this:
Array
0 => Some string
1 => Some other string, with comma
2 => And a third one with "double quotes"
);
You could either use implode with a different delimiter, e.g.
$delimiter = chr(9); // Tab
$delimiter = ';' // Semicolon. Excel's default
$delimiter = '|' // Pipe
$delimiter = "','"; // Comma plus Single Quote
If that is not working, run the String array through array_filter first and change the double quotes and comma via callback to whatever you think they should be, before exploding with just a comma delimiter:
$csv = implode(',' array_filter($array, 'my_csv_prepare'));
But keep in mind, this does not guarantee your consuming application to be able to read the CSV. It really depends on how they implemented CSV parsing. Like I said: no standard.
I am reading a CSV file with php. Many of the rows have a "check mark" which is really the square root symbol: √ and the php code is just skipping over this character every time it is encountered.
Here is my code (printing to the browser window in "CSV style" format so I can check that the lines break at the right place:
$file = fopen($uploadfile, 'r');
while (($line = fgetcsv($file)) !== FALSE) {
foreach ($line as $key => $value) {
if ($value) {
echo $value.",";
}
}
echo "<br />";
}
fclose($file);
As an interim solution, I am just finding and replacing the checkmarks with 1's manually, in Excel. Obviously I'd like a more efficient solution :) Thanks for the help!
fgetcsv() only works on standard ASCII characters; so it's probably "correct" in skipping your square root symbols. However, rather than replacing the checkmarks manually, you could read the file into a string, do a str_replace() on those characters, and then parse it using fgetcsv(). You can turn a string into a file pointer (for fgetcsv) thusly:
$fp = fopen('php://memory', 'rw');
fwrite($fp, (string)$string);
rewind($fp);
while (($line = fgetcsv($fp)) !== FALSE)
...
I had a similar problem with accented first characters of strings. I eventually gave up on fgetscv and did the following, using fgets() and explode() instead (I'm guessing your csv is comma separated):
$file = fopen($uploadfile, 'r');
while (($the_line = fgets($file)) !== FALSE) // <-- fgets
{
$line = explode(',', $the_line); // <-- explode
foreach ($line as $key => $value)
{
if ($value)
{
echo $value.",";
}
}
echo "<br />";
}
fclose($file);
You should setlocale ar written in documentation
Note:
Locale setting is taken into account by this function. If LANG is e.g. en_US.UTF-8, files in one-byte encoding are read wrong by this function.
before fgetcsv add setlocale(LC_ALL, 'en_US.UTF-8'). In my case it was 'lt_LT.UTF-8'.
This behaviour is reported as a php bug