small glitch while generating csv with newline character in php - php

I am simply generating a csv file based on data stored in a mysql table. The generated csv, when opened in excel, seems mostly ok, but whenever it has a newline character, excel puts the data on a new row. Any idea how to prevent that?
Sample data
line 1 some data
another data
CSV generation code:
header("Content-Type: text/csv; charset=UTF-8");
header("Content-Disposition: attachment; filename=\"".$MyFileName."\"");
$filename = $MyFileName;
$handle = fopen("temp_files/".$filename, "r");
$contents = fread($handle, filesize("temp_files/".$filename));
fclose($handle);
echo $contents;
exit;
content snippet I used to get rid of new line(didn't work):
$pack_inst = str_replace(',',' ',$get_data->fields['pack_instruction']);
$pack_inst = str_replace('\n',' ',$pack_inst);
$pack_inst = str_replace('\r',' ',$pack_inst);
$pack_inst = str_replace('\r\n',' ',$pack_inst);
$pack_inst = str_replace('<br>',' ',$pack_inst);
$pack_inst = str_replace('<br/>',' ',$pack_inst);
$pack_inst = str_replace(PHP_EOL, '', $pack_inst);
$pattern = '(?:[ \t\n\r\x0B\x00\x{A0}\x{AD}\x{2000}-\x{200F}\x{201F}\x{202F}\x{3000}\x{FEFF}]| |<br\s*\/?>)+';
$pack_inst = preg_replace('/^' . $pattern . '|' . $pattern . '$/u', ' ', $pack_inst);
$content .=','.$pack_inst;

According to RFC 4180, if a column's content contains the row delimiter (\r\n), the column delimiter (,) or the string delimiter (") then you must enclose the content inside double quotes ". When you do that, you must escape all " characters inside the content by preceding them with another ". So the following CSV content:
1: OK,2: this "might" work but not recommended,"3: new
line","4: comma, and text","5: new
line and ""double"" double quotes"
1: Line 2
Will produce 2 rows of CSV data, first one containing 5 columns.
Having said that, have a look at fputcsv() function. It will handle most gory details for you.

What you show is not the CSV generation code, it is simply the code that you have used to force a download to the browser. Regardless, the function that you need to sort this out is fputcsv(), which will automatically consider all sorts of edge cases that any code you write to convert tabular data to CSV format will likely not consider.
You say you are basing this on data in MySQL table, here is a basic framework for creating the CSV file, assuming the MySQLi extension used in a procedural manner:
<?php
// Connect to database and generate file name here
$fileName = 'file.csv';
// Get the data from the database
$query = "
SELECT *
FROM table_name
WHERE some_column = 'Some Value'
ORDER BY column_name
";
if (!$result = mysqli_query($db, $query)) {
// The query failed
// You may want to handle this with a more meaningful error message
header('HTTP/1.1 500 Internal Server Error');
exit;
} else if (!mysqli_num_rows($result)) {
// The query returned no results
// You may want to handle this with a more meaningful error message
header('HTTP/1.1 404 Not Found');
exit;
}
// Create a temporary file pointer for storing the CSV file
$tmpFP = fopen('php://temp', 'w+');
// We'll keep track of how much data we write to the file
$fileLength = 0;
// Create a column head row and write first row to file
$firstRow = mysqli_fetch_assoc($result);
$fileLength += fputcsv($tmpFP, array_keys($firstRow));
$fileLength += fputcsv($tmpFP, array_values($firstRow));
// Write the rest of the rows to the file
while ($row = mysqli_fetch_row($result)) {
$fileLength += fputcsv($tmpFP, $row);
}
// Send the download headers
header('Content-Type: text/csv; charset=UTF-8');
header('Content-Disposition: attachment; filename="'.$fileName.'"');
header('Content-Length: '.$fileLength);
// Free some unnecessary memory we are using
// The data might take a while to transfer to the client
mysqli_free_result($result);
unset($query, $result, $firstRow, $row, $fileName, $fileLength);
// Prevent timeouts on slow networks/large files
set_time_limit(0);
// Place the file pointer back at the beginning
rewind(tmpFP);
// Serve the file download
fpassthru($tmpFP);
// Close the file pointer
fclose($tmpFP);
// ...and we're done
exit;

Related

Exporting/Importing tables with different date format

Im having this problem, that when i export my table as csv file and open it on the excel: the date seems to have different format (in the database : yyyy-mm-dd, while on the excel dd/mm/yyyy)
Is there a way of solving these things to have the same format either when exporting or importing
This is my code when importing
elseif ($_POST["program"]=="dengue"){
$filename = $_FILES['file']['tmp_name'];
$file = fopen($filename, "r");
while (($getData = fgetcsv($file, 10000, ",")) !== FALSE)
{
$sql = "INSERT into lcho_dengue_activities (district_id,barangay_id,month,year,dengue_ind1,dengue_ind2,dengue_ind3,dengue_ind4,dengue_ind5,dengue_ind6,dengue_ind7,dengue_ind8,dengue_ind9,dengue_ind10,dengue_ind11,date)
values ('".$getData[0]."','".$getData[1]."','".$getData[2]."','".$getData[3]."','".$getData[4]."','".$getData[5]."','".$getData[6]."','".$getData[7]."','".$getData[8]."','".$getData[9]."','".$getData[10]."','".$getData[11]."','".$getData[12]."','".$getData[13]."','".$getData[14]."','".$getData[14]."')";
if(!mysqli_query($con, $sql))
{
echo ("Error description: " . mysqli_error($con));
}
else {
echo "<script type=\"text/javascript\">
alert(\"CSV File has been successfully Imported.\");
window.location = \"imports.php\"
</script>";
}
}
While this is on exporting
if($_POST["program"]=="dengue"){
// get records from database
$query = $db->query("SELECT * FROM lcho_dengue_activities ");
if($query->num_rows > 0){
$delimiter = ",";
$filename = "dengueactivities" . date('Y-m-d') . ".csv";
//create a file pointer
$f = fopen('php://memory', 'w');
//output each row of the data, format =line as csv and write to file pointer
while($row = $query->fetch_assoc()){
$lineData = array( $row['district_id'], $row['barangay_id'], $row['month'], $row['year'],$row['dengue_ind1'],$row['dengue_ind2'],$row['dengue_ind3'],$row['dengue_ind4'],$row['dengue_ind5'],$row['dengue_ind6'],$row['dengue_ind7'],$row['dengue_ind8'],$row['dengue_ind9'],$row['dengue_ind10'],$row['dengue_ind11'],$row['date']);
fputcsv($f, $lineData, $delimiter);
}
//move back to beginning of file
fseek($f, 0);
//set headers to download file rather than displayed
header('Content-Type: text/csv');
header('Content-Disposition: attachment; filename="' . $filename . '";');
//output all remaining data on a file pointer
fpassthru($f);
}
exit;
Need to have same format
You wrote in short:
"Im having this problem, that when i export my table as csv file and open it on the excel: the date seems to have different format (in the database : yyyy-mm-dd, while on the excel dd/mm/yyyy)"
Seems like that's just the nature of the beast. Remember that a .csv file is not an Excel file (which may store exact instructions on how to format everything). So when Excel opens a .csv file it tries to make some guesses about how to format columns of data and sometimes that may not be exactly what you intend.
A similar thing is trying to show something like 2 decimal places where Excel shows whatever it considers significant such as 0.12, 0.13, 0.1, 0. Another similar weird thing I've seen is if the first 10 or 20 rows or something in the .csv file are blank and then Excel often seems to have no clue on how to properly format the column.
To REALLY see exactly what is in a .csv file you can open it up in some kind of word processor such as a coding editor.

Writing into csv with php problems

I have a problem. I'm trying to get some data from a database into a .csv table.
$fn=fopen($path.$filename, "w");
$addstring = file_get_contents($path.$filename);
$addstring = 'Azonosito;Datum;Ido;Leiras;IP-cim;allomasnév;MAC-cim;Felhasznalonev;Tranzakcioazonosito;Lekerdezes eredmenye;Vizsgalat ideje;Korrelacios azonosito;DHCID;';
/*$addstring .= "\n";*/
$sql="select * from dhcpertekeles.dhcpk";
$result =mysqli_query($conn, $sql);
if ($result=mysqli_query($conn,$sql))
{
while ($row=mysqli_fetch_row($result))
{
$addstring .= "\n".$row[0].";".$row[1].";".$row[2].";".$row[3].";".$row[4].";".$row[5].";".$row[6].";".$row[7].";".$row[8].";".$row[9].";".$row[10].";".$row[11].";".$row[12].";";
};
};
/*file_put_contents($path.$filename, $addstring);*/
fwrite($fn, $addstring);
fclose($fn);
The data is in the following format:
The first addstring contains the column names, and has no issues
the second (addstring .=) contains the data:
ID($row[0]), Date($row[1]), Time($row[2]), Description($row[3]), IP($row[4]), Computer name($row[5]), MAC($row[6]), User($row[7])(empty), Transactionid($row[8]), query result($row[9]), query time($row[10]), correlation id($row[11])(empty), DHCID($row[12])(empty)
It is basically daily DHCP server data, uploaded to a database. Now, the code works, it does write everything i want to the csv, but there are 2 problems.
1, the code for some inexplicable reason, inserts an empty row into the csv table between the rows that contain the data. Removing $row[12] fixes this. I tried removing special characters, converting spaces into something that can be seen, and even converting empty string into something that can be seen. Yet nothing actually worked, i even tried file_puts_content(same for the second problem) instead of fwrite, but nothing. The same thing keeps happening. If i remove \n it will work, but the 2nd row onwards will be misplaced to the right by 1 column.
2, For some reason, the last 2 character is removed from the csv. The string that is to be inserted into the csv still contains said 2 characters before writing it to the file. Tried both fwrite and file_puts_content.
As for the .csv format, the data clumns are divided by ; and rows by \n.
Also tried reading the file with both libre office and excel thinking it might be excel that was splurging but no.
Try using fputcsv() function. I didn't test following code but I think it should work.
$file = fopen($path . $filename, 'w');
$header = array(
'Azonosito',
'Datum',
'Ido',
'Leiras',
'IP-cim',
'allomasnév',
'MAC-cim',
'Felhasznalonev',
'Tranzakcioazonosito',
'Lekerdezes eredmenye',
'Vizsgalat ideje',
'Korrelacios azonosito',
'DHCID'
);
fputcsv($file, $header, ';');
$sql = "select * from dhcpertekeles.dhcpk";
$result = mysqli_query($conn, $sql);
if ($result = mysqli_query($conn, $sql)) {
while ($row = mysqli_fetch_row($result)) {
fputcsv($file, $row, ';');
}
}
fclose($file);
The $addstring = file_get_contents($path.$filename) doesn't does nothing because you're overwriting that variable in the next line.
To remove the extra row on 12 did you tried removing the \n AND the \r with something like:
$row[12] = strtr($row[12], array("\n"=>'', "\r"=>''));
You can also check which ascii characters are you receiving in the $row[12] with this function taken form the php site:
function AsciiToInt($char){
$success = "";
if(strlen($char) == 1)
return "char(".ord($char).")";
else{
for($i = 0; $i < strlen($char); $i++){
if($i == strlen($char) - 1)
$success = $success.ord($char[$i]);
else
$success = $success.ord($char[$i]).",";
}
return "char(".$success.")";
}
}
Another tip can be the database it's returning UTF-8 or UTF-16 and you're losing some characters in the text file.
Try looking at that with the mb_detect_encoding function.

Enabling PHP errors when creating a CSV file from database

This method for exporting data on csv has worked previously on other projects, but I can not make this work on here, and I am not sure about how to enable erros for this case.
This PHP file creates a comma-separated file containing an initial row with a single tab ("ID"), and then it should be creating a row for each match on the SELECT query from DB
<?php
session_start();
ob_start();
include('conexionbbdd.php');
$file='informes/expositores_'.time().'.xls';
header("Content-Type: application/xls");
header("Content-Disposition: attachment; filename=$file");
header("Pragma: no-cache");
header("Expires: 0");
$output = fopen($file, 'w');
fwrite($output, chr(239) . chr(187) . chr(191));
fputcsv($output, array('ID'), "\t");
// fetch the data
$rows1 = mysqli_query($con, "SELECT ex_id FROM expositores WHERE ex_id = '26'");
// loop over the rows, outputting them
while ($row1 = mysqli_fetch_assoc($rows1)) {
fputcsv($output, $row1, "\t");
}
fclose($output);
echo $file;
ob_end_flush();
?>
In this particular case I've simplified this to maximu so, apart from the initial row, a unique row containg the "26" should be created (I've tested that the query works with PhpMyAdmin, there's an ID 26). But it does not.
It only creates correctly first row from this first fputcsv method:
fputcsv($output, array('ID'), "\t");
No other row seems to be fetched or placed on the CSV.
As the entire PHP file's aim is to create the CSV file, no error is shown because it does not open on a new window.
Output:
In order to solve this you will need to be able to view the errors. You can have a look in your error logs or add error reporting to the top of your file(s) right after your opening <?php tag error_reporting(E_ALL); ini_set('display_errors', 1);

PHP create a complex CSV file

I am in need to create a CSV file getting the data from a mySQL DB.
The fact is that I want the CSV tp be corrected labeled and not just writing the data like this:
id,name,url
1,thisismyname,thisismyurl
I need the CSV file to look well ordered and each data inserted in the relative column.
Also with the function I am going to add below I can only grab the data from the DB and write it to the CSV file as it is. But I need to work with the data and have the CSV labeled in this way:
Campaign Name:
Name of the campaign
Campaign Url:
Url of the campaign
Tot visits:
Tot of visits
Tot unique visits:
Tot of unique visits
id name url
1 thisname this url
2 thisname this url
3 thisname this url
4 thisname this url
5 thisname this url
This is the PHP code I have so far..I need to understand how to achieve a correct structure of the CSV with PHP and adding the lines in it the exact way I want..
Thanks for your help!
function genCSV($filename, $attachment = true, $headers = true) {
// send response headers to the browser
header('Content-Type: text/csv');
header('Content-Disposition: attachment;filename=' . $filename);
$fp = fopen('php://output', 'w');
$query = "SELECT * FROM campaigns";
$result = mysql_query($query) or die(mysql_error());
if ($headers) {
// output header row (if at least one row exists)
$row = mysql_fetch_assoc($result);
if ($row) {
fputcsv($fp, array_keys($row));
// reset pointer back to beginning
mysql_data_seek($result, 0);
}
}
while ($row = mysql_fetch_assoc($result)) {
fputcsv($fp, $row);
}
fclose($fp);
}
Here is a much less elegant solution than the one proposed by #Tom Regner.
I needed to backup certain database tables (all those with a given prefix) but not others. This method, though somewhat slow, allows you to select exactly which tables and which columns from those tables are copied. It was originally written to allow each piece of data to be AES encrypted before being entered into the file but there are other uses for it. As written here, the result is a CSV file with the first line containing the list of columns for the table and the rest containing the data in CSV. It will stand adaptation to output the result of any sql into CSV, if you like.
Obviously: mysqlidb = mysqli databse resource, backups/ = directory to put finished files in.
FWIIW, here is the code:
$sql="SHOW TABLES LIKE 'yourtable%'";
$result = $mysqlidb->query($sql);
$tableresult=$mysqlidb->query($sql);
while($tables=$tableresult->fetch_assoc())
{
$keys=array_keys($tables);
$tablename=$tables[$keys[0]];
echo "Writing $tablename <BR>";
$file=fopen("backups/$tablename.enc","w");
$cols=array();
$sql="SHOW COLUMNS FROM $tablename";
$result=$mysqlidb->query($sql);
while($row=$result->fetch_assoc())
{
$cols[]=$row['Field'];
}
fputcsv($file,$cols);
$sql="SELECT * FROM $tablename";
$result=$mysqlidb->query($sql);
while($row=$result->fetch_assoc())
{
fputcsv($file,$row);
}
fclose($file);
}

MySQL to Excel charset issue

I have a database set up which accepts user registrations and their details etc. I'm looking to export the database to an excel file using php.
The problem I am having is that some of the entrants have entered foreign characters in, such as Turkish, which has been written into the database 'incorrectly' - as far as I have ascertained, the charset was likely set up incorrectly when it was first made.
I have made my code to export the database into excel (below) but I cannot get the Excel document to show correctly regardless of how I try to encode the data
<?php
require_once('../php/db.php');
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=Download.xls");
header("Pragma: no-cache");
header("Expires: 0");
$query = "SELECT * FROM users";
$result = mysqli_query($link, $query);
if($result) {
$count = mysqli_num_rows($result);
for($i=0; $i<$count; $i++) {
$field = mysqli_fetch_field($result);
$header .= $field->name."\t";
while($row = mysqli_fetch_row($result)) {
$line = '';
foreach($row as $value) {
if((!isset($value)) OR ($value == "")) {
$value = "\t";
} else {
$value = str_replace('"', '""', $value);
$value = '"'.$value.'"'."\t";
}
$line .= $value;
}
$data .= trim($line)."\n";
}
$data = str_replace("\r", "", $data);
if($data == "") {
$data = "\n(0) Records Found!\n";
}
}
print mb_convert_encoding("$header\n$data", 'UTF-16LE', 'UTF-8');
} else die(mysqli_error());
?>
When I do this it comes up with an error when opening it up saying that Excel doesn't recognise the file type, it opens the document but its drawn boxes around all the Turkish characters its tried to write.
I'm no PHP expert this is just information I've kind of pieced together.
Can anyone give me a hand?
Much appreciated
Moz
First of all, you appear to be creating a tab-delimited text file and then returning it to the browser with the MIME-type application/octet-stream and the file extension .xls. Excel might work out that's tab-delimited (but it sounds from your error as though it doesn't), but in any case you really should use the text/tab-separated-values MIME type and .txt file extension so that everything knows exactly what the data is.
Secondly, to create tab-delimited files, you'd be very wise to export the data directly from MySQL (using SELECT ... INTO OUTFILE), as all manner of pain can arise with escaping delimiters and such when you try to cook it yourself. For example:
SELECT * FROM users INTO OUFILE '/tmp/users.txt' FIELDS TERMINATED BY '\t'
Then you would merely need to read the contents of that file to the browser using readfile().
If you absolutely must create the delimited file from within PHP, consider using its fputscsv() function (you can still specify that you wish to use a tab-delimiter).
Always use the .txt file extension rather than .csv even if your file is comma-separated as some versions of Excel assume that all files with the .csv extension are encoded using Windows-1252.
As far as character encodings go, you will need to inspect the contents of your database to determine whether data is stored correctly or not: the best way to do this is to SELECT HEX(column) ... in order that you can inspect the underlying bytes. Once that has been determined, you can UPDATE the records if conversions are required.

Categories