Writing into csv with php problems - php

I have a problem. I'm trying to get some data from a database into a .csv table.
$fn=fopen($path.$filename, "w");
$addstring = file_get_contents($path.$filename);
$addstring = 'Azonosito;Datum;Ido;Leiras;IP-cim;allomasnév;MAC-cim;Felhasznalonev;Tranzakcioazonosito;Lekerdezes eredmenye;Vizsgalat ideje;Korrelacios azonosito;DHCID;';
/*$addstring .= "\n";*/
$sql="select * from dhcpertekeles.dhcpk";
$result =mysqli_query($conn, $sql);
if ($result=mysqli_query($conn,$sql))
{
while ($row=mysqli_fetch_row($result))
{
$addstring .= "\n".$row[0].";".$row[1].";".$row[2].";".$row[3].";".$row[4].";".$row[5].";".$row[6].";".$row[7].";".$row[8].";".$row[9].";".$row[10].";".$row[11].";".$row[12].";";
};
};
/*file_put_contents($path.$filename, $addstring);*/
fwrite($fn, $addstring);
fclose($fn);
The data is in the following format:
The first addstring contains the column names, and has no issues
the second (addstring .=) contains the data:
ID($row[0]), Date($row[1]), Time($row[2]), Description($row[3]), IP($row[4]), Computer name($row[5]), MAC($row[6]), User($row[7])(empty), Transactionid($row[8]), query result($row[9]), query time($row[10]), correlation id($row[11])(empty), DHCID($row[12])(empty)
It is basically daily DHCP server data, uploaded to a database. Now, the code works, it does write everything i want to the csv, but there are 2 problems.
1, the code for some inexplicable reason, inserts an empty row into the csv table between the rows that contain the data. Removing $row[12] fixes this. I tried removing special characters, converting spaces into something that can be seen, and even converting empty string into something that can be seen. Yet nothing actually worked, i even tried file_puts_content(same for the second problem) instead of fwrite, but nothing. The same thing keeps happening. If i remove \n it will work, but the 2nd row onwards will be misplaced to the right by 1 column.
2, For some reason, the last 2 character is removed from the csv. The string that is to be inserted into the csv still contains said 2 characters before writing it to the file. Tried both fwrite and file_puts_content.
As for the .csv format, the data clumns are divided by ; and rows by \n.
Also tried reading the file with both libre office and excel thinking it might be excel that was splurging but no.

Try using fputcsv() function. I didn't test following code but I think it should work.
$file = fopen($path . $filename, 'w');
$header = array(
'Azonosito',
'Datum',
'Ido',
'Leiras',
'IP-cim',
'allomasnév',
'MAC-cim',
'Felhasznalonev',
'Tranzakcioazonosito',
'Lekerdezes eredmenye',
'Vizsgalat ideje',
'Korrelacios azonosito',
'DHCID'
);
fputcsv($file, $header, ';');
$sql = "select * from dhcpertekeles.dhcpk";
$result = mysqli_query($conn, $sql);
if ($result = mysqli_query($conn, $sql)) {
while ($row = mysqli_fetch_row($result)) {
fputcsv($file, $row, ';');
}
}
fclose($file);

The $addstring = file_get_contents($path.$filename) doesn't does nothing because you're overwriting that variable in the next line.
To remove the extra row on 12 did you tried removing the \n AND the \r with something like:
$row[12] = strtr($row[12], array("\n"=>'', "\r"=>''));
You can also check which ascii characters are you receiving in the $row[12] with this function taken form the php site:
function AsciiToInt($char){
$success = "";
if(strlen($char) == 1)
return "char(".ord($char).")";
else{
for($i = 0; $i < strlen($char); $i++){
if($i == strlen($char) - 1)
$success = $success.ord($char[$i]);
else
$success = $success.ord($char[$i]).",";
}
return "char(".$success.")";
}
}
Another tip can be the database it's returning UTF-8 or UTF-16 and you're losing some characters in the text file.
Try looking at that with the mb_detect_encoding function.

Related

Not getting expected value from MySQL

I'm embarrassed because this should be a pretty simple task, but I can't figure out why this is not working. I'm using a tab separated file to get values I need to populate a MySQL table. I have 2 MySQL tables, clients and data The clients table has an ID I need to fetch and use in the insert to the data table
<?php
// MySQL settings
define('DB_SERVER', 'localhost');define('DB_USERNAME', 'USER');
define('DB_PASSWORD', 'pass');define('DB_DATABASE', 'DB');
// connect to DB
if ($db = mysqli_connect(DB_SERVER,DB_USERNAME,DB_PASSWORD,DB_DATABASE)){}
else {echo 'Connection to DB failed';die();}
// load tab delim file
$file = "file.csv";// TSV actually
$handle = fopen($file, "r"); // Make all conditions to avoid errors
$read = file_get_contents($file); //read
$lines = explode("\n", $read);//get
$i= 0;//initialize
// loop over file, one line at a time
foreach($lines as $key => $value){
$cols[$i] = explode("\t", $value);
// get order ID for this URL
//$cols[$i]['6'] stores website URLs that match `salesurl` in the clients table
$getidsql = 'select `id` FROM DB.clients WHERE `salesurl` = \''. $cols[$i]['6'].'\'';
if ($result = mysqli_query($db, $getidsql)){
$totalcnt = mysqli_num_rows($result);
$idrow = mysqli_fetch_array($result);
echo '<h1>:'. $idrow['id'] .': '.$totalcnt.'</h1>'; //prints ':: 0'
} else {
echo mysqli_error($db);
echo 'OOPS<hr>'. $getidsql .'<hr>';
}
// if $idrow['id'] actually had a value, then
$insertqry = 'INSERT INTO `data` ......';
$i++;
} //end for each, file line loop
?>
The $getidsql query does work when copy pasted into PHPMyADMIN I get the id result, but within this script mysqli_num_rows is ALWAYS zero, and $idrow is never populated eg; NO ERRORS.. but no result (well, an empty result)
Turns out my code was working fine. My problem was with the data file I was working with. All the data had non-printable characters in it, in fact each character was followed by a non-ASCII character. Running this preg_replace prior to using it in my query solved the problem.
$data[$c] = preg_replace('/[\x00-\x08\x0B\x0C\x0E-\x1F\x80-\x9F]/u', '', $data[$c]);

PHP issues with explode() and SQL query

I have a code in PHP like the next:
//$menutype is defined in this point, and no problem with it
$sql = "SELECT struct FROM menutype WHERE id=$menutype;";
$result = mysql_query($sql);
list($struct) = mysql_fetch_row($result);
$menu = explode("\n", $struct); //Explode to make an array with each line
foreach ($menu as $index => $value)
{
//$barid is defined in the top of the document and no issue with it
creatabla($barid, $value);
}
function creatabla($barid, $tipo)
{
$tipo = trim($tipo, ' '); //trim to delete unwanted spaces
$sql = "SELECT name FROM products WHERE tipo LIKE '%$tipo%' AND restaurante='$barid';";
$result = mysql_query($sql);
while(list($name) = mysql_fetch_row($result))
{
echo "$name";
}
}
Similar 'struct' row struct:
Line1
Line2
Line3
Line4AndLastLine
No car return after Last Line.
Well, the code usually works fine, but If I modify the 'Struct' row, some lines won't be read fine, so I usually need to edit the struct row in the advaced editor of the phpmyadmin.
What can I do to solve this issue? Can I try other kind of filter in the sql statement? What can I do to improve the trim or the explode function to solve this issue?
Thanks you to all in advance.
I think I'm starting to understand the issue and I'm guessing you're right - I'm guessing there is a \r character on the end.
Right now you are calling this:
trim($tipo, ' ');
That deletes ONLY spaces. If you change it to simply:
trim($tipo);
Then it will delete a lot more types of whitespace, including (but not limited to) the \r character.
See HERE for more information on the trim() function.

How to run a mysql loop from a posted array and extract results to a CSV file

I have searched for hours to try and find a simple answer to this query I am having. I'm very sure it is covered in many ways, by other answers at least partially. I need a clear answer specific to what I am doing because I'm having difficulty trying to put together an answer that works for me from a bunch of other differently formatted and structured questions/answers.
I have a form which is posting a range of results intended to be used as keys for a mysql lookup loop. This is working fine - my post results in a successful array. For example:
$payarr = $_POST['pay'];
print_r($payarr);
Results in:
Array ( [0] => 12 [1] => 7 [2] => 1 )
Which is good given that the I want to run a mysqli process select the rows where claim_id is each of the values in $payarr. I then want to use fputcsv to write each of these rows fully, into a CSV file that is uniquely named by the days date and time that this is all run.
I have fragments of code that don't work and my frustration at trying to cobble something together that does work is getting a bit nuts. Can someone please show me how to do this effectively?
At the moment my code looks like this (but fails miserably):
<?php
include ("../conf/dbconfig.php");
include ("../conf/funcs.php");
include ("../conf/privs.php");
if($privs == 500) {
$view = $_POST['view'];
$payarr = $_POST['pay'];
$ts = date('Ymd-His');
print_r($payarr); //Testing to make sure our array to select from has arrived here ok.
//ob_start();
$fp = fopen('../csv/ResultsFile_'.$ts.'.csv', 'w');
foreach($payarr as $val) {
$result = mysqli_query($con, "'SELECT * FROM claims' WHERE claim_id='$val'");
//$row = mysqli_fetch_array($result, MYSQLI_ASSOC);
while ($row = mysqli_fetch_array($result)) {
echo $payarr;
echo $val;
//fputcsv($fp, $row);
print_r($row);
}
}
fclose($fp);
// return ob_get_clean();
} else {
header("Location: http://www.google.com.au/");
die;
}
?>
I'm willing (and happy) to rewrite the entire thing just as long as it works, so any help and suggestions are very appreciated!
Thanks in advance.
BTW - The $view value is not important in this process but is here as it will be passed back to the resulting header once this all works.
A few rough suggestions (you may need to adapt):
1) Change your query to use IN for the possible values, so you only have to execute one DB query:
$result = mysqli_query($con, "SELECT * FROM claims WHERE claim_id IN (" . implode(",",$payarr) . ")");
2) Make sure you are actually getting a DB result, instead of just assuming:
if(!$result || mysqli_num_rows($result) == 0) {
die("We didn't get any results from the DB!"); // Obviously you'll want better error handling
}
3) Now you can open your file, knowing that you need it. Make sure to verify it worked, since you could easily hit a permissions issue:
$fp = fopen('../csv/ResultsFile_'.$ts.'.csv', 'w');
if(!$fp) {
die("We couldn't open the CSV file for writing, check permissions!"); // Obviously you'll want better error handling
}
4) Now loop through your DB results and store them:
while ($row = mysqli_fetch_array($result)) {
fputcsv($fp, $row);
}
5) Does your CSV file need a header row? If so change mysqli_fetch_array() to mysqli_fetch_assoc() and stick this inside your loop:
while ($row = mysqli_fetch_assoc($result)) {
if(!isset($header)) {
$header = array_keys($row);
fputcsv($fp, $header);
}
fputcsv($fp, $row);
}
6) Only now should you close your file (in your code you do it inside the foreach loop):
fclose($fp);
7) Sanitize your $payarr. It could be as simple as:
$payarr = is_array($_POST['pay']) ? array_map('intval', $_POST['pay']) : array();
You may want to do more than that. But at least you're guaranteed to have an array with only integer values (and as long as you have no claim_id values of 0, it's harmless to have zeros in the array).
Hope that helps, at least track down where your code is failing.

small glitch while generating csv with newline character in php

I am simply generating a csv file based on data stored in a mysql table. The generated csv, when opened in excel, seems mostly ok, but whenever it has a newline character, excel puts the data on a new row. Any idea how to prevent that?
Sample data
line 1 some data
another data
CSV generation code:
header("Content-Type: text/csv; charset=UTF-8");
header("Content-Disposition: attachment; filename=\"".$MyFileName."\"");
$filename = $MyFileName;
$handle = fopen("temp_files/".$filename, "r");
$contents = fread($handle, filesize("temp_files/".$filename));
fclose($handle);
echo $contents;
exit;
content snippet I used to get rid of new line(didn't work):
$pack_inst = str_replace(',',' ',$get_data->fields['pack_instruction']);
$pack_inst = str_replace('\n',' ',$pack_inst);
$pack_inst = str_replace('\r',' ',$pack_inst);
$pack_inst = str_replace('\r\n',' ',$pack_inst);
$pack_inst = str_replace('<br>',' ',$pack_inst);
$pack_inst = str_replace('<br/>',' ',$pack_inst);
$pack_inst = str_replace(PHP_EOL, '', $pack_inst);
$pattern = '(?:[ \t\n\r\x0B\x00\x{A0}\x{AD}\x{2000}-\x{200F}\x{201F}\x{202F}\x{3000}\x{FEFF}]| |<br\s*\/?>)+';
$pack_inst = preg_replace('/^' . $pattern . '|' . $pattern . '$/u', ' ', $pack_inst);
$content .=','.$pack_inst;
According to RFC 4180, if a column's content contains the row delimiter (\r\n), the column delimiter (,) or the string delimiter (") then you must enclose the content inside double quotes ". When you do that, you must escape all " characters inside the content by preceding them with another ". So the following CSV content:
1: OK,2: this "might" work but not recommended,"3: new
line","4: comma, and text","5: new
line and ""double"" double quotes"
1: Line 2
Will produce 2 rows of CSV data, first one containing 5 columns.
Having said that, have a look at fputcsv() function. It will handle most gory details for you.
What you show is not the CSV generation code, it is simply the code that you have used to force a download to the browser. Regardless, the function that you need to sort this out is fputcsv(), which will automatically consider all sorts of edge cases that any code you write to convert tabular data to CSV format will likely not consider.
You say you are basing this on data in MySQL table, here is a basic framework for creating the CSV file, assuming the MySQLi extension used in a procedural manner:
<?php
// Connect to database and generate file name here
$fileName = 'file.csv';
// Get the data from the database
$query = "
SELECT *
FROM table_name
WHERE some_column = 'Some Value'
ORDER BY column_name
";
if (!$result = mysqli_query($db, $query)) {
// The query failed
// You may want to handle this with a more meaningful error message
header('HTTP/1.1 500 Internal Server Error');
exit;
} else if (!mysqli_num_rows($result)) {
// The query returned no results
// You may want to handle this with a more meaningful error message
header('HTTP/1.1 404 Not Found');
exit;
}
// Create a temporary file pointer for storing the CSV file
$tmpFP = fopen('php://temp', 'w+');
// We'll keep track of how much data we write to the file
$fileLength = 0;
// Create a column head row and write first row to file
$firstRow = mysqli_fetch_assoc($result);
$fileLength += fputcsv($tmpFP, array_keys($firstRow));
$fileLength += fputcsv($tmpFP, array_values($firstRow));
// Write the rest of the rows to the file
while ($row = mysqli_fetch_row($result)) {
$fileLength += fputcsv($tmpFP, $row);
}
// Send the download headers
header('Content-Type: text/csv; charset=UTF-8');
header('Content-Disposition: attachment; filename="'.$fileName.'"');
header('Content-Length: '.$fileLength);
// Free some unnecessary memory we are using
// The data might take a while to transfer to the client
mysqli_free_result($result);
unset($query, $result, $firstRow, $row, $fileName, $fileLength);
// Prevent timeouts on slow networks/large files
set_time_limit(0);
// Place the file pointer back at the beginning
rewind(tmpFP);
// Serve the file download
fpassthru($tmpFP);
// Close the file pointer
fclose($tmpFP);
// ...and we're done
exit;

MySQL to Excel charset issue

I have a database set up which accepts user registrations and their details etc. I'm looking to export the database to an excel file using php.
The problem I am having is that some of the entrants have entered foreign characters in, such as Turkish, which has been written into the database 'incorrectly' - as far as I have ascertained, the charset was likely set up incorrectly when it was first made.
I have made my code to export the database into excel (below) but I cannot get the Excel document to show correctly regardless of how I try to encode the data
<?php
require_once('../php/db.php');
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=Download.xls");
header("Pragma: no-cache");
header("Expires: 0");
$query = "SELECT * FROM users";
$result = mysqli_query($link, $query);
if($result) {
$count = mysqli_num_rows($result);
for($i=0; $i<$count; $i++) {
$field = mysqli_fetch_field($result);
$header .= $field->name."\t";
while($row = mysqli_fetch_row($result)) {
$line = '';
foreach($row as $value) {
if((!isset($value)) OR ($value == "")) {
$value = "\t";
} else {
$value = str_replace('"', '""', $value);
$value = '"'.$value.'"'."\t";
}
$line .= $value;
}
$data .= trim($line)."\n";
}
$data = str_replace("\r", "", $data);
if($data == "") {
$data = "\n(0) Records Found!\n";
}
}
print mb_convert_encoding("$header\n$data", 'UTF-16LE', 'UTF-8');
} else die(mysqli_error());
?>
When I do this it comes up with an error when opening it up saying that Excel doesn't recognise the file type, it opens the document but its drawn boxes around all the Turkish characters its tried to write.
I'm no PHP expert this is just information I've kind of pieced together.
Can anyone give me a hand?
Much appreciated
Moz
First of all, you appear to be creating a tab-delimited text file and then returning it to the browser with the MIME-type application/octet-stream and the file extension .xls. Excel might work out that's tab-delimited (but it sounds from your error as though it doesn't), but in any case you really should use the text/tab-separated-values MIME type and .txt file extension so that everything knows exactly what the data is.
Secondly, to create tab-delimited files, you'd be very wise to export the data directly from MySQL (using SELECT ... INTO OUTFILE), as all manner of pain can arise with escaping delimiters and such when you try to cook it yourself. For example:
SELECT * FROM users INTO OUFILE '/tmp/users.txt' FIELDS TERMINATED BY '\t'
Then you would merely need to read the contents of that file to the browser using readfile().
If you absolutely must create the delimited file from within PHP, consider using its fputscsv() function (you can still specify that you wish to use a tab-delimiter).
Always use the .txt file extension rather than .csv even if your file is comma-separated as some versions of Excel assume that all files with the .csv extension are encoded using Windows-1252.
As far as character encodings go, you will need to inspect the contents of your database to determine whether data is stored correctly or not: the best way to do this is to SELECT HEX(column) ... in order that you can inspect the underlying bytes. Once that has been determined, you can UPDATE the records if conversions are required.

Categories