Below is the script im using to export a MySQL database to a .csv file.
It seems to do fine with smaller outputs (under about 30,000 rows), but hangs on my larger outputs and just wont complete (1 million rows total).
Any ideas why?
Also, the PHPMyAdmin export utility dumps it incredibly quickly (just a few seconds). Wondering how it does it.
if(isset($_POST['submit'])) {
// Fetch Record from Database
$output = "";
$table = "tt_output";
$sql = mysqli_query($connection,"select * from {$table}");
$columns_total = mysqli_num_fields($sql);
// Get The Field Name
$query = "SHOW COLUMNS FROM {$table}"; $result_set = mysqli_query($connection,$query);
while ($result = mysqli_fetch_array($result_set)) {
$heading = $result[0];
$output .= trim($heading.',');
}
$output = substr($output,0,strlen($output)-1)."\r\n";
// Get Records from the table
while ($row = mysqli_fetch_array($sql)) {
for ($i = 0; $i < $columns_total; $i++) {
$output .='"'.$row["$i"].'",';
}
$output = substr($output,0,strlen($output)-1)."\r\n";
}
// Download the file
$filename = "output".".csv";
header("Pragma: public", true);
header("Expires: 0"); // set expiration time
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header('Content-Disposition: attachment; filename='.$filename);
header("Content-Transfer-Encoding: binary");
header('Connection: Keep-Alive');
echo $output;
exit;
}
On problem I can see is that you are reading everything into a variable / memory and only then start outputting it.
You should put your loop below the headers and output it right away:
...
header('Connection: Keep-Alive');
// Get Records from the table
while ($row = mysqli_fetch_array($sql)) {
for ($i = 0; $i < $columns_total; $i++) {
$output .='"'.$row["$i"].'",';
}
echo substr($output,0,strlen($output)-1)."\r\n";
}
Another possible problem is how you are building your row. If any of the fields contains an unescaped " character, it will break your csv. You might want to use fputcsv() to save the csv to a temporary file and only then send it to the client.
You can also save yourself a query as the column names will be the keys of your results arrays and if you really want to build the csv manually, you could use implode() instead of a loop but you would have to test if that is faster.
Related
I have a php script that exports a mysql database table to a csv file. For one of the fields of data, a user included an octothorp # in their text. For example, My Idea #1.
When my code gets to this piece of data, it stops at the # symbol. I'm guessing it hits the # and believes everything else that follows is a comment.
As a temporary solution, I am stripping out this symbol when adding the data to the database, but this is not ideal. Given the script below, how might I escape this symbol to prevent this from happening?
$query = "SELECT * FROM my_table";
$result = #mysqli_query($dbc, $query);
if (!$result) {
die('Couldn\'t fetch records');
}
$num_fields = mysqli_num_fields($result);
$headers = [];
for ($i = 0; $i < $num_fields; $i++) {
$field = mysqli_fetch_field_direct($result, $i);
// var_dump($field);
$headers[] = $field->name;
}
$fp = fopen('php://output', 'w');
if ($fp && $result) {
header('Content-Type: text/csv');
header('Content-Disposition: attachment; filename="export.csv"');
header('Pragma: no-cache');
header('Expires: 0');
fputcsv($fp, $headers);
while ($row = mysqli_fetch_array($result, MYSQLI_NUM)) {
fputcsv($fp, array_values($row));
}
die;
}
I have a script that downloads results from a MySql table into an CSV file like so
$result = $sql2;
$num_fields = mysql_num_fields($result);
$headers = array();
for ($i = 0; $i < $num_fields; $i++) {
$headers[] = mysql_field_name($result , $i);
}
$fp = fopen('php://output', 'w');
if ($fp && $result) {
header('Content-Type: text/csv');
header('Content-Disposition: attachment; filename="export.csv"');
header('Pragma: no-cache');
header('Expires: 0');
fputcsv($fp, $headers);
while ($row = mysql_fetch_assoc($result)) {
fputcsv($fp, array_values($row));
}
die;
}
Which works fine, the problem i am having is some of the values of columns contain & quot;. When printing this data on to the web page it shows as a real quote " but when downloading it into the CSV it prints as & quote;. I have tried to htmlspecialchars_decode() but the problem is still there.
Any help will be appreciated.
I have various images split in the database as blob files. I have written a php script which accesses these blob files and downloads them locally.
As there are many images I have put an loop which would download all images which are stored in the database, however when I run the script in the loop I get one big download and no separate images.
My code is as follows:
$query="SELECT `id` FROM `images` LIMIT 10";
$result=mysql_query($query);
$imageid = array();
while($row = mysql_fetch_assoc($result)){
//iterate over all the fields
foreach($row as $key){
$imageid[] = $key;
}
}
$fileName ='';
for ($i = 0; $i < count($imageid); $i++) {
$query="SELECT `filename` FROM `images` WHERE `id`=".$imageid[$i]."";
$result=mysql_query($query);
while($row = mysql_fetch_assoc($result)){
foreach($row as $key){
$fileName = $key;
}
}
header("Pragma: public");
header("Expires: 0"); // set expiration time
header("Cache-Control: must-revalidate, post-check=0,pre-check=0");
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header("Content-Transfer-Encoding: binary");
header('Content-type: application/octet-stream');
header('Content-Disposition: inline; filename='.$fileName.'');
$sql = "SELECT `data` FROM `images_data` WHERE `image_id`=".$imageid[$i]." ORDER BY id";
$result1=mysql_query($sql);
while($row = mysql_fetch_array($result1)) echo $row['data'];
}//END FOR LOOP
Is there any way I can download all of the images from running the script?
If I manually change the parameter of $imageid[$i] (eg 0,1,2,3 etc) the different images will download but its useless for the amount of images I have to use.
Any help is much appreciated!
You can get only on file once from one php script run.
In you case I think best way is, store all files from DB into temporal folder, when zip (or GZ) this folder, and after output this ZIP (or GZ) archive using corresponding MIME type in you header()
I'm new to PHP. I'm using the following code to generate an excel sheet from Mysql database using PHP.
<?php
include("../Connection/Connection.php");
$db_con=new Connection();
$db_con->get_connection(); //Opens a database connection. This function is contained in the Connection.php which is included above.
$result = mysql_query("SELECT * FROM country");
if (!$result) die('Couldn\'t fetch records');
$num_fields = mysql_num_fields($result);
$headers = array();
for ($i = 0; $i < $num_fields; $i++)
{
$headers[] = mysql_field_name($result , $i);
}
$fp = fopen('php://output', 'w');
if ($fp && $result)
{
header('Content-Type: text/csv');
header('Content-Disposition: attachment; filename="export.csv"');
header('Pragma: no-cache');
header('Expires: 0');
fputcsv($fp, $headers);
while ($row = mysql_fetch_array($result))
{
fputcsv($fp, array_values($row));
}
die;
}
?>
The above code is working and it generates the specified file export.csv but the problem is that it generates each column from MySql in this file twice. In other words, each column is duplicated twice (headers displayed are not duplicated though). What is wrong with this code? What changes should be made so that it displays each column exactly once?
Use mysql_fetch_row instead.
mysql_fetch_array fetches an array with BOTH string indexes and numeric indexes, so each value is fetched twice. Your code would work with either mysql_fetch_assoc or mysql_fetch_row.
i have a table which shows some mysql data, every entry has a checkbox to select individual entries, now i want to be able to export those selected entries into a xml or txt file, i tried this:
<?php
if ($_POST['exporttxt']) {
for ($i = 0; $i < count($_POST['checkbox']); $i++) {
$export_id = $checkbox[$i];
$sql = "SELECT * FROM table WHERE id='$export_id'";
$result = mysql_query($sql);
}
$output = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<root>\n";
if ($result->num_rows > 0) {
while ($myrow = $result->fetch_assoc())
{
$output .= "\t<row>\n";
foreach ($myrow as $_name => $_value)
{
$output .= "\t\t<$_name>$_value</$_name>\n";
}
$output .= "\t</row>\n";
}
}
$output .= "</root>";
}
header('content-type: text/xml');
header('content-disposition: attachment; filename=data_export.xml');
echo $output;
exit;
?>
But that didn't work at all, any hints ?
I've changed the code a bit, i am now using this
<?php
if ($_POST['exporttxt']) {
for($i=0;$i<count($_POST['checkbox']);$i++){
$export_id = $checkbox[$i];
$text = mysql_query("SELECT code FROM ticket WHERE id='$export_id'");
$text = mysql_fetch_assoc($text);
$text = $text["code"];
ob_end_flush();
header("Content-type: text/plain");
header("Content-disposition: attachment;filename=\"filename.txt\"");
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header("Content-Description: File Transfer");
header("Content-Length: ".strlen($output).";\n");
echo($text);
}
}
?>
I now get proper output on my screen, but it won't offer me to download the file ?
Your original code was querying each of the checkboxed IDs individually (a long and laborious way to handle it) and was trying to dump the results individually per row (not going to work well for you).
Try this. (NOTE: Untested, but should give you a good starting point to develop.)
if( $_POST['exporttxt'] ){
if( count( $_POST['checkbox'] )>0 ){
// If the checkbox values are meant to all be integers, you might want to perform some validation/sanitisation/filtering here
// Up to you to do that
// Collapse the IDs from the checkboxes into a comma-delimited string
$export_ids = implode( ',' , $_POST['checkbox'] );
// Template the SQL Query
$sqlTpl = 'SELECT code FROM ticket WHERE id IN ( %s )';
// Compile the SQL Query String
$sqlStr = sprintf( $sqlTpl , $export_ids );
// Execute the SQL Query
if( !( $sqlRes = mysql_query( $sqlStr ) ) ){
// SQL Error - Log it, Handle it
}elseif( mysql_num_rows( $sqlRes )==0) {
// No Rows Returned - Log it, Handle it
}else{
// We have results - process them
$text = array();
while( $r = mysql_fetch_assoc( $sqlRes ) ){
// Looping through the returned rows, adding them to the $text array
$text[] = $r['code'];
}
// Collapse the $text array down into a normal string, with one element per line
$output = implode( "\n" , $text );
// Output Handling from #narcisradu's answer
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: private",false);
header("Content-Transfer-Encoding: binary;\n");
header("Content-Disposition: attachment; filename=\"filename.txt\";\n");
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header("Content-Description: File Transfer");
header("Content-Length: ".strlen($output).";\n");
echo $output;
die; // Prevent any further output
}
}else{
// No Checkboxes Checked
echo 'You must select one or more checkboxes to export something, muppet.';
}
}
First of all:
for($i=0;$i<count($_POST['checkbox']);$i++){
$export_id = $checkbox[$i];
$sql = "SELECT * FROM table WHERE id='$export_id'";
$result = mysql_query($sql);}
This piece of code will actually do a query for every element in _POST['checkbox'] array but then you are using the result of the last query and you are not parsing the result of every query.
Please make sure the generated XML is well-formed and you actually have some data, by displaying directly into the browser.
After that you may try to force download:
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: must-revalidate, post-check=0, pre-check=0");
header("Cache-Control: private",false);
header("Content-Transfer-Encoding: binary;\n");
header("Content-Disposition: attachment; filename=\"".urlencode(basename($file_name))."\";\n");
header("Content-Type: application/force-download");
header("Content-Type: application/octet-stream");
header("Content-Type: application/download");
header("Content-Description: File Transfer");
header("Content-Length: ".strlen($output).";\n");
ob_end_flush();