The script in question takes an excel file of language vocabulary (French to English, etc) and creates XML (zips and downloads) to format a crossword generator we use.
It works, but I've been asked to remove any duplicate terms as an enhancement. Below is the original code in full, and then the new code to skip duplicate terms. With the new code, everything runs, but it creates a corrupt ZIP. Please see the before and after code and tell me what is going on:
Full before working code:
<?php
/** Error reporting */
error_reporting(E_ALL);
ini_set('display_errors', TRUE);
ini_set('display_startup_errors', TRUE);
define('EOL',(PHP_SAPI == 'cli') ? PHP_EOL : '<br />');
/** PHPExcel */
require_once 'Classes/PHPExcel.php';
require_once 'Classes/PHPExcel/IOFactory.php';
/** Functions */
require_once 'zip.php';
require_once 'named_to_number.php';
if ($_FILES["file"]["error"] > 0) {
echo "Error: " . $_FILES["file"]["error"] . "<br>";
}
/** Create Excel Object using PHPExcel **/
$inputFileName = $_FILES["file"]["tmp_name"];
$objPHPExcel = PHPExcel_IOFactory::load($inputFileName);
$objWorksheet = $objPHPExcel->getActiveSheet();
/** Get how many rows **/
$highestRow = $objWorksheet->getHighestRow();
/** Keeps track of chapters to make new files on change **/
$chapter = -1;
/** For keeping track of the files created to zip & delete. **/
$files_to_zip = array();
/** Iterates through every row, writing data from cells into XML files. **/
for ($row = 2; $row <= $highestRow; ++$row) {
//remove spaces
$term = str_replace(' ', '', $objWorksheet->getCellByColumnAndRow(1, $row)->getValue());
//skip terms if they are too long or contain a non-alpha character.
if (strlen($term) >= 17 || !preg_match('/^\p{L}+$/ui', $term)) {
continue;
}
//translates accented characters to numbered HTML code
$term = named_to_number(htmlentities($term, ENT_SUBSTITUTE , 'UTF-8'));
/** Checks first column to see if the chapter has changed.
If it has, the current file will be closed and a new one opened. **/
if ($chapter != $objWorksheet->getCellByColumnAndRow(0, $row)->getValue()){
fwrite ($f, "</words>\n</content>");
fclose($f);
if (strlen($objWorksheet->getCellByColumnAndRow(0, $row)->getValue()) < 2) {
$filename = 'ch0' . $objWorksheet->getCellByColumnAndRow(0, $row)->getValue() . '.xml';
} else {
$filename = 'ch' . $objWorksheet->getCellByColumnAndRow(0, $row)->getValue() . '.xml';
}
$f = fopen($filename, 'a');
/** Add to the list of files to zip and delete **/
array_push($files_to_zip, $filename);
fwrite ($f, "<content>\n<words>\n");
/** Update chapter value **/
$chapter = $objWorksheet->getCellByColumnAndRow(0, $row)->getValue();
}
/** Write terms **/
$data =
"<word><entry>" . $term .
"</entry><clue>" . $objWorksheet->getCellByColumnAndRow(2, $row)->getValue() .
"</clue></word>\n";
fwrite ($f, $data);
}
fwrite ($f, "</words>\n</content>");
fclose($f);
/** Removes any blank ch.xml files **/
if(($key = array_search('ch.xml', $files_to_zip)) !== false) {
unset($files_to_zip[$key]);
unlink('chapter.htm');
}
$zip = create_zip($files_to_zip, 'crossword.zip');
foreach($files_to_zip as &$del){
unlink($del);
}
header("Content-disposition: attachment; filename=crossword.zip");
header("Content-type: application/zip");
readfile("crossword.zip");
unlink("crossword.zip");
?>
Relevant snippet where code was added (comments only related to new code):
//array to be used to hold terms to check against
$used = array();
for ($row = 2; $row <= $highestRow; ++$row) {
$term = str_replace(' ', '', $objWorksheet->getCellByColumnAndRow(1, $row)->getValue());
//Add term to running array to check against to see if it exists
array_push($used, $term);
//Added a third condition to check the array to see if the term exists. I have also tried this using isset with array_flip.
if (strlen($term) >= 17 || !preg_match('/^\p{L}+$/ui', $term) || in_array($term, $used)) {
continue;
}
Again, the odd thing is that the script runs, but it just produces a bad zip. It is definitely that third conditional that is tripping the script up. I have tried it in its own if statement (just to be sure the syntax was right), but the problem persists.
Please help!
Thanks,
Mike
Related
I'm a beginner level developer learning php.The task that i need to do is upload a 6gb CSV file which contains data, into the data base.I need to access the data i.e reading the file through controller.php file and then splitting that huge CSV file into 10,000 row output CSV files and writing data into those output CSV files. I have been through this task a week already and dint figure it out yet.Would you guys please help me in solving this issue.
<?php
namespace App\Http\Controllers;
use Illuminate\Queue\SerializesModels;
use App\User;
use DateTime;
use Illuminate\Http\Request;
use Storage;
use Validator;
use GuzzleHttp\Client;
use GuzzleHttp\RequestOptions;
use Queue;
use App\model;
class Name extends Controller
{
public function Post(Request $request)
{
if($request->hasfile('upload')){
ini_set('auto_detect_line_endings', TRUE);
$main_input = $request->file('upload');
$main_output = 'output';
$filesize = 10000;
$input = fopen($main_input,'r');
$rowcount = 0;
$filecount = 1;
$output = '';
// echo "here1";
while(!feof($input)){
if(($rowcount % $filesize) == 0){
if($rowcount>0) {
fclose($output);
}
$output = fopen(storage_path(). "/tmp/".$main_output.$filecount++ . '.csv','w');
}
$data = fgetcsv($input);
print_r($data);
if($data) {
fputcsv($output, $data);
}
$rowcount++;
}
fclose($output);
}
}
}
Maybe it's because you are creating a new $output file handler for each iteration.
I've made some adjustments, so that we only create a file when the rowCount = 0 and close it when the fileSize is reached. Also the rowCount has to be reset to 0 each time we close the file.
public function Post(Request $request)
{
if($request->hasfile('upload')){
ini_set('auto_detect_line_endings', TRUE);
$main_input = $request->file('upload');
$main_output = 'output';
$filesize = 10000;
$input = fopen($main_input,'r');
$rowcount = 0;
$filecount = 1;
$output = '';
// echo "here1";
while(!feof($input)){
if ($rowCount == 0) {
$output = fopen('php://output', storage_path(). "/tmp/".$main_output.$filecount++ . '.csv','w');
}
if(($rowcount % $filesize) == 0){
if($rowcount>0) {
fclose($output);
$rowCount = 0;
continue;
}
}
$data = fgetcsv($input);
print_r($data);
if($data) {
fputcsv($output, $data);
}
$rowcount++;
}
fclose($output);
}
}
Here is working example of splitting CSV file by the amount of lines (defined by$numberOfLines). Just set your path in $filePath and run the script in shell for example:
php -f convert.php
script code:
convert.php
<?php
$filePath = 'data.csv';
$numberOfLines = 10000;
$file = new SplFileObject($filePath);
//get header of the csv
$header = $file->fgets();
$outputBuffer = '';
$outputFileNamePrefix = 'datasplit-';
$readLinesCount = 1;
$readlLinesTotalCount = 1;
$suffix=0;
$outputBuffer .= $header;
while ($currentLine = $file->fgets()) {
$outputBuffer .= $currentLine;
$readLinesCount++;
$readlLinesTotalCount++;
if ($readLinesCount >= $numberOfLines) {
$outputFilename = $outputFileNamePrefix . $suffix . '.csv';
file_put_contents($outputFilename, $outputBuffer);
echo 'Wrote ' . $readLinesCount . ' lines to: ' . $outputFilename . PHP_EOL;
$outputBuffer = $header;
$readLinesCount = 0;
$suffix++;
}
}
//write remainings of output buffer if it is not empty
if ($outputBuffer !== $header) {
$outputFilename = $outputFileNamePrefix . $suffix . '.csv';
file_put_contents($outputFilename, $outputBuffer);
echo 'Wrote (last time)' . $readLinesCount . ' lines to: ' . $outputFilename . PHP_EOL;
$outputBuffer = '';
$readLinesCount = 0;
}
you will not be able to convert such amount of data in one php execution if it is run form web because of the maximum execution time of php scripts that is usually between 30-60sec and there is a reason for that - don't event try to extend it to some huge number. If you want your script to run even for hours you need to call it from command line, but you also can call it similar way from another script (for example the controller you have)
You do that this way:
exec('php -f convert.php');
and that's it.
The controller you have will not be able to tell if the whole data was converted because before that happens it will be terminated. What you can do is to write your own code in convert.php that updates some field in database and other controller in your application can read that and print to the user the progress of the runnig convert.php.
The other approach is to crate job/jobs that you can put in the queue and can be run by job manager process with workers that can take care for the conversion but I think that would be an overkill for your need.
Keep in mind that if you split something and on different location join you may have problem of getting something wrong in that process the method that would assure you that you split, transferred, joined your data successfully is to calculate HASH ie SHA-1 of the whole 6GB file before split, send that HASH to destination where all small parts of data needs to be combined, combine them into one 6GB file, calculate HASH of that file and compare with the one that was send. Keep in mind that each of small parts of your data after splitting has their own header to be CSV file easy to interpret (import), where in the original file you have only one header row.
I'm using Symfony2.3.4, PHP5.6.3 and PHPExcel 1.8.0.
When I tried to read an excel file it works OK for almost all cells.
If the cell contains a very large number, when I read it and show the value in an html view it outputs false.
I tried to use a custom value binder like Mark Baker instructed here but I couldn't make it work, it just comes as a boolean right from the beginning.
IMPORTANT:
The excels I'm trying to load in the html are downloaded(generated) from another site and I noticed when you try to open them with Microsoft Excel, it first prompts you with a warning window telling the user that the FILE EXTENSION AND THE FILE FORMAT DO NOT MATCH, although if you choose to open it anyway, it opens fine.
I think that's what's causing the problem, I'm almost sure(I can't contact the guys who implemented the other site's download function) they did something like this:
$objWriter = \PHPExcel_IOFactory::createWriter($objPHPExcel, $ext == 'xlsx' ?
'Excel5' : 'Excel2007');
when they should have done something like this:
$objWriter = \PHPExcel_IOFactory::createWriter($objPHPExcel, $ext == 'xls' ?
'Excel5' : 'Excel2007');
making the EXTENSION and the FORMAT match, as instructed in the PHPExcel's docs.
If you need any specific clarification please ask.
My code to load the file into the html:
public function uploadAction() {
$request = $this->getRequest();
$form = $this->createFormBuilder()
->add('file', 'file')
->getForm();
if ($request->getMethod() == 'POST'){
$form->submit($request);
$file = $form['file'];
$file->getData()->move(
'uploads', $form['file']->getData()->getClientOriginalName());
$ext = pathinfo($file->getData()->getClientOriginalName(), PATHINFO_EXTENSION);
$name = pathinfo($file->getData()->getClientOriginalName(), PATHINFO_BASENAME);
//$objReader = \PHPExcel_IOFactory::createReader('xlsx' == $ext ? 'Excel2007' : 'Excel5');
$objReader = \PHPExcel_IOFactory::createReaderForFile('uploads/' . $name);
$objReader->setReadDataOnly(true);
$objPHPExcel = $objReader->load('uploads/' . $name);
$activeSheet = $objPHPExcel->getActiveSheet();
$rowIter = $activeSheet->getRowIterator();
foreach ($rowIter as $key => $row) {
$columns = array();
$cellIterator = $row->getCellIterator();
$cellIterator->setIterateOnlyExistingCells(false);
foreach ($cellIterator as $cell)
$columns[] = $cell->getCalculatedValue();
}
}
}
NOTE: I really don't know the difference between:
$objReader = \PHPExcel_IOFactory::createReader('xlsx' == $ext ? 'Excel2007' : 'Excel5');
and
$objReader = \PHPExcel_IOFactory::createReaderForFile('uploads/' . $name);
I DO know I can't use the first because of the problem I described above about the files being ill-generated and so. If I try to use it, the browser goes:
The filename uploads/<name>.xls is not recognised as an OLE file.
Can anyone point me to a workaround, because it's now me on the hook and I'm supposed to make it work somehow. Maybe there's nothing wrong with the files and it's me doing something wrong. Please help, this is causing me problems with dates too but one step at a time.
EDIT:
This is but the read function in OLERead.php.
I was browsing it and var_dump-ing all I could get my hands on.
As you can see there are two var_dumps in the code below, those output:
string '<div>
' (length=8)
string '��ࡱ�' (length=8)
Which doesn't happen when I try it with a regular .xls file created manually:
string '��ࡱ�' (length=8)
string '��ࡱ�' (length=8)
I guessed you could use this better than me if it helps at all. Thanks again.
public function read($sFileName) {
// Check if file exists and is readable
if (!is_readable($sFileName)) {
throw new PHPExcel_Reader_Exception("Could not open " . $sFileName . " for reading! File does not exist, or it is not readable.");
}
// Get the file identifier
// Don't bother reading the whole file until we know it's a valid OLE file
$this->data = file_get_contents($sFileName, FALSE, NULL, 0, 8);
////VAR_DUMPSSSSSSSSSSSS
var_dump($this->data);
var_dump(self::IDENTIFIER_OLE);
die();
// Check OLE identifier
if ($this->data != self::IDENTIFIER_OLE) {
throw new PHPExcel_Reader_Exception('The filename ' . $sFileName . ' is not recognised as an OLE file');
}
// Get the file data
$this->data = file_get_contents($sFileName);
// Total number of sectors used for the SAT
$this->numBigBlockDepotBlocks = self::_GetInt4d($this->data, self::NUM_BIG_BLOCK_DEPOT_BLOCKS_POS);
// SecID of the first sector of the directory stream
$this->rootStartBlock = self::_GetInt4d($this->data, self::ROOT_START_BLOCK_POS);
// SecID of the first sector of the SSAT (or -2 if not extant)
$this->sbdStartBlock = self::_GetInt4d($this->data, self::SMALL_BLOCK_DEPOT_BLOCK_POS);
// SecID of the first sector of the MSAT (or -2 if no additional sectors are used)
$this->extensionBlock = self::_GetInt4d($this->data, self::EXTENSION_BLOCK_POS);
// Total number of sectors used by MSAT
$this->numExtensionBlocks = self::_GetInt4d($this->data, self::NUM_EXTENSION_BLOCK_POS);
$bigBlockDepotBlocks = array();
$pos = self::BIG_BLOCK_DEPOT_BLOCKS_POS;
$bbdBlocks = $this->numBigBlockDepotBlocks;
if ($this->numExtensionBlocks != 0) {
$bbdBlocks = (self::BIG_BLOCK_SIZE - self::BIG_BLOCK_DEPOT_BLOCKS_POS) / 4;
}
for ($i = 0; $i < $bbdBlocks; ++$i) {
$bigBlockDepotBlocks[$i] = self::_GetInt4d($this->data, $pos);
$pos += 4;
}
for ($j = 0; $j < $this->numExtensionBlocks; ++$j) {
$pos = ($this->extensionBlock + 1) * self::BIG_BLOCK_SIZE;
$blocksToRead = min($this->numBigBlockDepotBlocks - $bbdBlocks, self::BIG_BLOCK_SIZE / 4 - 1);
for ($i = $bbdBlocks; $i < $bbdBlocks + $blocksToRead; ++$i) {
$bigBlockDepotBlocks[$i] = self::_GetInt4d($this->data, $pos);
$pos += 4;
}
$bbdBlocks += $blocksToRead;
if ($bbdBlocks < $this->numBigBlockDepotBlocks) {
$this->extensionBlock = self::_GetInt4d($this->data, $pos);
}
}
$pos = 0;
$this->bigBlockChain = '';
$bbs = self::BIG_BLOCK_SIZE / 4;
for ($i = 0; $i < $this->numBigBlockDepotBlocks; ++$i) {
$pos = ($bigBlockDepotBlocks[$i] + 1) * self::BIG_BLOCK_SIZE;
$this->bigBlockChain .= substr($this->data, $pos, 4 * $bbs);
$pos += 4 * $bbs;
}
$pos = 0;
$sbdBlock = $this->sbdStartBlock;
$this->smallBlockChain = '';
while ($sbdBlock != -2) {
$pos = ($sbdBlock + 1) * self::BIG_BLOCK_SIZE;
$this->smallBlockChain .= substr($this->data, $pos, 4 * $bbs);
$pos += 4 * $bbs;
$sbdBlock = self::_GetInt4d($this->bigBlockChain, $sbdBlock * 4);
}
// read the directory stream
$block = $this->rootStartBlock;
$this->entry = $this->_readData($block);
$this->_readPropertySets();
}
The difference between
$objReader = \PHPExcel_IOFactory::createReader('xlsx' == $ext ? 'Excel2007' : 'Excel5');
and
$objReader = \PHPExcel_IOFactory::createReaderForFile('uploads/' . $name);
The first is trusting that the extension is correct for the actual format of the file, that a file with an extension of .xlsx really is an OfficeOpenXML-format file or an extension of .xls really is a BIFF-format file, and then telling PHPExcel to use the appropriate reader.
This isn't normally a problem unless it isn't (for example) just HTML markup in a file with an .xls or .xlsx extension.... then you're selecting the wrong Reader for the actual format of the file; and this is what MS Excel itself is telling you with its message that "FILE EXTENSION AND THE FILE FORMAT DO NOT MATCH"
The second is using PHPExcel's identify() method to work out what format the file really is (irrespective of what it claims to be based on a false extension), and then selecting the appropriate Reader for that format.
EDIT
Unsure exactly how large your large numbers are, but I'll take a look at the HTML Reader and see if I can identify why it should be giving a boolean false instead of an actual numeric value
i have found this answer ,
PHP Converting CSV to XLS - phpExcel error
but i have tried it in Laravel 4 and i am not able to get it to work , any help would be appreciated.
My Code
public function CsvExcelConverter($filename){
$objReader = Excel::createReader('CSV');
$objReader->setDelimiter(";");
$objPHPExcel = $objReader->load('uploads/'.$filename);
$objWriter = Excel::createWriter($objPHPExcel, 'Excel5');
//new file
$new_filename = explode('.',$filename);
$new_name = $new_filename[1];
$objWriter->save($new_name.'.xls');
return $new_name.'.xls';
}
thank for the answers, but for some reason we cant seem to set the delimiter on load but i have found that you can set it in the config file .
vendeor/maatwebsite/excel/src/config/csv.php
then just specify the delimiter. this way when loading the file it actually separates each entry and when converting it each entry is in its own cell.
thanks for all the help.
/* Get the excel.php class here: http://www.phpclasses.org/browse/package/1919.html */
require_once("../classes/excel.php");
$inputFile=$argv[1];
$xlsFile=$argv[2];
if( empty($inputFile) || empty($xlsFile) ) {
die("Usage: ". basename($argv[0]) . " in.csv out.xls\n" );
}
$fh = fopen( $inputFile, "r" );
if( !is_resource($fh) ) {
die("Error opening $inputFile\n" );
}
/* Assuming that first line is column headings */
if( ($columns = fgetcsv($fh, 1024, "\t")) == false ) {
print( "Error, couldn't get header row\n" );
exit(-2);
}
$numColumns = count($columns);
/* Now read each of the rows, and construct a
big Array that holds the data to be Excel-ified: */
$xlsArray = array();
$xlsArray[] = $columns;
while( ($rows = fgetcsv($fh, 1024, "\t")) != FALSE ) {
$rowArray = array();
for( $i=0; $i<$numColumns;$i++ ) {
$key = $columns[$i];
$val = $rows[$i];
$rowArray["$key"] = $val;
}
$xlsArray[] = $rowArray;
unset($rowArray);
}
fclose($fh);
/* Now let the excel class work its magic. excel.php
has registered a stream wrapper for "xlsfile:/"
and that's what triggers its 'magic': */
$xlsFile = "xlsfile://".$xlsFile;
$fOut = fopen( $xlsFile, "wb" );
if( !is_resource($fOut) ) {
die( "Error opening $xlsFile\n" );
}
fwrite($fOut, serialize($xlsArray));
fclose($fOut);
exit(0);
If you use the maatwebsite/excel library in Laravel, you can only use native PHPExcel instance methods, not static methods. To convert from CSV to excel, this code can be found at Documentation page
Excel::load($filename, function($file) {
// modify file content
})->setFileName($new_name)->store('xls');
In theory, you should create your custom class to set delimiter:
class CSVExcel extends Excel {
protected $delimiter = ';';
}
and now you could use:
CSVExcel::load('csvfilename.csv')->setFileName('newfilename')->export('xls');
But the problem is, that $delimiter isn't used in this case. Delimiter support seems to be added not long time ago, so maybe there is a bug or it needs to be used in the other way. I've added issue just in case for that: https://github.com/Maatwebsite/Laravel-Excel/issues/262
I need a script that is finding and then replacing a sertain line in a CSV like file.
The file looks like this:
18:110327,98414,127500,114185,121701,89379,89385,89382,92223,89388,89366,89362,89372,89369
21:82297,79292,89359,89382,83486,99100
98:110327,98414,127500,114185,121701
24:82297,79292,89359,89382,83486,99100
Now i need to change the line 21.
This is wat i got so far.
The first 2 to 4 digits folowed by : ar a catergory number. Every number after this(followed by a ,) is a id of a page.
I acces te id's i want (i.e. 82297 and so on) from database.
//test 2
$sQry = "SELECT * FROM artikelen WHERE adviesprijs <>''";
$rQuery = mysql_query ($sQry);
if ( $rQuery === false )
{
echo mysql_error ();
exit ;
}
$aResult = array ();
while ( $r = mysql_fetch_assoc ($rQuery) )
{
$aResult[] = $r['artikelid'];
}
$replace_val_dirty = join(",",$aResult);
$replace_val= "21:".$replace_val_dirty;
// file location
$file='../../data/articles/index.lst';
// read the file index.lst
$file1 = file_get_contents($file);
//strip eerde artikel id van index.lst
$file3='../../data/articles/index_grp21.lst';
$file3_contents = file_get_contents($file3);
$file2 = str_replace($file3_contents, $replace_val, $file1);
if (file_exists($file)) {
echo "The file $filename exists";
} else {
echo "The file $filename does not exist";
}
if (file_exists($file3)) {
echo "The file $filename exists";
} else {
echo "The file $filename does not exist";
}
// replace the data
$file_val = $file2;
// write the file
file_put_contents($file, $file_val);
//write index_grp98.lst
file_put_contents($file3, $replace_val);
mail('info#', 'Aanbieding catergorie geupdate', 'Aanbieding catergorie geupdate');
Can anyone point me in the right direction to do this?
Any help would be appreciated.
You need to open the original file and go through each line. When you find the line to be changed, change that line.
As you can not edit the file while you do that, you write a temporary file while doing this, so you copy over line-by-line and in case the line needs a change, you change that line.
When you're done with the whole file, you copy over the temporary file to the original file.
Example Code:
$path = 'file';
$category = 21;
$articles = [111182297, 79292, 89359, 89382, 83486, 99100];
$prefix = $category . ':';
$prefixLen = strlen($prefix);
$newLine = $prefix . implode(',', $articles);
This part is just setting up the basics: The category, the IDs of the articles and then building the related strings.
Now opening the file to change the line in:
$file = new SplFileObject($path, 'r+');
$file->setFlags(SplFileObject::DROP_NEW_LINE | SplFileObject::SKIP_EMPTY);
$file->flock(LOCK_EX);
The file is locked so that no other process can edit the file while it gets changed. Next to that file, the temporary file is needed, too:
$temp = new SplTempFileObject(4096);
After setting up the two files, let's go over each line in $file and compare if it needs to be replaced:
foreach ($file as $line) {
$isCategoryLine = substr($line, 0, $prefixLen) === $prefix;
if ($isCategoryLine) {
$line = $newLine;
}
$temp->fwrite($line."\n");
}
Now the $temporary file contains already the changed line. Take note that I used UNIX type of EOF (End Of Line) character (\n), depending on your concrete file-type this may vary.
So now, the temporary file needs to be copied over to the original file. Let's rewind the file, truncate it and then write all lines again:
$file->seek(0);
$file->ftruncate(0);
foreach ($temp as $line) {
$file->fwrite($line);
}
And finally you need to lift the lock:
$file->flock(LOCK_UN);
And that's it, in $file, the line has been replaced.
Example at once:
$path = 'file';
$category = 21;
$articles = [111182297, 79292, 89359, 89382, 83486, 99100];
$prefix = $category . ':';
$prefixLen = strlen($prefix);
$newLine = $prefix . implode(',', $articles);
$file = new SplFileObject($path, 'r+');
$file->setFlags(SplFileObject::DROP_NEW_LINE | SplFileObject::SKIP_EMPTY);
$file->flock(LOCK_EX);
$temp = new SplTempFileObject(4096);
foreach ($file as $line) {
$isCategoryLine = substr($line, 0, $prefixLen) === $prefix;
if ($isCategoryLine) {
$line = $newLine;
}
$temp->fwrite($line."\n");
}
$file->seek(0);
$file->ftruncate(0);
foreach ($temp as $line) {
$file->fwrite($line);
}
$file->flock(LOCK_UN);
Should work with PHP 5.2 and above, I use PHP 5.4 array syntax, you can replace [111182297, ...] with array(111182297, ...) in case you're using PHP 5.2 / 5.3.
I have some PHP code that runs a query on a database, saves the results to a csv file, and then allows the user to download the file. The problem is, the csv file contains page HTML around the actual csv content.
I've read all the related questions here already, including this one. Unfortunately my code exists within Joomla, so even if I try to redirect to a page that contains nothing but headers, Joomla automatically surrounds it with its own navigation code. This only happens at the time of download; if I look at the csv file that's saved on the server, it does not contain the HTML.
Can anyone help me out with a way to force a download of the actual csv file as it is on the server, rather than as the browser is editing it to be? I've tried using the header location, like this:
header('Location: ' . $filename);
but it opens the file in the browser, rather than forcing the save dialog.
Here's my current code:
//set dynamic filename
$filename = "customers.csv";
//open file to write csv
$fp = fopen($filename, 'w');
//get all data
$query = "select
c.firstname,c.lastname,c.email as customer_email,
a.email as address_email,c.phone as customer_phone,
a.phone as address_phone,
a.company,a.address1,a.address2,a.city,a.state,a.zip, c.last_signin
from {$dbpre}customers c
left join {$dbpre}customers_addresses a on c.id = a.customer_id order by c.last_signin desc";
$votes = mysql_query($query) or die ("File: " . __FILE__ . "<br />Line: " . __LINE__ . "<p>{$query}<p>" . mysql_error());
$counter = 1;
while ($row = mysql_fetch_array($votes,1)) {
//put header row
if ($counter == 1){
$headerRow = array();
foreach ($row as $key => $val)
$headerRow[] = $key;
fputcsv($fp, $headerRow);
}
//put data row
fputcsv($fp, $row);
$counter++;
}
//close file
fclose($fp);
//redirect to file
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=".$filename);
header("Content-Transfer-Encoding: binary");
readfile($filename);
exit;
EDITS
Full URL looks like this:
http://mysite.com/administrator/index.php?option=com_eimcart&task=customers
with the actual download link looking like this:
http://mysite.com/administrator/index.php?option=com_eimcart&task=customers&subtask=export
MORE EDITS
Here's a shot of the page that the code is on; the generated file still is pulling in the html for the submenu. The code for the selected link (Export as CSV) is now
index.php?option=com_eimcart&task=customers&subtask=export&format=raw
Now here is a screenshot of the generated, saved file:
It shrank during the upload here, but the text highlighted in yellow is the html code for the subnav (list customers, add new customer, export as csv). Here's what my complete code looks like now; if I could just get rid of that last bit of html it would be perfect.
$fp= fopen("php://output", 'w');
$query = "select c.firstname,c.lastname,c.email as customer_email,
a.email as address_email,c.phone as customer_phone,
a.phone as address_phone, a.company, a.address1,
a.address2,a.city,a.state,a.zip,c.last_signin
from {$dbpre}customers c
left join {$dbpre}customers_addresses a on c.id = a.customer_id
order by c.last_signin desc";
$votes = mysql_query($query) or die ("File: " . __FILE__ . "<br />Line: " . __LINE__ . "<p>{$query}<p>" . mysql_error());
$counter = 1;
//redirect to file
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=customers.csv");
header("Content-Transfer-Encoding: binary");
while ($row = mysql_fetch_array($votes,1)) {
//put header row
if ($counter == 1){
$headerRow = array();
foreach ($row as $key => $val)
$headerRow[] = $key;
fputcsv($fp, $headerRow);
}
//put data row
fputcsv($fp, $row);
$counter++;
}
//close file
fclose($fp);
UPDATE FOR BJORN
Here's the code (I think) that worked for me. Use the RAW param in the link that calls the action:
index.php?option=com_eimcart&task=customers&subtask=export&format=raw
Because this was procedural, our link was in a file called customers.php, which looks like this:
switch ($r['subtask']){
case 'add':
case 'edit':
//if the form is submitted then go to validation
include("subnav.php");
if ($r['custFormSubmitted'] == "true")
include("validate.php");
else
include("showForm.php");
break;
case 'delete':
include("subnav.php");
include("process.php");
break;
case 'resetpass':
include("subnav.php");
include("resetpassword");
break;
case 'export':
include("export_csv.php");
break;
default:
include("subnav.php");
include("list.php");
break;
}
So when a user clicked on the link above, the export_csv.php file is automatically included. That file contains all the actual code:
<?
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=customers.csv");
header("Content-Transfer-Encoding: binary");
$fp= fopen("php://output", 'w');
//get all data
$query = "select
c.firstname,c.lastname,c.email as customer_email,
a.email as address_email,c.phone as customer_phone,
a.phone as address_phone,
a.company,a.address1,a.address2,a.city,a.state,a.zip, c.last_signin
from {$dbpre}customers c
left join {$dbpre}customers_addresses a on c.id = a.customer_id order by c.last_signin desc";
$votes = mysql_query($query) or die ("File: " . __FILE__ . "<br />Line: " . __LINE__ . "<p>{$query}<p>" . mysql_error());
$counter = 1;
while ($row = mysql_fetch_array($votes,1)) {
//put header row
if ($counter == 1){
$headerRow = array();
foreach ($row as $key => $val)
$headerRow[] = $key;
fputcsv($fp, $headerRow);
}
//put data row
fputcsv($fp, $row);
$counter++;
}
//close file
fclose($fp);
This is a piece of sample code that I just cooked up to help you out. Use it as an action method in your controller.
function get_csv() {
$file = JPATH_ADMINISTRATOR . DS . 'test.csv';
// Test to ensure that the file exists.
if(!file_exists($file)) die("I'm sorry, the file doesn't seem to exist.");
// Send file headers
header("Content-type: text/csv");
header("Content-Disposition: attachment;filename=test.csv");
// Send the file contents.
readfile($file);
}
This alone will not be enough, because the file you download will still contain the surrounding html. To get rid of it and only receive the csv file's contents you need to add format=raw parameter to your request. In my case the method is inside the com_csvexample component, so the url would be:
/index.php?option=com_csvexample&task=get_csv&format=raw
EDIT
In order to avoid using an intermediate file substitute
//set dynamic filename
$filename = "customers.csv";
//open file to write csv
$fp = fopen($filename, 'w');
with
//open the output stream for writing
//this will allow using fputcsv later in the code
$fp= fopen("php://output", 'w');
Using this method you have to move the code that sends headers before anything is written to the output. You also won't need the call to the readfile function.
Add this method to your controller:
function exportcsv() {
$model = & $this->getModel('export');
$model->exportToCSV();
}
Then add a new model called export.php, code below. You will need to change or extend the code to your situation.
<?php
/**
* #package TTVideo
* #author Martin Rose
* #website www.toughtomato.com
* #version 2.0
* #copyright Copyright (C) 2010 Open Source Matters. All rights reserved.
* #license http://www.gnu.org/copyleft/gpl.html GNU/GPL
*/
//No direct acesss
defined('_JEXEC') or die();
jimport('joomla.application.component.model');
jimport( 'joomla.filesystem.file' );
jimport( 'joomla.filesystem.archive' );
jimport( 'joomla.environment.response' );
class TTVideoModelExport extends JModel
{
function exportToCSV() {
$files = array();
$file = $this->__createCSVFile('#__ttvideo');
if ($file != '') $files[] .= $file;
$file = $this->__createCSVFile('#__ttvideo_ratings');
if ($file != '') $files[] .= $file;
$file = $this->__createCSVFile('#__ttvideo_settings');
if ($file != '') $files[] .= $file;
// zip up csv files to be delivered
$random = rand(1, 99999);
$archive_filename = JPATH_SITE.DS.'tmp'.DS.'ttvideo_'. strval($random) .'_'.date('Y-m-d').'.zip';
$this->__zip($files, $archive_filename);
// deliver file
$this->__deliverFile($archive_filename);
// clean up
JFile::delete($archive_filename);
foreach($files as $file) JFile::delete(JPATH_SITE.DS.'tmp'.DS.$file);
}
private function __createCSVFile($table_name) {
$db = $this->getDBO();
$csv_output = '';
// get table column names
$db->setQuery("SHOW COLUMNS FROM `$table_name`");
$columns = $db->loadObjectList();
foreach ($columns as $column) {
$csv_output .= $column->Field.'; ';
}
$csv_output .= "\n";
// get table data
$db->setQuery("SELECT * FROM `$table_name`");
$rows = $db->loadObjectList();
$num_rows = count($rows);
if ($num_rows > 0) {
foreach($rows as $row) {
foreach($row as $col_name => $value) {
$csv_output .= $value.'; ';
}
$csv_output .= "\n";
}
}
$filename = substr($table_name, 3).'.csv';
$file = JPATH_SITE.DS.'tmp'.DS.$filename;
// write file to temp directory
if (JFile::write($file, $csv_output)) return $filename;
else return '';
}
private function __deliverFile($archive_filename) {
$filesize = filesize($archive_filename);
JResponse::setHeader('Content-Type', 'application/zip');
JResponse::setHeader('Content-Transfer-Encoding', 'Binary');
JResponse::setHeader('Content-Disposition', 'attachment; filename=ttvideo_'.date('Y-m-d').'.zip');
JResponse::setHeader('Content-Length', $filesize);
echo JFile::read($archive_filename);
}
/* creates a compressed zip file */
private function __zip($files, $destination = '') {
$zip_adapter = & JArchive::getAdapter('zip'); // compression type
$filesToZip[] = array();
foreach ($files as $file) {
$data = JFile::read(JPATH_SITE.DS.'tmp'.DS.$file);
$filesToZip[] = array('name' => $file, 'data' => $data);
}
if (!$zip_adapter->create( $destination, $filesToZip, array() )) {
global $mainframe;
$mainframe->enqueueMessage('Error creating zip file.', 'message');
}
}
}
?>
Then go to your default view.php and add a custom buttom, e.g.
// custom export to set raw format for download
$bar = & JToolBar::getInstance('toolbar');
$bar->appendButton( 'Link', 'export', 'Export CSV', 'index.php?option=com_ttvideo&task=export&format=raw' );
Good luck!
You can use Apache's mod_cern_meta to add HTTP headers to static files. Content-Disposition: attachment. The required .htaccess and .meta files can be created by PHP.
Another way to output CSV data in a Joomla application is to create a view using CSV rather than HTML format. That is, create a file as follows:
components/com_mycomp/views/something/view.csv.php
And add content similar to the following:
<?php
// No direct access
defined('_JEXEC') or die;
jimport( 'joomla.application.component.view');
class MyCompViewSomething extends JViewLegacy // Assuming a recent version of Joomla!
{
function display($tpl = null)
{
// Set document properties
$document = &JFactory::getDocument();
$document->setMimeEncoding('text/csv');
JResponse::setHeader('Content-disposition', 'inline; filename="something.csv"', true);
// Output UTF-8 BOM
echo "\xEF\xBB\xBF";
// Output some data
echo "field1, field2, 'abc 123', foo, bar\r\n";
}
}
?>
Then you can create file download links as follows:
/index.php?option=com_mycomp&view=something&format=csv
Now, you would be right to question the 'inline' part in the Content-disposition. If I recall correctly when writing this code some years ago, I had problems with the 'attachment' option. This link which I just googled now seemed familiar as the driver for it: https://dotanything.wordpress.com/2008/05/30/content-disposition-attachment-vs-inline/ . I've been using 'inline' ever since and am still prompted to save the file appropriately from any browsers I test with. I haven't tried using 'attachment' any time recently, so it may work fine now of course (the link there is 7 years old now!)