exporting large files to excel using PHPExcel - php

I am trying to export around 40,000 rows of Mysql data in PHP(Laravel4) using PHPExcel library.
Below is my code:
($patList is an array of result columns)
set_time_limit ( 3000 );
$cacheMethod = PHPExcel_CachedObjectStorageFactory:: cache_to_phpTemp;
$cacheSettings = array( 'memoryCacheSize' => -1);
PHPExcel_Settings::setCacheStorageMethod($cacheMethod, $cacheSettings);
$objPHPExcel = new PHPExcel();
$i = 1;
$patList = $result[0];
for ($r = 0; $r < count($patList); $r++) {
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue("A$i", $patList[$r][0])
->setCellValue("B$i", $patList[$r][1])
->setCellValue("C$i", $patList[$r][2])
->setCellValue("D$i", $patList[$r][1])
->setCellValue("E$i", $patList[$r][2])
->setCellValue("F$i", $patList[$r][1])
->setCellValue("G$i", $patList[$r][2])
->setCellValue("H$i", $patList[$r][2])
->setCellValue("I$i", $patList[$r][1])
->setCellValue("J$i", $patList[$r][2])
->setCellValue("K$i", $patList[$r][5]);
$i++;
}
$objPHPExcel->getActiveSheet()->setTitle('Name of Sheet 1');
header('Content-Type: application/vnd.ms-excel');
header('Content-Disposition: attachment;filename="result.xls"');
header('Cache-Control: max-age=0');
ob_clean();
flush();
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel5');
$objWriter->save('php://output');
The above code runs fine if there are 3-4 columns in the excel and 15,000 rows. However, if I increase the no. of rows to 30,000 or the no. of columns to 10, the excel doesn't get generated.

$cacheSettings = array( 'memoryCacheSize' => -1);
isn't sensible.... I don't even know if using a value of -1 will work; but if it does, it will mean that you're storing everything in memory and nothing in php://temp
The memoryCacheSize value tells the cache stream how much data should be stored in memory before switching data out to php://temp ; so
$cacheSettings = array( 'memoryCacheSize' => '8MB');
would tell the cache stream to use 8MB of memory, and then if additional data needed to be stored to use php://temp instead

Related

Large excel file is not generating, getting error: ERR_INVALID_RESPONSE

Using PHPExcel library in Codeigniter.
Small excel file of about 20k row lines are generating perfectly, while in the case of large (like 43k row lines) file it gets:
This site can’t be reached
The webpage at
https://exmple.com/
might be temporarily down or it may have moved permanently to a new web address.
ERR_INVALID_RESPONSE
I've tried-
ini_set('max_input_vars', 19999);
set_time_limit ( 6000 );
ini_set('max_execution_time', 6000);
memory_get_usage(true);
but didn't get result.
Codeigniter Version: 3.1.11
PHP Version: 7.4
Part of code if needed (its not exact code):
public function test(){
$this->load->library('Excel');
ob_start();
// Create new PHPExcel object
$objPHPExcel = new PHPExcel();
$exlHeading = array(
'font' => array(
'bold' => true,
'size' => 12,
'name' => 'Verdana')
);
// Set document properties
$objPHPExcel->getProperties()->setCreator("BigDream India")
->setLastModifiedBy("TEST")
->setTitle("REPORT")
->setSubject("ATTENDANCE REPORT")
->setDescription("Attendance Monthly Report")
->setKeywords("ATT_REPORT")
->setCategory("Excel Sheet");
for($i=0; $i<=40000; $i++){
$objPHPExcel->setActiveSheetIndex(0)->setCellValue('A'.$i, 'Test content is here.');
$objPHPExcel->setActiveSheetIndex(0)->mergeCells("A$i:I$i");
$objPHPExcel->getActiveSheet()->getStyle("A$i:I$i")->applyFromArray($exlHeading);
}
$objPHPExcel->getActiveSheet()->setTitle('Attendance Monthly Report');
$objPHPExcel->setActiveSheetIndex(0);
ob_end_clean();
$filename = 'MyOfficeGuardian-Monthly_Report.xls';
header('Content-type: application/vnd.ms-excel');
header('Content-Disposition: attachment;filename="' . $filename . '"');
header('Cache-Control: max-age=0');
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel5');
$objWriter->save('php://output');
}
Issue resolved by just updating upload_max_filesize value from 2mb to 20mb in php.ini file.

PHPExcel Clone Worksheet - Efficient use of memory

I need to clone the first worksheet a few times, accordingly to the amount of rows, but something may be wrong.
The code is:
public function downloadFile()
{
date_default_timezone_set('America/Sao_Paulo');
if(file_exists("xpto.xlsx")){
$objPHPExcel = PHPExcel_IOFactory::load("xpto.xlsx");
$sheets = 3;//3 is enough to throw the error
for($i = 0; $i<$sheets; $i++){
$objClonedWorksheet = clone $objPHPExcel->getSheet(0);
$objClonedWorksheet->setTitle('Sheet ' . $i);
$objClonedWorksheet->setCellValue('A1', 'Test ' . $i);
$objPHPExcel->addSheet($objClonedWorksheet);
}
$objPHPExcel->setActiveSheetIndex(0);
$filename = 'file.xlsx';
header('Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
header('Content-Disposition: attachment;filename="'.$filename.'"');
header('Cache-Control: max-age=0');
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel2007');
ob_end_clean();
$ret = $objWriter->save('php://output');
exit;
}
}
But I got an exhausted memory error. Than I tried the most commented solution (that is actually an workaround) that is to add
ini_set('memory_limit', '-1');
I added this line just after the load function and it worked, but I don't think it is a good solution to use on a SaaS application. I don't even think most hosts (AWS, for example) will allow me to use that.
I also tried to clone the sheet before the for loop, but when use addSheet, I realized that this function doesn't create a new object and when I change the name of the sheet (by the second iteration of the for loop), it changes the last sheet created, throwing an "already existing sheet with the same name" error.
Trying to use one of the links #rhazen listed, I changed the for loop to:
$objFromSheet = $objPHPExcel->getSheet(0);
$sheets = 3;
for($i = 1; $i<=$sheets; $i++){
$objToSheet = $objPHPExcel->createSheet($i);
foreach($objFromSheet->getRowIterator() as $row){
$cellIterator = $row->getCellIterator();
$cellFrom = $cellIterator->current();
$cellTo = $objToSheet->getCell($cellFrom->getCoordinate());
$cellTo->setXfIndex($cellFrom->getXfIndex());
$cellTo->setValue($cellFrom->getValue());
}
}
But it seems not to work either. Is there a misunderstanding about Iterator or XfIndex?
The solution is in the edited question. Thanks for those who helped.

Selective download with PHPExcel

I'm using PHPExcel to download some data stored in MySQLi. I made an algorithm that is working for every data base (in theory). I have tested it with some of them and it was working fine.
I extracted names of the columns in an array: column_names and then, I'm adding titles and data to the excel report.
// Adding titles
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue('A1',$bigTitle);
$counter = 0;
$let = 'a';
while ($counter <= count($column_names)){
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue(strtoupper($let).'3', $column_names[$counter]);
$let++;
$counter++;
}
//Adding data
$i = 4;
while ($row = $result->fetch_array()) {
$counter = 0;
$let = 'a';
while ($counter <= count($column_names)){
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue(strtoupper($let).$i, $row[$column_names[$counter]]);
$let++;
$counter++;
}
$i++;
}
I'm connecting to the database using
$conexion = new mysqli('localhost','user','pass','SAT_dbname',21);
I cloned "SAT_db1" database to "SAT_db2". They have exactly the same structure but different information. The download is working if I'm using
$conexion = new mysqli('localhost','user','pass','SAT_db1',21);
But it's not working if I'm using
$conexion = new mysqli('localhost','user','pass','SAT_db2',21);
I don't know what is wrong if they're the same with different names. Is not PHPExcel working with cloned databases? What else could it be?
The error shows up in the browser as "File not found".
EDIT
I was testing the download all day and I finally found something: I can download when I have few registers. When I have a few more, I can't.
I'm sending the file to the browser:
header('Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet');
header('Content-Disposition: attachment;filename="Report.xlsx"');
header('Cache-Control: max-age=0');
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel2007');
$objWriter->save('php://output');
Still haven't found a solution.
PHPExcel has limits with cache. I had to change these limits manually. I used this code before creating PHPExcel object:
set_time_limit(0) ;
$cacheMethod = PHPExcel_CachedObjectStorageFactory:: cache_to_phpTemp;
$cacheSettings = array( 'memoryCacheSize' => '500MB');
PHPExcel_Settings::setCacheStorageMethod($cacheMethod, $cacheSettings);
I assume if you have bigger files, you can expand "memoryCacheSize".

export file php to xls uses class PHPExcel

I want to export some file php to XLS file use class PHPEXcel, I don't used this before.
notif on my browser:
"Fatal error: Allowed memory size of 25165824 bytes exhausted (tried to allocate 1056 bytes) in C:\AppServ\www\kjjp2\Classes\PHPExcel\Cell.php on line 1124"
code:
<?php
include "config/koneksi.php";
error_reporting(E_ALL);
require_once 'Classes/PHPExcel.php';
// Create new PHPExcel object
$objPHPExcel = new PHPExcel();
$query = "SELECT * FROM `tabeldata`";
$hasil = mysql_query($query);
// Set properties
$objPHPExcel->getProperties()->setCreator("Erik")
->setLastModifiedBy("Erik")
->setTitle("Office 2007 XLSX ")
->setSubject("Office 2007 XLSX ")
->setDescription("Document for Office 2007 XLSX, generated using PHP classes.")
->setKeywords("office 2007 openxml php")
->setCategory("Test result file");
// Add some data
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue('A1', 'Jenis Report')
->setCellValue('B1', 'Pembayaran')
->setCellValue('C1', 'No')
->setCellValue('D1', 'Cabang')
//and some files
->setCellValue('AG1', 'Surveyor');
$rowNya = 3;
$no = 0;
while($row=mysql_fetch_array($hasil)){
$no = $no +1;
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue("A$rowNya", $row['jenReport'])
->setCellValue("B$rowNya", $row['pembayaran'])
->setCellValue("C$rowNya", $row['no'])
->setCellValue("D$rowNya", $row['cabang'])
->setCellValue("E$rowNya", $row['namaSales'])
->setCellValue("F$rowNya", $row['jenLaporan'])
//and some files
->setCellValue("AG$rowNya", $row['surveyor']);
$rowNya = $rowNya + 1;
}
// Rename sheet
$objPHPExcel->getActiveSheet()->setTitle('Simple');
// Set active sheet index to the first sheet, so Excel opens this as the first sheet
$objPHPExcel->setActiveSheetIndex(0);
// Redirect output to a client’s web browser (Excel5)
header('Content-Type: application/vnd.ms-excel');
header('Content-Disposition: attachment;filename="database.xls"');
header('Cache-Control: max-age=0');
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel5');
$objWriter->save('php://output');
exit;
?>
You are running out of RAM that can be used by PHP. You can set how much RAM that PHP will use in php.ini. You currently have this limit set to 24MB, which is pretty low. Try increasing it.
ini_set('memory_limit', '256M');
PHPExcel is known for being memory hungry. The website has a discussions with some workarounds, see http://phpexcel.codeplex.com/discussions/242712?ProjectName=phpexcel
You could also consider using another library like the old Spreadsheet_Excel_Writer lib (which has its drawbacks as well)
http://pear.php.net/package/Spreadsheet_Excel_Writer/redirected

PHPexcel cell caching

Am working with large excel file I have removed limits is there a way I can read through my cells much quicker. I have 6000 rows and reducing the tmp folder in my wamp folder.
set_time_limit(0);
ini_set('memory_limit', '-1');
include'../Classes/PHPExcel.php';
include'../Classes/PHPExcel/IOFactory.php';
$cacheMethod = PHPExcel_CachedObjectStorageFactory::cache_to_phpTemp;
$cacheSettings = array( ' memoryCacheSize ' =>'8MB');
PHPExcel_Settings::setCacheStorageMethod($cacheMethod, $cacheSettings);
$objReader = new PHPExcel_Reader_Excel2007();
$objPHPExcel = $objReader->load('Book1.xlsx');
$highestRowEM = $objPHPExcel->getActiveSheet()->getHighestRow();
echo $highestRowEM;
for($i=0;$i<$highestRowEM;$i++){
$varval=$objPHPExcel->getActiveSheet()->getCell('A'.$i)->getValue();
if($varval==""){
break;
}
}
echo $i;
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel2007');
$objWriter->save('Book1.xlsx');
What to clean up workbooks with worksheets with , in them
`include'../Classes/PHPExcel.php';
include'../Classes/PHPExcel/IOFactory.php';
set_time_limit(0);
ini_set('memory_limit', '-1');
$xlsxfiles=$_SESSION['file'];
echo $xlsxfiles;
echo "<br>";
$objReader = PHPExcel_IOFactory::createReader('Excel2007');
$objPHPExcel = PHPExcel_IOFactory::load('../upload/'.$xlsxfiles);
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, "Excel2007");
////Validation
foreach ($objPHPExcel->getWorksheetIterator() as $worksheet) {
$highestRow = $worksheet-> getHighestDataRow();
$columnhighest=$worksheet->getHighestDataColumn();
$columnhighestval=array_search($columnhighest, $alphabet);
echo 'Worksheet - ' , $worksheet->getTitle()." Number of rows: ".$highestRow."<br>";
for($cl=0;$cl<$highestRow+1;$cl++){
for ($ga=1;$ga<$columnhighestval;$ga++){
$letters=strtoupper($alphabet[$ga]);
$clean=$worksheet->getCell($letters.$cl)->getValue();
$cleandone=str_replace(','," ",$clean);
$worksheet->setCellValue($letters.$cl,$cleandone);
}
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel2007');
}
}
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, "Excel2007");
$objWriter->setOffice2003Compatibility(true);
$objWriter->save('../upload/'.$xlsxfiles);
echo "Done";
Tip 1:
replace
for($i=0;$i<$highestRowEM;$i++){
$varval=$objPHPExcel->getActiveSheet()->getCell('A'.$i)->getValue();
with
$objSheet = $objPHPExcel->getActiveSheet();
for($i=0;$i<$highestRowEM;$i++){
$varval = $objSheet->getCell('A'.$i)->getValue();
Then you're not calling $objPHPExcel->getActiveSheet() in every iteration of the loop.
Then tell us what you're actually trying to do with the worksheet data, and we may be able to help speed it up a bit more
EDIT
Don't set the cell value unless you have to;
Don't instantiate the Writer until you need it, and definitely not in the loop;
Instead of using $letters and $alphabet, use the routines built-into PHPExcel, or take advantage of PHP's Perl-style character incrementor;
Don't do maths in your loop comparisons
////Validation
foreach ($objPHPExcel->getWorksheetIterator() as $worksheet) {
$highestRow = $worksheet->getHighestDataRow();
$columnhighest=$worksheet->getHighestDataColumn();
$columnhighest++;
echo 'Worksheet - ' , $worksheet->getTitle()." Number of rows: ".$highestRow."<br>";
for($cl = 0; $cl <= $highestRow; $cl++){
for ($ga='A'; $ga !== $columnhighest; $ga++){
$clean=$worksheet->getCell($ga.$cl)->getValue();
$cleandone=str_replace(','," ",$clean);
if($clean != $cleandone) {
$worksheet->setCellValue($ga.$cl, $cleandone);
}
}
}
}
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, 'Excel2007');

Categories