Create big zip archive in PHP - php

In a current PHP project, I need to bundle up a bunch of PDF files in some sort of archive so the user can download them all together. Because zip is most common and even the most basic non-IT-Windows guys know about it, I was going for a zip archive.
My code looks like the following
$invoices = getRequestedInvoices(); // load all requested invoices, this function is just for demonstration
// Create a temporary zip archive
$filename = tempnam("tmp", "zip");
$zip = new \ZipArchive();
$zip->open($filename, \ZipArchive::OVERWRITE);
foreach($invoices as $invoice)
{
// create the pdf file and add it to the archive
$pdf = new InvoicePdf($invoice); // this is derived from \ZendPdf\PdfDocument
$zip->addFromString($pdf->getFilename(), $pdf->render()); // for clarification: the getFilename method creates a filename for the PDF based on the invoice's id
}
$zip->close();
header('Content-Type: application/zip');
header('Content-Length: ' . filesize($filename));
header('Content-Disposition: attachment; filename="invoices.zip"');
readfile($filename);
unlink($filename);
exit;
This script works fine if the server has enough memory. Unfortunately our productive system is very limited so the script only works with a few PDF files, but most of the time it runs out of memory and aborts. Adding unlink($pdf) at the end of the foreach loop didn't help, so my guess is the ZipArchive object is using up the memory.
I am trying to add as little dependencies as possible to the project, so I would love to be able to solve this with PHPs (PHP 5.4) own functions or functions from Zend Framework 2. I was looking for some way of directly streaming the archive (the zip:// stream wrapper looked good at first, but it's read-only), but that seems to be impossible with zip archives.
Does anyone have an idea? Maybe a different but also widely known type of archive that allows streaming? Compression is not a must

I had to find a quick solution to this problem, so despite trying to avoid it, I had to use an external dependency.
I found the ZipStream class from the PHPZip project (https://github.com/Grandt/PHPZip) which does the job just fine.
$zip = new \ZipStream("invoices.zip");
foreach($invoices as $invoice)
{
$pdf = new InvoicePdf($invoice);
$zip->addFile($pdf->render(), $pdf->getFilename());
}
$zip->finalize();
exit;

Related

How to debug/time my php function? Create Zip file

Have a php function to generate a zip file "on fly". I have the files on one server (AWS S3) but the PHP function to generate the zip-file is on another server/web hosting. I have noticed that it takes long time to generate the zip-file and I get a corrupt zip file if there are many files when I create zip file. I want to troubleshoot/debug where it takes "stop", what the missing link is if there are many files (the limit seems to be 20 files, which is not many).
How can I find out where in my function it fails if I have more than 20 files to generate my zip-file from?
Can I add a timer to every row?
Can I find out if it is memory or something else on my shared hosting (where I have the php function)? Or if it is something with S3.
My php function to generate the zip file from files on AWS S3
<?php
$imageQueryResult = $this->getUserImages('download', array('i.image_name'));
if(!empty($imageQueryResult)) {
$imageUrl = $this->getFromAmazon('download', $imageQueryResult);
$imageNumber = 1;
$zipName = 'tt-'.date("Y-m-d").'.zip';
//create new zip object
$zip = new ZipArchive();
//create a temp file & open it
$tmp_file = tempnam('.','');
$zip->open($tmp_file, ZipArchive::CREATE);
//loop through each file
foreach($imageUrl as $image){
//Get extension
$ext = pathinfo(parse_url($image, PHP_URL_PATH), PATHINFO_EXTENSION);
//download file
$download_file = file_get_contents($image);
//add it to the zip
$zip->addFromString(basename($imageNumber.'-tt.'.$ext),$download_file);
$imageNumber++;
}
//close zip
$zip->close();
//send the file to the browser as a download
header('Content-disposition: attachment; filename='.$zipName);
header('Content-Length: ' . filesize($tmp_file));
header('Content-type: application/zip');
readfile($tmp_file);
ignore_user_abort(true);
unlink($tmp_file);
?>
You can try to use
http://php.net/manual/en/ziparchive.getstatusstring.php
ZipArchive::getStatusString — Returns the status error message, system and/or zip messages
You don't know how long it will take to compress all your files. So you'll have to check the maximal execution time and the memory that consumes that script.
If the problem is with the time, the solution might be to do this in chunks:
open the archive for writing
add N files per iteration
close the archive
Repeat until you you have files
Keep in mind that with this approach the more files you have, the more memory you need to store the temporary results

php zlib: How to dynamically create an in-memory zip files from string variables?

this is what I need
$a=array('folder'=>'anyfolder','filename'=>'anyfilename','filedata'=>'anyfiledata');
I need to create a variable $zip with compressed data from $a and output this $zip into browser window with
header('Content-type: application/octetstream');
echo $zip;
All libs that I found, they pack files and create zip files physically on disk. I need to do it dynamically, using memory. Is there any examples of how to do it using pure php's zlib and without studying the zip format specifications?
UPD:
could it be done with CreateZIP class?
UPUPD:
Yes. it could =)
Yes, CreateZIP class does it.
require_once('CreateZipFile.php');
$zip = new CreateZipFile;
$zip->addFile('anyfiledata', 'anyfilename');
$zip->addFile('anyfiledata2', 'anyfolder/anyfilename.anyext');
$zip->addDirectory('anyemptydirwouldbecreatedinthiszipfile');
header('Content-disposition: attachment; filename='.'zipfile.zip'.'');
header('Content-type: application/octetstream');
echo $zip->getZippedfile();
die();
great thanx to Er. Rochak Chauhan

How to modify excel file using PHPExcel in symfony2

In my project I use symfony2 PHPExcel wrapper https://github.com/liuggio/ExcelBundle
With the example from the link above I can create new excel files. However this file has no style or markup at all. So I created a excel template where I want to input some data.
I know how to load an excel file:
$excelObj = $this->get('xls.load_xls2007')
->load($this->get('kernel')
->getRootDir() . '/../web/excel-template.xlsx');
//custom modifications on excel file
Now I need to create a response. But in the doc of ExcelBundle there is no information on how to do that. They just show how response work for a excel file that is created by code.
I tried:
$excelService->setExcelObj($excelObj);
$response = $excelService->getResponse();
//the rest is equal to the code in the doc
but it gives me a blank excel document.
Any ideas how to make a response with a loaded excel file?
you can do this by
// Read the file
$objReader = PHPExcel_IOFactory::createReader($fileType);
$objPHPExcel = $objReader->load($fileName);
// Change the file
$objPHPExcel->setActiveSheetIndex(0)
->setCellValue('A1', 'Hello')
->setCellValue('B1', 'World!');
// Write the file
$objWriter = PHPExcel_IOFactory::createWriter($objPHPExcel, $fileType);
$objWriter->save($fileName);
if you dont understand please comment..
I would save the file to disk and redirect the user to the on-disk version personally. This will allow several things
Your web server to serve files instead of PHP, a good thing from a performance and memory usage standpoint.
Decouple your architecture a bit to allow for future changes such as moving the creation and loading of Excel files to asynchronous operations.
The ability to use http://wiki.nginx.org/XSendfile (There is an Apache module also).
The user can re-download the file or pause and resume download without recreating it.
To do this you will want to
Save the file to a web accessible temp directory after its created
Redirect the user to that file location
Create a cron or some other job that deletes older files in the temp directory.
The tempfile api (http://us2.php.net/tmpfile) might be useful here.
the new version 2.* of PHPExcelbundle could help you.
Is now possible:
//read
$phpExcelObject = $this->get('phpexcel')->createPHPExcelObject('file.xls');
$phpExcelObject->setActiveSheetIndex(0)
->setCellValue( 'C6', 'some text' )
->setCellValue( 'D6', 'some text2' );
$writer = $this->get('phpexcel')->createWriter($phpExcelObject, 'Excel5');
$writer->save('file.xls');
// or
$response = $this->get('phpexcel')->createStreamedResponse($writer);

dompdf Customization Question

I'm trying to optimize dompdf to do something a bit strange. As it is on my server, dompdf is generating a pdf file (for the client requesting the file) from a php/html file stored somewhere on the server. This is cool because it doesn't bog the server down with pdf files, but the problem I have is that I want someone to be able to export a group of PDFs and receive them in a zip file or something similar.
Is there a way to make dompdf export a group of PDF files, based on the filenames of the php/html files, to a zip file or something so the person requesting it can download it?
Let me know if more information is needed.
Thank you!!
DOMPDF only handles a single file at a time. But you could write a PHP script to accept the list of files and then use DOMPDF to parse each one separately. Since DOMPDF can return the rendered PDF as a string you could write each file out to a temporary directory then archive the results when you're done. Or, if you're using dompdf.php you could use exec() to process each HTML document in a similar loop.
Here's a simple example showing one way to do what you want:
$files_html = array('docs/file1.html','docs/file2.html');
foreach ($files_html as $file) {
exec('dompdf.php ' . $file);
}
$zip = new ZipArchive();
$zip->open('docs/pdfs.zip', ZipArchive::CREATE);
$files_pdf = glob('docs/*.pdf');
foreach ($files_pdf as $file) {
$zip->addFile($file);
}
$zip->close();
header('Content-Description: File Transfer');
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename=pdfs.zip);
header('Content-Transfer-Encoding: binary');
readfile('docs/pdfs.zip');
There are some discussions of using DOMPDF to batch process files in the forum and issue tracker.

PHP ZIP files on the fly

What's the easiest way to zip, say 2 files, from a folder on the server and force download? Without saving the "zip" to the server.
$zip = new ZipArchive();
//the string "file1" is the name we're assigning the file in the archive
$zip->addFile(file_get_contents($filepath1), 'file1'); //file 1 that you want compressed
$zip->addFile(file_get_contents($filepath2), 'file2'); //file 2 that you want compressed
$zip->addFile(file_get_contents($filepath3), 'file3'); //file 3 that you want compressed
echo $zip->file(); //this sends the compressed archive to the output buffer instead of writing it to a file.
Can someone verify:
I have a folder with test1.doc, test2.doc, and test3.doc
with the above example - file1 (file2 and file3) might just be test1.doc, etc.
do I have to do anything with "$filepath1"? Is that the folder directory that holds the 3 docs?
Sorry for my basic question..
Unfortunately w/ PHP 5.3.4-dev and Zend Engine v2.3.0 on CentOS 5.x I couldn't get the code above to work. An "Invalid or unitialized Zip object" error message was all I could get. So, in order to make it work, I had to use following snippet (taken from the example by Jonathan Baltazar on PHP.net manual, at the ZipArchive::open page):
// Prepare File
$file = tempnam("tmp", "zip");
$zip = new ZipArchive();
$zip->open($file, ZipArchive::OVERWRITE);
// Stuff with content
$zip->addFromString('file_name_within_archive.ext', $your_string_data);
$zip->addFile('file_on_server.ext', 'second_file_name_within_archive.ext');
// Close and send to users
$zip->close();
header('Content-Type: application/zip');
header('Content-Length: ' . filesize($file));
header('Content-Disposition: attachment; filename="file.zip"');
readfile($file);
unlink($file);
I know this is different than working w/ memory only - unless you have your tmp area in ram ;-) - but maybe this can help out someone else, who's struggling with the solution above, like I was; and for which performance penalty is not an issue.
Your code is very close. You need to use the file name instead of the file contents.
$zip->addFile(file_get_contents($filepath1), 'file1');
should be
$zip->addFile($filepath1, 'file1');
http://us3.php.net/manual/en/function.ziparchive-addfile.php
If you need to add files from a variable instead of a file you can use the addFromString function.
$zip->addFromString( 'file1', $data );
This works for me (nothing is written to disk)
$tmp_file = tmpfile(); //temp file in memory
$tmp_location = stream_get_meta_data($tmp_file)['uri']; //"location" of temp file
$zip = new ZipArchive;
$res = $zip->open($tmp_location, ZipArchive::CREATE);
$zip->addFile($filepath1, 'file1');
$zip->close();
header('Content-type: application/zip');
header('Content-Disposition: attachment; filename="download.zip"');
echo(file_get_contents($tmp_location));
If you have access to the zip commandline utility you can try
<?php
$zipped_data = `zip -q - files`;
header('Content-type: application/zip');
header('Content-Disposition: attachment; filename="download.zip"');
echo $zipped_data;
?>
where files is the things you want to zip and zip the location to the zip executable.
This assumes Linux or similar, of course. In Windows you might be able to do similar with another compression tool, I guess.
There's also a zip extension, usage shown here.
maraspin's Answer is what I tried. Strangely, I got it working by removing the header line that references the file size:
header('Content-Length: ' . filesize($file));
With the above change, the code works like a breeze!
Without this change, I used to get the following error message:
End-of-central-directory signature not found. Either this file is not a zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive.
Test environment:
OS: Ubuntu 10.10
Browser: Firefox
And the default LAMP stack.
To create ZIP files on the fly (in memory), you can use ZipFile class from phpMyAdmin:
PMA\libraries\ZipFile.php
An example of how to use it in your own application:
https://stackoverflow.com/a/9648562/767871
Note: Your ZIP files will be limited by PHP's memory limit, resulting in corrupted archive if you exceed the limit.
itsols
If you want to insert the 'Content-Length' do it like this:
$length = filesize($file);
header('Content-Length: ' . $length);
I don't know why, but it crashes if you put it in the same line.
On Unix systems (and maybe others),
you can apply a simple trick to #maraspin's answer by deleting the file entry for the file ("unlinking" it from its inode), and send its data using a handle opened previously. This is basically the same thing tmpfile() does; this way you can "temporarify" any file.
The code is the same as maraspin's up to the very last lines:
// Prepare File
$file = tempnam("tmp", "zip");
$zip = new ZipArchive();
$zip->open($file, ZipArchive::OVERWRITE);
// Stuff with content
$zip->addFromString('file_name_within_archive.ext', $your_string_data);
$zip->addFile('file_on_server.ext', 'second_file_name_within_archive.ext');
// Close and send to users
$zip->close();
header('Content-Type: application/zip');
header('Content-Length: ' . filesize($file));
header('Content-Disposition: attachment; filename="file.zip"');
// open a handle to the zip file.
$fp = fopen($file, 'rb');
// unlink the file. The handle will stay valid, and the disk space will remain occupied, until the script ends or all file readers have terminated and closed.
unlink($file);
// Pass the file descriptor to the Web server.
fpassthru($fp);
As soon as the script finishes, or terminates abnormally, or the application pool is cycled, or the Apache child gets recycled -- the "disappearance" of the file will be formalized and its disk space released.

Categories