i want to create a big zip-file with thousands (10000+) pdf-files and upload it to a external sftp server.
Currently the process always results in a timeout (already modified the php.ini), but the zip-file gets created never-the-less, but we can never be sure if everything went well and when it is finished.
I looked around but found no satisfactory response. Is there a possibility to prevent timeouts, keep the process spinning and know when it is finished? Maybe via AJAX? =/
Thanks in advance
For upload, you can try the ZipStream class from the PHPZip project (https://github.com/Grandt/PHPZip)
$zip = new \ZipStream("invoices.zip");
foreach($invoices as $invoice)
{
$pdf = new InvoicePdf($invoice);
$zip->addFile($pdf->render(), $pdf->getFilename());
}
$zip->finalize();
exit;
For upload, if you have tried this (https://www.sitepoint.com/upload-large-files-in-php/), then you can google for "jquery file upload"
Related
I have been stuck on this for hours. Maybe my mind is tired. I hope someone can assist me. I am developing a Laravel application which connects to an external application that creates and edits invoices. One of my routes is supposed to allow a user to download a PDF invoice from this external application. My controller resembles the code below:
public function download(Invoice $invoice)
{
// Creates an instance of the remote invoice
$remoteInvoiceFile = RemoteInvoiceSoftware::find($invoice->id);
return response()->streamDownload(function () use ($remoteInvoiceFile ) {
// This calls the remote server and the server responds with the PDF file contents
$file = $remoteInvoiceFile->download();
echo $file;
}, 'file.pdf');
}
I read about the streamDownoad function from the Laravel documentation and implemented it the same way it was displayed. But I am getting an error where I am able to download a PDF file, however, not only is the file less than 5KB (the original invoice file is about 60KB), I also get an error when I try to open it. Something about the file being corrupt or not parsed well.
When I echo $remoteInvoiceFile->download() without using the streamDownload I get something like this:
Please help me figure what's going on and how I can fix this. Thank you!!
I am trying to generate an archive on-the-fly in PHP and send it to the user immediately (without saving it). I figured that there would be no need to create a file on disk as the data I'm sending isn't persistent anyway, however, upon searching the web, I couldn't find out how. I also don't care about the file format.
So, the question is:
Is it possible to create and manipulate a file archive in memory within a php script without creating a tempfile along the way?
I had the same problem but finally found a somewhat obscure solution and decided to share it here.
I came accross the great zip.lib.php/unzip.lib.php scripts which come with phpmyadmin and are located in the "libraries" directory.
Using zip.lib.php worked as a charm for me:
require_once(LIBS_DIR . 'zip.lib.php');
...
//create the zip
$zip = new zipfile();
//add files to the zip, passing file contents, not actual files
$zip->addFile($file_content, $file_name);
...
//prepare the proper content type
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=my_archive.zip");
header("Content-Description: Files of an applicant");
//get the zip content and send it back to the browser
echo $zip->file();
This script allows downloading of a zip, without the need of having the files as real files or saving the zip itself as a file.
It is a shame that this functionality is not part of a more generic PHP library.
Here is a link to the zip.lib.php file from the phpmyadmin source:
https://github.com/phpmyadmin/phpmyadmin/blob/RELEASE_4_5_5_1/libraries/zip.lib.php
UPDATE:
Make sure you remove the following check from the beginning of zip.lib.php as otherwise the script just terminates:
if (! defined('PHPMYADMIN')) {
exit;
}
UPDATE:
This code is available on the CodeIgniter project as well:
https://github.com/patricksavalle/CodeIgniter/blob/439ac3a87a448ae6c2cbae0890c9f672efcae32d/system/helpers/zip_helper.php
what are you using to generate the archive? You might be able to use the stream php://temp or php://memory to read and write to/from the archive.
See http://php.net/manual/en/wrappers.php.php
Regarding your comment that php://temp works for you except when you close it, try keeping it open, flushing the output, then rewind it back to 0 and read it.
Look here for more examples: http://us.php.net/manual/en/function.tmpfile.php
Also research output buffering and capturing: http://us.php.net/manual/en/function.ob-start.php
You need to use ZipArchive::addFromString - if you use addFile() the file is not actually added until you go to close it. (Horrible bug IMHO, what if you are trying to move files into a zip and you delete them before you close the zip...)
The addFromString() method adds it to the archive immediately.
Is there really a performance issue here, or does it just offend your sense of rightness? A lot of processes write temporary files and delete them, and often they never hit the disk due to caching.
A tempfile is automatically deleted when closed. That's it's nature.
There are only two ways I can think of to create a zip file in memory and serve it and both are probably more trouble than they are worth.
use a ram disk.
modify the ziparchive class to add a method that does everything the close() method does, except actually close the file. (Or add a leave-open parameter to close()).
This might not even be possible depending on the underlying C libraries.
I have a function that uploads a file into a web storage and prior to saving the file on the storage system if the file is a pdf file i would like to determine how many pages a pdf file has.
Currently i have the following:
$pdftext = file_get_contents($path);
$num = preg_match_all("/\/Page\W/", $pdftext, $dummy);
return $num;
Where $path is the temporary path that i use with fopen to open the document
This function works at times but is not reliable. I know theres also this function
exec('/usr/bin/pdfinfo '.$pdf_file.' | awk \'/Pages/ {print $2}\'', $output);
But this requires the file to donwloaded on the server. Any ideas or suggestions to accomplish this?
PHP is a server-side language, meaning all processing happens on your server. There's no way for PHP to determine details of a file on the client side, it has no knowledge of it neither the required access to it.
So the answer to your question as it is now is: It's not possible. But you probably have a goal in mind why you want to check this, sharing this goal might help to get more constructive answers/suggestions.
As Oldskool already explained this is not possible with PHP on the client side. You would have to upload the PDF file to the server and then determine the amount of pages. There are libraries and command line tools that could accomplish this.
In case you don't want to upload the PDF file to the server (which seems to be the case here) you could use the pdf.js library. Now the client is able to determine the amount of pages in a PDF document on its own.
PDFJS.getDocument(data).then(function (doc) {
var numPages = doc.numPages;
}
There are other libraries as well but I'm not certain about their browser support (http://www.electronmedia.in/wp/pdf-page-count-javascript/)
Now you just submit the amount of pages from javascript to your php file that needs this information. In order to achive this you simply use ajax. In case you don't know ajax, just google it there are enough examples out there.
As a side note; Always remember to not trust the client. The client is able to modify the page count and send a completely different one.
For those of you running linux servers this actually is possible. You need the pdfinfo extension installed and using the function
$pages = exec('/usr/bin/pdfinfo '.$pdf_file.' | awk \'/Pages/ {print $2}\'', $output);
outputs the correct page number where $pdf_file is the temporary path on the server upon upload.
The reason it wasnt working for me was because i didnt have the PDFinfo installed.
I'm creating a pdf file on fly using fpdf of PHP from the database but when there are a lot of data in my db , either the file cant be created completely and just half of it comes out, or nothing will happen or I get run-time error what should I do to be able to create large PDF files without error?
large pdf :
The code is like :
require('lib\fpdf.php');
require ('lib\pdf.php');
include ('lib\bukaDb.php');
$pdf = new PDF('P','pt','A4');
$pdf->SetFont('Arial','',8);
$pdf->AliasNbPages();
$pdf->Open();
$pdf->setY($pdf->tMargin);
for ($i=0; $i<count($var); $i++){
//Do something , show data
//$pdf->Ln(20);
}
// Finally, produce the PDF!!
$pdf->Output();
And my table has hundreds of rows, I cant use set time limit function, any other recommendation?
Your script is probably timing out. Try disabling the built-in time limit:
set_time_limit(0);
If you host your site or app on a shared-hosting server though, make sure you are allowed to do this. Some web hosters (including mine) frown upon using lots of server resources for tasks like this.
We create PDF files (using fpdf) with hundreds of pages without any problem, so I guess it's your code that is collecting / displaying data that causes the request to time out.
When your HTTP request takes long to complete (more than 10 seconds) then it's probably your script simply taking too much time to complete.
I really think the problem is what happens inside that for loop. Try to create a simple dummy PDF with about the same amount of pages. If that works, you know that you have to search the problem inside the for loop.
Move you script to crontab and set saved pdf as file. Once your script finished, set it to send email telling you that the process finished.
As rough pseudo-code:
require('lib\fpdf.php');
require ('lib\pdf.php');
include ('lib\bukaDb.php');
$pdf = new PDF('P','pt','A4');
$pdf->SetFont('Arial','',8);
$pdf->AliasNbPages();
$pdf->Open();
$pdf->setY($pdf->tMargin);
for ($i=0; $i<count($var); $i++){
//Do something , show data
//$pdf->Ln(20);
}
// Finally, produce the PDF!!
$savefile = "/path/to/output/dir/output.pdf";
$pdf->Output($savefile,'I');
mail('admin#example.com', 'PDF Generator', "PDF has been generated at $savefile");
Then, all you have to do is, save it to somewhere on your server and set it's path as crontab:
/path/to/php/cli/php /path/to/your/saved/php/pdf_generator.php
Please note that you have to:
make sure your $savefile directory is writable by script.
use php cli, not by accessing script via browser as it will also affected by time limit.
I am trying to generate an archive on-the-fly in PHP and send it to the user immediately (without saving it). I figured that there would be no need to create a file on disk as the data I'm sending isn't persistent anyway, however, upon searching the web, I couldn't find out how. I also don't care about the file format.
So, the question is:
Is it possible to create and manipulate a file archive in memory within a php script without creating a tempfile along the way?
I had the same problem but finally found a somewhat obscure solution and decided to share it here.
I came accross the great zip.lib.php/unzip.lib.php scripts which come with phpmyadmin and are located in the "libraries" directory.
Using zip.lib.php worked as a charm for me:
require_once(LIBS_DIR . 'zip.lib.php');
...
//create the zip
$zip = new zipfile();
//add files to the zip, passing file contents, not actual files
$zip->addFile($file_content, $file_name);
...
//prepare the proper content type
header("Content-type: application/octet-stream");
header("Content-Disposition: attachment; filename=my_archive.zip");
header("Content-Description: Files of an applicant");
//get the zip content and send it back to the browser
echo $zip->file();
This script allows downloading of a zip, without the need of having the files as real files or saving the zip itself as a file.
It is a shame that this functionality is not part of a more generic PHP library.
Here is a link to the zip.lib.php file from the phpmyadmin source:
https://github.com/phpmyadmin/phpmyadmin/blob/RELEASE_4_5_5_1/libraries/zip.lib.php
UPDATE:
Make sure you remove the following check from the beginning of zip.lib.php as otherwise the script just terminates:
if (! defined('PHPMYADMIN')) {
exit;
}
UPDATE:
This code is available on the CodeIgniter project as well:
https://github.com/patricksavalle/CodeIgniter/blob/439ac3a87a448ae6c2cbae0890c9f672efcae32d/system/helpers/zip_helper.php
what are you using to generate the archive? You might be able to use the stream php://temp or php://memory to read and write to/from the archive.
See http://php.net/manual/en/wrappers.php.php
Regarding your comment that php://temp works for you except when you close it, try keeping it open, flushing the output, then rewind it back to 0 and read it.
Look here for more examples: http://us.php.net/manual/en/function.tmpfile.php
Also research output buffering and capturing: http://us.php.net/manual/en/function.ob-start.php
You need to use ZipArchive::addFromString - if you use addFile() the file is not actually added until you go to close it. (Horrible bug IMHO, what if you are trying to move files into a zip and you delete them before you close the zip...)
The addFromString() method adds it to the archive immediately.
Is there really a performance issue here, or does it just offend your sense of rightness? A lot of processes write temporary files and delete them, and often they never hit the disk due to caching.
A tempfile is automatically deleted when closed. That's it's nature.
There are only two ways I can think of to create a zip file in memory and serve it and both are probably more trouble than they are worth.
use a ram disk.
modify the ziparchive class to add a method that does everything the close() method does, except actually close the file. (Or add a leave-open parameter to close()).
This might not even be possible depending on the underlying C libraries.