I have a problem with PDF generation using MPDF 6.
The project consists of many data forms stored in a database.
And we have the reports section where users can generate PDFs from the search results.
Some tables are generating PDFs just fine. Others show the spinner hanging for very long time without a result.
The memory limit is set very high (550M) but still, even small data tables get stuck at the spinner.
What am I doing wrong ?
Thank you in advance !
Related
I am using dompdf in laravel for generating pdf of large records, but it is taking more than 3 minutes to download.
Although I have tried many solutions like :
Remove all CSS and external links also there is no inline CSS only simple HTML table.
The query is working fine and executing in milliseconds.
I have increased the ini_memory limit too but didn't work.
Need to download around 3K-4K records.
I'm looking for some advice, to be specific in the generation of data using phpmyadmin and mysql.
I have a website (local) that register ideas from different people, those ideas are registered in a database.
I'm able to retrieve all the data into a table that i have in a reports section, the problem is that when there are at least 1500 records it gets slow (i know the amount of data printed in the table makes it slow).
I generate some charts from that data using charts.js, which is good until there are a lot of records in the webpage. (I tested putting 5000 records, 1500 records, and 2000 (which is the max amount we get with this ideas)).
So, what do I want to know?
What is an efficient way of showing that amount of data to the user without the page getting extremely slow.
What is the best practice to generate charts, from a table that gets its data from database or directly getting the data from the database, if so, which chart framework would you recommend?
I have read about pagination, to retrieve just part of the data by page, but my export to excel use the table data to export it, so it would ruin the exporting option.
I'm really sorry if this is no place for asking this programming question, but I cant think of a way of doing this in an efficient way.
Here is a picture of part of my webpage (in top you can see some charts), (below you can see some records of the table), the middle is just some report generation from the date/week and the export to excel option.
Tech used in the webpage:
PHP
JQUERY
HTML5
CSS3
mysql
Any help will be appreciated.
I have a report generation PHP program which used to work fine before. I have used 2 3rd party libraries in the program: Google image chart library ( returns image if I supply values in url ) and tcpdf ( for pdf generation ). I am using mysql not mysqli for queries. There are lots of queries and loops in the page.
Before it used to take less than 3 minutes to generate the report, I am using an ajax call to generate the report which gives a completed message once the file generation is done. This program saves the pdf file in a folder and I have a link with same name to download the file.
Recently when I checked its not generating properly.
Error was TCPDF unable to get the image. This was because of the google chart library not returning the image properly. When I access the chart url in browser it gives me the image without any issue but If I give it in an image src inside a php file, its not showing. So I decided to save the file in a folder using functions like file_get_contents,file_put_contents and link it in image src. This part is now working correctly I can see the image.
But now the problem is it is taking a lot of time to generate the report, even in local environment. I tried to generate the report without the chart priniting but even then its taking time. In between it was 25 minutes n all and now its close to 10 minutes to generate a 40 page pdf file.
I really don't know why its taking so much time. All of this was working fine before and now its not working. Only thing that changed was google image chart library but now even without(commented that part and checked) that also its taking time.
How do I speed this up ? Is there any way to check which part of program is slow.
Tried xdebug but its output file is more than 400 mb and webgrind is not able to process it.
Please help.
Your next step is to troubleshoot performance.
Is TCPDF doing a lot of work you don't need done? Presumably you've seen the tips from TCPDF's author on increasing performance, and put them into practice. http://www.tcpdf.org/performances.php
Are some of your MySQL queries inefficient? Obtain an interactive connection to your MySQL server, using phpMyAdmin or a similar command-line tool. While your pdf-creation process is running, repeatedly issue this command
SHOW FULL PROCESSLIST
It presents an INFO column showing the active MySQL query for each connection. It also shows each query's elapsed time in milliseconds. If you have queries that run for many hundreds of milliseconds, you might consider using MySQL's
EXPLAIN command to analyze those queries. Often adding an appropriate index to a MySQL table can dramatically speed things up.
Is the machine running your PDF program short on RAM? use a performance monitor like *nix top or Windows perfmon to take a look.
Is your 40-page report, simply put, a huge job to create? If so, you might consider switching to a faster report-generation program than PHP + TCPDF.
Sorted out.
The issue is with the database, one of the tables has more 120000 records in it. Deleted irrelevant records, not a permanent solution but now it generates the same thing in 2.1 minutes.
Now I can't do the same thing in my production server. I would love to get your inputs on how to optimize the database.
Thank You
I am using DOMPDF to render HTML tables. However, because large tables we'll say greater that 150 rows render extremely slowly. I know this is a known issue for DOMPDF.
Does anyone know of any good work around?
Maybe using a different html to pdf converter?
How to write html code that is formatted like a table without using a table?
Any idea?
Using big and unnecessary css-files caused slowness for me. I was using 8000 line css-file. Example 1-page pdf rendered 5.5 seconds.
I removed my css-links from html-template and made a custom css-file containing only things I need ie. bootstrap's grid, table and panel css.
Now rendering takes less than 1 second so over 80% render time improvement.
I've got a report that can generate over 30,000 records if given a large enough date range. From the HTML side of things, a resultset this large is not a problem since I implement a pagination system that limits the viewable results to 100 at a given time.
My real problem occurs once the user presses the "Get PDF" button. When this happens, I essentially re-run the portion of the report that prints the data (the results of the report itself are stored in a 'save' table so there's no need to re-run the data-gathering logic), and store the results in a variable called $html. Keep in mind that this variable now contains 30,000 records of data plus the HTML needed to format it correctly on the PDF. Once I've got this HTML string created, I pass it to TCPDF to try and generate the PDF file for the user. However, instead of generating the PDF file, it just craps out without an error message (the 'Generating PDf...') dialog disappears and the system acts like you never asked it to do anything.
Through tests, I've discovered that the problem lies in the size of the $html variable being passed in. If the report under 3K records, it works fine. If it's over that, the HTML side of the report will print but not the PDF.
Helpful Info
PHP 5.3
TCPDF for PDF generation (also tried PS2PDF)
Script Memory Limit: 500 MB
How would you guys handle this scale of data when generating a PDF of this size?
Here is how I solved this issue: I noticed that some of the strings that I was having in my HTML output had some slight encoding issues - I ran htmlentities on those particular strings as I was querying the database for them and that cleared the problem.
Don't know if this was what was causing your problem, but my experience was very similar - when I was trying to output an HTML table that had a large size, with about 80.000 rows, TCPDF would display the page header but nothing table-related. This behaviour would be the same with different sets of data and different table structures.
After many attempts I started adding my own pagination - every 15 table rows, I would break the page and add a new table to the following page. That's when I noticed that every once and a while I would get blank pages between a lot of full and correct ones. That's when I realised that there must be a problem with those particular subsets of data, and discovered the encoding issue. It may be that you had something similar and TCPDF was not making it clear what your problem was.
Are you using the writeHTML method?
I went through the performance recommendations here: http://www.tcpdf.org/performances.php
It says "Split large HTML blocks in smaller pieces;".
I found that if my blocks of HTML went over 20,000 characters the PDF would take well over 2 minutes to generate.
I simply split my html up into the blocks and called writeHTML for each block and it improved dramatically. A file that wouldn't generate in 2 minutes before now takes 16 seconds.
TCPDF seems to be a native implementation of PDF generation in PHP. You may have better performance using a compiled library like PDFlib or a command-line app like htmldoc. The latter will have the best chances of generating a large PDF.
Also, are you breaking the output PDF into multiple pages? I.e. does TCPDF know to take a single HTML document and cut it into multiple pages, or are you generating multiple HTML files for it to combine into a single PDF document? That may also help.
I would break the PDF into parts, just like pagination.
1) Have "Get PDF" button on every paginated HTML page and allow downloading of records from that HTML page only.
2) Limit the maximum number of records that can be downloaded. If the maximum limit reaches, split the PDF and let the user to download multiple PDFs.