If some php page is running some long process such as sleep or while loop that makes it take a while till it loads, does it affect on other processes from the same page ?,
I noticed when i try to open the same page with different short process, it also takes so long to load and to be clear it doesn't load before the first one (long process) does,
is it true or something's wrong with my code and how to prevent it ?
i think it has something to do with cache, i don't wanna mess up though before getting a tip or an answer
PHP run in a single process, each time you access the page, it start the process, process, and finish.
Each process won't affect the others.
I noticed when i try to open the same page with different short
process, [...] it doesn't load before the first one (long process) does
The most common reasons:
Your scripts use PHP sessions "as is", which use file locking. The file locking mechanism ensures only one script at a time can edit the session data of each user, but this does mean that provided two requests from the same user happen simultaneously, a second script will not start before the first has finished if they both rely on sessions (two different users have different session files though, so they can't collide)
The browser automatically detects the page is taking long and delays subsequent requests intentionally in the background — I believe this is something Google Chrome does by default.
Both cases are relatively safe however, because the delay is only present in case the same user is trying to load several pages simultaneously which is not usual — different users will not see delays regardless how long the actual page takes to load.
More on the topic in this excellent SO answer.
Related
Seeing some strange behavior in my application. Any time I have a long running script waiting for a response in the browser, any other page in the application will spin/hang after a click (even for very simple//static pages) until the original page finishes loading. This is a PHP5.3 based application using native PHP sessions on Apache 2.2.x.
This happens in multiple browsers in all our dev, qa, and production instances. I'm not sure where to start looking. Any advice?
Get all the data you need from the $_SESSION, and call session_write_close() before you are going to do something that takes a long time. Due to a PHP script still being active with that session, it locks the session (after all, it may write some data to that session that the next request needs).
If you need to write something to the session after you've done your long job, you can just call session_start() once again (provided you've not generated output yet), write to it, and after that the script may end, or maybe you repeat this cycle a few times.
Also, don't call session_start() if you don't need it, I see you mention 'static' pages. If a page does not need session data, avoid the overhead & locks it creates.
So here's the lowdown:
The client i'm developing for is on HostGator, which has limited their max_execution_time to 30 seconds and it cannot be overridden (I've tried and confirmed it cannot be via their support and wiki)
What I'm have the code doing is take an uploaded file and...
loop though the xml
get all feed download links within the file
download each xml file
individually loop though each xml array of each file and insert the information of each item into the database based on where they are from (i.e. the filename)
Now is there any way I can queue this somehow or split the workload into multiple files possibly? I know the code works flawlessly and checks to see if each item exists before inserting it but I'm stuck getting around the execution_limit.
Any suggestions are appreciated, let me know if you have any questions!
The timelimit is in effect only when executing PHP scripts through a webserver, if you execute the script from CLI or as a background process, it should work fine.
Note that executing an external script is somewhat dangerous if you are not careful enough, but it's a valid option.
Check the following resources:
Process Control Extensions
And specifically:
pcntl-exec
pcntl-fork
Did you know you can trick the max_execution_time by registering a shutdown handler? Within that code you can run for another 30 seconds ;-)
Okay, now for something more useful.
You can add a small queue table in your database to keep track of where you are in case the script dies mid-way.
After getting all the download links, you add those to the table
Then you download one file and process it; when you're done, you check them off (delete from) from the queue
Upon each run you check if there's still work left in the queue
For this to work you need to request that URL a few times; perhaps use JavaScript to keep reloading until the work is done?
I am in such a situation. My approach is similar to Jack's
accept that execution time limit will simply be there
design the application to cope with sudden exit (look into register_shutdown_function)
identify all time-demanding parts of the process
continuously save progress of the process
modify your components so that they are able to start from arbitrary point, e.g. a position in a XML file or continue downloading your to-be-fetched list of XML links
For the task I made two modules, Import for the actual processing; TaskManagement for dealing with these tasks.
For invoking TaskManager I use CRON, now this depends on what webhosting offers you, if it's enough. There's also a WebCron.
Jack's JavaScript method's advantage is that it only adds requests if needed. If there are no tasks to be executed, the script runtime will be very short and perhaps overstated*, but still. The downsides are it requires user to wait the whole time, not to close the tab/browser, JS support etc.
*) Likely much less demanding than 1 click of 1 user in such moment
Then of course look into performance improvements, caching, skipping what's not needed/hasn't changed etc.
I'm building a feature of a site that will generate a PDF (using TCPDF) into a booklet of 500+ pages. The layout is very simple but just due to the number of records I think it qualifies as a "long running php process". This will only need to be done a handful of times per year and if I could just have it run in the background and email the admin when done, that would be perfect. Considered Cron but it is a user-generated type of feature.
What can I do to keep my PDF rendering for as long as it takes? I am "good" with PHP but not so much with *nix. Even a tutorial link would be helpful.
Honestly you should avoid doing this entirely from a scalability perspective. I'd use a database table to "schedule" the job with the parameters, have a script that is continuously checking this table. Then use JavaScript to poll your application for the file to be "ready", when the file is ready then let the JavaScript pull down the file to the client.
It will be incredibly hard to maintain/troubleshoot this process while you're wondering why is my web server so slow all of a sudden. Apache doesn't make it easy to determine what process is eating up what CPU.
Also by using a database you can do things like limit the number of concurrent threads, or even provide faster rendering time by letting multiple processes render each PDF page and then re-assemble them together with yet another process... etc.
Good luck!
What you need is to change the allowed maximum execution time for PHP scripts. You can do that by several means from the script itself (you should prefer this if it would work) or by changing php.ini.
BEWARE - Changing execution time might seriously lower the performance of your server. A script is allowed to run only a certain time (30sec by default) before it is terminated by the parser. This helps prevent poorly written scripts from tying up the server. You should exactly know what you are doing before you do this.
You can find some more info about:
setting max-execution-time in php.ini here http://www.php.net/manual/en/info.configuration.php#ini.max-execution-time
limiting the maximum execution time by set_time_limit() here http://php.net/manual/en/function.set-time-limit.php
PS: This should work if you use PHP to generate the PDF. It will not work if you use some stuff outside of the script (called by exec(), system() and similar).
This question is already answered, but as a result of other questions / answers here, here is what I did and it worked great: (I did the same thing using pdftk, but on a smaller scale!)
I put the following code in an iframe:
set_time_limit(0); // ignore php timeout
//ignore_user_abort(true); // optional- keep on going even if user pulls the plug*
while(ob_get_level())ob_end_clean();// remove output buffers
ob_implicit_flush(true);
This avoided the page load timeout. You might want to put a countdown or progress bar on the parent page. I originally had the iframe issuing progress updates back to the parent, but browser updates broke that.
I am making a Warehouse management system.
The orders come in a CSV in the morning that my script then executes.
It places a php-made barcode on the top of each order. the sample CSV i am using has around 100 unique orders on, so when i load the page that will then print orders off the server is getting 100+ requests and (im guessing) some of the images time out.
When i view source and open the link to the ones that don't work it loads the image, leading me to think i need to somehow disable the timout method on the browser.
My only other idea is to load the barcodes through javascript.
Any suggestions?
I think what enygma may be getting at is the limited processing time php scripts have. Sometimes they get cut off after 30 seconds. Generating all of those images at one time might run over, causing your script to be killed on the server and stop sending data. Your idea of loading them in javascript is probably your best bet, as long as you only do a few at a time or do them serially.
If you start a session in php, the session is locked and cannot be accessed by another php script until released.
Based on you generating images with php - that's quite likely the cause of what you see.
There are other questions which go into a bit more detail of how php and sessions work; but most likely that's the direct cause for some of your images not being received - the requests are in a single, serial queue being processed in turn because each script reads the session and doesn't release it until it's finished. The requests at the end of the queue hit a time limit one way or another and return nothing.
Therefore, ensure that you call:
session_write_close();
as soon as you can in all scripts that need access to the session to prevent them from blocking all other php requests, or better still don't use the session at all (e.g. if you're using the session for authorization just include a hash in the url and compare to that for image requests).
I have a PHP script something like:
$i=0;
for(;$i<500;++i) {
//Do some operation with files numbered 0 to 500;
}
The thing is, the script works and displays the end results, but the operation takes a while and watching a blank screen can be frustrating. I was thinking if there is some way I can continuously update the page at the client's end, detailing which file is currently being worked upon. That is, can I display and continuously update what is the current value of $i?
The Solution
Thanks everyone! The output buffering is working as suggested. However, David has offered valuable insight and am considering that approach as well.
You can buffer and control the output from the PHP script.
However, you may want to consider the scalability of this design. In general, heavy processes shouldn't be done online. Your particular case may be an edge in that the wait is acceptable, but consider something like this as an alternative for an improved user experience:
The user kicks off a process. This can be as simple as setting a flag on a record in the database or inserting some "to be processed" records into the data.
The user is immediately directed to a page indicating that the process has been queued.
An offline process (either kicked off by the PHP script on the server or scheduled to run regularly) checks the data and does the heavy processing.
In the meantime, the user can refresh the page (manually, by navigating elsewhere and coming back to check, or even use an AJAX polling mechanism to update the page) to check the status of the processing. In this case, it sounds like you'd have several hundred records in a database table queued for processing. As each one finishes, it can be flagged as done. The page can just check how many are left, which one is current, etc. from the data.
When the processing is completed, the page shows the result.
In general this is a better user experience because it doesn't force the user to wait. The user can navigate around the site and check back on progress as desired. Additionally, this approach scales better. If your heavy processing is done directly on the page, what happens when you have many users or the data processing load increases? Will the page start to time out? Will users have to wait longer? By making the process happen outside of the scope of the website you can offload it to better hardware if needed, ensure that records are processed in serial/parallel as business rules demand (avoid race conditions), save processing for off-peak hours, etc.
Check out PHP's Output Buffering.
Try to use:
flush();
http://php.net/manual/ru/function.flush.php
Try the flush() function. Calling this function forces PHP to send whatever output it has so far to the client, instead of waiting for the script to end.
However, some web servers will only send the output once the entire page is done being built, so calling flush() would have no effect in this case.
Also, browsers themselves buffer input, so you may run into problems there. For example, certain versions of IE won't start displaying the page until 256 bytes has been received.