I've got a php script that queries an external API that allows a limited amount of queries in a time period. To handle this I sleep my script for 60 seconds if I get a message back that I've hit this limit and then wake up and check again then repeat this until I can get more data back from the API.
The problem seems to be that sometimes during that sleep (somewhat rarely) the script will either crash, or never return. I'm not certain which yet all I can tell is that the script it doesn't pick back up and restart processing.
I'm looking for any tips or ideas on what could be happening so I have an idea of what to look for to fix this.
Thanks.
Can you post some of the code?
Since you are not sure if the script is crashing or never returning, it is possible that there is another area of your script that isn't hitting the sleep command, and just finishes executing.
check the max execution time in php.ini
$time=ini_get('max_execution_time');
$slept=0;
while(!dataready()){
sleep(1);
$slept++;
if($slept>=$time-4) die("script timed out");
}
Related
I have a php script which is runtime should take about 20-25 minutes, and approximately 15 minutes during execution time it simply re-executing for no apparent reason.
I gave it a token to make sure it kills the second execution before it runs again, but it's not the solution I prefer to use in the long run.
Some details about the script:
It ignites after a socket connection (socket does not timeout nor restarting at any point of the session.)
I gave the php script ignore_user_abort(true) in order to prevent the script from ending when user goes somewhere else.
I don't get any execution runtime errors, or any errors at all... In the client side I get a GET error message to the script after 15 minutes, which is actually the second execution try. It's still running and socket is still delivering data, though it appears the script has ended.
There are no two separated requests
I'm not sure what could be the problem of it, hopefully one of the masters here could hint with a clue?
Much thanks!
PHP is running as an Apache module.
The script start with a: ini_set('max_execution_time', 300);
What it does is basically connecting to a database, doing a big SELECTquery and looping through the results, writing them to a file and echoing a "result ok" after each write with explicit flush();
There is no sleep() call.
This is a "test" script made from a co-worker of mine for backup purposes and is intended to run up to a few hours! I thought I was aware of script execution time limit and thought his script would time out after 300 seconds...
But it didn't !
It's invoked from a web browser. The page is left open and we can see the results being echoed in real-time.
Why doesn't it time out?
Even stranger, one of the test issued a "Maximum execution time of 300 seconds exceeded" but this appeared at least after 2 hours of execution!
What's going on here? Is there something to understand between max_execution_time and flush() or a browser window being kept opened?
As you can see on the man page for the set_time_limit function, here the total execution time you are setting only affects the actual script. The time spent on database queries or any other external calls is not counted (if the OS is not Windows).
The only thing I can think of that may cause this is if PHP is running in safe mode. ini_get('max_execution_time') should tell you if it's actually being set.
Edit: Spotted your comment above.
echo ini_get('max_execution_time'); // says 300
If 300 is reported, and your not on Windows, #mishu is likely right. Your SELECT query is probably taking hours.
I have an action that occurs when a user goes to a url. The action involves a tremendous amount of insert-update ot a MYSQL database and takes a good minute to finish. I was wondering if this action continues even if a user switches to another page? I would expect so since PHP is server side. But i would like to have a clear explanation on this. Anyone?
No, the default operation of Apache and PHP is to terminate the script when the user disconnects. This termination can happen anytime after the connection is broken, so your script may run for hours, or could die instantly. Generally the kill occurs when the script attempts to do any output.
If you need to keep the script running in spite of user disconnects, then you'll have to use
ignore_user_abort(TRUE);
I have a php script run as a cron job that executes a set of simple tasks that loops for each user in the database and takes about 30 mins to complete. This process starts over every hour and needs to be as fast and efficient as possible. The problem Im having, is like with any server script, execution time varies and I need to figure out the best cron time settings.
If I run cron every minute, I need to stop the last loop of the script 20 seconds before the end of the minute to make sure that the current loop finishes in time. Over the course of the hour this adds up to a lot of wasted time.
Im wondering if its a bad idea to simple remove the php execution time limit and run the script once an hour and let it run to completion.... is this a bad idea?
Instead of setting the max_execution_time you could also use set_time_limit() to reset the counter on every loop. This will ensure your script is never running out of time unless there is something serious hanging within the current loop (and taking longer than the max_execution_time).
Basically this should make your script run as long as it needs while giving it a 30 seconds timeout between two set_time_limit() calls.
Assuming you'd like the work done ASAP, don't use cron. Cron is good for things that need to happen at specific times. It's often abused to simulate a background process that would ideally process work as soon as work appears. You should probably write a daemon that runs continuously. (Note: you could also look at a message/work-queue type system, there are nice libraries out there to do this too)
You can write a daemon from scratch using the pcntl functions (since you don't care about multiple worker processes, it's super-easy to get a process running in the background.), or cheat and just make a script that runs forever and run it via screen, or leverage some solid library code like PEAR's System:Daemon or nanoserv
Once the daemonization stuff is taken care of, all you really care about is having a loop that runs forever. You'll want to take care that your script doesn't leak memory, or consume too many resources.
Generally, you can do something like:
<?PHP
// some setup code
while(true){
$todo = figureOutIfIHaveWorkToDo();
foreach($todo as $something){
//do stuff with $something
//remember to clean up resources so you don't leak memory!
usleep(/*some integer*/);
}
usleep(/* some other integer */);
}
And it'll work pretty well.
Setting the time limit to 0 and letting it do its thing is fairly typical of PHP based cronjobs (in my experience), but this is also the point when you should ask yourself a few important questions, such as "Should I rewrite this job in a compiled language?" and "Am I using all of my tools (database, etc) to their maximum efficiency?"
That said, maybe better than completely removing the time limit would be to set it to the upper limit you actually want. If that means 48 minutes, then set_time_limit(48 * 60);
I really think you shouldn't set the time out to 0, that is just looking for trouble. At most, set it to 59*60 seconds, but setting it to 0 might cause security problems, if a script hangs, it will hang almost forever until the server host stops the execution. It is considered bad practice to do so.
I have used the php command-line interface for similar long running tasks in the past. You probably do not want to remove the execution time limit for any request.
Sounds like a great idea if there's little chance that it will take more than an hour. Note, however, that the wrong bug can be a really good way of making it take longer than expected..
To avoid all sorts of nasty problems, you should have a guard file with the process ID of the script. On startup, you should check to make sure the file doesn't exist, or if it does that the process ID in the file doesn't exist (through a kill( pid, 0 ) call). If these conditions are met, create a new file with the script's PID and delete the file when you're done.
This is the same trick that many daemons use to ensure it isn't already running. If the daemon was killed suddenly, the file will still exist but the PID of the process therein is unlikely to be running.
Depending on what your script does, it can lead to problems if you remove the time limit. If per example, you are polling an external server that is unresponsive while the job is running, and that your cron takes 2 hours instead of 30 minutes to complete, you may get a stack of PHP processes being fired up even if the previous ones haven't completed yet. This can cause system instability and crashes.
You probably have two options:
Make sure that no other instance of your script is running beforehand, otherwise exit() on start.
Consider changing your cronjob into a daemon.
Does it have to run hourly like clockwork?
If not split the job (you mentioned it was more than one simple task) do each task every hour?
Or split it per user, do A-M on hour, then N-Z the next?
For example, there is a very simple PHP script which updates some tables on database, but this process takes a long time (maybe 10 minutes). Therefore, I want this script to continue processing even if the user closed the browser, because sometimes users do not wait and they close the browser or go to another webpage.
If the task takes 10 minutes, do not use a browser to execute it directly. You have lots of other options:
Use a cronjob to execute the task
periodically.
Have the browser
request insert a new row into a
database table so that a regular
cronjob can process the new row and
execute the PHP script with the
appropriate arguments
Have the
browser request write a message to
queue system, which has a subscriber
listening for such events (which then
executes the script).
While some of these suggestions are probably overkill for your situation, the key, combining feature is to de-couple the browser request from the execution of the job, so that it can be completed asynchronously.
If you need the browser window updated with progress, you will need to use a periodically-executed AJAX request to retrieve the job status.
To answer your question directly, see ignore_user_abort
More broadly, you probably have an architecture problem here.
If many users can initiate this stuff, you'll want the web application to add jobs to some kind of queue, and have a set number of background processes that chew through all the work.
The PHP script will keep running after the client terminates the connection (not doing so would be a security risk), but only up to max_execution_time (set in php.ini or through a PHP script, generally 30 seconds by default)..
For example:
<?php
$fh = fopen("bluh.txt", 'w');
for($i=0; $i<20; $i++) {
echo $i."<br/>";
fwrite($fh,$i."\n");
sleep(1);
}
fclose($fh);
?>
Start running that in your browser and close the browser before it completes. You'll find that after 20 seconds the file contains all of the values of $i.
Change the upper bound of the for loop to 100 instead of 20, and you'll find it only runs from 0 to 29. Because of PHP's max_execution_time the script times out and dies.
if the script is completely server based (no feedback to the user) this will be done even if the client is closed.
The general architecture of PHP is that a clients send a request to a script that gives a reply to the user. if nothing is given back to the user the script will still execute even if the user is not on the other side anymore. More simpler: their is no constant connection between server and client on a regular script.
You can make the PHP script run every 20 minutes using a crontab file which contains the time and what command to run in this case it would be the php script.
Yes. The server doesn't know if the user closed the browser. At least it doesn't notice that immediately.
No: the server probably (depending of how it is configured) won't allow for a php script to run for 10 minutes. On a cheap shared hosting I wouldn't rely on a script running for longer than a reasonable response time.
A server-side script will go on what it is doing regardless of what the client is doing.
EDIT: By the way, are you sure that you want to have pages that take 10 minutes to open? I suggest you to employ a task queue (whose items are executed by cron on a timely basis) and redirect user to a "ok, I am on it" page.