I am working on cron jobs for my php app and planning to use cron via wget/curl.
Some of my php cron jobs can take 2-3 hours. How to let php work 2-3 hours from cron tab ? Is it good practice to run such long time jobs via cron wget/curl ? Anyone got experience on this ? I also have email queue and i needs to be run every 10 seconds, but cron tab is minute level. Any suggest on this case ?
Thanks for reading.
When you use wget/curl, you are requesting for a page from a webserver. Every webserver will have a time out period, so this might time out.
Also some of the hosting providers may stop the process running beyond certain minutes ( basically done to control the rogue threads).
So it is not advisable to schedule using wget/curl if the job takes more than few minutes.
Try scheduling it using actual scheduler. You can run php from command line
php [options] [-f] [--]
[args...]
php command should be on the path.
You can use the following at the start of your script to tell PHP to effectively never time out:
set_time_limit(0);
What may be wiser is, if crontab is going to run the script every 24 hours, set the timeout to 24 hours so that you don't get two copies running at the same time.
set_time_limit(24*60*60);
Crontab only allows minute-level execution because, that's the most often you should really be launching a script - there are startup/shutdown costs that make more rapid scheduling inefficient.
If your application requires a queue to be checked every 10 seconds a better strategy might be to have a single long-running script that does that checking and uses sleep() occasionally to stop itself from hogging system resources.
On a UNIX system such a script should really run as a daemon - have a look at the PEAR System_Daemon package to see how that can be accomplished simply.
I've just run into the problem with a long running PHP script executed via cron and wget. The script terminates after 15 minutes due to the default timeout in wget.
However I believe this method does in fact work if you set up the correct timeouts. The PHP script you run needs to be set to have an unlimited run time. Best to set this at the start of your long running script.
set_time_limit(0);
When using wget you also need to remove its timeout by passing -T0. e.g.
wget -q0 -T0 http://yourhost/job.php
Be very careful not to overload your server with long running scripts.
If using wget to call the long running cron job, consider some of it's defaults that aren't desirable in this situation. By default it'll --tries 20 times if no data is read within --read-timeout of 900 seconds. So at a maximum set tries to 1 to ensure the cron job is only called once, since you of course wouldn't want a long running script called multiple times while it's still running.
wget -t 1 https://url/
If on cPanel and you don't care for any output emails after each cron job run, also use the following flags -O and -q to hide output, and redirect errors with 2>&1.
wget -O /dev/null -q -t 1 https://url/ 2>&1
Related
I faced very strange problem on my hosting.
I have script and it can be triggered using URL like this
https://mywebsite.com/script.php
I need this script to be executed one time in two days.
So I created Cron job just like my hosting provider advised me.
wget -O /dev/null -q 'https://mywebsite.com/script.php'
It uses wget because script requires some extra scripts - so my hosting provider said that I need to create task like this.
It worked fine for about a month but for few weeks I have a problem.
For some reason that I and my hosting provider can't understand when I run script by opening URI in browser - script executed fine (I know it because of emails that are sended in 4 different steps of execution). But when Cron execute this scripts it executes infinitely - so I continue to receive emails for numerous times until I rename script or delete it.
Script execution time is about 2-3 minutes. So when I run it from URL and wait till it finishes - I get error on the screen that time of request (60 sec) is over. But I know that scripts executes fine till the last step.
What is the problem?
Wget
I had the same problem at some point with a php based cronjob. The Problem was, that wget itself can have a timeout. If this timeout is reached, wget will try again and again.
Try to use some wget options to make sure it runs as you want it to run.
Example:
wget -O /dev/null --tries=1 --timeout=600 'https://mywebsite.com/script.php'
--tries tells how many times it will try to execute if a timeout appears.
--timeout specifies the max exec. time in seconds.
Those options can be specified at cronjob level as well.
PHP Cronjobs
If possible it will may be a betther choice to let PHP run your cronjob directly. If you know the servers php directory you could create a cronjob like
/usr/bin/php /srv/www/yousite/html/script.php
In this case you have no third party programm like wget to rely on. If this helps depends on how the cronjob is built. If your cronjobs uses $_SERVER variables for example, this would not work.
There are some settings you want to check, before you use any PHP file as cronjob.
Keep in mind that the php configuration set within the php.ini could also have an impact on unwanted errors on PHP Cronjobs in general. In the php.ini there is a value called "max_execution_time" where the max seconds to process a php request is defined.
An other setting you might want to get your eye on is the "memory_limit" wich is also defined within the php.ini configuration. This configuration defines the max. memory a php request can use. As your cronjob seems to run for 2-3 minutes, that could mean that maybe a lot of data is stored in memory while you use it.
Be aware, that any request uses those limits. If you set them to high it may will cause problems with CPU load on your server, or with too many spawned php processes.
If you have a shared hosting service or something similar, you may not be able to change any of those settings.
I have a Debian linux server that runs a large number of cron jobs via a 3rd party scheduler. Many of these cron jobs coincide at exactly on the hour xx:00:00 and half hour xx:30:00 which causes large spikes on the load.
While I am not observing any abnormal behaviour of the server, the more jobs that are added, the higher the load is becoming.
The jobs are small PHP CLI commands run locally on the server that take <30s to run.
I was thinking of adding random sleep xx; values to the beginning of each PHP CLI command of up to around 120s so that some of the commands are more spread out.
The current command launched is
/usr/bin/php /var/www/html/test.php
which I was thinking of changing to, for example,
sleep 45; /usr/bin/php /var/www/html/test.php
My question is, is this just 'false economy' as once the command is loaded with the sleep xx; added to it, is it just wasting resource sitting there counting down and waiting to execute and should I take the approach of "if it ain't broke, don't try and fix it"? Is it better to just let the server queue up the commands and let it process them in due course?
Thank you.
I have the task in cron: every 3 hours to run php script.
I tried with WGET and PHP as well, but got the same problem. Sometimes script works more than 2 minutes, but mostly 30 seconds enough. And if execution time more than 60 seconds it get dropped (504 getway) and cron run this script again. And then again and again. Fatal overload in several hours.
I tried this with a huge amount of different syntax, but fail:
php -q /var/www/webmy/data/www/website.com/news.php
/usr/bin/wget -O - -q -t 1 http://website.com/news.php
How can I manage the command with only 1 try to run my script? I don't need it to run million times in every 60 seconds. Any limitations?
Maybe I can limit/drop execution time to 20 seconds to prevent any inappropriate script running. I just need to run script, I don't need system to wait any time, the script finishes task in any way.
You can set the maximum execution time directly in the php script with set_time_limit ( int $seconds );
see more here
First of all, the wget suggestion is a bad one. If you're going to use a PHP script for a cron task, you're better off running it as a command line script, thus running it directly and not through a web server.
Assuming you don't rely on server information or GET/POST variables in your script: have you tried running it once manually? Is anything strange happening when you do so?
A simple crontab entry like the one below will run your script only once every three hours:
* */3 * * * php /path/to/script.php
If you want guaranteed avoiding of two scripts running in same time, you should use some kind of locking, e.g. simplest file locking with flock
Gist
I am trying to run a cron job in PHP. But the exec and system functions didn't work. (Because the server administrator doesn't allow me to use these two functions due to the security reasons) However, I still need to use PHP to run it, because the running time is decided by the formula and it's uncertain. As a result, how can I run the cron job without using exec or system function in PHP?
My thought
The only way I have come up with is adding the cron job via the cPanel, which is set to “once per minute.” That means setup a cron job for every minute to check the target PHP page whether it has anything to do at that moment.
Issue
There's a possible issue if it always checks the page every minute per day, won't it damage the host? (Maybe it will cause a damage to CPU or maybe occupy the memory space, etc.) If so, how to improve it?
You can call a bash script containing a call to PHP-CLI and then the file:
#!/usr/bin/php
/path/to/my/php/file.php
Go to the command line of the server and type "which php". Replace /usr/bin/php above with that which is returned. Then you call this bash script in your CRON.
As you say you have a cpanel server, im guessing its a linux box. You run a cronjob from a linux box by typing crontab -e and then to run a php script. */5 * * * * /usr/bin/php /path/to/file.php to un it every 5 minutes.
I was wondering how to make a php daemon script that runs one time at the day?
Do you know any good frameworks with benefits?
or is it just small code?
Thanks
I was wondering how to make a php deamon script that runs one time at
the day?
In order to do this, get familiar with cron jobs. A cron job is a function that gets executed by the server on a time interval. Usually you'd edit your "crontab" by executing crontab -e
Then, once inside, you'd write the interval you want, followed by the command.
Typically it looks like:
30 18 * * * rm /home/someuser/tmp/* > /home/someuser/cronlogs/clean_tmp_dir.log
Since its PHP, you can either a) run your php command as a php cli command, OR b) you can make the command get executed when a particular page is run... and just execute that in cron via a curl -X GET 'http://url/' (etc.)
Also, note that you can write all of your stuff in a shell script file and actually run that file as your cron command... that reduces line-item complexity
cron
Sorry I haven't closed this one.
I actually discovered that my host didn't allowed cron jobs running. So I found a relevant homepage that offer a free service to make a request for me when I needed. In my case, I have specified a url link that should be requested to my RESTful API each day.
The link is here and works like a charm :)