I have this mail script I have to run a few times.
To start the script I will use cron, but the script has to run 2 or 3 more times (with an hour apart).
What's the best way to do this?
To use the sleep command for an hour, or at the end of the script, place some code, so that the script will create a new cron job to run it self after an hour?
Thanks
Unless there's some cost savings in keeping the script running in memory, you're better off using cron to invoke it every hour, as needed.
0 0-2 * * * /usr/local/bin/mail-script.php
You can choose multiple hours using the - syntax, or the comma syntax:
0 0,1,2,3 * * * /usr/local/bin/mail-script.php
If it needs to maintain some form of state, use a temporary file to keep saved state.
Do:
> man 5 crontab
To see if your *nix handles the above cases.
Finally, unless you know the script has to run only 2-3 times, you're better off putting the logic about whether to "run or not to run" in the PHP script itself, and then just run it every hour.
One advantage of using sleep() is that it could be more portable. For example, on many systems I work with, users are not allowed to have their own cron jobs - so writing your program to take care of its own timer-ness might be an advantage.
An alternative to sleep() might be using SIGALRM (so your script catches an interrupt and executes code at a certain interval - when that interrupt is thrown.)
I mean, I'd recommend using cron - but here are some alternatives!
I'm not sure either approach (sleeping for an hour, or creating cron jobs from php) is ideal, how about a cron job that runs every hour anyway, then your php script checks whether it should run?
Why not just set the cron criteria so it fires those specific times? Cron is pretty flexible in that aspect.
Update your question with when you want it to fire off and I can give an example.
Related
I'm new to this cronjobs and I want few emails (2k to 3k mails to be more precise) to be sent at specific time and date which are in the database table.Currently to achieve this, I'm calling my mail function file(sendmail.php) for every minute using cron job and comparing the current time and the time which comes from the db table, if true the mail will be sent.By doing this I'm afraid there will be some effect on the performance.
Can we schedule cronjob right after the insert query in the php script. So that I can pass those time and date variables to it?
Does calling the file for every minute in the cron job is a good practice? Will the performance get effected because my application will be used by 25 users at a time?
Although by calling the file for every minute achieves my task, but still want to know if there are any better ways.
Thank you in advance.
for every minute using cron
If you're firing off cron jobs every minute then you're doing some thing wrong. There are problems with jitter, and concurrency.
comparing the current time and the time which comes from the db table
Does that mean you are doing the tie check outside of the DBMS? That would be very silly.
Can we schedule cronjob right after the insert query in the php script
Yes, although you'd need to use sudo to create privilege separation. However you having (potentially) thousands of cron jobs is a very bad idea.
While there is a lot missing from your problem statement, based on what you have said, I'd suggest having a cron job running once every (say) 15 minutes, polling the database for the emails to be sent in that time window - with the time comparison and concurrency locking done in the database.
I'm currently working on a browser game with a PHP backend that needs to perform certain checks at specific, changing points in the future. Cron jobs don't really cut it for me as I need precision at the level of seconds. Here's some background information:
The game is multiplayer and turn-based
On creation of a game room the game creator can specify the maximum amount of time taken per action (30 seconds - 24 hours)
Once a player performs an action, they should only have the specified amount of time to perform the next, or the turn goes to the player next in line.
For obvious reasons I can't just keep track of time through Javascript, as this would be far too easy to manipulate. I also can't schedule a cron job every minute as it may be up to 30 seconds late.
What would be the most efficient way to tackle this problem? I can't imagine querying a database every second would be very server-friendly, but it is the direction I am currently leaning towards[1].
Any help or feedback would be much appreciated!
[1]:
A user makes a move
A PHP function is called that sets 'switchTurnTime' in the MySQL table's game row to 'TIMESTAMP'
A PHP script that is always running in the background queries the table for any games where the 'switchTurnTime' has passed, switches the turn and resets the time.
You can always use a queue or daemon. This only works if you have shell access to the server.
https://stackoverflow.com/a/858924/890975
Every time you need an action to occur at a specific time, add it to a queue with a delay. I've used beanstalkd with varying levels of success.
You have lots of options this way. Here's two examples with 6 second intervals:
Use a cron job every minute to add 10 jobs, each with a delay of 6 seconds
Write a simple PHP script that runs in the background (daemon) to adds an a new job to the queue every 6 seconds
I'm going with the following approach for now, since it seems to be the easiest to implement and test, as well as deploy on different kinds of servers/ hosting, while still acting reliably.
Set up a cron job to run a PHP script every minute.
Within that script, first do a query to find candidates that will have their endtime within this minute.
Start a while-loop, that runs until 59 seconds have passed.
Inside this loop, check the remianing time for each candidate.
If teh time limit has passed, do another query on that specific candidate to ensure the endtime hasn't changed.
If it has, re-add it to the candidates queue as nescessary. If not, act accordingly (in my case: switch the turn to the next player).
Hope this will help somebody in the future, cheers!
Here's what I'm trying to accomplish in high-level pseudocode:
query db for a list of names (~100)
for each name (using php) {
query a 3rd party site for xml based on the name
parse/trim the data received
update my db with this data
Wait 15 seconds (the 3rd party site has restrictions and I can only make 4 queries / minute)
}
So this was running fine. The whole script took ~25 minutes (99% of the time was spent waiting 15 seconds after every iteration). My web host then made a change so that scripts will timeout after 70 seconds (understandable). This completely breaks my script.
I assume I need to use cronjobs or command line to accomplish this. I only understand the basic us of cronjobs. Any high level advice on how to split up this work in a cronjob? I am not sure how a cronjob could parse through a dynamic list.
cron itself has no idea of your list and what is done already, but you can use two kinds of cron-jobs.
The first cron-job - that runs for example once a day - could add your 100 items to a job queue.
The second cron-job - that runs for example once every minute in a certain period - can check if there are items in the queue, execute one (or a few) and remove it from the queue.
Note that both cron-jobs are just triggers to start a php script in this case and you have two different scripts, one to set the queue and one to process part of a queue so almost everything is still done in php.
In short, there is not much that is different. Instead of executing the script via modphp or fcgi, you are going to execute it via command line php /path/to/script.php.
Because this is a different environment than http, some things obviously don't work. Sessions, cookies, get and post variables. Output gets send to stdout instead of the browser.
You can pass arguments to your script by using $argv.
I'm aware of cron jobs to execute commands at a certain time, but what if that time is not constant? For instance, suppose a user asks for a reminder email exactly 1hr after signing up for something, is there an easy way to go about doing this?
Timing is critical. I am actually trying to create AI that will essentially act on its own but only at variable points during the day. Any help would be appreciated!
You can use at to schedule jobs for specific times. cron is for repeating jobs, at is for one-shot/oddball interval ones. Both have a resolution of 1 minute, though, so you can't specify a start period with seconds granularity.
The command's available on both Unix/Linux and Windows.
Here a workable flow:
user Requests email in 1 hour
You insert into the a table action (action_id, time)
On the PHP server create a cron job to check the action in the action table every minute, then do the action that need to be done at that time
That is a simple example from the request. It might get a bit more complex then that.
EDIT : this suggestion might be good only if you need to be very precise with the time management!
if you dont wanna use the cron triggers and you are not comfortable with them here are two php scheduling libraries..
1) http://www.php.brickhost.com/
2) http://www.phpjobscheduler.co.uk/
Try them if you like:
I need some implementation advice. I have a MYSQL DB that will be written to remotely for tasks to process locally and I need my application which is written in PHP to execute these tasks imediatly as they come in.
But of course my PHP app needs to be told when to run. I thought about using cron jobs but my app is on a windows machine. Secondly, I need to be constantly checking every few seconds and cron can only do every minute.
I thought of writing a PHP daemon but I am getting consued on hows its going to work and if its even a good idea!
I would appreciate any advice on the best way to do this.
pyCron is a good CRON alternative for Windows:
Since this task is quite simple I would just set up pyCron to run the following script every minute:
set_time_limit(60); // one minute, same as CRON ;)
ignore_user_abort(false); // you might wanna set this to true
while (true)
{
$jobs = getPendingJobs();
if ((is_array($jobs) === true) && (count($jobs) > 0))
{
foreach ($jobs as $job)
{
if (executeJob($job) === true)
{
markCompleted($job);
}
}
}
sleep(1); // avoid eating unnecessary CPU cycles
}
This way, if the computer goes down, you'll have a worst case delay of 60 seconds.
You might also want to look into semaphores or some kind of locking strategy like using an APC variable or checking for the existence of a locking file to avoid race conditions, using APC for example:
set_time_limit(60); // one minute, same as CRON ;)
ignore_user_abort(false); // you might wanna set this to true
if (apc_exists('lock') === false) // not locked
{
apc_add('lock', true, 60); // lock with a ttl of 60 secs, same as set_time_limit
while (true)
{
$jobs = getPendingJobs();
if ((is_array($jobs) === true) && (count($jobs) > 0))
{
foreach ($jobs as $job)
{
if (executeJob($job) === true)
{
markCompleted($job);
}
}
}
sleep(1); // avoid eating unnecessary CPU cycles
}
}
If you're sticking with the PHP daemon do yourself a favor and drop that idea, use Gearman instead.
EDIT: I asked a related question once that might interest you: Anatomy of a Distributed System in PHP.
I'll suggest something out of the ordinary: you said you need to run the task at the point the data is written to MySQL. That implies MySQL "knows" something should be executed.
It sounds like perfect scenario for MySQL's UDF sys_exec.
Basically, it would be nice if MySQL could invoke an external program once something happened to it.
If you use the mentioned UDF, you can execute a php script from within - let's say, INSERT or UPDATE trigger.
On the other hand, you can make it more resource-friendly and create MySQL Event (assuming you're using appropriate version) that would use sys_exec to invoke a PHP script that does certain updates at predefined intervals - that reduces the need for Cron or any similar program that can execute something at predefined intervals.
i would definately not advise to use cronjobs for this.
cronjobs are a good thing and very useful and easy for many purposes, but as you describe your needs, i think they can produce more complications than they do good. here are some things to consider:
what happens if jobs overlap? one takes longer to execute than one minute? are there any shared resources/deadlocks/tempfiles? - the most common method is to use a lock file, and stop the execution if its occupied right at the start of the program. but the program also has to look for further jobs right before it completes. - this however can also get complicated on windows machines because they AFAIK don't support write locks out of the box
cronjobs are a pain in the ass to maintain. if you want to monitor them you have to implement additional logic like a check when the program last ran. this however can get difficult if your program should run only on demand. the best way would be some sort of "job completed" field in the database or delete rows that have been processed.
on most unix based systems cronjobs are pretty stable now, but there are a lot of situatinos where you can break your cronjob system. most of them are based on human error. for example a sysadmin not exiting the crontab editor properly in edit mode can cause all cronjobs to be deleted. a lot of companies also have no proper monitoring system for the reasons stated above and notice as soon as their services experience problems. at this point often nobody has written down/put under version control which cronjobs should run and wild guessing and reconstruction work begins.
cronjob maintaince can be further complicated when external tools are used and the environment is not a native unix system. sysadmins have to gain knowledge of more programs and they can have potential errors.
i honestly think just a small script that you start from the console and let open is just fine.
<?php
while(true) {
$job = fetch_from_db();
if(!$job) {
sleep(10)
} else {
$job->process();
}
}
you can also touch a file (modify modification timestamp) in every loop, and you can write a nagios script that checks for that timestamp getting out of date so you know that your job is still running...
if you want it to start up with the system i recommend a deamon.
ps: in the company i work there is a lot of background activity for our website (crawling, update processes, calculations etc...) and the cronjobs were a real mess when i started there. their were spread over different servers responsible for different tasks. databases were accessed wildly accross the internet. a ton of nfs filesytems, samba shares etc were in place to share resouces. the place was full of single points of failures, bottlenecks and something constantly broke. there were so many technologies involved that it was very difficult to maintain and when something didnt work it needed hours of tracking down the problem and another hour of what that part even was supposed to do.
now we have one unified update program that is responsible for literally everyhing, it runs on several servers and they have a config file that defines the jobs to run. eveyrthing gets dispatched from one parent process doing an infinite loop. its easy to monitor, customice, synchronice and everything runs smoothly. it is redundant, it is syncrhonized and the granularity is fine. so it runs parallel and we can scale up to as many servers as we like.
i really suggest to sit down for enough time and think about everything as a whole and get a picture of the complete system. then invest the time and effort to implement a solution that will serve fine in future and doesnt spread tons of different programs throughout your system.
pps:
i read a lot about the minimum interaval of 1/5 minutes for cronjobs/tasks. you can easily work around that with an arbitrary script that takes over that interval:
// run every 5 minutes = 300 secs
// desired interval: 30 secs
$runs = 300/30; // be aware that the parent interval needs to be a multiple of the desired interval
for($i=0;$i<$runs;$i++) {
$start = time();
system('myscript.php');
sleep(300/10-time()+$start); // compensate the time that the script needed to run. be aware that you have to implement some logic to deal with cases where the script takes longer to run than your interavl - technique and problem described above
}
This looks like a job for a job server ;) Have a look at Gearman. The additional benefit of this approach is, that this is triggered by the remote side, when and only then there is something to do, instead of polling. Especially in intervals smaller than (lets say) 5 min polling is not very effective any more, depending on the tasks the job performs.
The quick and dirty way is to create a loop that continuously checks if there is new work.
Psuedo-code
set_ini("max_execution_time", "3600000000");
$keeplooping = true;
while($keeplooping){
if(check_for_work()){
process_work();
}
else{
sleep(5);
}
// some way to change $keeplooping to false
// you don't want to just kill the process, because it might still be doing something
}
Have you tried windows scheduler(comes with Windows by default)? In this you will need to provide php path and your php file path. It works well
Can't you just write a java/c++ program that will query for you through a set time interval? You can have this included in the list of startup programs so its always running as well. Once a task is found, it can handle it on a separate thread even and process more requests and mark others complete.
The most simple way is to use embed Windows schedule.
Run your script with php-cli.exe with filled php.ini with extensions your script needs.
But I should to say that in practice you don't need so short time interval to run your scheduled jobs. Just make some tests to get best time interval value for yours one. It is not recommended to setup time interval less than a 1 minute.
And another little advise: make some lock file at the beginning of your script (php flock function), check for availability to write into this lock file to prevent working of two or more copies same time and at the end of your script unlink it after unlocking.
If you have to write output result to DB try to use MySQL TRIGGERS instead of PHP. Or use events in MySQL.