I have the following (simplified) batch email process set up under a Windows Server:
User performs action that requires an email be sent;
The data necessary to create the email is inserted into an SQL Server table;
Once every 6 hours, Task Scheduler calls a PHP file which goes through the table, creating and sending out each of the outstanding emails.
This works quite well, however the application owners would like certain sorts of email sent out more regularly, in this case, every 20 minutes.
My first thought was to set up another Task Scheduler entry, but that raises the issue of what happens every 6th hour, when both tasks will be run at the same time. It will also require creating another PHP file, which isn't really a problem, but is annoying.
The other alternative I considered was to set the scheduler to every 20 minutes, and incorporate the 'what do I send, and when' logic into the batch file itself - if it's 12AM, 6AM, 12PM or 6PM perform both sets of emails, otherwise just perform the 20 minute one. That does, however, require hardcoding those times, and doesn't seem like it should be the first resort.
Is there a better way to accomplish this?
After thinking about it a little bit more, I realised that I could accomplish what I wanted via the application of the PHP time() function and some modulo arithmetic:
$runTime = time(); //Set time the program was run, seconds since epoch
$modTimeHour = $runTime % 3600; //3600 seconds in an hour
$modTimeTwenty = $runTime % 1200; //1200 seconds in twenty minutes
//Task Scheduler doesn't always run exactly on the dot, so give it some leeway
if ($modTimeHour < 5 || $modTimeHour > 3595) {
//send emails - category 1
}
if ($modTimeTwenty < 5 || $modTimeTwenty > 1195) {
//send emails - category 2
}
By getting the seconds since epoch, and checking whether the modulo of the number of seconds in the time periods I'm interested in is within a certain range, I can have many different 'streams' all going out at their proper times.
Related
Reason
I've been building a system that pulls data from multiple JSON sources. The data being pulled is constantly changing and I'm recording what the changes are to a SQL database via a PHP script. 9 times out of 10 the data is different and therefore needs recording.
The JSON needs to be checked every single second. I've been successfully using a cron task every minute with a PHP function that loops 60 times over.
The problem I'm now having is that the more JSON sources I want to check the slower the PHP file runs, meaning the next cron get's triggered before the previous has finished. It's all starting to feel way too unstable and hacky.
Question
Assuming the PHP script is already the most efficient it can be, what else can be done?
Should I be using multiple cron tasks?
Should something else other then PHP be used?
Are cron tasks even suitable for this sort of problem?
Any experience, best practices or just plan old help will be very much appreciated.
Overview
I'm monitoring for active race sessions and recording each driver and then each lap a driver completes. Laps are recorded only once a driver crosses the start/finish line and I do not know when race sessions may or may not be active or when a driver crosses the line. Therefore I have been checking every second for new data to record.
Each venue where a race session may be active has a separate URL to receive JSON data from. The more venue's I add to my system to monitor the slower the script takes to run.
I've currently 19 venues and the script takes circa 12 seconds to complete. Since I'm running a cron job every minute and looping the script every second. I'm assuming I have at the very least 12 scripts running every second. It just doesn't seem like the most efficient way to do it to me. Of course, it worked a charm back when I was only checking 1 single venue.
There's a cycle to your operations. It is.
start your process by reading the time witn $starttime = time();.
compute the next scheduled time by taking the time plus 60 seconds. $nexttime = $starttime + 60;
do the operations you must do (read a mess of json feeds)
compute how long is left in the minute $timeleft = $nexttime - time();.
sleep until the next scheduled time if ($timeleft > 0) sleep ($timeleft);
set $starttime = $nexttime.
jump back to step 2.
Obviously, if $timeleft is ever negative, you're not keeping up with your measurements. If $timeleft is always negative, you will get further and further behind.
The use of cron every minute is probably wasteful, because it takes resources to fire up a new process and get it going. You probably want to make your process run forever, and use a shell script that monitors it and restarts it if it crashes.
This is all pretty obvious. What's not so obvious is that you should keep track of your individual $timeleft values for each minute over your cycle of measurements. If they vary daily, you should track for a whole day. If they vary weekly you should track for a week.
Then you should should look at the worst (smallest) values of $timeleft. If your 95th percentile is less than about 15 seconds, you're running out of resources and you need to take action. You need a margin like 15 seconds, so your system doesn't move into overload.
If your system has zero tolerance for late sampling of data, you should look at the single worst value of $timeleft, not the 95th percentile. You should give yourself a more generous margin than 15 seconds.
So-called hard real time systems allocate a time slot to each operation, and crash if the operation exceeds the time slot. In your case the time slot is 60 seconds and the operation is reading a certain number of feeds. Crashing is pretty drastic, but measuring is mandatory.
The simplest action to take is to start running multiple worker processes. Give some of your feeds to each process. php runs single-threaded so multiple processes probably will help, at least until you get to three or four of them.
Then you will need to add another computer, and divide your feeds among worker processes on those multiple computers.
A language environment that parses JSON faster than php does might help, but only if the time it takes to parse the JSON is more important than the time it takes to wait for it to arrive.
I have been trying to do random cron jobs where I choose the year month date and hour but the minute is randomised.
My first attempt was to run the cron every min and then compare a random date with todays date:
I inserted a random date into a database column fake_time in the format 2014-10-26 17 rand(0,59). In the php page where I run the cron every min:
if($row["fake_time"] == date("Y-m-d H i")){
//do stuff
}
And this worked perfectly. But then I found out that I can't run the cron every min because my hostor (hostgator) wont allow me to! Have you got any ideas on how I can do this any other way?
Or should i just set it up on https://www.easycron.com/ instead?
I think you are being limited by the number of cron jobs you can run in a day, IIRC hostgator has a daily limit for basic plan. To work around this limitation, IMO, you have two choices:
Go to sleep for 60 seconds
Basically, run the cron job at the required hour every day, and check for your condition, if it is not True, then go to sleep for 60 seconds.
if($row["fake_time"] == date("Y-m-d H i")){
//do stuff
} else {
sleep(60);
}
This way, you have a single cron job, though it runs for a long while. In case you can run the cron job only daily, and you want to run at random hour as well as minute, you can change your logic and go to sleep for 3600 seconds for hourly sleeps, and then go for minutely sleeps of 60 seconds.
You might need to setup set_time_limit accordingly.
Set up easycron
In case your cron jobs are terminated abruptly because the time limit can't be set, you will need to hit using easycron service.
In this case, put the above script code in a php file, say script.php, and schedule a cronjob to hit with a get request on this script. Your command in this case will look something like
wget your.domain.com/script.php
If there are no technical problems, you can do it with your php script (no install required).
if(rand(1, 5) == 1){
// do your staff
}
If you think that the script will not be executed often enough, you can reduce the difference rand(1,3)
I have a function callUpdate() that needs to be executed after every update in the webpage admin.
callUpdates execute some caching (and takes up to 30 sec..) so it is not important to execute it immediately but in reasonable amount of time after the last update lets say 60 sec.
The goal is to skip processing if the user (users) make several consecutive changes in a short amount of time.
here is my current code:
//this in separate stand alone script that is called asynchronous way
//so hanging for 1min does not and block the app.
function afterUpdate(){
$time = time();
file_put_contents('timer.txt', $time);
sleep(60);
if (file_get_contents("timer.txt") == $time) {
callUpdate();
}
}
My concerns here are bout the sleep function .. if it takes too much resources
(if I make 10 quick saves, this will start 10 PHP processes running for almost 60 sec each ..)
And what will happen if 2 users call simultaneously file_put_contents() on the same file.
Please tell me if there is better approach and if there are some major issues in mine.
NOTE: data between sessions can be stored only in a file
and there I have limited access to the server setup "APC settings and such"
I'm currently working on a browser game with a PHP backend that needs to perform certain checks at specific, changing points in the future. Cron jobs don't really cut it for me as I need precision at the level of seconds. Here's some background information:
The game is multiplayer and turn-based
On creation of a game room the game creator can specify the maximum amount of time taken per action (30 seconds - 24 hours)
Once a player performs an action, they should only have the specified amount of time to perform the next, or the turn goes to the player next in line.
For obvious reasons I can't just keep track of time through Javascript, as this would be far too easy to manipulate. I also can't schedule a cron job every minute as it may be up to 30 seconds late.
What would be the most efficient way to tackle this problem? I can't imagine querying a database every second would be very server-friendly, but it is the direction I am currently leaning towards[1].
Any help or feedback would be much appreciated!
[1]:
A user makes a move
A PHP function is called that sets 'switchTurnTime' in the MySQL table's game row to 'TIMESTAMP'
A PHP script that is always running in the background queries the table for any games where the 'switchTurnTime' has passed, switches the turn and resets the time.
You can always use a queue or daemon. This only works if you have shell access to the server.
https://stackoverflow.com/a/858924/890975
Every time you need an action to occur at a specific time, add it to a queue with a delay. I've used beanstalkd with varying levels of success.
You have lots of options this way. Here's two examples with 6 second intervals:
Use a cron job every minute to add 10 jobs, each with a delay of 6 seconds
Write a simple PHP script that runs in the background (daemon) to adds an a new job to the queue every 6 seconds
I'm going with the following approach for now, since it seems to be the easiest to implement and test, as well as deploy on different kinds of servers/ hosting, while still acting reliably.
Set up a cron job to run a PHP script every minute.
Within that script, first do a query to find candidates that will have their endtime within this minute.
Start a while-loop, that runs until 59 seconds have passed.
Inside this loop, check the remianing time for each candidate.
If teh time limit has passed, do another query on that specific candidate to ensure the endtime hasn't changed.
If it has, re-add it to the candidates queue as nescessary. If not, act accordingly (in my case: switch the turn to the next player).
Hope this will help somebody in the future, cheers!
I have a PHP-based site with bookings/appointments stored in a MySQL database.
I want to set an e-mail notification to be sent to each person who made a booking exactly 24 hours before their booking. Bookings are added through a regular PHP form.
I know I could add a script to, for instance, the index page that checks the database for any bookings that are in the final 24 hours, but that's unreliable since I won't be getting much traffic at first. So the index page could go hours without visits, and the notification would be hours late.
The other solution that came to mind is to set a cron job that runs every minute, calls a PHP script which checks whether any e-mails should be sent and sends them. But I'm not sure if this is overkill in a way; does anyone have a better solution than having something run in the background every minute?
To sum it up - is there a way to do this without cron?
Triggering the job from a web page is a very bad idea, for two reason: (1) if you don't get traffic to your site, the job doesn't run; (2) if you get a lot of notifications, the job will be slowing down the response to the web requests (assuming you invoke the job synchronously).
I would strongly discourage you from running a job every minute either - it definitely will be an overkill. Instead, think whether you really need "exactly 24 hours" as the interval or would "between 22 and 26 hours" be ok.
We have a similar requirements - and went about it by setting a job that runs every 4 hours and checks what notifications need to be sent for events starting between 22 and 26 hours form the time the script runs. This way, the script is only execute 6 times in a day and everything gets sent correctly.
If 4 hours approximation is not good enough, then think to the largest interval that's appropriate. I'm sure 1 hour should be sufficient. Have a script run once an hour (from cron) and check for events starting between 23 and 24 hours from the time of the run.
Remember that once your email is sent, it doesn't end up in the recipient's inbox immediately: sometimes it takes a few seconds, but sometimes it may take an hour or even more - so an extra hour difference in your script won't be a problem.
You don't need to use cron as an interval'd timer. You can set a very specific date and time when you want your job done.
Here's an article on the subject
For instance:
0 0 18 5 * <php command here>
Will run every May 18th at midnight. That's more than enough time to clear it before the next iteration (next year).
there is no way other than setting cron or sending request to server through periodic calls...below is post similar to your question, you may get idea.
Live redirect based on periodic server calls with JSON or AJAX
Thanks
a cron job every minute has no sense ! but you can do a cron job every hour because i think it doesn't meter a hour difference or 2 hours . without cron it isn't any other way . it will take about 2 second (at max) to complete so it is worth