i have script which must execute after every n minutes. n minutes is dynamic so that i could not set a cron job to call the script (at a specific time).
so what i did was i stored the time after every n minutes in an array so that when the script is executed, it will first check whether the current time is in the array. if it is found in the array, it continues to executes otherwise it exits.
to execute the script, i must use a cron job to run every minute to check the time in the array. unfortunately, my web host only allows 5 minutes as the least interval. so every time the script is called, i check whether the values between $current_time and $current_time + (4*60) // 4 minutes is found in the array. if it is, and if needed, i use time_sleep_until to delay the script until the time reaches the value found in the array.
so if my script executes at 10:05 and the value found in the array is 10:06, i let the script sleep until 10:06 before it continues to execute. however, if the sleep time is more than a minute or so, i get a Mysql server gone away.
how can i prevent this? or is there a better way to do this?
thanks!
A couple choices, which is better I do not know.
One, is make sure your script works with CLI and after that minute is up call it with the http://www.php.net/exec function (if your host allows it).
Two, is setup a script, with a possible hash as a key and use a header redirect after the minute is up, this would call the script brand new so a new MySQL connection is made.
A third option is to set the script up like in two, except setup a schedule task / cron job on your computer that opens that page (it would have to be in the webroot) and calls it every minute or however you want. This is not a set method, but depends on how much your computer is on.
Fourth, similar to the third but use a free cron job hosting service like: http://www.onlinecronjobs.com/en
Hope that helps. If I think of other options I will update.
Related
I'm currently working on a browser game with a PHP backend that needs to perform certain checks at specific, changing points in the future. Cron jobs don't really cut it for me as I need precision at the level of seconds. Here's some background information:
The game is multiplayer and turn-based
On creation of a game room the game creator can specify the maximum amount of time taken per action (30 seconds - 24 hours)
Once a player performs an action, they should only have the specified amount of time to perform the next, or the turn goes to the player next in line.
For obvious reasons I can't just keep track of time through Javascript, as this would be far too easy to manipulate. I also can't schedule a cron job every minute as it may be up to 30 seconds late.
What would be the most efficient way to tackle this problem? I can't imagine querying a database every second would be very server-friendly, but it is the direction I am currently leaning towards[1].
Any help or feedback would be much appreciated!
[1]:
A user makes a move
A PHP function is called that sets 'switchTurnTime' in the MySQL table's game row to 'TIMESTAMP'
A PHP script that is always running in the background queries the table for any games where the 'switchTurnTime' has passed, switches the turn and resets the time.
You can always use a queue or daemon. This only works if you have shell access to the server.
https://stackoverflow.com/a/858924/890975
Every time you need an action to occur at a specific time, add it to a queue with a delay. I've used beanstalkd with varying levels of success.
You have lots of options this way. Here's two examples with 6 second intervals:
Use a cron job every minute to add 10 jobs, each with a delay of 6 seconds
Write a simple PHP script that runs in the background (daemon) to adds an a new job to the queue every 6 seconds
I'm going with the following approach for now, since it seems to be the easiest to implement and test, as well as deploy on different kinds of servers/ hosting, while still acting reliably.
Set up a cron job to run a PHP script every minute.
Within that script, first do a query to find candidates that will have their endtime within this minute.
Start a while-loop, that runs until 59 seconds have passed.
Inside this loop, check the remianing time for each candidate.
If teh time limit has passed, do another query on that specific candidate to ensure the endtime hasn't changed.
If it has, re-add it to the candidates queue as nescessary. If not, act accordingly (in my case: switch the turn to the next player).
Hope this will help somebody in the future, cheers!
Here's what I'm trying to accomplish in high-level pseudocode:
query db for a list of names (~100)
for each name (using php) {
query a 3rd party site for xml based on the name
parse/trim the data received
update my db with this data
Wait 15 seconds (the 3rd party site has restrictions and I can only make 4 queries / minute)
}
So this was running fine. The whole script took ~25 minutes (99% of the time was spent waiting 15 seconds after every iteration). My web host then made a change so that scripts will timeout after 70 seconds (understandable). This completely breaks my script.
I assume I need to use cronjobs or command line to accomplish this. I only understand the basic us of cronjobs. Any high level advice on how to split up this work in a cronjob? I am not sure how a cronjob could parse through a dynamic list.
cron itself has no idea of your list and what is done already, but you can use two kinds of cron-jobs.
The first cron-job - that runs for example once a day - could add your 100 items to a job queue.
The second cron-job - that runs for example once every minute in a certain period - can check if there are items in the queue, execute one (or a few) and remove it from the queue.
Note that both cron-jobs are just triggers to start a php script in this case and you have two different scripts, one to set the queue and one to process part of a queue so almost everything is still done in php.
In short, there is not much that is different. Instead of executing the script via modphp or fcgi, you are going to execute it via command line php /path/to/script.php.
Because this is a different environment than http, some things obviously don't work. Sessions, cookies, get and post variables. Output gets send to stdout instead of the browser.
You can pass arguments to your script by using $argv.
i have a big script written in php, which should import a lot of informations in a prestashop installation, using webservices, this script is written in "sections" I mean, there is a function that import the categories, another one that import products, then manufacturers, and so on, there are about 7 - 10 functions called in the main script. Basically I assume that this script must run for about an hour, passing from a function to the next one and so on since it arrives at the last function, then return some values and stops until the next night.
i would like to understand if it could be better :
1) impose a time limit of 30 minutes everytime i enter a new function (this will prevent the timeout)
2) make a chain of pages, each one with a single function call (and of course the time limit)
or any other idea... i would like to :
know if a function has been called (maybe using a global variable?)
be sure that the server will execute the function in order (so the pages chain)...
i hope to have beeen clear, otherwise i'll update the question.
edits:
the script is executed by another server that will call a page, the other server is "unkown" from me, so I simply know only that this page is called (they could also call the function by going on the page) but anyway i have no controll on it.
For any long running scripts, I would run it through the commandline, probably with a cronjob to kick it off. If it's triggered from the outside, I would create a job queue (for example in the database) where you insert a new row to signify that it should run, along with any variable input params. Then the background job would run - say - every 5 minutes, check if there's a new job in the queue. If there's not, just exit. If there is, mark that it has begun work and start processing. When done, mark that it's done.
1 hour of work is a looooooooong time though. Nothing you can do to optimise that?
You can increase the time limit for execution of a script as much as you want using :
set_time_limit(seconds);
And also for long running scripts you need a more memory. you can increase the memory limit using :
ini_set('memory_limit','20M');
And second other thing you have to make sure is that you are running your script on a dedicated server because if you are using a shared server you server will kill automatically long running scripts.
Any of the users can trigger an event, as many times they wish. Each time this event is triggered, a row in a mysql table is created with the timestamp, their userid and other useful information. What I need to do is launch a script which runs exactly 12 hours after the event is triggered i.e. for each row in the table.
How can I achieve this in an automated and efficient fashion?
You could use a cron job which every minute launches the script.
In the script you should first fetch the row(s) and check if it's OK to run (if 12 hours passed) then continue, else stop.
I don't know if this will be very efficient, it depends on your number of entries in the database, but technically it's not expensive to just check if the current date matches a date fetched from the database + 12 hrs, I cannot say more because you didn't give too much details about your data.
You'd probably be better off with a cronjob
http://en.wikipedia.org/wiki/Cron
Potentially, you could look into MySQL event scheduler. Although It might not fit your needs, hard to really tell on the details given
http://dev.mysql.com/doc/refman/5.1/en/events.html
something like
CREATE EVENT myTimedEvent ON SCHEDULE EVERY INTERVAL 5 minutes DO CALL updateRows();
updateRows checks your criteria (12hours ago), if it is, perform whatever action you want to do. This requires your MySQL to be # version 5.1+ however
You would probably best have a cron job which runs every minute and checks for any rows in the database > 12 hours old that have not yet been processed the script and process them. This wont give you EXACTLY a 12 hour difference, but would give you a difference within a minute of that.
You would probably also want to make sure that script would be able to run within a few seconds such that you don't have overlap of the script running twice at the same time.
This could be done using CronJobs. If you have root access to your Server or a server administration toolkit that offers cronjob managment you would do it on your server. otherwise use an online cronjob service (google for cronjob online).
the cronjob then triggers a php script on your server in your defined interval, like every minute or every 5 minutes.
this script then selects all rows from your mysql table which are older then 12 hours (WHERE timestamp <= NOW() - INTERVAL 12 hour ) and performs your desired actions on each result and delete the result from the table (or mark it done),
just make sure that the fetching and the actions itself are faster than your cronjob interval, otherwise you would have two or more scripts working on the same rows.
The easy way for me is to make the page head contains
<head>
<meta http-equiv="refresh" content="43200"> <!-- 43200 for (12 hours × 60 mintes × 60 seconds) -->
</head>
This method is very helpful to avoid server time out, which you can't avoid if you are using only PHP code
Important if you start the script using submit button, it's recommended to make the form action="other_page" not the same page, and sure you should save your input values as cookie using setcookie function and grab it as a variable in the action page using $cookieVar = $_COOKIE['the_cookie_name'] ;
You may need to increase or decrease the $cookieVar value and update the value again using setcookie every time your head code http-equiv do the refresh automatically after the specific seconds (43200 in our example) depends on what you want to do
Note that if you make your action start automatically without pressing submit button you can do the action in the same page.
I did that idea before, I had 180,000 images in one folder and the server didn't allow me to download it all because it was showing me only first 7998 images, so I create a script to zip each 1000 image in a file outside the images folder and to avoid time out, I made the code refresh each 60 second , finally I got only 180 zip file :)
I am writing a task which will be ran by as a Cron Job. I want the function to check if a file has been downloaded to the server yet. The time stamp check is fine, however what is the best way to make the check loop run every 15 minutes? Could I use sleep and then check the current time stamp againsed the time stamp I want the loop to terminate (5am)?
Cheers.
This is what cron does. Crontab allows you to set an interval to run a script. It is easy to specify 15 minute intervals. This way you won't have to worry about sleeping or error recovery or anything in your php script, just your logic.
http://www.electrictoolbox.com/run-cron-command-every-15-minutes/