Hi all
I am using Zend framework for my PHP project. Basically I have few actions that I want to run automatically. I will be using a cron job to do the trick. The cron job will run a php script file.
Till now everything seems normal. Now I have created a table in my database and have stored the actions that I need to run in it. For example I need to do the following:
1-Create a sample file (5 times).
2-Upload all sample files (1 time).
3-send mail (continuous).
So I will store in my table the controller and action for (creating a sample file) and set its repeating time to 5 and set also its execution time. The same will be done for the other two actions.
Now the script file is run every minute. In the script file I will select all records that from the table that have an execution time equal to the current time then run them.
Now to make things just clear this system actually works but I was thinking about a better or improved scheduler techniques. Storing the scheduled actions in a table sound like a good idea but I was wondering if there was a superior approach ?!
This seems fine to me. You're using a cron job to run a PHP script which selects matching records to be executed from a database table. Storing data, as long as it's in a normalised form in the database, will be an efficient way. Monitor the load on your server and adjust from there if you need to.
My aproach is have a cyclic script that "empties" (set jobs as "finished") the table. And there are more CRONs with different timing that fill the table. So the workflow is like this:
minute: nothing to do
minute: nothing to do
hour: added export feed update
minute: export feed update finished
minute: nothing to do
minute: nothing to do
minute: nothing to do
minute: nothing to do
...
minute: nothing to do
five_hours: added "clear the cache"
minute: clear cache finished
minute: nothing to do
minute: nothing to do
...
Related
Does anybody know how WordPress stores it's CronJob events? I'm developing a plugin with multiple concurrent CronJobs, which behaves really strange. When configuring the plugin the first Event will generate some page data over a period of roughly 10-15 mins and is split into multiple packages. These packages will reschedule themselves to get the maximum running time, without hitting the script execution limit. However when the first CronJob is executed, the user can start a second one (not the same one, it's from another section), which will always result in the second one being scheduled, staying in standby and getting removed after the first one has finished an execution.
We had problems with long running CronJobs and the database cache before: Some of our data is bundled into an option and inserting data into this package will overwrite changes made outside of the CronJob. Maybe something similar is happening here. For reference: The reshedule of the first CronJob happens inside said CronJob. Could that be a problem too?
This is how the error is behaving:
Init
Cron 1 is sheduled to a past timestamp.
Cron 1 is starting.
Cron 2 is sheduled to a past timestamp.
Cron 1 is working.
Cron 1 is finished.
Cron 1 is resheduled to a new timestamp.
Cron 2 gets removed from the event queue.
Cron 1 is starting...
I have checked everything that correlates to the scripts themselves: The events are properly registered, have a unique argument (just in case) and even pull a new version of the database options they change, before doing so. Limits are set beforehand and every related function is wrapped in a try-catch-block.
My questions so far: Does anybody know what can cause a CronJob do get deleted (besides "wp_clear_sheduled_hook")? Does WordPress store the events as an option? Can a CronJob overwrite these settings, when it is running for a long time?
Thanks for your help and greetings
SOLUTION: Thanks #kyon147 for pointing out that WordPress is using the wp-options table to store information about the sheduled events. In case anyone has similar problems: Wordpress will load ALL options into it's cache, when it is called. Meaning when starting Cron1 the "cron"-array with your events might look like this:
array('cron1' => 'time')
When something is changing this option while the script is still runing, this change will not be reflected to the script. Meaning the array will still be as above, even when an event is added from another script/session. So when resheduling the event INSIDE Cron1 WordPress took the array above, not the new one. This resulted in the changes being reset to the state, when Cron1 was started and thus the event appearing missing.
I have to populate and update one of my MySql database table using a complex and expensive query, based on selection from other table's data. This table doesn't need to be always fully updated when i make a query on it, but i'd like to have a cyclic update every 5 minutes.
This automatic update should be infinite and i need to be sure that it never stops.
After some research, i've found some solution, but i don't know which is better for security and performance.
One of these could be my goal:
Don't create table and make complex query from php every time to get the desired result
Create a php script that repeats cyclically and update table db, maybe using Cron Job.
Update table using a sql event
I think that first solution could be to expensive since query is complex and there could be many request every second, but the result is always updated. I don't have experience about Cron Job, so i can't know if it could be a good idea or not. For the third solution, i still don't have database privileges to run events, but i'd like to know if it could be a valid solution.
All other solutions are welcome, thanks.
Do not use cron. Think about what will happen if one instance goes beyond 5 minutes and the next starts up. Eventually you will have hundreds of copies bogged down stumbling over each other.
Instead have a single job in a loop doing the update. (OK, you could have a cron job to perform a "keep-alive" task of restarting the query if it dies.)
The job would
CREATE TABLE new ...
INSERT INTO new SELECT complex-stuff...
RENAME TABLE real TO old, new TO real;
DROP TABLE old;
loop.
I would opt for Cron Job.
It doesn't clog any request, since it's executed from the operating system.
You can define which user executes the script (cron -u apache -e).
Easy to define interval. (i.e. every 5 minutes */5 * * * * php /path/to/script.php).
It's loggable.
Additional Notes
I had a cron job running under root and it worked just fine. My problem was that the project had a private logging mechanism that each log file would be created by apache user. By running it from root, sometimes the file would be created by root and after that, the scripts being executed by apache would not be able to APPEND to the log.
I also had an emailing script that would run once every 2 minutes that got stuck for 1h. Turns out, because of a bug in the application, an invalid email address (somethingwithoutatsign.com) was inserted into the database, which made the PHPMailer library throws errors. After that, I added a catch block that would send an email to me whenever an exception was thrown. Now, if the script stops running because of bad execution, I get to know right away.
My host (blueangelhost.com) claims that I can't use the event scheduler because it takes up too many resources. I have access to cron jobs in cPanel, but I've tried and they don't seem to work.
So, my question: Is there any kind of efficient PHP code that will automatically truncate a MySQL table in a database?
Well, if it needs to be automatic, or at a specific time, not really. But you could have your website trigger the script when someone gets on it, here's the approach you could use:
On a script that is run on every page (header, menu,footer, layout):
Check in DB or file, the date of the last truncate;
If the date is yesterday, run the truncate;
Change DB or file and put current date;
This way, it will execute once a day. But never at the same time, and not if no one walks on your website for a whole day.
What is the best way to update my database with data every say 2 hours. I am going to be calling an API that returns JSON, and then I want to update that database. It is bringing back snow conditions.
Thanks,
Ryan
It depends what kind of server you are on, but if it is linux based, you can use cron to run your php script every 2 hours.
The cron entry would be something like:
0 */2 * * * /path/to/script
And the script could be a shell script with the form (just an example for my server...):
#!/usr/local/bin/php
<?php
?>
You can create events in MySQL: http://dev.mysql.com/doc/refman/5.1/en/create-event.html
CREATE EVENT e_hourly
ON SCHEDULE EVERY 2 HOUR
COMMENT 'a comment'
DO BEGIN
//Hourly action
END
Running non-SQL command from within the DB-server is not recommended though due to security issues.
(this is only possible through a custom UDF like: http://bernardodamele.blogspot.com/2009/01/command-execution-with-mysql-udf.html#!/2009/01/command-execution-with-mysql-udf.html )
On windows the at command or scheduled tasks can run every 2 hours:
at: http://support.microsoft.com/kb/313565
scheduled tasks: http://windows.microsoft.com/en-US/windows7/schedule-a-task
On Linux crontab is your friend: http://kevin.vanzonneveld.net/techblog/article/schedule_tasks_on_linux_using_crontab/
Use cron and write some script that gets data from api and insert it to database.
I'm creating a web application where every row of a table needs to be processed. I'm spawning one child PHP process per table row. I'm implementing a safety mechanism, so if a PHP process is interrupted processing a row, a new PHP process will spawned to process said row. To do this I'm going to create a new table where all PHP processes check in every 10 seconds or so. I need MySQL to delete all rows that haven't been checked into for 5 minutes or more, so my application will know to create a new PHP child to process that row.
I know it's possible to get MySQL to run queries on an interval, but I don't know how.
~Enter stackoverflow~
Edit: I was hoping to learn how to do this 100% MySQL. Is there no way to set MySQL to run a query every hour, or at a specific time each day or such?
Crontab. You can run the query directly using the mysql client (mysql -uusername -ppassword dbname -e 'query here') or schedule a PHP script which runs the query.
DELETE FROM table WHERE checked_into < CURRENT_TIMESTAMP - INTERVAL 5 MINUTE
MySQL Events are tasks that run according to a schedule. Therefore, we sometimes refer to them as scheduled events. ... Conceptually, this is similar to the idea of the Unix crontab (also known as a “cron job”) or the Windows Task Scheduler.
http://dev.mysql.com/doc/refman/5.1/en/events-overview.html
And here is the lovely syntax: http://dev.mysql.com/doc/refman/5.1/en/create-event.html
One way to run MySQL queries on a certain interval would be to set up a cron job. Assuming you've got full access to your webserver, this should be doable. You'd just make a PHP page that does the SQL operations you want to occur every X time interval, and then set the script to run on that interval via cron jobs. More specifics: http://en.wikipedia.org/wiki/Cron
I think what you are looking for is an event scheduler, first introduced in MySQL 5.1.
On a side note, maybe you should redesign your program a little to avoid the extra layer of event scheduler:
Instead of deleting a row, where a process has not checked in for a while, just have a column with a check in timestamp. Then if some row has a very old check in timestamp, you can spawn a new PHP process for it.