I am developing software that is similar to fantasy football applications.
I have user accounts with information pertaining to each of them stored in a relational database in mysql. I have a php script that I want to run after all of the games are finished to update every member's stats at once (there could be thousands of users).
After researching it seems like the best solution would be to set up a cron tab using Unix.
I am wondering if this would be the correct solution?
If I set up this crontab to run every Tuesday would it run the PHP script which would update the user information?
Is a cron tab easy to set up with Unix for a simple website?
Setting up crons is really simple. If you have shell access, simply type #crontab -e and it will bring you to the cron settings. Check out this link: Crontab - Quick Reference.
What you simply do is create a PHP script and link it through the cron tab. If you are running a server with cPanel, there is a built-in cron manager too, if you prefer to do it in a GUI.
You did not give enough information for me to say anything conclusive about this, but you want to balance the script out so that it either 1) runs frequently enough so that it has to update small enough changes--for example, if 100 records are changed every hour, running the script every day will mean it has to update 2400 records, whereas if you run it once a week, it has to update 16800 recods, which probably is so severe that it will raise your server load. And/ or 2) break up your updates in chunks. For example if records only change once a week, but at that time 16800 records change, then you might want to either have your scripts run for 1000 updates and put it to sleep for a while, run some more, sleep, etc (this is a bad habit, but it is the simplest I suppose), or you run your cron script frequently, every time it does 1000 updates, and at the end of it, it will leave a trace (for example, the last ID number) of the last change it has updated into the database so that the script can pick up on that the next time it runs.
If you want me to draft up some more elaborate examples the illustrate the two different approaches, please let me know.
Related
Context
I'm currently implementing a feature to schedule notifications for a specific period through a web form using PHP and Firebase.
To send the notification I use Firebase and it sends notifications to Android/Ios.
To schedule the notification I use the AT linux service, as it seems to suit better than cron, as cron runs at certain frequencies and AT does not, it runs at a specific time.
man page about the AT: man page AT
Sample code
/usr/bin/php `send_notification.php` | at 2021-07-11 15:40
This will create a file on linux that will run in the period 2021-07-11 15:40 only once.
Problems
The AT service, like CRON, creates files inside a directory on the operating system that represent the jobs.
1 - If a machine on AWS is scaled, jobs would likely be duplicated and consequently send notifications more than once. (Note: I don't know much about machine scaling, but I believe it should happen)
2 - And if the machine is in downtime due to the inclusion of some functionality or something like that, I believe that the way it is currently the job would not be executed.
3 - Another problem, but not the main one, would be if I was using a docker container. As Ubuntu + PHP are inside the container, the job files would probably be lost if I restarted the container, so in this case I believe that a solution would be to use volume, but that would not be my problem now, as currently the application uses only one machine on AWS EB with the PHP image.
Doubts
Is there any solution I can apply to solve this duplicate job problem using PHP?
Is the approach using AT the most suitable? I see a lot of people talking to use CRON, but CRON will run the job several times and for me that's not what I'm looking for.
I think you need a place where scheduled and finished notifications will be persisted, independently on what you are using, cron or at.
If I had such a task, I would stay with a solution like this: run special script, "scheduler.php" each 1 (or more, e.g. 5) mins by cron, which will check some log file(or remote database in case of several machines) and look if there are any new lines. If new line present and it contains timestamp in the past and status "sceduled", than script will lock it and run your "sender.php". After that it will mark the line as "done". Each line in a storage should contain a timestamp to run and one of three statuses "scheduled", "running" and "done".
With such approach you could plan new notifications by adding a line with needed time and status "scheduled" to the storage. Note, that there can be a little delay between scheduled time and actual notification depending on the cron interval, but I suppose it is not critical.
This will allow you to run any number of crons on different machines and guarantee that each job will be done once.
Important: if you will adopt this scheme, be sure that your scheduler.php reads and updates a storage in a single atomic operation, to prevent race conditions between several crons. File locks, or "select for update" will do.
For the checking of online users who doesn't log out properly but close the browser, I want to run a cron job in every 2/3 second. (I am updating database in every 10 second when logged in)
Will it harmful for server?
Cron lowest possible frequency is 1 minute so you cannot fire anything more often with it. As for overkill - it may or may not be there, but you need to review the code and load it produces yourself.
You can't run cron jobs more frequently than once a minute so this isn't possible from a cron anyways. You'd have to have a process running in a loop with sleep to achieve this, but the idea is overkill anyways - once a minute is fine.
I've got a database that processes an order - the Order is inserted at the beginning of the process but if the process is not complete (they left the page etc) the "confirmed" field is not filled in, this allows the admin to know which orders were complete or not.
At the moment I don't have the luxury of adjusting the way the website manages the data, I can however add some sort of weekly cleanout of these database fields where "confirmed" = 0.
How would I go about doing this automatically? Is there a way to run a query like this on a weekly, monthly, yearly basis?
Edit:
What if I added that the website is stored on a restricted, shared server where I have no access to the box itself?
You can use CRON to perform this job. But if you need to clean your database from incompleted orders only to show completed ones for humans and you don't export them to other places (e.g. manager visits backend once per day and looks for completed orders manually), i suggest you not to hire CRON for this. You can add few lines of code which will remove old entries from database each time backend was accessed.
DELETE queries are executed really fast so there will be no problems with perfomance. Real data deletion will be performed only by request. Also you don't need to have CRON (some cheap hosting providers do not give access to this service).
I usually create a PHP page that does the action and then schedule that to run using cURL and a CRON job.
If it's a Unix-like box, I'd put the cleanup in a PHP CLI script (that is, a PHP script that runs from the command line rather than in a web server module) and invoke that from a cronjob. Cron is a daemon for this exact purpose: Run things at predefined intervals. Cronjobs are defined in a crontab file, usually in /etc/crontab (for root) and ~/.crontab (for regular users); many distributions offer tools to safely edit this file.
Try man cron and man crontab for details. Also: http://en.wikipedia.org/wiki/Cron
setup a cron job and do a wget to run the script you want to automate. look at the link for more info http://www.phpmyvisites.us/faq/crontab-launch-automatically-statistics-archiving-process-every-62.html.
Write a short PHP script that updates/deletes the mysql records,
Then, if you have access to the linux/unix box, type crontab -e, you will then be prompted with the option to choose which editor you would like to view the current cronjobs, select the default and add this line:
0 0 * * 0 /usr/bin/php /path/to/script.php
Please check if /usr/bin/php exists before you set this up (you will need to install php-cli incase it does not exist, or the file might be also at /usr/sbin/php).
This cronjob will run once a week, if you want to change the frequency, use the manpages to learn how.
Also, are you sure you would like to run this script automatically? from what I understand you will need someone to first see the incomplete orders and only then to clean them up. if you run this cronjob you might be in a situation where someone did not complete his order but any evidence to that was deleted.
Hope this helps,
I am using MYSQL as my database and PHP as my programming language.I wanted to run a cron job which would run until the current system date matches the "deadline(date)" column in my database table called "PROJECT".Once the dates are same an update query has to run which would change the status(field of project table) from "open" to "close".
I am not really sure if cron jobs are the best way or I could use triggers or may be something else.Also I am using Apache as my web server and my OS is windows vista.
Also which is the best way to do it? PHP scheduler or cron jobs or any other method? can anybody enlighten me?
I think your concept needs to change.
PHP cannot schedule a job, neither can MySQL. Triggers in MySQL execute when a mysql query occurs, not at a specific time. Neither
This limitation usually isn't a problem in web development. The reason is because your PHP application should control all data going in and out. Usually, this means just the HTML that displays that data, or other formats to users, or other programs.
In your case you can think about it this way. The deadline is a set date. You can treat it as data, and save it to your database. When the deadline occurs is not important, it is that the data you have sent in your database is viewed correctly.
When a request is made to your application, check if the date of the deadline is in the past, if it is, then display that the project is closed - or update that the project is closed, just before display.
There really is no reason to update data independantly of your PHP application.
Usually, the only things you want to schedule are jobs that would affect your application in terms of load, or that need to be done only once, or where concurrency or time is an issue.
In your case none of those apply.
PS: I haven't tried PHPscheduler but I can guess it isn't a true scheduler. Cron is a deamon that sleeps until a given task is due in its queue, executes the task, then sleeps till the next one is due (at least thats what it does in the current algorithm). PHP cannot do that without the sockets and fork extensions, as special setup. So PHPscheduler is most likely just checking if a date for a task has expired, on each load of a webpage (whenever PHP executes a page). This is no different then you just checking if the date on the project has expired, without the overhead of PHPScheduler.
I would always go for a cron job for anything scheduling related.
The big bonus point is that you can echo info out as well and it get's emailed to you.
You'll find once you start using cronjobs, it's hard to stop.
cron does not exist, per se, in vista, but what does exist is the standard windows scheduling manager which you can run with a command line like "php -q -f myfile.php" which will execute the php file at the given time.
you can also use a port of the cron program, there are many out there.
if it is not critical to the second, any windows scheduling application will do, just be sure to have you PHP bin path in your PATH variable for simplicity.
For Windows CRON jobs I cannot recommend PyCron enough.
While CRON and Windows Scheduled Tasks are the tried and true ways of scheduling jobs/tasks to run on a regular basis, there are use cases where having a different scheduled task in CRON/Windows can become tedious. Namely when you want to let users schedule things to run, or for instances where you prefer simplicity/maintainability/portability/etc or all of the above.
In cases where I prefer to not use CRON/Windows for scheduled tasks, I build into the application a task scheduling system. This still requires 1 CRON job or Windows Task to be scheduled. The idea is to store Job details in the database (job name, job properties, last run time, run interval, anything else that is important for your implementation). You then schedule a "Master" job in CRON or Windows which handles running all of your other jobs for you. You'll need this master job to run at least as often as your shortest interval; if you want to be able to schedule jobs that run every minute the master job needs to run every minute.
You can then launch each scheduled job in the background from PHP with minimal effort (if you want). In memory constrained systems you can monitor memory usage or keep track of the PIDs (various methods) and limit to N jobs running at a given time.
I've had a great deal of success with this method, YMMV however based on your needs and your implementation.
how about PHPscheduler..R they not better than cronjobs? I think crons would be independent of the application hence would be difficult if one has to change the host..i am not really sure though..It would be great if anyone can comment on this!! Thanks!
I need to reset a MySQL Field value automatically at midnight. It is a specific column in a specific row in a table. I know how to do this in PHP but I do not know how to execute the PHP Script at midnight without someone having to do it themselves. Do you have any viable solutions?
Edit:--------------------
Preferably without using Cron Jobs.
If you are running on linux you would use a cronjob
On most distros there is a command called crontab that schedules tasks for you and a specific format you need:
0 0 * * * php /path/to/file.php
EDIT:
Wrote this before you edited yours :p I'm not sure there is any other way. Do you have a specific reason not to use a cronjob?
You may not even need to specify the php part if the php file is marked as executable i.e
0 0 * * * /path/to/file.php
Just do "chmod +x /path/to/file.php" form the linux command line
There are web based cron services too. Basically you set up an account and then they visit an URL of your choosing at a regular interval. On your server you set up a page that does what you need to get done. Preferably give it a unlikely-to-find-name like myjob789273634ahhh8s2nhw8sghusgf874wfu.php. You get the idea. (Remember that PHP-scripts timeout after like 30secs.)
Here's a google search:
**oops I'm new so I can't post URL apparently. Just search for "web based cron".
Good luck
/0
You could write a job scheduler into your program that runs jobs in a cron-like way. It would require a user to interact with the system to trigger, but it might be good enough depending on your needs. This is much more complicated than just running a cronjob, and does not ensure prefect timing (since it wont run until a user hits a page).
You'd probably need to add a table into you database that would list the job, the time you want them done, and a locking flag to avoid concurrent attempts to run the job. Each time your script runs, you'd check this table for overdue jobs and run them as needed.
Asking how to reliably set off a script at the same time every night without cron (or a scheduled task, on Windows) is like asking how to make a dynamic website without a server-side language.
If your app absolutely relies on a script running exactly at midnight, cron is a requirement. If your users have a hosting company that (stupidly) does not permit cron, they're going to be out of luck.