Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I have a table, that consists of all the info to run a campaign. The info includes, the time interval to check the campaign (10 mins, 15 mins etc.) and other information to check whether it meets the specific requirement or not, to run the campaign.
At the moment, what I am planning to do is:
Add my code in one php file
In the code, go through all the rows of the table
Check if it's the time to check the campaign or not (via the interval)
If it's the time to check the campaign, then go through other details of the table and based on the set conditions, send an email or SMS.
I am planning to run a cron job which goes through this php file, after every 10 minutes (As it's the shortest check interval)
I need suggestions, whether it's the proper solution or not OR if someone has any better and efficient solution?
That's a decent starting point...
If it's the time to check the campaign...
Keep in mind that sometimes a cron process takes longer than expected, gets stuck or your system crashes in the middle. Ideally your process will keep track of a.) what it's doing, and b.) when it did it. And be able to fix problems like skipped or stuck processing.
It could be that you never want to send a message that late. Then again you may want to make sure all of the missed messages get sent. Your code should be able to handle this case automatically to some degree. Maybe automatically do anything that's should've been done in the last hour but wasn't and ignore anything older than that. For older stuff you'd have to manually run the script. Make sure your script has command line arguments that simplify you forcing it to run for prior time intervals and specific campaign IDs. This will make your life way easier after a disaster.
I suggest that you have some kind of reporting so you can keep track of your processing in real time. Pretty simple if you're writing state info to your database. Add on an end of processing timestamp and you can even see how long your cron jobs are running. If you don't want to use this state info in your cron job you can just write it to a log file instead of a database. And in that case (if needed) you would use a lock file to indicate when a cron job is running and prevent other cron jobs from starting at the same time. Regardless, it's good practice to write a log file so you have a record of what happened. Imagine if your cron job sent an email but crashed while attempting to write the state to the database. You'd at least have a log line to help you investigate later.
I am planning to run a cron job which goes through this php file,
after every 10 minutes (As it's the shortest check interval)
So the speed of your script will vary with the amount of data, latency of external services (assuming your script talks directly to such services). I would start with a much longer cron job start interval - assuming that your client/use case allows for that. If you follow the suggestion above to have your script automatically handle skipped times this isn't a problem. The more stuff you're processing the more time your script will eventually need. So on day 1 it might only need 1 second. But on day 300 it might need 15 minutes? (At that point you could decide that you want to have multiple processes/threads running at the same time with each one focused on a single campaign or range of campaigns. Who knows...) But you'll know because you have reports/alerts/logs on the start/end processing times.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I'm developing a tool for my company, that in very broad strokes is intended to message users some information (a link to a picture) if they decide they want to be notified when it comes online.
If it were just alerting them when it's online is easy, because you don't have to schedule it, just check the list to see if anyone wants to be messaged about the picture when it comes online, and do it.
But we also have "We've got your petition, the picture is not here yet but we'll message you when it is" kind of message, and a few "not here yet" that I have to launch days later if the picture isn't online yet. And all these scheduled jobs need to be canceled if in any moment the picture comes online, and we send the message with it's link.
I'll try to explain it in as much detail as I can:
The user asks to be notified
Our web takes note of the petition. (adds it to the DB)
Check if the file is already online. At this point in time the file might not have been uploaded yet, but may be mere seconds away from it. So if the picture is not online yet when the petition is made, we want to wait 1-2 minutes to send the "Picture not yet here" message. Just so we don't send a "not yet here" message and 5 seconds later a "here's your picture" one.
We want to wait a few hours (1-3), to send a new message asking them to be patient.
After another set amount of time (7 days, approx) we want to send a last message, letting them know that the picture might never reach them because it's not being uploaded.
In any given time, even after point 5, if the picture comes online we want to cancel all these schedules and send the message with the picture.
I've been trying to learn how to do this, and through my search I've learned of three possible ways to achieve this functionality:
Option A: A single Cronjob executing every minute, that sweeps the database table searching if it's time to send one of those messages.
This option is easy to understand, although I'm afraid it might tax the database too much. I can use the shifty control panel that 1and1 has to set up that single Cronjob and call it a day.
Option B: Programatically write Cronjobs for every message that gets scheduled.
This sounds like it would be more "efficient", or at least less taxing on the DB, but I'm not sure Cronjob is supposed to work like that. It's normally used to schedule tasks that repeat themselves, isn't it? This would need a whole lot of functions to work (read the Crontab, add a line, search a line, edit a line, delete a line). Problem being here that I don't know how to edit the crontab that's on 1and1 servers via php. I have tried to contact them but their support has not been helpful at all.
Option C: The "at" function in linux.
This I just learned about. It looks like it would do what I want: schedule a task that happens only once, and it's structure seems pretty easy to handle. The problem here is threefold: 1- I don't know if PHP can execute Command Lines, 2- I don't know if the server at 1and1 has the "at" program installed, 3- I don't know if I can get a Command Line to execute a PHP file with the arguments to make it work.
And if any of these can be done, I don't know how.
As you see there are plenty of things I don't know about, but I've been trying to inform myself and learn. I just ask here because I'm at the end of the rope.
These options I listed are not an exhaustive list, they are just the methods I've found.
Which method would serve my purpose better? And how to do it?
Relevant facts:
Our host and database are located within 1and1, in a virtual server (meaning, we don't have a complete server for us, but share one with other clients)
Although we have "Unlimited" database space and queries, there is still a hard limit of how many queries you can do in a certain limit.
I'm new-ish to using linux, and I have not worked with PHP for years (until I've got this job!), so it would be better if your explanation doesn't assume deep knowledge on my part.
Programatically write Cronjobs for every message that gets scheduled.
God no, the biggest issue you'll have with that is that you will have to anticipate in advance what kinds of messages you'll have to send when, and that clashes with your ability to easily cancel messages. You will generally have to worry about a lot of state to manage (more on that later), which is a lot of hassle. You also need to ensure your scheduled jobs are cleaned up again afterwards, since cron can only set repeating tasks.
The "at" function in linux.
This is basically cron but for non-repeating tasks. That's better, but is still stateful. Especially with shared hosts it's also somewhat unpredictable whether your code will always execute on the same machine or when a machine might reboot. In those circumstances you may lose your scheduled jobs, so this is a no-go.
A single Cronjob executing every minute, that sweeps the database table searching if it's time to send one of those messages.
Yes, this is the way to do it. The biggest advantage here is that it's stateless, in the sense that it will always pick up on the exact current contents of your database, so it allows you to easily manage what your job should be doing on its next run and not having to anticipate that at the time you schedule an event.
I'm afraid it might tax the database too much.
It's one query per minute (if you write it well). Presumably every single page load of your website will incur one or multiple queries, and a properly built site should be able to handle hundreds to thousands of loads per second. One more query per minute isn't going to tank it. If it does, you have bigger issues.
I would personally choose the option A, I'm using it already on a project I worked on.
In your case, having the data on a shared hosting, I would create a cronjob that runs every minute (using an online service) and hits a php file somewhere in your folders, checking in a database table if anything must be done.
You should write some code that handles all the notifications you want to send and when, creating, for each of them, a row in the db table with the time of execution and all the details ready to be used to create the notification and to send it out.
The entire thing would work more or less as follow:
- Something happens that requires the creation of a notification to be sent out in 5 minutes: the row is created in the db table with the unix time or date of 5 minutes from now.
- A notification needs to be sent out 3 days from now, you use the same procedure as above.
The cronjob runs every minute and checks for expired orders (anything with date <= now), if any, a script takes care of these rows and execute the orders (sending out only the notifications required).
The database wouldn't be bothered too much, having to perform only 1 query per minutes (only checking for expired orders).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
PHP newbie here. My question is related to the logic/best practice for executing a certain task.
I have a news event/announcement being displayed on a website and needs to be removed/expire after a certain date/time. I have set a expiration date/time in the database, when a user visits the website after the date/time has passed a query is triggered and the news event/announcement is set to "0" (which is hidden).
I would like to know if this the best practice of accomplishing it or is there a better way?
Thanks.
The method you mention is usually effective, especially for small applications, but not a best practice. The reason it's not a best practice is because:
Issues
You are making the user wait for task execution
If there is no activity on your website, these tasks will not be done
If anything happens while executing a task, your user will receive the errors
The first two might not seem to matter much, but that's only because the tasks you have now are very fast and non-critical. However, if a task would take a second or two, that would mean the user now has to wait 2 extra seconds before he sees the page, which is bad.
Likewise, if nobody visits your site for a week and there's a list of 15 tasks that need to be done, the user now would have to wait for 30 seconds. It might even time out the whole page; which would mean your tasks are now unfinished and the user is annoyed with getting a timeout for seemingly no reason.
In addition, if one of your tasks is time critical, it still won't be done. For example if the task is to send someone a reminder email after 24 hours but nobody logs in, the mail won't be sent.
The last one is also a problem; both because this makes it hard to see when a task fails (as the error is logged as a user problem, if at all) and because your user (again for no reason) is now looking at an error screen.
Solution
If you want to use the best practice, move all these sorts of tasks to either a Scheduled Task (under windows) or a Cronjob (under unix). This means you have a system service that periodically starts up and executes a PHP script that can do maintenance to your site, such as removing these news messages, sending out emails, or other things.
This has a number of advantages:
The server will always be there on time to run the tasks when they need to be run
You can disable timeouts and upgrade memory availability to run intensive tasks
Users will not have to wait for anything to complete
You can add special logging to the server, so that you know when these important but hidden tasks fail
Most providers allow you to set these kinds of tasks even on cheap hosting packages.
By far the simplest way to do this is to have a publisheduntil field with a date time. Each time you get a list of events to be shown on a page check this field eg
Select * from my_table where published = true and published_until > todays_date
This is avery simple way of ensuring that events disappear when they should.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I’m looking at ways to send out 1 email every 1 min. I’ve looked at the example below where the top answer is to use PHP sleep() function.
However, I’ve also found suggestions that sleep() might slow down the server.
I’m not looking for exact answers but general approaches would be great.
However, I've also found suggestions that sleep might slow down the
server.
Yes, and hitting the pause button on a movie playing on your computer will slow down the duration of the film based on the amount of time you pause the movie.
The purpose of sleep is to put a pause in your script. As described in the official PHP documentation:
Delays the program execution for the given number of seconds.
So yes, it slows down your server. But only on content or pages where sleep is active.
So if this is a fronted script with sleep in it, it slows down the ability for anyone to view content via the PHP script that uses sleep. Place it in the middle of a page where HTML is rendering with a 1 second delay & your page now takes 1 second longer to render.
If this is a backend process only you really know about or trigger, no big deal. It’s a background process anyway so it will just expectedly slow things down in that realm.
That said, let’s look at your core question which is the first sentence of your post:
I’m looking at ways to send out 1 email every 1 min.
Then what you are looking for is a cron job which is a timed job on a Unix/Linux system. An entry for a cron job for something sending mails out every minute might be something like this:
* * * * * /full/path/to/php /full/path/to/your/php/script.php
But that is superficial. It basically just triggers the script.php every minute. Then within your script.php you would have to create some core logic that would control what happens each time it’s triggered. If you are using a database, then maybe you could create a last_sent field where you sent a time stamp of the last time a mail was sent to a recipient and then you act on that. But again, the logic is based on your core needs.
But at the end of the day, I am not too clear how sleep would factor into any of this. Might be worth it to take a step back and better architect your script to fit your needs knowing what cron is, what sleep is & what they are as well are not.
It is generally done with a separated worker and a queue manager.
That's it: you have a queue manager (i.e. RabbitMQ) that a sending email worker is bound to,
Then when you need to send 10 emails you put all of them to the corresponding queue at once in the script that serves HTTP response. This step is immediate.
Then a worker reads emails one by one and sends them with required delay. This step takes some time but we don't care.
Hello fellow programmers! :)
I want to be able to set up some php script to be run after some events, triggered by user. Let's say, user creates a forum thread, that should be closed after 48 hours automatically. It is equivalent to an update to MySQL row:
UPDATE threads SET closed = '1' WHERE threads.id = 'x'.
Hence, this problem should not necessarily be solved exclusively with php.
This kind of questions pop up from time to time, but everything I found was to set up a cron job to run every 'x' amount of time, that checks if the time has come to close the thread. The problem is, that running this checks often cause higher system load than if you schedule a script to be run once at a given time. Not to forget, that there could be hundreds or even thousands of threads, each with it's own time to be closed. We can avoid checking every single thread by creating some sort of queue, for instance in MySQL, so the script selects from the DB entries with "time_to_close < NOW()" and closes these. Another drawback is, that I would like the thread to be closed exactly after 48 hours. In that case the script should be run every second and should take very little time to be executed completely.
Alternatively to cron job I think following method can also be useful:
check at every access to the Thread if it should be closed. This also causes higher load, especially if the thread is accessed very often.
So is there any efficient way to schedule a (php) script run depending on the time of a specific event? While writing this question I stumbled upon MySQL event scheduler. Together with procedures, that can provide additional flow control (close thread only if there was no activity since 48 hours) I think my idea can be implemented. I am not familiar with these functions of MySQL, so I would appreciate any help on this topic.
With best regards,
system__failure.
I know a lot of websites compensate for this kind of behavior by doing this on a per user request basis. The overhead is not that bad and your records are always displaying correctly (unless you have a design problem.) This also works because most hosts don't give you cron access. It is very rare you will need to schedule a job in php. There are a few exceptions like report generations every hour. But trying to catch user actions with cron is not a good idea.
I have a few ideas about this but here is what I need to do and just wanted some second opinions really.
I am writing a small auction site in PHP/SQL, but I have come up against a hurdle.
When an item finishes, much like eBay, I need to be able to tell that it's finished and send out the emails to who has won it and who has sold it.
The only way I can think of is to schedule a piece of code to keep checking what auctions have ended but surely there is a better way?
The solution can be in multiple parts :
A script that is launched via Cron (every 5 minutes could be good, even less...). It detects the finished auction and put them in a queue.
A script, that pretty much runs continuously, and that processes items in the queue.
Note that :
You have to ensure that an auction is still open before displaying the page ! (a simple test) That way people can't join in after it closes.
For each script, you can use PHP, or any other language
Advantages :
The cron job is very fast, low on resources, and if there are a lot of auction to process, there is no risk it will be run in parallel (and then conflicts)
The queue system ensure that your system won't crash because there is too much going on... It will process the queue as fast as possible, but if it is not fast enough, the website will continue to run. You can however end up with emails being sent hours or days after the auction is closed. But the "limit" is way more predictible, and won't crash your system.
You can extend it in the future with multithreading processing of the queue, distributed processing... This is a scalable architecture.
This architecture is fun.
Additionnal informations :
Regarding the daemon script, I doesn't have to run continuously. What you can do is : at the end of the cron job, if there are items in the queue, then it checks if the other script (processing) is running. If yes then exit. If the other script is not running, it launches it...
The daemon script gets an item out of the queue and process it. At the end, if there are still items in the queue, it processes it, else, it exits.
With this system, everything is optimal and everyone loves each other !
To check if the other script is running, you can use a file and write in it "1" or "0" (= running / not running). The first script reads it, the second writes it... You can also use the database to do it. Or you can maybe use system tools or shell command...
Please be kind to share the SQL script that query the highest bidder based on the bidding end date (how to know the bidding is over) and award the product to the highest bidder
I would setup a cron job to run every 10-20-30-60 minutes etc to send out emails and update the auction details.
If you're script is fast, running it every minute or so may be alright.
Be aware that many shared hosting will only allow you to send out a certain number of emails per hour.
Do these emails need to be sent out instantly?,
I can see 2 possible problems and goals you are trying to achive:
Visual: You want that when a user browse your website, without updating or refreshing the page, it keeps updating the page so that if an audition ends, it appears something like "Audition ended, the item goes to...".
Solution: You should use Javascript and AJAX. (I assume you are already using it for countdowns or something). Make an AJAX call every 5 seconds (could be enough) and update the content.
Pratical: You want that if an audition is ended an user cannot join it. Solution: You can do it just with PHP and mysql. You could create a fields where you store the audition start timestamp and then make a simple if (time() >= ($timestamp + $duration)) {} (where $timestamp is the start of the audition and $duration is the duration of the audition) to block possible bad users trying to do it.