I have a web application where users can create topics and also comment on other topics (similar to what we have here on stackoverflow). I want to be able to send notifications to participating users of a discussion.
I know the easiest way to go about it is to hook the notification to the script executed when a user interacts with a discussion. In as much as that seems very easy, I believe its not the most appropriate way as the user will need to wait till all the emails notifications (notification script finishes execution) are sent till he gets the status of his action.
Another alternative I know of is to schedule the execution of the notification script using cronjob. In order for the notification to be relevant, the script will be scheduled to execute every 3 to 7 minutes so as to make sure the users get notification in a reasonable time.
Now my concern is, will setting cronjob to run a script every 3 minutes consume reasonable system resource putting into consideration my application is still running on a shared hosting platform?
Also, am thinking is it possible to have a scenario where by the comment script will trigger or notify a notification script to send notifications to specified email addresses while the comment script continues it's execution without having to wait for the completion of the notification script. If this can be achievable, then I think it will be the best choice for me.
Thank you very much for your time.
Unless your notification script is enormously resource-intensive and sends dozens or hundreds of messages out on each run, I would not worry about scheduling it every 3-7min on a shared host. Indeed, if you scheduled it for 3 minutes and found performance sagging on your site, then increase it to 4min for a 25% reduction in resources. It's pretty unlikely to be a problem though.
As far as starting a background process, you can achieve that with a system call to exec(). I would direct you to this question for an excellent answer.
IMO adding a "hook" to each "discussion interaction" is by far the cleanest approach, and one trick to avoid making users wait is to send back a Content-Length header in the HTTP response. Well-behaved HTTP clients are supposed to read the specified number of octets and then close the connection, so if you send back your "status" response with the proper Content-Length HTTP header (and set ignore_user_abort) then the end user won't notice that your server-side script actually continues on its merry way, generating email notifcations (perhaps even for several minutes) before exiting.
Related
I am using the PHPMailer library to handle the sending of emails from within my application.
The problem is, when some emails are triggered to be sent (such as when a contact form has been submitted, a new user registers, etc), it could take 1-3 seconds for the page to load while the email is sending. If there is ever a problem sending the mail, the delay can be more.
I was thinking of saving any emails that need to be sent into a pending_emails table in my database, then just have a cron job ran every minute which would send out all those emails, then remove them from the table.
My question is, does this seem like a logical thing to do? Are there any potential resource concerns I should have with a cron job running every minute vs sending the email in runtime? (I need to run the cron job often, as someone may be waiting on an urgent message, for example "reset password" email)
You got everything right already.
Sending at runtime, just when you respond to the user's HTTP request, is the easiest thing to do. But the response is slowed down a bit by this, of course. That's not too bad in a small application, because sending email is faster than one might think. It definitely works.
Implementing a message queue is the more elegant and scalable approach, of course. But it takes a little more work. Your idea of using a pending_emails database table is totally valid. There are libraries and components for such queues, but you don't have to use them.
This is a very opinion based question so you're going to get a lot of different, conflicting answers because there are some who might tell you its ok to make a user wait 1-3 seconds since its not that long but I tend to disagree with that. What I typically do instead, however, is use a Queue.
There are ways to create a queue WITHOUT using 3rd party software, but there are some excellent tools out there such as RabbitMQ, Iron.io or Beanstalkd which can be extremely helpful to performing tasks in the background. These services push your task into a queue and these items in the queue are processed in a timely manner in the background, but the user gets an almost immediate response (depending on what you're doing). This is how I usually handle more resource intensive tasks, like sending an email, in the background to avoid holding up the response to a user.
Best of luck.
Look into threading (PHP Threading). I would suggest you create a new thread which invokes the sending of the email. This way, you can return a response to the user without waiting for the email to be sent, and the email sending process would run in parallel in another thread.
My script sends notification emails of a new comment, this could be to 50 members and 50 emails need to be sent, which could take 20 seconds which is way to long for the user to wait! What's the best way in PHP to do this, is there a way to do asynchronously?
A simple way might be to store the necessary information (email addresses, content) in a database and have a batch process run every minute or so with a cron job. The batch process can query the database for pending emails and, if any are to be sent, go through them and then delete the database entries.
is there a way to do asynchronously?
Yes, there is!
exec('wget PATH_TO_YOUR_SCRIPT_THAT_SENDS_THE_NEWSLETTER > /dev/null &');
Note that the database alternative is a pretty good one too. But this should work too if you're on Linux (and doesn't require a database).
I'd use something like RabbitMQ. Your website acts like a producer sending the email requests to Rabbit; then have a consumer running that processes the requests from Rabbit.
Advantages - if your consumer falls over then when you restart it will pickup from where it left off (last acknowledged request).
Indeed it can be done asynchronously.
The simplest way is to insert the email data into a database rather than actually running the emails, and then have a cron job that periodically actually sends the emails.
There are of course other ways too, but that will probably be the most straight forward.
You can use cURL POST to start an asynchronous script. Set the timeout to a short period so your script can resume after the POST request has been made. You can set the email information in the POST request or store it in a data base table.
Now that all of the browsers I like have almost full support for Server Sent Events, I wanted to try implementing it on a site I've been putting off because I hate polling. But I have initial hesitation that I was hoping I could get some help on.
Here is my use case:
User goes to a form, something time-based and competitive, in this case class registration. All things being equal, they have a list of about 30 - 40 classes they are eligible for, and in order to minimize instances of "she logged in first but he hit save first but he didn't mean to hit save but she already chose another class" etc, I want to make the form real-time, so that when someone selects an option, it goes straight into the db and anyone else viewing the form sees that it is filling up. (I'll deal with the stress of people changing their minds later).
So, in a polling scenario, I had to deal with the AJAX calls having to check on the status of 40 spots and update them and setting an interval that could potentially still create collisions.
But with Server Sent Events, I can have the listener get just the spots that need updating, which seems better, but here's where I get stuck:
Is there any risk of the listener getting overloaded? Let's say the script sends 15 messages, back-to-back, about a status change. I see vague mentions of how user agents should handle queued tasks, but it's not clear if that's for establishing a connection or handling server-sent messages
Is this basically just passing the burden of polling from the browser to the server? Does the script have to check the DB every second for changes? Is there any way for the script to be aware or notified when change has occurred? Let's assume that seat requests are sent to requests.php via ajax and that updates.php pushes events back to the browser. Is there a standard and/or clever way for updates to idle until requests has made a commit?
The only solution I can think of is for requests.php to write the committed changes to a flat file (commits.xml perhaps) and updates.php just polls the file size every half-second, thereby keeping the workload to a minimum.
Any better/smarter/more obvious solutions out there?
Polling your database for changes is not a good idea. Instead, you should do inter-process PUB/SUB on the server. To do that, you can use a message queue like RabbitMQ, ZeroMQ or Redis PUB/SUB.
In my application, the user needs to register through a form, where I have to send three mails and do some other (huge) database checks. It takes a lot of time, is it possible to make the whole task as background process or some other alternates is there?
If your database activities take too long then you need to rethink your design. However if the delay is due to emails, then just store the emails in DB or in files. Create a cron job that sends out these queued emails every 5/10/15 minutes(and then delete them).
maybe you can once a user is registered flag him as pending in your database.
Then you could defer the work in a python or php routine running in the background continuously who would look for any pending request, do the check, send the emails and finally update the database accordingly.
the user during this time would be in a registered but pending status, but at least from a visitor point of view, he is not stuck waiting for everything to be processesed.
You cannot make a PHP script that has been started over a webserver process a background process.
I would check if I can optimize the database (probably, you have insufficient indizes), and if that doesn't fly, build a second process that gets started regularily (maybe once every five minutes or so) on the CLI side with a cronjob, showing the user a "Thank you for your registration" page...
As per my comment elsewhere, spawning a long running process from PHP is a practical solution bearing in mind a few caveats if the performance problems are unavoidable.
However "send 3 mails" should not take an appreciable amount of time (I don't know what the database checks are). You need to spend some time looking at optimizing the existing process.
Other ways to solve the problem would be conventional batch processing, offloading the heavy lifting to a multi-process/multi-threaded daemon via a network call or asynchronous messaging system, or even a single threaded job processor using a message queue.
I currently have an IRC bot written in C++ which monitors a page written in php for changes and then outputs these changes to the IRC channel.
However the current method is rather in-effective as it just constantly polls the page once every 10 seconds and compares it to the last seen version to check if anything has changed.
I can decrease the page check interval to about 2-3 seconds before the IRC bot starts to take a performance hit, however this isn't ideal.
Often the page I am monitoring can change multiple times within the 10 second period, so a change could be missed, what would be a better method to get the data from the page? considering I control both the page written in PHP, and the IRC bot, but they are on different servers.
The sole purpose of this page is to pass data to the IRC bot, so it could be completely re-implemented as something else if that would be a better solution; the IRC bot also monitors multiple versions of this page to check for different things.
If the data generated by PHP isn't somehow pushed on a stream (broadcast or feed), you don't have any other choice than polling the page, unfortunately.
What you could do is push the data from PHP using broadcast, or make a persistent connection from the bot to the PHP script, or make the PHP calculate the differences itself.
The PHP script should send a message to a public port or path that your IRB bot listens on, containing information about any posts made. This way, you are notified only when a message arrives.
One note about doing these sorts of things, beware if there are a lot of posts within a short period; if concurrency is important, you'll want to implement this using a proper MQ service like 0MQ/RabbitMQ/InsertMQFrameworkNameHere to ensure the messages arrive in order and are guaranteed sending and receiving.
If you need to monitor every change, then have your PHP page "push" data to your bot rather than your IRC bot "pull" data from the page (through polling). This can be done over any network socket, even something like a HTTP POST request from your PHP page to your bot over port 80.
A good alternative to polling is Comet. Here are examples (for JavaScript though): http://www.zeitoun.net/articles/comet_and_php/start.
I would suggest this approach:
when you retrieve your page, specify a very long timeout, say 10 minutes (bear with me for a moment);
if you have a new page, let the server return it; otherwise just don't send a reply
if there is no page, the client will wait for up to 10 minutes before giving up (timing out); but, if during this time a new page is there, your server can reply to the request and pass the page to the client;
in case the timeout fires, you simply send another request with the same long timeout.
Hope I could explain it clearly. The only tricky point is how your web page (PHP) can hold the wait when a request arrives if there is no new data to send back.
This can be easily accomplished like this:
if ($newDataAvailable) {
file_put_contents($data, $request_uri);
return;
}
while (!$newDataAvailable) {
usleep(10000);
$newDataAvailable = <check_for_data>;
}
//-- here data is available
<build response using get_file_contents($uri)>
<send response>