I am working on a blueprint to send and Receive email messages.I looking for best practice.
We have to send , for example 1000 messages on click of a button. How to Use SNS and SQS to send messages right away? Is it a good idea to create one SNS request to create 1000 message in a queue?
Is there any way to schedule 1000 messages ?
We receive messages, whats the best way to use SNS and SQS to process the messages?
It would probably be best to put 1 message on a queue on the button click so the user doesn't have to wait. Then have a different process that take messages off the queue and sends the emails or queues up another 1000 messages, one for each email.
Don't think of it as sending 1000 emails with the click of a button, thinking of it as scheduling an email job.
Don't know all that you are trying to do, but you probably also don't want to do a 1000 message load while the user is waiting for a response after a button click.
I would consider, have 2 SQS queues (at least), the first queue is used to schedule the batch job, on a mouse click you insert a single job, i.e. 'send this email to 1000 people' as Travis R suggested.
Another job could poll Q1, see the 'send these 1000 people this email, and it could then create 1000 SQS messages into Q2, one per email address.
A third process (perhaps multiple of them), would watch Q2 and send a single email from the list, and delete the message. Using this method you could throttle you send rate down quite a bit to adapt to your ISP's restrictions, either with the SQS settings, or by limiting how often you 'email a single email' job is schedule to run.
Also consider using amazons SES and setting up another process to monitor bounces and complaints - which you are bound to have if you are kicking off 1000+ emails at a time.
Related
I am developing a Laravel application that uses ajax in making requests. This application needs to send bulk emails. I have read the documentation on Queues and understand basically that it'll help run jobs at the background. What practice do you think will be best?
Should I send the emails via an ajax request while not freezing the user?
Should I use Laravel Queues alone?
Should I combine both?
My current thought:
If I use just Queues I might face difficulties updating the user on the progress of the task.
I recommend to use queues whenever you are performing bulk actions like sending emails, sms, notifications, etc. This will better your user experience since they won't have to wait (freeze screen) and move onto other tasks.
Should I send the emails via an ajax request while not freezing the user? Yes
If I use just Queues I might face difficulties updating the user on the progress of the task.
That is correct. You won't be able to show progress. Rather you can show appropriate message to the user like 'emails sent successfully' and let the queue jobs do the rest of the tasks for you.
I have a question
I have a simple system where a User creates a Booking
After a user has created a booking, an email and a SMS will be sent to the user.
I have though of multiple ways to approach this
After the booking was made, send the email and the SMS(blocking)
After the booking was made, Fire an event that is queueable and has two listeners. One sends the email. The second one sends the SMS.
After the booking was made, Fire and event, that will queue 2 jobs, one that sends the email, the second one sends the sms.
Option 1 is a no-go since it is blocking and doesn't have a retry to my Transactional Email provider or SMS provider.
Option 2 is do-able
Option 3 is do-able
In the future I want to be able to send the same email/sms without the BookingConfirmation event, so I will eventually need jobs that can be queue'd without the event.
Or is the event part just overhead and can I queue the jobs directly from the Controller instead of firing the event?
I am basically asking if I should use the Events at all or just use Jobs?
Thanks!
Events are useful because you can fire one event and have multiple action independently.
That means if in the future you want to have a third action, if you use events you can just add another event callback, meanwhile if you skip the events and directly put the jobs into the queue, you have to modify the controller to put the third action.
Convenience lies in the fact you don't have to alter any method in order to add new functionality.
In the end of the day, it's up to the developer. Events provide better code between each action. But performance wise, I think it is the same.
I would put all the tasks as Jobs in a queue, you then fire events (last line of job) when the Job is done, so you can do any follow up actions or things you want to do to inform the use.
So then you can have a job for each task and keep them simple, and also less exceptions or errors to track on each one.
The system I am using at the moment for example uses the event to trigger the next job, but other tasks I fire multiple jobs at once.
Eg.
user clicks booked.
dispatch job email
job email is queued
Job email is processed
Job email sends event
Event picked up by listener
Listener dispatch Job SMS
job SMS is queued
Job SMS is processed
Job SMS sends event
Event picked up by listener
anything else you want to do...
might sound a lot of steps but all are short and then simple to follow and update.
I am working on a web-application to make an SMS Reminder service, which takes various inputs from the user, like the user's name, his number, and the time he wants the reminder. The reminder is then sent through an SMS. I have the SMS Gateway part figured out for which I am using Zeep Mobile's API. I wanted to know how I can send an SMS on the time input by the user.
The database would have the user-id and the time, and I need to get my application to send an sms at the time. Any tutorials on similar lines would be great help.
Thanks in advance
Let's say your reminder interval is 1 minute and you are deploying on Linux.
1) Set up a cron-job to check your database every minute for possible reminders
2) If there are reminders to be sent, execute your sending script.
3) Mark sent reminders with a status (or similar) so you don't send them again.
Depending on what server your application is deployed, you would need to run some kind of service that would run continuously and send SMS whenever one is due.
One note though, since repeated database access is costly, you may like to load up Reminders that would need to be sent in a short time. The database should be accesses periodically with a reasonable time interval.
I'm coding a team collaboration web app in PHP, and I have a few events that users get notified about through email and/or SMS. The current way I'm doing it is as follows:
Every user has his notification settings in the database as boolean variables.
Say users would be notified when someone comments on the team's page. When the function that posts a comment is called, the same function would contain extra code that checks "who wants to be notified about this?" and then sends notifications to them (which slows down the function a bit).
Is there a more efficient/faster/flexible way to setup notifications? maybe through a script that runs via a cron job? or shall I just keep doing it this way?
I appreciate your help.
I implemented a similar method to the one you're following on a website with a multi-table approach. The users table held the contact information along with opt-in, opt-out options while an event table held the instructions to notify. Several other events were hard coded because of their importance. The thing that set the site apart a bit was a "workflow" area on the user's dashboard that also showed the user what action items they had. We found that most users ignored the emails and dealt directly with that dashboard workflow area. You'd be surprised how many times people change emails or just ignore them altogether.
With 280,000 users and daily visits in the tens of thousands, there was no performance issue noticed. However, the process of queuing emails can be inefficient if you're not careful, so take particular time to benchmark your mail sending functions--its as easy as echoing out microtime before and after the mail send is accomplished--to evaluate its effectiveness. On my current company's site, such improvements yielded a 800% reduction of email queuing time (queuing being the process of generating the emails and submitting them via php mailer to the mail system for distribution)
I'd say have a table that is a queue of notifications. Let the function that post a comment still check "who wants to be notified about this?" but then just log entries containing the messages in this table. Then have a separate process work from the queue i.e. your cron job suggestion.
Depending on your database you may perhaps make use of database events or triggers instead of a cron job. This however have the requirement that your database allow you to put code in your database that will send the SMS or Email. This poses a security risk normally which you may or may not be concerned about in your setup.
Is there any way i can send out about 3000+ emails from one php script request without overloading a dedicated IP... the max would be 500 per hour?
If you dont get me.. here is detailed :)
I can only send out 500 emails via the mail() function in PHP per hour via my dedicated IP, is there any way i could send out for example 3000 rows of emails pulled from an email address but stagger the mail() functions out for 500 per hour...
Thanks already!
Create 2 tables, one for the email message and one for the list of recipients.
Then create a script to be run by cron that checks if there is a new message in the message table and if so sends a batch of email to the next set of recipients. Marking each recipient after the mail is sent.
Then you create a web interface for your client to create a message and attach recipients to the message once the user marks the message as ready to go your cron job picks it up and processes it.
If there aren't any messages to be sent your cron job doesn't do anything.
You could sleep between the calls, or, if they're already in a database, put a field in there that says when they were sent. Then you select the ones that haven't been sent, and start from there.
I would put a field in the DB to show when the last email was sent to each user and what email it was. I would also have another DB table to show each email you sent and if it has been sent to all users yet.
User Table:
Id, UserName, Email, etc, DateTimeOfLastEmail, LastEmailId
Email Table:
Id, EmailSubject, EmailContent, DateTimeSent, SentToAll(True/False), DateTimeOfFinish
Thanks for all the answers!
The best way i found was actually to simply sleep() between calls using the sleep() as i tested 400 mails, this took 17 seconds :)
It is unlikely the user will send more than the 450 limit ... but if they do i have an if statement before the while() ends checking if there are more than 450 rows, if so it will sleep between each... this works without fiddly databases :)
Thanks!
well after doing some math you could send an email every 8.3 seconds (498/hr) but it doesn't solve the problem. I think another approach would be to use a DB, query for the 500 and have a CRON job run the script every hour.
So in the DB table you could have the script update a field after the email has been sent so the next cron job will query and get the next 500 emails that need to be sent.