we send messages with Amazon SES using SES API. Our send rate is now 90 messages per second. But we are getting throttling exception even when we don't reach this limit but just trying to approach it.
Right now we can steadily send at the rate of 30 messages per second. The question is how to send faster.
Let me dive into some more details and clarify the question.
It might take from 0.3 to 3 seconds for a single API send request to complete. This is why if we would send messages sequentially we would hardly get 1 message per second speed.
Luckily we can send messages in parallel and this is what we are doing.
For each thread we check that it does not send more then allowed number of messages per second. For example if we have 40 threads then we don't allow each thread to send more then 2 messages per second. Yes, this is not optimal.
We register every message send and the time when the API request has completed (when we got the response from API). This allows to get some statistics.
When we restrict the sending limit to be less then allowed limit (like 60 instead of 90) everything works fine.
When we try to send at maximum limit we start getting throttling errors. Like when have speed of 80 requests per second reached we start getting exceptions.
This allows me to put the questions:
Q: How to send messages with highest allowed speed?
Let's start with another question - 'How does SES calculate the number of messages to check send rate?'
Let me guess. When we submit a new request they look at the number of requests submitted during last second from current moment and if this number is less then our limit the request is accepted.
But wait. If we have 40 threads and each thread can not send more then 2 messages per second then we can never reach the limit. But we do get the exceptions.
Research
There is a great blog post on Amazon SES blog about handling the limits. We are trying to adopt this approach but have not succeed yet.
We are using PHP in our application and PHP SES SDK.
I guess this is quite common task but for some reason I'm not lucky to find the complete solution.
Any help or ideas would be greatly appreciated. Thank you.
The key take away is:
A variety of factors can affect your send rate, e.g. message size,
network performance or Amazon SES availability.
Based of what you've said it seems like you're using a bit of fuzzy logic to try and calculate how many messages you're sending. This won't be perfect so if your AWS limit is 90p/s then getting setting your code lower, e/g to 60p/s makes sense (once again this all depends on how accurate your estimates are).
You should consider the other approaches as you've mentioned, such as "Exponential backoff" as described in the link you provided.
Another thing you can consider is taking advantage of a queue, like SQS. This way you can pick tasks off the list as quick as possible, and if you're a little too quick you can always back off and then jump back on the queue as soon as possible.
Related
I have two bots on my server but since 4 days ago both of them stopped working.
I checked the script on the other server and I'm pretty sure there is nothing wrong with it. also I talked to my hosting provider and seems that there is nothing wrong with it too.
What's going wrong?
Update : "I'm using webhooks"
When sending messages inside a particular chat, avoid sending more than one message per second. We may allow short bursts that go over this limit, but eventually you'll begin receiving 429 errors.
If you're sending bulk notifications to multiple users, the API will not allow more than 30 messages per second or so. Consider spreading out notifications over large intervals of 8—12 hours for best results.
Also note that your bot will not be able to send more than 20 messages per minute to the same group.
I am using some API which is free.
I am using PHP script which is using fopen to download JSON from API.
When I make to many requests(eg. 2 requests every minute) API is blocking my PHP server IP.
Is there a way to solve it and possibility to make more requests (I don't want to DDoS attack)?
Is there better solution than use of many PHP servers with different IP's?
This is a quite abstract question as we don't know the actual api you are talking about.
But, usually, if an api implement a rate limit, it shows this kind of header in it's answer:
X-Rate-Limit-Limit: the rate limit ceiling for that given request
X-Rate-Limit-Remaining: the number of requests left for the 15 minute window
X-Rate-Limit-Reset: the remaining window before the rate limit resets in UTC epoch seconds
Please check the docs (this one is from twitter, https://dev.twitter.com/rest/public/rate-limiting).
I am using the Geocoding API, and am receiving OVER_QUERY_LIMIT and I have enabled my billing account, which should give me over 100k queries. I am doing about 2500 or less. It seems to happen when I am processing many items in a php loop, but not for each item - for example.
OK
OK
OVER_QUERY_LIMIT
OK
OK
So it doesn't appear I am actually over the limit, but that's the XML returned for the transaction. If I process the same transaction in a URL it works with no issue.
Ideas?
Pace your application or submit the requests in smaller groups. Solutions include using a cron job to distribute the requests throughout the day, or adding a small delay between requests.
From: https://developers.google.com/maps/documentation/business/articles/usage_limits
If you exceed the usage limits you will get an OVER_QUERY_LIMIT status
code as a response.
This means that the web service will stop providing normal responses
and switch to returning only status code OVER_QUERY_LIMIT until more
usage is allowed again. This can happen:
Within a few seconds, if the error was received because your
application sent too many requests per second.
I'm writing an application for processing large amounts of google analytics data at once, and I keep bumping on the
(403) Quota Error: User Rate Limit Exceeded
error.
I have done some research and I've found out that while the limit is 10queries/second/user,
it defaults to 1. So I adjusted it to 10 in the Google console, but without any luck.
I've also added sleep(0.5) between every other call I make, which would make it impossible for 10 requests to be done in 1 second, but also without any luck.
This seems very weird to me, and that's why I'm wondering if it might be possible that 1 call with multiple dimensions/metrics/sort filters, might be treated as multiple requests?
Edit: I've also looked into the UserIp and the quotaUser standard query parameters, but I'm unsure how I can add those to my request (I'm using the API to make the calls:
$analytics->data_ga->get($query);
). If I understand correctly, these parameters can be used to split your quota over the users you're querying data for. In my case that won't be helpful at all (Correct me if I'm wrong), because the problem is I'm hitting the per second-cap, and I'm not querying for more than one user in the same second.
Any help would be greatly appreciated
You are correct that userid won't affect the 10 per second quota. In my earlier work, I tried the approach of adding a sleep between calls but learned the hard way that that isn't the correct solution. The correct solution is to look for the quota error and when found then add the sleep call. And if you still get a quota error then add a larger sleep call. I try three times in my code. There are other protocol errors - "backend error" I think - where Google's advice is to just try again.
I have a web application where users can create topics and also comment on other topics (similar to what we have here on stackoverflow). I want to be able to send notifications to participating users of a discussion.
I know the easiest way to go about it is to hook the notification to the script executed when a user interacts with a discussion. In as much as that seems very easy, I believe its not the most appropriate way as the user will need to wait till all the emails notifications (notification script finishes execution) are sent till he gets the status of his action.
Another alternative I know of is to schedule the execution of the notification script using cronjob. In order for the notification to be relevant, the script will be scheduled to execute every 3 to 7 minutes so as to make sure the users get notification in a reasonable time.
Now my concern is, will setting cronjob to run a script every 3 minutes consume reasonable system resource putting into consideration my application is still running on a shared hosting platform?
Also, am thinking is it possible to have a scenario where by the comment script will trigger or notify a notification script to send notifications to specified email addresses while the comment script continues it's execution without having to wait for the completion of the notification script. If this can be achievable, then I think it will be the best choice for me.
Thank you very much for your time.
Unless your notification script is enormously resource-intensive and sends dozens or hundreds of messages out on each run, I would not worry about scheduling it every 3-7min on a shared host. Indeed, if you scheduled it for 3 minutes and found performance sagging on your site, then increase it to 4min for a 25% reduction in resources. It's pretty unlikely to be a problem though.
As far as starting a background process, you can achieve that with a system call to exec(). I would direct you to this question for an excellent answer.
IMO adding a "hook" to each "discussion interaction" is by far the cleanest approach, and one trick to avoid making users wait is to send back a Content-Length header in the HTTP response. Well-behaved HTTP clients are supposed to read the specified number of octets and then close the connection, so if you send back your "status" response with the proper Content-Length HTTP header (and set ignore_user_abort) then the end user won't notice that your server-side script actually continues on its merry way, generating email notifcations (perhaps even for several minutes) before exiting.