I am having trouble retrieving all messages through the Gmail API PHP Library. I use listUsersThreads to retrieve all threads to do either a full or partial mailbox sync on a user's account. The initial full sync processes and returns the message ids I need, which I then use to store mail meta headers (from, to, date, subject). A subsequent call using listUserHistory from the last history id, allows me to do a partial sync to only retrieve the latest messages. From the data I have stored, I then display a full message conversation log between two parties, ordered by date to show the conversation. Clicking the message will then query the API to retrieve email body, which I then display.
The issue is that based on the messages I have saved, looking at the MIME content, there are messages in the MIME body, that I do not have on my database. I then also tried to query the API using a search query, and still there are missing messages, not returned by the API.
A previous developer used mimecast to get the messages, and querying that database does in fact return the messages that I'm missing.
How is the Google Gmail API not giving me all messages between sender and receiver? The MIME body clearly shows messages that are not available when querying the API, and I don't understand why, or how to find the missing messages.
Any assistance would be appreciated.
So, for in case this issue comes up for anyone else, I believe that it has something to do with expired history items. I stand to be corrected, as this can only be proved after I've had my implementation running for more than two weeks.
If you're considering running a mailbox sync, there's a good chance that you'll be missing messages, especially if those messages were sent from a client other than Inbox or Gmail. History items are kept for two weeks on average, so by syncing a mailbox, you'll be syncing everything from when the account was activated, but expired history items will not be available.
In theory, this means that you should have the full email conversation while a partial sync is executed. You should have all the MIME headers that you need as and when the communication takes place, provided that, like me, you have push notifications enabled through the Google Cloud Services console to alert your systems to run a partial sync process on any given account.
If your partial sync is executed manually, or possibly through CRON on an interval, rather than through push notifications, you'll need to make sure that the interval is configured to sync while all history items are still available and accessible.
The downside of this, though, is that even if you do have trace of all communication with their message IDs, a lookup on an expired message to retrieve email body will fail with a 404 status code, and you will not be able to retrieve email body contents for some messages.
Thus, if your processes relies heavily on what's in the body of the email, you should also store body content locally during a partial sync I really only need the MIME headers, although I do lookup message contents when needed, but it won't cause major problems for me if I was unable to retrieve the body of any given message.
I should be able to confirm this theory within a month from now, so if you think my theory is incorrect, please feel free to make me the wiser. :)
Related
I build an online shop when my customers buy something from the shop, when they reach checkout, I use hyperpay API, so when they pay by visa, its a request from my server to hyperpay server then I have the result if it the success I send an email message to customer and store payment and products data in another server with api.
not always but sometimes, my server stop after sending emails, or stop before sending data to the second server, or I get a response from hyperpay and the operation cut, I don't know why.
So I think it's a time-out problem, but I am still not sure.
and another problem, my jquery scripts response take a long time.
so I want to know, the problem with my codes or from my server?
There could numerous reason behind for that to happen. As per your info, it looks like the time to send the email is taking longer or sometime it halts. Then you can do this things step by step:
Check laravel.log inside storage directory to check if any relevant information.
If the SMTP or whatever means of connection you are using to send the email needs to debugged.
You should send heavy tasks such as email sending to queue: https://laravel.com/docs/8.x/mail#queueing-mail so the other tasks can be executed without any problem while the SMTP call is being made.
If none of this help, you should look into your both web server and mail server
I have a PHP web app with a section to send invoice by email to all partners.
Each partner could have more than 1 email.
For example: COMPANY-1 with emails: email_1#company1.com and email_2#company1.com
I send all with phpmailer.
I save into my database the send-response from server mail ( in this case gmail).
If sent, i'll save true, else false - foreach email.
But some emails sometimes are incorrect (due to error typing when insert) or are dismissed by the company (in this case the company do not inform us and we still send to this deleted email) and I like to retrieve these "problems" using web app.
(If I login into google webmail I can see the email returned for each sent to an not existing email, with some advice/error code inside... but i don't know how to retrieve using PHP)
I tried using
$inbox = imap_open($hostname,$username,$password);
$emails = imap_search($inbox,'ALL');
This script works, but using ALL i get timeout.
I don't know the best method to retrieve these email of "NOT DELIVERED EMAILS FOR SOME PROBLEMS".
My first ideas was to set a custom Message-ID in the header when I send:
$messageID = "<" . md5($_SERVER['HTTP_HOST'].(idate("U")-1000000000).uniqid()).'-'.idate("U") . '#' . $_SERVER['HTTP_HOST'] . '>';
$mail->MessageID = $messageID;
and then in my script, using a command like:
$messageID = retrieve_from_database(...);
$emails = imap_search($inbox,'HEADER ' . $messageID);
but it doesn't exist a query HEADER
I'd like in my web app to show, for each email sent, eventually problem of bounced or undelivered but I don't know how and what I need to "ask" to imap_search to retrieve responses associated to the original sent email.
I read criterias list from docs, but I don't find anything usefull I think: https://www.php.net/manual/en/function.imap-search.php
Example of a scenario I need:
I send an email with Message-ID: <test#testXYZ> from my_email#mywebapp.com to email_not_existing#test.com. The server response true cause sent is done.
I insert a new entry on my database table emails_sent with send result true: INSERT INTO emails_sent (ID, email_dest, sent, MessageID, when) VALUES (455, 'email_not_existing#test.com', TRUE, $messageID, '2019-05-30 8:00:00')
The gmail daemon reply to my_email#mywebapp.com with a message of undelivered email
In my web app, when I go into page of sent emails, foreach ID from emails_sent table, i need to search on imap eventually response email with errors. In this example case when I ask for ID 455, I need to search on imap_search messages replyes associated with my email sent with MessageID assigned by me (or other method i don't know....)
I would advise you to get away from shared hosting and get a proper server (try scaleway.com). As you're discovering, it means you don't get sufficient control.
I'd recommend against using IMAP as your database - as you're finding it's not good enough for the kind of searching you want to do. If you import all the bounced messages into your own DB, you're free to parse and search in whatever way you like, and it's also easy to get all messages from IMAP.
The most efficient way to handle bounces is to get your SMTP server to pipe bounce messages directly into a script. This will give you the complete bounced message, along with environment variables that tell you more about the delivery. That of course requires that you have config-level access to your mail server, or a hosting provider with a clue.
Be warned that bounce handling is an exceedingly unpleasant task, and is very difficult to do well. You may want to look at a 3rd-party bounce handler like Boogie Tools bounce studio to save you some sanity! Aside from some very broad top-level standards, there is very little consistency at a lower level, and bounces are often useless. For example, MS Exchange can produce bounces that do not contain the bounced message's original target address at all! The way around this is to use VERP, where every address (or even every message) gets its own unique return path, allowing you to know where a message bounced from even if the message content contains no clues.
I'm creating an app which will allow all users to initiate conversation with page owner using Chat-to-SMS service.
Problem is that I'm not sure how to track conversations since there would be one conversation chain from SMS Provider <-> page owner. When owner clicks reply, it needs to know which session user it needs to send the message to.
Basically, I need to do opposite what is being explained here: https://www.twilio.com/docs/quickstart/php/sms/tracking-conversations
How should I accomplish this? I can't quite wrap my head around this.
Twilio developer evangelist here.
If you are sending all the chat messages to just one SMS number then there is no easy way to tie replies back to the original message. (As a quick experiment, if you open an SMS conversation in your own phone and try to reply to any message that wasn't the latest one, you'll see it's not possible.)
There are workarounds though.
You could, when forwarding the message, generate an ID for it. Then get your page owner to include that ID when responding to that message, that way you can route the message back to the original sender and strip out the ID.
Alternatively, when replying you could always respond to the last message that came in. This relies on there not being much traffic, allowing the page owner to respond before the next message arrives. This is error prone though.
Another alternative is purchasing a new number for each new conversation. You could expire the number after a predetermined amount of time. This is made much easier with Twilio Proxy, which was announced recently and is currently in preview.
Let me know if that helps at all.
Hi I need to build a dashboard which shows stats for our automation campaigns such as sends, opens, bounces etc.
The Mailchimp api only returns for the entire period the automation has been active. I want to see the stats for a particulier date, however I can't find an API method that accepts a date range or groups results per date. The lists/activity method is exactly what I need, but it doesn't include e-mails sent using an automation :(
I have emailed the Mailchimp support team and they recommend me to use the /reports/email-activity method. This returns a long list of all open and bounce activities for an automation. I have tried to use this, but this is also not a good solution. This means I would have to store thousands of events in MySQL and run complicated group by and where queries to get my data. Furthermore this only includes the open, click and bounce events, thus this data is worthless without knowing how much e-mails were sent on the date of an event because you can't calculate open rates.
Does anyone know a way the retrieve stats for just one date? What I am looking for is a Mailchimp version of Mandrill method: https://mandrillapp.com/api/docs/messages.JSON.html#method=search-time-series.
Thank you for you time and help
for the campaigns exist "Response body parameters"
before_send_time
since_send_time
campaigns?since_send_time=2016-02-21T15:41:36+00:00&before_send_time=2017-03-10T15:41:36+00:00
I am using the Esendex Rest API http://developers.esendex.com/APIs/REST-API/inbox and I am trying to get all the messages in the inbox from a particular phone number.
This is basically so if I do a send out of SMS's and someone replies I am able to retrieve the reply.
I can currently get all the messages in the inbox into an array and sort them, but this is too slow with over 4000 messages.
Does anyone know if I can filter by phone number, or at least date-received in my API requests? Even if I could get the last weeks worth of SMS's.
You can try to use the conversation API if you want to filter by a phone number. The drawback is that you can get only the latest 15 messages, also that there is no reliable way to detect if a reply has been received from a phone number other than the inbox API. Apart from this I wasn't able to find anything that can help you. Their API is very restrictive and minimal.
There's currently no way to do this via the REST API but I'll see if we can add it over the next couple of weeks (I work for Esendex). As you note the functionality already exists in the Conversations API albeit in a more minimal form.
In the meantime you could achieve this by checking all messages that arrive either by polling the Inbox or using Push Notifications.