POSTFIX/sendmail suddenly adding extra line breaks - php

Over the weekend our MTA (POSTFIX) suddenly started prepending line breaks to the boundaries of our messages.
We have several PHP templates that define multipart/alternative messages and define the headers.
Here is the PHP mailer format, which was working friday, then just suddenly stopped on monday.
$headers = "From: name <our#example.com>\r\n" .
"Reply-To: name <our#example.com>\r\n" .
"MIME-Version: 1.0\r\n" .
"Content-Type: multipart/alternative; boundary=\"09127kjhd821\"";
$txt = "\r\n\r\n--09127kjhd821\r\n" .
"Content-Type: text/plain; charset=UTF-8\r\n" .
"Content-Transfer-Encoding: quoted-printable\r\n\r\n" .
"Text Message";
$html = "\r\n\r\n--09127kjhd821\r\n".
"Content-Type: text/html; charset=UTF-8\r\n" .
"Content-Transfer-Encoding: base64\r\n\r\n" .
chunk_split( base64_encode( "HTML Message") );
$body = $txt . $html . "\r\n\r\n--09127kjhd821--";
mail(
"someone#example.com",
"=?UTF-8?B?" . base64_encode( "Subject" ) . "?=",
$body,
$headers
);
Comparing the original mails to the broken ones i see the following
Broken:
Date: Fri, 3 Aug 2012 16:52:39 -0400 (EDT)
--09127kjhd821
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Original (Working)
Date: Tue, 31 Jul 2012 12:36:45 -0400 (EDT)
--09127kjhd821
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
As you can see the line breaks are off pretty much doubling for each \r and \n, nothing was changed to my knowledge that would cause this.
Any suggestions or help is greatly appreciated.

The only thing I can come up with is the POSTFIX and PHP issues with converting LF to CRLF so when CRLF is defined in the message body for boundaries and headers it is being converted to CRCRLF.
Saving the message source however only displays as CRLF in the hex editor though, which may be a conversion in the editor I use or email client download process.
I still don't understand why it just suddenly changed as it was working fine previously.
The only difference I can think of is maybe due to the scripts line-endings that may have changed from CRLF to LF causing a conversion to take place and thus breaking emails that contain CRLF in the body.
I ultimately fixed the issue by changing the sendmail_path in php.ini to
sendmail_path="/usr/bin/dos2unix|/usr/sbin/sendmail -t -i"

Related

Why do my "From" and "Reply-To" headers keep getting included in my mail sent from PHP?

I have this code for sending mail in PHP:
$headers = "Content-type: text/html;\r\n";
$headers .= 'From: "Registration" <registration#site.com>' . "\r\n";
$headers .= "Reply-To: registration#site.com\r\n";
$headers .= "Return-Path: registration#site.com\r\n";
$mailresult = mail($to, $subject, $message, $headers, '-f registration#site.com');
The mail sends, the from address is set correctly, and it's working almost perfectly.
The problem is that the mail includes the From:, Reply-To:, and Return-Path: fields in the body of the mail. Here is a snapshot of what it looks like in my Gmail interface:
Here is what the raw headers look like:
X-PHP-Originating-Script: 33:mailtest.php
Content-type: text/html;
Message-Id: <20130316153738.AA739118073#server.com>
Date: Sat, 16 Mar 2013 15:37:38 +0000 (UTC)
From: registration#site.com
From: Registration <registration#site.com>
Reply-To: registration#site.com
Return-Path: registration#site.com
<html>
The first From: seems to be within the actual headers. The second From:, and everything after that down to to <html> is in the body.
How do I get these header fields out of the email body and into the header where they belong?
http://php.net/manual/en/function.mail.php
additional_headers (optional)
[...]
Note:
If messages are not received, try using a LF (\n) only. Some Unix
mail transfer agents (most notably qmail) replace LF by CRLF
automatically (which leads to doubling CR if CRLF is used). This
should be a last resort, as it does not comply with RFC 2822.
Remove the semicolon in the first header (text/html).

How can I send email with UTF-8 encoding?

I have inherited a script that sends some content out in three languages (all on same content - repeated) however when recieved the content characters are broken for what i assume is a UTF-8 issue.
Am i right all i need to do is change the charset part to utf-8, or does anything else need to change like the 7bit part ?
you can see where I inserted one UTF-8 reference (not tested yet)
there was something here http://bitprison.net/php_mail_utf-8_subject_and_message which seems to reference base encoding, but I'm not sure if I need that here ?
// Contruct message body.
$body = "";
// Add message for non-mime clients.
$body .= "This is a multi-part message in MIME format.\n";
// Add text body.
$body .= "\n--$boundary\nContent-Type: text/plain; charset=UTF-8; format=flowed\nContent-Transfer-Encoding: 7bit\n\n" . $textContent;
// Add HTML body.
$body .= "\n--$boundary\nContent-Type: text/html; charset=ISO-8859-1; format=flowed\nContent-Transfer-Encoding: 7bit\n\n" . $htmlContent;
mail( $row["email"], "Update Your ArtsDB Listing", $body, $headers );
I looked on another post on here for an a example.
$body .= "\n--$boundary\nContent-Type: text/plain; charset=UTF-8; format=flowed\nContent-Transfer-Encoding: 8bit\n\n" . $textContent;
// Add HTML body.
$body .= "\n--$boundary\nContent-Type: text/html; charset=UTF-8; format=flowed\nContent-Transfer-Encoding: 8bit\n\n" . $htmlContent;
You are using Content-Type: text/plain; charset=UTF-8 to tell the mail reader that such message part uses UTF-8, which is fine, but... What does the $textContent variable contain? That's the important bit. According to Content-Transfer-Encoding: 7bit, it's a 7 bit encoding so it can't be raw UTF-8. However, you are not using any of the usual 7-bit encodings used for e-mail. Otherwise, there would be a (e.g.) Content-Transfer-Encoding: quoted-printable header.
To sum up, you need to:
Have a source string that contains valid UTF-8.
Pick a encoding for the transfer, such as quoted_printable_encode().
Add a header to tell which transfer encoding you chose.
You could also send the raw UTF-8 as-is and set Content-Transfer-Encoding: 8bit but I would not recommend it. You risk breaking the SMTP standard just by sending very long lines. Also, you have no idea of what kind of legacy programs this will go through.
E-mail is harder than it seems, that's why sooner or later you end up using a third-party library: PHP Mailer, Swift Mailer, PEAR Mail...
Content-Transfer-Encoding: 7bit
This makes no sense - there is no direct mapping between 7bit data and an 8+bit representation. You need to change the mime headers to state what encoding you are using.
For SMTP the transfer encoding should be a 7 bit ascii charset. To change your utf8 data you need to encode this - common encodings are base64 and quoted printable (PHP provides encode and decode fns for both).
Why not just use a good lib like phpmailer or swiftmailer

sending mail from php: headers are interpreted as body?

When I send mail from php with \r\n as line break in the headers (as it should be according to documentation)
$headers = "From: $email\r\n";
$headers .= "Reply-To: Just me <$email>\r\n";
$headers .= 'Content-type: text/plain; charset=iso-8859-1' . "\r\n";
$headers .= "Content-Transfer-Encoding: 8bit\r\n";
$subject = "Hello world";
$body = "<html><p>Hey, whats up?</p></html>";
mail($to, $subject, $body, $headers);
Some mail clients will interpret \r\n as being two line breaks. So for this mail() above the real mail content would look like this:
X-Message-Delivery: Vj0LEdMMtPAYT0xO0Q9MTtTQ0w9MA==
X-Message-Status: n
Received: from server75.publicompserver.de ([92.43.108.63]) by snt0-mc2-f13.Snt0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675);
Thu, 9 Dec 2010 12:09:22 -0800
Message-ID: <40177C.70807#justme.org>
[lots of other headers]
Date: Thu, 09 Dec 2010 21:09:32 +0100
X-OriginalArrivalTime: 09 Dec 2010 20:09:22.0873 (UTC) FILETIME=[F88C3A90:01CB97DC]
From: $email
Reply-To: Just me <$email>
Content-type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: 8bit
<html><p>Hey, whats up?</p></html>
Now some clients (googlemail for example) will ignore these extra linebreaks. Others (thunderbird) will interpret the first extra linebreak as being the end of the headers and will interpret the rest of the header lines as being part of the body (losing header information, in this case rendering the mail as text instead of html).
I've seen the same problem from other mail-sending websites, too.
What is happening here? \r\n is the correct line break according to doc, or is something else going wrong here?
And how can it be resolved? Changing the line break to \n instead of \r\n seems to help, but since the docs say "thou should use \r\n" this can't be right, can it?
This is mentioned in the documentation at http://php.net/manual/en/function.mail.php: "If messages are not received, try using a LF (\n) only. Some poor quality Unix mail transfer agents replace LF by CRLF automatically (which leads to doubling CR if CRLF is used). This should be a last resort, as it does not comply with » RFC 2822."
So, as you said: it's not right, but it's reality.
As mentioned in your previous question. Each platform delimits lines in a different way (CRLF or LF). It's best to leave this decision up to PHP via the PHP_EOL constant and PHP will take care of it.
In your code, you're using \r\n for some of the header lines, and only \n for others. Maybe this problem wouldn't occur if you were consistent.

Which line break in php mail header, \r\n or \n?

I've seen a lot of examples using the php mail function. Some of them use \r\n as line break for the header, some use \n.
$headers = "From: Just Me\n";
$headers .= "Reply-To: Just me <$email>\n";
vs
$headers = "From: Just Me\r\n";
$headers .= "Reply-To: Just me <$email>\r\n";
which one is correct?
Sometimes I've had cases where \r\n is used and part of the header is interpreted by some email clients as mail text (losing these header information) - is this because \r\n is wrong?
The CRLF \r\n, should be used according to the php documentation. Also, to conform to the RFC 2822 spec lines must be delimited by the carriage return character, CR \r immediately followed by the line feed, LF \n.
Since \r\n is native to Windows platforms and \n to Unix, you can use the PHP_EOL­Docs constant on Windows, which is the appropriate new line character for the platform the script is currently running on.
Just in case a search engine picks this up and saves someone else the frustration I went through: here's an additional curiousity.
On php 5.2x on Linux, I had \r\n on my email headers in php mail(), after an upgrade to php 5.3.3, the formatting and sending mysteriously failed. Removing the \r fixed the script (after examining many many other possibilities).
As stated above, \r\n is what you should use according to the RFC, but this breaks your headers on several mail systems (f.i. Outlook 2003). Even though \n is not the 'proper' line break to use, in my experience it works correctly on all mail systems I've encountered so far. Because of this, I always use just \n.
The RFC formally mandates CRLF (\r\n) but using Unix breaks (\n) for headers will save you a lot of hassle. Some mail servers, such as qmail, will reject your message if it uses \r\n.
Source: experience, confirmed by this note: http://www.php.net/function.mail#40204
My experience:
HTML emails were working in web clients, but breaking in MS based desktop clients (entourage, outlook). Was using \r\n. Removed the \r on the MIME-Version only and now works across the board.
I've had the problem of gmail misunderstanding \r\n headers, but simply leaving the header line breaks at \n was not enough in my case, because in that case some versions of Outlook showed emails as empty.
The solution in https://stackoverflow.com/a/7960957 (I chose to install postfix 2.9 on lucid from a ppa) coupled with using \n seems to work everywhere now.
I changed my script to use PHP_EOL instead which seems to work -- like this:
//Set Content-type header
$headers = "MIME-Version: 1.0" . PHP_EOL;
$headers .= "Content-type: text/html; charset=iso-8859-1" . PHP_EOL;
//Additional headers
$headers .= "From: $from" . PHP_EOL;
$headers .= "Cc: $cc" . PHP_EOL;
$headers .= "Content-type: text/html" . PHP_EOL;
$headers .= "Bcc: $bcc" . PHP_EOL;
NB. Be sure to us " instead of ' as the latter doesn't seem to work!
> $mail = new PHPMailer;
$mail->isSMTP();
**$mail->isHTML(true);**
Insert this code after working
> all html tag <br> <p> in $mail->Body='Hello<br> how are you ?<b>';

PHP difference between \r\n and \n

Simple question...
I have seen people tell me to use "\r\n" in various places and others tell me to use "\n" in the same place. I'm sure one is right and one is wrong. Example - when designing mail() headers:
Tutorial #1:
//define the headers we want passed. Note that they are separated with \r\n
$headers = "From: webmaster#example.com\r\nReply-To: webmaster#example.com";
//add boundary string and mime type specification
$headers .= "\r\nContent-Type: multipart/mixed; boundary=\"PHP-mixed-".$random_hash."\"";
Tutorial #2 (notice the header argument):
mail($to, $subject, $body,
"From: " . $from . "\n" .
"bcc: " . $bcc . "\n" .
"MIME-Version: 1.0\n" .
"Content-Type: multipart/alternative;\n" .
" boundary=" . $mime_boundary_header)
I am confused, but clearly it makes somewhat of a difference, because with one, my headers worked, and with the other they only sometimes work.
\r\n are end of line characters for Windows systems.
\n is the end of line character for UNIX systems.
These characters are invisible. From my own experience \n is usually okay for Windows as well.
Some prefer to use PHP_EOL constant instead of these characters for portability between platforms.
echo 'hi' . PHP_EOL;
echo "hi\n";
$headers = "From: webmaster#example.com" . PHP_EOL
. "Reply-To: webmaster#example.com";
As per RFC 821 (the SMTP protocol), line endings should always be \r\n (<CR><LF>) in mail headers and content, but in practice it shouldn't matter as most mail servers handle all three type of line endings correctly (supposedly, some old UNIX-based ones actually choke on \r\n but not \n).
In addition to Yada's answer, here is an explaining blog entry: https://blog.codinghorror.com/the-great-newline-schism/
[Update 2017-05-24: Fixed the link]

Categories