PHP Imap Body encoding - php

I have a question. I send out an E-Mail via "PHPMailer" Class. Then I answered with Outlook 2013 to this mail. Afterwords im Trying to read the Email via PHP with "imap_fetchbody". But I only get
PGh0bWwgeG1sbnM6dj0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTp2bWwiIHhtbG5zOm89InVy bjpzY2hlbWFzLW1pY3Jvc29mdC1jb206b2ZmaWNlOm9mZmljZSIgeG1sbnM6dz0idXJuOnNjaGVt YXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6d29yZCIgeG1sbnM6bT0iaHR0cDovL3NjaGVtYXMubWlj cm9zb2Z0LmNvbS9vZmZpY2UvMjAwNC8xMi9vbW1sIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcv VFIvUkVDLWh0bWw0MCI+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIg Y29udGVudD0idGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjxtZXRhIG5hbWU9IkdlbmVyYXRv ciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUgKGZpbHRlcmVkIG1lZGl1bSkiPg0KPCEtLVtp ZiAhbXNvXT48c3R5bGU+dlw6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kb1w6KiB7 YmVoYXZpb3I6dXJsKCNkZWZhdWx0I1ZNTCk7fQ0Kd1w6KiB7YmVoYXZpb3I6dXJsKCNkZWZhdWx0 I1ZNTCk7fQ0KLnNoYXBlIHtiZWhhdmlvcjp1cmwoI2RlZmF1bHQjVk1MKTt9DQo8L3N0eWxlPjwh W2VuZGlmXS0tPjxzdHlsZT48IS0tD[...]
Stuff like this - I tried many decoding options in the imap_fetchbody function, but I am not getting it - this is my code:
$server='{IPADDRESS:143/novalidate-cert}INBOX';
$adresse='cloud#MYDOMAIN';
$password='MYPASSWORD';
$mbox = imap_open($server, $adresse, $password, OP_READONLY, 1, array('DISABLE_AUTHENTICATOR' => 'PLAIN')) or die(var_dump(imap_errors()));
$no = 1;
$headers = imap_headers($mbox);
$text = imap_fetchbody($mbox, $no, 1);
for($i = 0; $i < count($headers); ++$i)
{
$string = imap_fetchbody($mbox, $i+1, 1);
echo $string;
}
Any Ideas?

I found a solution myself. It seems that those messages come with different Mime Types, etc. So this function helps to find the mime type and offers an easy "getBody" function - see this page:
https://www.sitepoint.com/exploring-phps-imap-library-1/

Related

PHP webcrawler programmed in Visual Studio Code has problems with unknown class, how do I fix that?

and thanks in advance. I try to build a webscraper with PHP and I use Visual Studio Code.
When I run the following code, the following problem shows up:
Use of unknown class: 'Goutte\Client'
Does anyone know how to solve that issue?
I have googled all over the place, looked at SO and asked the forbidden one, but still after three days I have not achieved any progress. (I am also a noob, so maybe it is not as difficult to solve as I think).
Looking forward to your feedback and tips.
<?php
require 'vendor/autoload.php';
use Goutte\Client;
// Initialize the Goutte client
$client = new Client();
// Create a new array to store the scraped data
$data = array();
// Loop through the pages
for ($i = 0; $i < 3; $i++) {
// Make a request to the website
$crawler = $client->request('GET', 'https://ec.europa.eu/info/law/better-regulation/have-your-say/initiatives_de?page=' . $i);
// Find all the initiatives on the page
$crawler->filter('.initiative')->each(function ($node) use (&$data) {
// Extract the information for each initiative
$title = $node->filter('h3')->text();
$link = $node->filter('a')->attr('href');
$description = $node->filter('p')->text();
$deadline = $node->filter('time')->attr('datetime');
// Append the data for the initiative to the data array
$data[] = array($title, $link, $description, $deadline);
});
// Sleep for a random amount of time between 5 and 10 seconds
$sleep = rand(5,10);
sleep($sleep);
}
// Open the output file
$fp = fopen('initiatives.csv', 'w');
// Write the header row
fputcsv($fp, array('Title', 'Link', 'Description', 'Deadline'));

PHP imap_fetchbody

I have been trying to fetch message but unsuccessful.
$body = imap_fetchbody($inbox, $email_id, 0);
the messages without attachments are good and I have output but with attachments
gives some complicated outputs out of which both html and plain message are encoded with some (Content-Type) which is a part of gmail messages
You can use the following code to get the plain text part of a multipart email body:
<?php
//get the body
$body = imap_fetchbody($inbox, $email_id, 0);
//parse the boundary separator
$matches = array();
preg_match('#Content-Type: multipart\/[^;]+;\s*boundary="([^"]+)"#i', $body, $matches);
list(, $boundary) = $matches;
$text = '';
if(!empty($boundary)) {
//split the body into the boundary parts
$emailSegments = explode('--' . $boundary, $body);
//get the plain text part
foreach($emailSegments as $segment) {
if(stristr($segment, 'Content-Type: text/plain') !== false) {
$text = trim(preg_replace('/Content-(Type|ID|Disposition|Transfer-Encoding):.*?\r\n/is', '', $segment));
break;
}
}
}
echo $text;
?>
$body = imap_fetchbody($inbox, $email_id, 1.0);
this seems to be the only one working for me. I think the first integer in the last parameter represents the section of the email, so if it starts with a zero it will contain all the header information. If it starts with a one then it contains the message information. Then the second integer followed by the period is the section of that section. So when I put zero it shows information, but when I put one or two it doesn't show anything for some emails.
This helped
$body = imap_fetchbody($inbox, $email_id, 1.1);

Reading mail in PHP?

I'm using pop3class for retrieving mails from pop3 server. when I call
$pop3->RetieveMessage($messageNumber,$header,$body,-1);
I receive header and body of the message in $header and $body variables.
when I
Print_r($body);
I've lot of HTML code in output. Can someone help me in which class should
I use for dealing with the received mail, because I don't know the returned
data type of the mail.
i think this link can help u in getting the incoming mails
http://ca.php.net/imap
$mb = imap_open("{host:port/imap}","username", "password" );
$messageCount = imap_num_msg($mb);
for( $MID = 1; $MID <= $messageCount; $MID++ )
{
$EmailHeaders = imap_headerinfo( $mb, $MID );
$Body = imap_fetchbody( $mb, $MID, 1 );
doSomething( $EmailHeaders, $Body );
}

Send Mail from raw body for testing purposes

I am developing a PHP application that needs to retrieve arbitrary emails from an email server. Then, the message is completely parsed and stored in a database.
Of course, I have to do a lot of tests as this task is not really trivial with all that different mail formats under the sun. Therefore I started to "collect" emails from certain clients and with different contents.
I would like to have a script so that I can send out those emails automatically to my application to test the mail handling.
Therefore, I need a way to send the raw emails - so that the structure is exactly the same as they would come from the respective client. I have the emails stored as .eml files.
Does somebody know how to send emails by supplying the raw body?
Edit:
To be more specific: I am searching for a way to send out multipart emails by using their source code. For example I would like to be able to use something like that (an email with plain and HTML part, HTML part has one inline attachment).
--Apple-Mail-159-396126150
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
The plain text email!
--=20
=20
=20
--Apple-Mail-159-396126150
Content-Type: multipart/related;
type="text/html";
boundary=Apple-Mail-160-396126150
--Apple-Mail-160-396126150
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=iso-8859-1
<html><head>
<title>Daisies</title>=20
</head><body style=3D"background-attachment: initial; background-origin: =
initial; background-image: =
url(cid:4BFF075A-09D1-4118-9AE5-2DA8295BDF33/bg_pattern.jpg); =
background-position: 50% 0px; ">
[ - snip - the html email content ]
</body></html>=
--Apple-Mail-160-396126150
Content-Transfer-Encoding: base64
Content-Disposition: inline;
filename=bg_pattern.jpg
Content-Type: image/jpg;
x-apple-mail-type=stationery;
name="bg_pattern.jpg"
Content-Id: <4BFF075A-09D1-4118-9AE5-2DA8295BDF33/tbg.jpg>
/9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAASAAA/+IFOElDQ19QUk9GSUxFAAEB
[ - snip - the image content ]
nU4IGsoTr47IczxmCMvPypi6XZOWKYz/AB42mcaD/9k=
--Apple-Mail-159-396126150--
Using PHPMailer, you can set the body of a message directly:
$mail->Body = 'the contents of one of your .eml files here'
If your mails contain any mime attachments, this will most likely not work properly, as some of the MIME stuff has to go into the mail's headers. You'd have to massage the .eml to extract those particular headers and add them to the PHPMailer mail as a customheader
You could just use the telnet program to send those emails:
$ telnet <host> <port> // execute telnet
HELO my.domain.com // enter HELO command
MAIL FROM: sender#address.com // enter MAIL FROM command
RCPT TO: recipient#address.com // enter RCPT TO command
<past here, without adding a newline> // enter the raw content of the message
[ctrl]+d // hit [ctrl] and d simultaneously to end the message
If you really want to do this in PHP, you can use fsockopen() or stream_socket_client() family. Basically you do the same thing: talking to the mailserver directly.
// open connection
$stream = #stream_socket_client($host . ':' . $port);
// write HELO command
fwrite($stream, "HELO my.domain.com\r\n");
// read response
$data = '';
while (!feof($stream)) {
$data += fgets($stream, 1024);
}
// repeat for other steps
// ...
// close connection
fclose($stream);
You can just use the build in PHP function mail for it. The body part doesnt have to be just text, it can also contain mixed part data.
Keep in mind that this is a proof of concept. The sendEmlFile function could use some more checking, like "Does the file exists" and "Does it have a boundry set". As you mentioned it is for testing/development, I have not included it.
<?php
function sendmail($body,$subject,$to, $boundry='') {
define ("CRLF", "\r\n");
//basic settings
$from = "Example mail<info#example.com>";
//define headers
$sHeaders = "From: ".$from.CRLF;
$sHeaders .= "X-Mailer: PHP/".phpversion().CRLF;
$sHeaders .= "MIME-Version: 1.0".CRLF;
//if you supply a boundry, it will be send with your own data
//else it will be send as regular html email
if (strlen($boundry)>0)
$sHeaders .= "Content-Type: multipart/mixed; boundary=\"".$boundry."\"".CRLF;
else
{
$sHeaders .= "Content-type: text/html;".CRLF."\tcharset=\"iso-8859-1\"".CRLF;
$sHeaders .= "Content-Transfer-Encoding: 7bit".CRLF."Content-Disposition: inline";
}
mail($to,$subject,$body,$sHeaders);
}
function sendEmlFile($subject, $to, $filename) {
$body = file_get_contents($filename);
//get first line "--Apple-Mail-159-396126150"
$boundry = $str = strtok($body, "\n");
sendmail($body,$subject,$to, $boundry);
}
?>
Update:
After some more testing I found that all .eml files are different. There might be a standard, but I had tons of options when exporting to .eml. I had to use a seperate tool to create the file, because you cannot save to .eml by default using outlook.
You can download an example of the mail script. It contains two versions.
The simple version has two files, one is the index.php file that sends the test.eml file. This is just a file where i pasted in the example code you posted in your question.
The advanced version sends an email using an actual .eml file I created. it will get the required headers from the file it self. Keep in mind that this also sets the To and From part of the mail, so change it to match your own/server settings.
The advanced code works like this:
<?php
function sendEmlFile($filename) {
//define a clear line
define ("CRLF", "\r\n");
//eml content to array.
$file = file($filename);
//var to store the headers
$headers = "";
$to = "";
$subject = "";
//loop trough each line
//the first part are the headers, until you reach a white line
while(true) {
//get the first line and remove it from the file
$line = array_shift($file);
if (strlen(trim($line))==0) {
//headers are complete
break;
}
//is it the To header
if (substr(strtolower($line), 0,3)=="to:") {
$to = trim(substr($line, 3));
continue;
}
//Is it the subject header
if (substr(strtolower($line), 0,8)=="subject:") {
$subject = trim(substr($line, 8));
continue;
}
$headers .= $line . CRLF;
}
//implode the remaining content into the body and trim it, incase the headers where seperated with multiple white lines
$body = trim(implode('', $file));
//echo content for debugging
/*
echo $headers;
echo '<hr>';
echo $to;
echo '<hr>';
echo $subject;
echo '<hr>';
echo $body;
*/
//send the email
mail($to,$subject,$body,$headers);
}
//initiate a test with the test file
sendEmlFile("Test.eml");
?>
You could start here
http://www.dreamincode.net/forums/topic/36108-send-emails-using-php-smtp-direct/
I have no idea how good that code is, but it would make a starting point.
What you are doing is connecting direct to port 25 on the remote machine, as you would with telnet, and issuing smtp commands. See eg http://www.yuki-onna.co.uk/email/smtp.html for what's going on (or see Jasper N. Brouwer's answer).
Just make a quick shell script which processes a directory and call it when you want e.g. using at crontab etc
for I in ls /mydir/ do cat I | awk .. | sendmail -options
http://www.manpagez.com/man/1/awk/
You could also just talk to the mail server using the script to send the emls with a templated body..
Edit: I have added the code to Github, for ease of use by other people. https://github.com/xrobau/smtphack
I realise I am somewhat necro-answering this question, but it wasn't answered and I needed to do this myself. Here's the code!
<?php
use PHPMailer\PHPMailer\PHPMailer;
use PHPMailer\PHPMailer\SMTP;
class SMTPHack
{
private $phpmailer;
private $smtp;
private $from;
private $to;
/**
* #param string $from
* #param string $to
* #param string $smtphost
* #return void
*/
public function __construct(string $from, string $to, string $smtphost = 'mailrx')
{
$mail = new PHPMailer(true);
$mail->isSMTP();
$mail->SMTPDebug = SMTP::DEBUG_SERVER;
$mail->SMTPAutoTLS = false;
$mail->Host = $smtphost;
$this->phpmailer = $mail;
$this->from = $from;
$this->to = $to;
}
/**
* #param string $helo
* #return SMTP
*/
public function getSmtp(string $helo = ''): SMTP
{
if (!$this->smtp) {
if ($helo) {
$this->phpmailer->Helo = $helo;
}
$this->phpmailer->smtpConnect();
$this->smtp = $this->phpmailer->getSMTPInstance();
$this->smtp->mail($this->from);
$this->smtp->recipient($this->to);
}
return $this->smtp;
}
/**
* #param string $data
* #param string $helo
* #param boolean $quiet
* #return void
* #throws \PHPMailer\PHPMailer\Exception
*/
public function data(string $data, string $helo = '', bool $quiet = true)
{
$smtp = $this->getSmtp($helo);
$prev = $smtp->do_debug;
if ($quiet) {
$smtp->do_debug = 0;
}
$smtp->data($data);
$smtp->do_debug = $prev;
}
}
Using that, you can simply beat PHPMailer into submission with a few simple commands:
$from = 'xrobau#example.com';
$to = 'fred#example.com';
$hack = new SMTPHack($from, $to);
$smtp = $hack->getSmtp('helo.hostname');
$errors = $smtp->getError();
// Assuming this is running in a phpunit test...
$this->assertEmpty($errors['error']);
$testemail = file_get_contents(__DIR__ . '/TestEmail.eml');
$hack->data($testemail);

xml parse error: 'Invalid character'

I'm using the google weather api for a widget.
All is fine and dandy except that today I encountered a problem that I cannot solve.
When called with this location:
http://www.google.com/ig/api?weather=dunjkovec,medimurska,croatia&hl=en
I get this error:
XML parse error 9 'Invalid character' at line 1, column 169 (byte index 199)
I suspect that the problem is here: Nedelišće
The code block is this one:
$parser = xml_parser_create('UTF-8');
xml_parser_set_option($parser, XML_OPTION_CASE_FOLDING, 0);
xml_parser_set_option($parser, XML_OPTION_SKIP_WHITE, 1);
$ok = xml_parse_into_struct($parser, $data, $values);
if (!$ok) {
$errmsg = sprintf("XML parse error %d '%s' at line %d, column %d (byte index %d)",
xml_get_error_code($parser),
xml_error_string(xml_get_error_code($parser)),
xml_get_current_line_number($parser),
xml_get_current_column_number($parser),
xml_get_current_byte_index($parser));
}
$data is the content of the xml and $values is empty.
Can someone help me? Thank you very much!
EDIT----------------------------------
After reading Hussein's post I discovered that the problem is in the way the file gets retrieved.
I tried file_get_contents and cURL. Both returns:
that is the line that creates problems. Or so I thought! I tried this html_entity_decode($data,ENT_NOQUOTES,'UTF-8') and it wasn't working, so I made a discover, I can't echo the contents of the xml, I can only print_r them and see the results in the html source! With any other location in the world it works, only this one creates problems... I wanna cry :-(
EDIT 2--------------------------------
For anybody that cares. I fixed the problem with this lines of code after retrieving the xml file from the api:
$data = mb_convert_encoding($data, 'UTF-8', mb_detect_encoding($data, 'UTF-8, ISO-8859-1', true));
$data = html_entity_decode($data,ENT_NOQUOTES,'UTF-8');
then parse the xml, it works like a charm.
I marked hussein's answer because it got me on the right track.
After reading at your problem, I tried same thing on my machine.
What I did is
1. Downloaded xml file on my local machine from the URL you posted.
2. Used your xml parsing script to prepare structure from XML.
Amazingly it worked perfectly on my machine, even though XML has Nedelišće keyword.
So, I see the problem in the way of reading XML file.
It would be easy to debug if you can tell me the way you are reading the xml form google api.
Are you using CURL?
EDIT -----------------------------------------------
Hi 0plus1,
I have prepared one helper function to convert those special chars to html for making it able for parsing..
I am pasting entire code here. Use following script..
function utf8tohtml($utf8, $encodeTags)
{
$result = '';
for ($i = 0; $i < strlen($utf8); $i++)
{
$char = $utf8[$i];
$ascii = ord($char);
if ($ascii < 128)
{
// one-byte character
$result .= ($encodeTags) ? htmlentities($char , ENT_QUOTES, 'UTF-8') : $char;
} else if ($ascii < 192)
{
// non-utf8 character or not a start byte
} else if ($ascii < 224)
{
// two-byte character
$result .= htmlentities(substr($utf8, $i, 2), ENT_QUOTES, 'UTF-8');
$i++;
} else if ($ascii < 240)
{
// three-byte character
$ascii1 = ord($utf8[$i+1]);
$ascii2 = ord($utf8[$i+2]);
$unicode = (15 & $ascii) * 4096 +
(63 & $ascii1) * 64 +
(63 & $ascii2);
$result .= "&#$unicode;";
$i += 2;
} else if ($ascii < 248)
{
// four-byte character
$ascii1 = ord($utf8[$i+1]);
$ascii2 = ord($utf8[$i+2]);
$ascii3 = ord($utf8[$i+3]);
$unicode = (15 & $ascii) * 262144 +
(63 & $ascii1) * 4096 +
(63 & $ascii2) * 64 +
(63 & $ascii3);
$result .= "&#$unicode;";
$i += 3;
}
}
return $result;
}
$curlHandle = curl_init();
$serviceUrl = "http://www.google.com/ig/api?weather=dunjkovec,medimurska,croatia&hl=en";
// setup the basic options for the curl
curl_setopt($curlHandle , CURLOPT_URL, $serviceUrl);
curl_setopt($curlHandle , CURLOPT_HEADER , 0);
curl_setopt($curlHandle , CURLOPT_HTTPHEADER , array("Cache-Control: no-cache","Content-type: application/x-www-form-urlencoded;charset=UTF-8"));
curl_setopt($curlHandle , CURLOPT_FOLLOWLOCATION , true);
curl_setopt($curlHandle , CURLOPT_RETURNTRANSFER , true);
curl_setopt($curlHandle , CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)');
$data = curl_exec($curlHandle);
// echo $data;
$data = utf8tohtml($data , false);
echo $data;
$parser = xml_parser_create("UTF-8");
xml_parser_set_option($parser, XML_OPTION_TARGET_ENCODING, "UTF-8");
xml_parser_set_option($parser, XML_OPTION_CASE_FOLDING, 0);
xml_parser_set_option($parser, XML_OPTION_SKIP_WHITE, 1);
$ok = xml_parse_into_struct($parser, $data, $values);
if (!$ok) {
$errmsg = sprintf("XML parse error %d '%s' at line %d, column %d (byte index %d)",
xml_get_error_code($parser),
xml_error_string(xml_get_error_code($parser)),
xml_get_current_line_number($parser),
xml_get_current_column_number($parser),
xml_get_current_byte_index($parser));
}
echo "<pre>";
print_r($values);
echo "</pre>";
Hope this will help.
Thanks!
Hussain.
The Content-Type header field in the response specifies the content to be encoded with ISO 8859-1 (see response on Web-Sniffer.net) and not UTF-8. So either specify ISO-8859-1 as encoding or omit that parameter and xml_parser_create tries to identify the encoding.
Again, which php version are you using? xml_parser_create takes encoding as a parameter, but only for output, not input in some versions. http://www.php.net/manual/en/function.xml-parser-create.php
You might want to consider creating an empty utf-8 string and then filling it with the XML retrieved from Google, or explicitly converting the string to UTF-8.
string utf8_encode ( string $data )
Google is correctly informing us the data is UTF-8, but only in the header, not in the actual XML.

Categories