We retrieve and parse e-mails via PHP5-imap, and when parsing e-mails, we noticed that some of the e-mails contain special characters which are not filtered out automatically (ex: =2E or = before a line break)
What is the best way to filter these characters out ?
We're using this section of code to get the mail body:
$structure = imap_fetchstructure(static::$_connection, $msg_id);
$unparsed = imap_fetchbody(static::$_connection, $msg_id, $i + 1);
if ($structure->parts[$i]->encoding == 3) { // 3 = BASE64
$parsed = base64_decode($unparsed);
}
elseif ($structure->parts[$i]->encoding == 4) { // 4 = QUOTED-PRINTABLE
$parsed = quoted_printable_decode($unparsed);
}
else {
$parsed = $unparsed;
}
Related
I am currently trying to get the UNSEEN/UNREAD messages from my server. Currently, I have this:
$openmail = imap_open($dns, $email, $password) or die("Cannot Connect " . imap_last_error());
if ($openmail) {
/* grab emails */
$emails = imap_search($openmail, 'UNSEEN');
if ($emails) {
//For every e-mail.
$tot = imap_num_msg($openmail);
for ($i = $tot; $i > 0; $i--) {
$structure = imap_fetchstructure($openmail, $i);
if (isset($structure->parts) && is_array($structure->parts) && isset($structure->parts[1])) {
$part = $structure->parts[1];
$message = imap_fetchbody($openmail, $i, 2, FT_PEEK);
if ($part->encoding == 3) {
$message = imap_base64($message);
} else if ($part->encoding == 1) {
$message = imap_8bit($message);
} else {
$message = imap_qprint($message);
}
}
$header = imap_header($openmail, $i);
$from = imap_utf8($header->fromaddress);
$subject = $header->Subject;
$subject = substr($subject, 0, 150);
$date = $header->Date;
}
/* Print out the Unseen messages in here! */
} else {
/* No unseen messages */
echo "No unseen";
}
}
I've tried sending multiply emails to my mailserver, refreshed the page with the above script. But I keep getting the "No unseen".
I've tried to output the $emails but it's empty. It can't find anything.
If I try to just get ALL the messages (no unseen filter), I can see the emails I've sent to the mailbox, although, they're all marked as read.
Your code $message = imap_fetchbody($openmail, $i, 2, FT_PEEK); uses hardcoded part offsets, i.e. it assumes that a message is always multipart/alternative with text/plain and text/html. That's a very wrong assumption.
You have to look at the output of the BODYSTRUCTURE to see what the MIME structure of your mail is.
With that out of the way, be sure that none of your commands use the BODY fetch operation; you have to use BODY.PEEK. I have no idea how this is accessible within the PHP c-client bindings.
I ma trying to send arabic text from the clickatell sms provider using php, but all i have been able to send yet is garbled text. The text I am using for testing is "غثس هفس ".
I have tried encoding the message text to Windows-1256,1250 and ISO-8859-6 formats using iconv but everytimeit just sends garbled texts.
Can anyone please give me some pointers to what I am missing?
Thanks in advance
Ok, this is for whoever comes on this path later. Found the solution here
Retaining code from Esailija's answer
<?php
$text = "غثس هفس خن";
$arr = unpack('H*hex', iconv('UTF-8', 'UCS-2BE', $text));
$message = strtoupper($arr['hex']);
$username = '';
$password = '';
$API_ID = '';
$to = '';
$url = "http://api.clickatell.com/http/auth?user=$username&password=$password&api_id=$API_ID";
$ret = file($url);
$sess = explode(":",$ret[0]);
if ($sess[0] == "OK") {
$sess_id = trim($sess[1]);
$url = "http://api.clickatell.com/http/sendmsg?session_id=$sess_id&to=$to&text=$message&unicode=1";
$ret = file($url);
$send = explode(":",$ret[0]);
if ($send[0] == "ID") {
echo "success - message ID: ". $send[1];
} else {
echo "send message failed";
}
} else {
echo "Authentication failure: ". $ret[0];
}
Based on this, the default is GSM but you can also choose UCS2.
So:
<?php
//Make sure the PHP source file is physically
//saved in utf-8
$text = rawurlencode(iconv( "UTF-8", "UCS-2", "غثس هفس "));
$url = "http://api.clickatell.com/http/auth?user=username&password=password&api_id=API_ID&encoding=UCS2";
$ret = file($url);
$sess = explode(":",$ret[0]);
if ($sess[0] == "OK") {
$sess_id = trim($sess[1]);
$url = "http://api.clickatell.com/http/sendmsg?session_id=$sess_id&to=$to&text=$text&encoding=UCS2";
$ret = file($url);
$send = explode(":",$ret[0]);
if ($send[0] == "ID") {
echo "successnmessage ID: ". $send[1];
} else {
echo "send message failed";
}
} else {
echo "Authentication failure: ". $ret[0];
}
I rewrote ClickATell code for Vtiger 6 so that it can accommodate UTF-8 Extended characters. I use Turkish. I use the below code for conversion. I hope you can port.
/**
* Function to handle UTF-8 Check and conversion
* #author Nuri Unver
*
*/
public function smstxtcode($data){
$mb_hex = '';
$utf = 0;
for($i = 0 ; $i<mb_strlen($data,'UTF-8') ; $i++){
$c = mb_substr($data,$i,1,'UTF-8');
$o = unpack('N',mb_convert_encoding($c,'UCS-4BE','UTF-8'));
$hx = sprintf('%04X',$o[1]);
$utf += intval(substr($hx,0,2));
$mb_hex .= $hx;
}
if ($utf>0)
{
$return=$mb_hex;
$utf=1;
}
else
{
$return=utf8_decode($data);
$utf=0;
}
return array($utf,$return);
}
The you call this function with your message. The response you get it is an array saying wheter to send unicode or normal text depending on the message and also the text to be sent. If there are no extended characters, it just sends it as plain text with unicode=0 in order to save characters. If message contains extended characters, it converts the message to hexcode and sends it as unicode.
This code just does the calculations. You need to implement your own code to port it to your system. For demonstration this is the code I use for Vtiger to extract data and send the message:
/**
* Function to handle SMS Send operation
* #param <String> $message
* #param <Mixed> $toNumbers One or Array of numbers
*/
public function send($message, $toNumbers) {
if(!is_array($toNumbers)) {
$toNumbers = array($toNumbers);
}
$params = $this->prepareParameters();
$smsarray = $this->smstxtcode($message);
$params['text'] = $smsarray[1];
$params['unicode'] = $smsarray[0];
$params['to'] = implode(',', $toNumbers);
$serviceURL = $this->getServiceURL(self::SERVICE_SEND);
Short answer: you need to do two things:
a) Convert your Arabic message:
$data = iconv("UTF-8", "UCS-2BE", $data);
$data = bin2hex($data);
Use the resulting string as your message text
b) When sending via the HTTP API, include the unicode parameter:
&unicode=1
I have been on this for days and days, and am at the point that I have pulled out so many hairs that I now have just one hair left on my head. That hair is my last bit of pride. But seriously though, I have found dozens of answers but none seem to apply to my problem!
I have an e-mail form for a website I made. The site and form are made in Flash (AS3), the script for processing the e-mail is an external php file. The e-mail form works just fine, except for when I use certain characters:
% is not shown in the e-mail, including any text directly behind it
when a &, < or > is present, the form will say 'sending..' but not go beyond that point; I don't receive any e-mail.
All (or most at least) other characters like !##$^*_+-=~` are no problem.
I have already made sure both AS3 and php codes have
"MIME-Version: 1.0; Content-Type: text/html; charset=utf-8" is included in my sending if check in the php file;
the textfields in AS3 are set to htmlText instead of just text.
My scripts:
mail.php
if( $yourName == true ) {
$sender = $fromEmail;
$yourEmail = "myemail#example.com"; // Here i of course use my own email address
$ipAddress = $_SERVER['REMOTE_ADDR']; // This gets the user's ip Address
$emailMsg = "Van: $sender\r\n" .
"Name: $yourName\r" .
"Subject: $yourSubject\n\n" .
"$yourMsg\n\n\n\n" .
"------------------------------\r" .
"Sent from IP-address $ipAddress\r" .
"X-Mailer: PHP/" . phpversion();
# these are three (out of many) things I tried to work around the problem #
//$emailMsg = str_replace( '&', "&", $emailMsg );
//$emailMsg = htmlspecialchars($emailMsg, ENT_QUOTES);
//$emailMsg = mysql_real_escape_string($emailMsg);
$return = "From: $sender\r\n";
if( mail($yourEmail, "$yourSubject", $emailMsg, $return, "MIME-Version: 1.0; Content-Type: text/html; charset=utf-8")) {
echo "sentStatus=yes";
}
else {
echo "sentStatus=no";
}
}
?>
FormScript.as
package {
/*required imports*/
public class FormScript extends Sprite {
/*here are the variable declarations*/
public function FormScript() {
sendbtn.buttonMode = true;
sendbtn.addEventListener(MouseEvent.CLICK, submit);
resetbtn.buttonMode = true;
resetbtn.addEventListener(MouseEvent.CLICK, reset);
urlRequest.method = URLRequestMethod.POST;
/*here are are some positionings and addchilds*/
function init():void {
//Set all fields to empty
yourName.htmlText = "";
fromEmail.htmlText = "";
yourSubject.htmlText = "";
yourMsg.htmlText = "";
valid.text = "";
}
function submit(e:MouseEvent):void {
//Check to see if any of the fields are empty
if(yourName.htmlText == "" || fromEmail.htmlText == "" ||
yourSubject.htmlText == "" ||yourMsg.htmlText == "" ) {
valid.text = "All fields must be filled in";
}//Check if you're using a valid email address
else if(!checkEmail(fromEmail.htmlText)) {
valid.text = "Please enter a valid e-mail address";
}
else {
valid.text = "Sending..";
var emailData:String =
"name=" + yourName.htmlText +
"&from=" + fromEmail.htmlText +
"&subject=" + yourSubject.htmlText +
"&msg=" + yourMsg.htmlText;
var urlVars:URLVariables = new URLVariables(emailData);
urlVars.dataFormat = URLLoaderDataFormat.TEXT;
urlRequest.data = urlVars; varLoad.load( urlRequest );
varLoad.addEventListener(Event.COMPLETE, thankYou );
}
}
function reset(e:MouseEvent):void {
init(); //call the initial clear function
}
function checkEmail(s:String):Boolean {
//yourMsg.text = escape("&");
//This tests for correct email address
var p:RegExp = /(\w|[_.\-])+#((\w|-)+\.)+\w{2,4}+/;
var r:Object = p.exec(s);
if( r == null ) {
return false;
}
return true;
}
function thankYou(e:Event):void {
var loader:URLLoader = URLLoader(e.target);
var sent = new URLVariables(loader.data).sentStatus;
//valid.text = sent;
if( sent == "yes" ) {
valid.text = "Thank you for your e-mail!"; timer = new Timer(500);
timer.addEventListener(TimerEvent.TIMER, msgSent);
timer.start();
}
else {
valid.text = "Something went wrong, please try again";
}
}
function msgSent(te:TimerEvent):void {
if(timer.currentCount >= 10) {
init();
timer.removeEventListener(TimerEvent.TIMER, msgSent);
}
}
}
}
}
Keywords:ampersand special characters symbols less-than less than greater-than greater than please don't edit this, it's for others to find this question because you can't search for an '&' and such.
The most obvious culprit here is messy way you're creating the emailData string. As a first step I'd recommend reformatting it to the following:
var urlVars:URLVariables = new URLVariables();
urlVars.name = yourName.htmlText;
urlVars.from = fromEmail.htmlText;
urlVars.subject = yourSubject.htmlText;
urlVars.msg = yourMsg.htmlText;
I think this will automatically URI encode the values, but if not, use encodeURI() as suggested by Mark Knol.
Within Flash, the values need to be encoded, otherwise the querystring could be corrupted.
var emailData:String =
"name=" + encodeURI(yourName.htmlText) +
"&from=" + encodeURI(fromEmail.htmlText) +
"&subject=" + encodeURI(yourSubject.htmlText) +
"&msg=" + encodeURI(yourMsg.htmlText);
Try to use
$emailMsg = utf8_decode($emailMsg);
I decode all my strings I get from Flash.
If this doesn't help, use
$emailMsg = urldecode($emailMsg);
Or both :D
i have read the mail in php with imap_fetchbody() and decode it with imap_qprint().
$body = imap_fetchbody($mbox, $no, 1);
$body = imap_qprint($body);
but there is difference between original mail and output mail
Original mail:
Plain Text
This is a testing mail to check
first second third fourth fifth sixth Home
this is yellow background
website: http://bizved.com
testing of quote
Thank you Hiren Manek Bhuved Solutions
output mail
Plain Text
This is a testing mail to check
first second third fourth fifth sixth Home
this is yellow background
website:�http://bizved.com
testing of�quote
Thank you �Hiren Manek Bhuved Solutions
can anybody gives solution?
thanks in advance.
I always had the same problem with imaps. I'dont guarantee for anything but you may want to give this a try:
function utf8_imap_header_decode($str)
{
$tmp = imap_mime_header_decode($str);
if (!mb_check_encoding($tmp[0]->text, 'UTF-8'))
return utf8_encode($tmp[0]->text);
return $tmp[0]->text;
}
function utf8_imap_body_decode($str)
{
return utf8_encode(quoted_printable_decode($str));
}
I have make the following solution as mail headers already its character set.
$st = imap_fetchstructure($mbox, $no);
$part = $st->parts[$partno];
$body = imap_fetchbody($mbox, $no, $partno);
$body = imap_qprint($body);
$charset = 'UTF-8';
if(!empty($part->parameters)){
for ($k = 0, $l = count($part->parameters); $k < $l; $k++) {
$attribute = $part->parameters[$k];
if($attribute->attribute == 'CHARSET'){
$charset = $attribute->value;
}
}
}
//echo $charset;
$bodytext = mb_convert_encoding($body,'UTF-8',$charset);
This not full solution. It's only for character encoding. Mail has different parts for plain text, html text, attachments, etc. for that you should have different handling for each type.
I just started today working with PHP's IMAP library, and while imap_fetchbody or imap_body are called, it is triggering my Kaspersky antivirus. The viruses are Trojan.Win32.Agent.dmyq and Trojan.Win32.FraudPack.aoda. I am running this off a local development machine with XAMPP and Kaspersky AV.
Now, I am sure there are viruses there since there is spam in the box (who doesn't need a some viagra or vicodin these days?). And I know that since the raw body includes attachments and different mime-types, bad stuff can be in the body.
So my question is: are there any risks using these libraries?
I am assuming that the IMAP functions are retrieving the body, caching it to disk/memory and the AV scanning it sees the data.
Is that correct? Are there any known security concerns using this library (I couldn't find any)? Does it clean up cached message parts perfectly or might viral files be sitting somewhere?
Is there a better way to get plain text out of the body than this? Right now I am using the following code (credit to Kevin Steffer):
function get_mime_type(&$structure) {
$primary_mime_type = array("TEXT", "MULTIPART","MESSAGE", "APPLICATION", "AUDIO","IMAGE", "VIDEO", "OTHER");
if($structure->subtype) {
return $primary_mime_type[(int) $structure->type] . '/' .$structure->subtype;
}
return "TEXT/PLAIN";
}
function get_part($stream, $msg_number, $mime_type, $structure = false, $part_number = false) {
if(!$structure) {
$structure = imap_fetchstructure($stream, $msg_number);
}
if($structure) {
if($mime_type == get_mime_type($structure)) {
if(!$part_number) {
$part_number = "1";
}
$text = imap_fetchbody($stream, $msg_number, $part_number);
if($structure->encoding == 3) {
return imap_base64($text);
} else if($structure->encoding == 4) {
return imap_qprint($text);
} else {
return $text;
}
}
if($structure->type == 1) /* multipart */ {
while(list($index, $sub_structure) = each($structure->parts)) {
if($part_number) {
$prefix = $part_number . '.';
}
$data = get_part($stream, $msg_number, $mime_type, $sub_structure,$prefix . ($index + 1));
if($data) {
return $data;
}
} // END OF WHILE
} // END OF MULTIPART
} // END OF STRUTURE
return false;
} // END OF FUNCTION
$connection = imap_open($server, $login, $password);
$count = imap_num_msg($connection);
for($i = 1; $i <= $count; $i++) {
$header = imap_headerinfo($connection, $i);
$from = $header->fromaddress;
$to = $header->toaddress;
$subject = $header->subject;
$date = $header->date;
$body = get_part($connection, $i, "TEXT/PLAIN");
}
Your guess seems accurate. IMAP itself is fine. What you do with the contents is what's dangerous.
What's dangerous about virus e-mails is that users might open a .exe attachment or something, so bad attachments and potentially evil HTML are what's being checked. As long as your code handling attachments doesn't tell the user to open them and this is just automatic processing or whatever, you're good to go. If you're planning on outputting HTML contents, be sure to use something like HTML Purifier.
The AV is detecting these signatures as they pass through the networking stack, most likely. You should be able to tell the source of the detection from the messages Kaspersky is giving you.