PHP encoding problems - php

After a webhook (XML) trigger, I have a PHP code doing the following treatment:
$xmlData = fopen('php://input' , 'rb');
while (!feof($xmlData)) { $xmlString .= fread($xmlData, 4096); }
fclose($xmlData);
file_put_contents('orders/order' . date('m-d-y') . '-' . time() . '.xml', $xmlString, FILE_APPEND);
And I also transfer this info to a database:
$xml = new SimpleXMLElement($xmlString);
$address1 = trim($xml->{'billing-address'}->address1);
$sql="INSERT INTO `Customers`(`address1`)
VALUES
('$address1')";
My problem is that character is not properly transported for the xml file and the database.
Original statement:
São Paulo
XML file saved on the server:
<?xml version="1.0" encoding="UTF-8"?>
<address1>São Paulo</address1>
Information on database (utf8_general_ci):
São Paulo
Everything seems to be proper set to UTF-8 but I still have this character problems.

When you establish database connection run the query:
SET NAMES 'utf8
That should help. Also make sure the data coming from your "web hook" is in that format.

I think the php functions you are using are not intended to manage multibytes string. Try putting
mb_internal_encoding("UTF-8");
at the beginning of your php code and have a look to the Multibyte String Functions page on php.net to understand if you need to change some function in favour of its corresponding multibyte version.
As a last chance you might try to iconv the string before inserting it to the database:
$address1 = trim($xml->{'billing-address'}->address1);
$address1 = iconv(iconv_get_encoding($string), "UTF-8", $address1);

Related

PHP & JSON : Trying to write JSON to Mysql

I am having an issue where my PHP script opens a file with JSON code and needs to insert it into a MySQL database.
For some reason it only displays some of the output from the JSON.
Here is my code
$json = json_decode(file_get_contents('data.json'), true);
$data = $json;
// VAR's
$system = $data['System'];
$cid_from = $data["From"];
$cid_to = $data['To'];
//DEBUG USAGES
$array = print_r($data, true);
////// THIS ONE WORKS FINE
echo $data["System"];
////// THIS ONE DOESN'T WORK
echo $data["To"];
file_put_contents('output/json-local.txt',$array . "\r\n", FILE_APPEND);
////// BUT HERE IT ACTUALLY WORKS
file_put_contents('output/cli-from.txt',$data['From']. "\r\n", FILE_APPEND);
file_put_contents('output/cli-to.txt',$data['To']. "\r\n", FILE_APPEND);
// file_put_contents('json-sysid-local.txt',$systemid . "\r\n", FILE_APPEND);
Here is the contents of data.json
{"action":"call-data-record",
"System":"48130b83e2232f0ecd366a92d4d1261d",
"PrimaryCallID":"n1bWEfCdHcf#MSS.MTN.CO.ZA-b2b_1",
"CallID":"0440b807#pbx",
"From":"<sip:+27722080036#xxx.co.za>",
"To":"<sip:27102850816#xxx.co.za>",
"Direction":"O",
"RemoteParty":"",
"LocalParty":"",
"TrunkName":"",
"TrunkID":"",
"Cost":"",
"CMC":"",
"Domain":"xxx.co.za",
"TimeStart":"2018-08-14 16:03:21",
"TimeConnected":"",
"TimeEnd":"2018-08-14 16:03:23",
"LocalTime":"2018-08-14 18:03:21",
"DurationHHMMSS":"0:00:00",
"Duration":"0",
"RecordLocation":"",
"RecordUsers":"",
"Type":"hunt",
"Extension":"100",
"ExtensionName":"100",
"IdleDuration":"",
"RingDuration":"2",
"HoldDuration":"0",
"IvrDuration":"0",
"AccountNumber":"400",
"IPAdr":"",
"Quality":"VQSessionReport: CallTerm\r\nLocalMetrics:\r\nCallID:0440b807#pbx\r\nFromID:<sip:27102850816#xxx.co.za>\r\nToID:<sip:+27722080036#xxxx.co.za>;tag=1460166964\r\nx-UserAgent:Vodia-PBX/57.0\r\nx-SIPmetrics:SVA=RG SRD=91\r\nx-SIPterm:SDC=OK SDR=OR\r\n"}
Your "To" data is encapsulated in <>. This causes your browser to interpret it as an HTML tag and not display any content.
You can (should!) escape the special HTML control characters:
echo htmlspecialchars($data["To"]);
See http://php.net/htmlspecialchars
Edit: It doesn't hurt to precautionary add this to your other outputs aswell. If the string doesn't contain such characters, it will simply be returned onchanged. You eliminate possible XSS attack vectors this way.
The browser source clearly shows "To":"" is being written by PHP to the browser output correctly but the browser is interpreting as an HTML opening tag hence ignoring the rest of the content.
Wrap your output in the PHP htmlspecialchars() function to see the output as in the file.
Add - echo "TO : ".htmlspecialchars($data["To"]);

XML character encoding issue with PHP

I have code which is creating an XML, my only problem is with the encoding of words like á, olá and ção.
These characters dont appear correctly and when I try reading the XML I get an error displayed relating to that character.
$dom_doc = new DOMDocument("1.0", "utf-8");
$dom_doc->preserveWhiteSpace = false;
$dom_doc->formatOutput = true;
$element = $dom->createElement("hotels");
while ($row = mysql_fetch_assoc($result)) {
$contact = $dom_doc->createElement( "m" . $row['id'] );
$nome = $dom_doc->createElement("nome", $row['nome'] );
$data1 = $dom_doc->createElement("data1", $row['data'] );
$data2 = $dom_doc->createElement("data2", $row['data2'] );
$contact->appendChild($nome);
$contact->appendChild($data1);
$contact->appendChild($data2);
$element->appendChild($contact);
$dom_doc->appendChild($element);
What can I change to fix my problem, I am using utf-8???
Please try to put directly 'á', 'olá' or 'ção' in your script.
$data1 = $dom_doc->createElement("data1", 'ção');
If you don't have problem, this is probably the data you get from mysql that are wrongly encoded.
Are you sure your mysql outputs correct UTF-8?
To know that, make your PHP dump your data in an HTML document with meta tag set to UTF-8 and see if the characters display correctly.
You can also call :
$data1 = $dom_doc->createElement("data1", mb_detect_encoding($row['data']));
and see what encoding is detected by PHP for your data.
If you can't convert the data from your database, or change its settings, you can use mb_convert to do it on-the-fly : http://www.php.net/manual/en/function.mb-convert-encoding.php
You are using utf-8, the 8-bit unicode encoding format. Even though it properly supports all 1,112,064 code points in Unicode its possible that there is an issue here.
Try UTF-16 as the standard, just an idea. See below:
$dom_doc = new DOMDocument("1.0", "utf-16");
OR
$dom_doc = new DOMDocument("1.0", "ISO-10646");

Ampersands in database

I am trying to write a php function that goes to my database and pulls a list of URLS and arranges them into an xml structure and creates an xml file.
Problem is, Some of these urls will contain an ampersand that ARE HTML encoded. So, the database is good, but currently, when my function tries to grab these URLS, the script will stop at the ampersands and not finish.
One example link from database:
http://www.mysite.com/myfile.php?select=on&league_id=8&sport=15
function buildXML($con) {
//build xml file
$sql = "SELECT * FROM url_links";
$res = mysql_query($sql,$con);
$gameArray = array ();
while ($row = mysql_fetch_array($res))
{
array_push($row['form_link']);
}
$xml = '<?xml version="1.0" encoding="utf-8"?><channel>';
foreach ($gameArray as $link)
{
$xml .= "<item><link>".$link."</link></item>";
}
$xml .= '</channel>';
file_put_contents('../xml/full_rankings.xml',$xml);
}
mysql_close($con);
session_write_close();
If i need to alter the links in the database, that can be done.
You can use PHP's html_entity_decode() on the $link to convert & back to &.
In your XML, you could also wrap the link in <![CDATA[]]> to allow it to contain the characters.
$xml .= "<item><link><![CDATA[" . html_entity_decode($link) . "]]></link></item>";
UPDATE
Just noticed you're actually not putting anything into the $gameArray:
array_push($row['form_link']);
Try:
$gameArray[] = $row['form_link'];
* #Musa looks to have noticed it first, for due credit.
Look at this line
array_push($row['form_link']);
you never put anything in the $gameArray array, it should be
array_push($gameArray, $row['form_link']);
You need to use htmlspecialchars_decode. It will decode any encoded special characters in string passed to it.
This is most likely what you are looking for:
http://www.php.net/manual/en/function.mysql-real-escape-string.php
Read the documentation, there are examples at the bottom of the page...
'&' in oracleSQL and MySQL are used in queries as a logical operator which is why it is tossing an error.
You may also want to decode the HTML...

Creating/Writing an XML file in PHP?

I currently have a script written in PHP where I connect to a database in phpMyAdmin, and then parse all of the table values into an XML document.
Here is how the script works:
$xmlBody .= "<XML>";
$sql_news = mysql_query("SELECT * FROM news_table");
$xmlBody .= "<News_Table>";
//this while loop churns out the values of our "news_table" table
while($row_news = mysql_fetch_array($sql_news)){
// Set DB variables into local variables for easier use
$id_news = $row_news["news_id"];
$author_news = $row_news["news_author"];
$time_news = $row_news["news_time"];
$title_news = $row_news["news_title"];
$content_news = $row_news["news_content"];
$desc_news = $row_news["news_description"];
$image_news = $row_news["news_image"];
$xmlBody .= '
<News_Table_Entry>
<DataID>' . $id_news . '</DataID>
<DataAuthor>' . $author_news . '</DataAuthor>
<DataTime>' . $time_news . '</DataTime>
<DataTitle>' . $title_news . '</DataTitle>
<DataContent>' . $content_news . '</DataContent>
<DataDesc>' . $desc_news . '</DataDesc>
<DataImage>' . $image_news . '</DataImage>
</News_Table_Entry>';
} // End while loop
$xmlBody .= "</News_Table>";
mysql_close(); // close the mysql database connection
$xmlBody .= "</XML>";
echo $xmlBody;
?>
How do I create and output this as an external XML file? I've successfully got this script working, but using all of the methods for writing out to XML isn't working. Using the
echo 'Wrote: ' . $doc->save("/tmp/test.xml") . ' bytes'; // Wrote: 72 bytes
Function isn't working, along with the fwrite function as well. I've been working at trying to figure this out for a few days, but none of the solutions I've been told to try out have worked. Any help or insight would be greatly appreciated!
According to your code, you're already building the XML content yourself. XML files are just regular text files, so in this case you don't need any of the special XML functions that validate and render. Instead, you can simply save your text to the .xml file:
file_put_contents('/tmp/test.xml', $xmlBody);
file_put_contents allows you to forego all the fopen/fwrite functions, so it's the easiest way to write content to disk.
On the other hand, if you do want to learn to build a structured XML document with all the bells and whistles of consistency, look up SimpleXML or XMLWriter. A little more overhead that way, but doing all the markup by hand can be unwieldy, especially when one typo can invalidate your whole document.
This will help you
$xml_msg_in = fopen('/tmp/test.xml',"w");
fwrite($xml_msg_in,$xmlBody);
fclose($xml_msg_in);
Have you tried file_put_contents('/path/to/output.xml', $xmlBody);
Just to understand, does your echo $xmlBody; line at least work as expected?
And if you are trying to use the XML file somewhere without success, then note you're missing the XML header as the first line of the XML file.
<?xml version="1.0"?>
Note that depending on the data in your database, this header may need to specify a wider encoding such as UTF-16. But you should choose what character sets you want to allow in that database.
<?xml version="1.0" encoding="UTF-16"?>
And what is the trailing ?> at the end of your code sample?

PHP simplexml_load_file with special chars in URL

I'm attempting to retrieve a local weather forecast, based on the IP of the user.
I'm using geoplugin.net to get the user location and feed the city and country name to the Google Weather API.
//Get user IP
$ip = $_SERVER['REMOTE_ADDR'];
$geolocation = unserialize(file_get_contents('http://www.geoplugin.net/php.gp?ip='.$ip));
$geo_city = $geolocation['geoplugin_city'];
$geo_country = $geolocation['geoplugin_countryName'];
$file = "http://www.google.com/ig/api?weather=".$geo_city.",".$geo_country;
$xml = simplexml_load_file($file);
//Echo content of retrieved XML for debugging purposes
echo "<pre>";
print_r($xml);
echo "</pre>";
It works well for most cases, but when I try it on my own IP, I get Søborg, Denmark (which is not 100% accurate, but close enough) and that gives me an almost empty response from the weather API.
The main suspect in the case, is the dastardly "ø"-character.
The XML that I want can be seen here: http://www.google.com/ig/api?weather=S%C3%B8borg,Denmark
The XML that I'm getting can be seen here: http://www.google.com/ig/api?weather=S
When I type this URL into the browser it works fine:
http://www.google.com/ig/api?weather=Søborg,Denmark
When I use this version it works as well (in the browser):
http://www.google.com/ig/api?weather=S%C3%B8borg,Denmark
but this version returns the forecast for Borg,Syddanmark:
http://www.google.com/ig/api?weather=S%26oslash%3Bborg,Denmark
None of the above returns the desired result, when fed to the simplexml_load_file().
As stated, I suspect that it is a character set issue, but I can't figure out what to do about it.
What is the correct way to solve it?
I know that I can use latitude and longtitude as parameters for Google Weather API instead, but that's just circumventing the problem, not solving it.
If you URL-decode S%26oslash%3Bborg you'll see that this string corresponds to Søborg which gives us Søborg after we decode HTML entities like so:
$city = 'S%26oslash%3Bborg,Denmark';
echo $city = rawurldecode($city);
//prints Søborg,Denmark
echo $city = html_entity_decode($city, 0, 'UTF-8');
//prints Søborg,Denmark
echo $city = rawurlencode($city);
//prints S%C3%B8borg%2CDenmark
And then:
$xml = file_get_contents('http://www.google.com/ig/api?weather='.$city);
$xml = mb_convert_encoding($xml, 'UTF-8');
$xml = simplexml_load_string($xml);
echo $xml->weather->forecast_information->city['data'];
Outputs expected:
Søborg, Capital Region of Denmark
It does indeed sound like a character set issue. Have you tried converting the URL to another encoding, e.g. using iconv, before passing the result into simplexml_load_file()?
Try this out:
$file = "http://www.google.com/ig/api?weather=" . $geo_city . "," . $geo_country;
$data = file_get_contents($file);
$data = mb_convert_encoding($data, "UTF-8", "ISO-8859-2");
$xml = simplexml_load_string($data);
echo "<pre>"; print_r($xml); echo "</pre>";
It's taken from this maybe similar thread: https://stackoverflow.com/a/5136549/949476

Categories