I have code which is creating an XML, my only problem is with the encoding of words like á, olá and ção.
These characters dont appear correctly and when I try reading the XML I get an error displayed relating to that character.
$dom_doc = new DOMDocument("1.0", "utf-8");
$dom_doc->preserveWhiteSpace = false;
$dom_doc->formatOutput = true;
$element = $dom->createElement("hotels");
while ($row = mysql_fetch_assoc($result)) {
$contact = $dom_doc->createElement( "m" . $row['id'] );
$nome = $dom_doc->createElement("nome", $row['nome'] );
$data1 = $dom_doc->createElement("data1", $row['data'] );
$data2 = $dom_doc->createElement("data2", $row['data2'] );
$contact->appendChild($nome);
$contact->appendChild($data1);
$contact->appendChild($data2);
$element->appendChild($contact);
$dom_doc->appendChild($element);
What can I change to fix my problem, I am using utf-8???
Please try to put directly 'á', 'olá' or 'ção' in your script.
$data1 = $dom_doc->createElement("data1", 'ção');
If you don't have problem, this is probably the data you get from mysql that are wrongly encoded.
Are you sure your mysql outputs correct UTF-8?
To know that, make your PHP dump your data in an HTML document with meta tag set to UTF-8 and see if the characters display correctly.
You can also call :
$data1 = $dom_doc->createElement("data1", mb_detect_encoding($row['data']));
and see what encoding is detected by PHP for your data.
If you can't convert the data from your database, or change its settings, you can use mb_convert to do it on-the-fly : http://www.php.net/manual/en/function.mb-convert-encoding.php
You are using utf-8, the 8-bit unicode encoding format. Even though it properly supports all 1,112,064 code points in Unicode its possible that there is an issue here.
Try UTF-16 as the standard, just an idea. See below:
$dom_doc = new DOMDocument("1.0", "utf-16");
OR
$dom_doc = new DOMDocument("1.0", "ISO-10646");
Related
This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 7 months ago.
I'm using json_encode($data) to an data array and there's a field contains Russian characters.
I used this mb_detect_encoding() to display what encoding it is for that field and it displays UTF-8.
I think the json encode failed due to some bad characters in it like "ра▒". I tried alot of things utf8_encode on the data and it will by pass that error but then the data doesn't look correct anymore.
What can be done with this issue?
The issue happens if there are some non-utf8 characters inside even though most of them are utf8 chars. This will remove any non-utf8 characters and now it works.
$data['name'] = mb_convert_encoding($data['name'], 'UTF-8', 'UTF-8');
If you have a multidimensional array to encode in JSON format then you can use below function:
If JSON_ERROR_UTF8 occurred :
$encoded = json_encode( utf8ize( $responseForJS ) );
Below function is used to encode Array data recursively
/* Use it for json_encode some corrupt UTF-8 chars
* useful for = malformed utf-8 characters possibly incorrectly encoded by json_encode
*/
function utf8ize( $mixed ) {
if (is_array($mixed)) {
foreach ($mixed as $key => $value) {
$mixed[$key] = utf8ize($value);
}
} elseif (is_string($mixed)) {
return mb_convert_encoding($mixed, "UTF-8", "UTF-8");
}
return $mixed;
}
Please, make sure to initiate your Pdo object with the charset iso as utf8.
This should fix this problem avoiding any re-utf8izing dance.
$pdo = new PDO("mysql:host=localhost;dbname=mybase;charset=utf8", 'user', 'password');
With php 7.2, two options allow to manage invalid UTF-8 direcly in json_encode :
https://www.php.net/manual/en/function.json-encode
json_encode($text, JSON_INVALID_UTF8_IGNORE);
Or
json_encode($text, JSON_INVALID_UTF8_SUBSTITUTE);
you just add in your pdo connection charset=utf8
like below line of pdo connection:
$pdo = new PDO("mysql:host=localhost;dbname=mybase;charset=utf8", 'user', 'password');
hope this will help you
Remove HTML entities before JSON encoding. I used html_entity_decode() in PHP and the problem was solved
$json = html_entity_decode($source);
$data = json_decode($json,true);
Do you by any chance have UUIDs in your result set? In that case the following database flag will help:
PDO::DBLIB_ATTR_STRINGIFY_UNIQUEIDENTIFIER => true
If your data is well encoded in the database for example, make sure to use the mb_ * functions for string handling, before json_encode. Functions like substr or strlen do not work well with utf8mb4 and can cut your text and leave a malformed UTF8
I know this is kind of an old topic, but for me it was what I needed. I just needed to modify the answer 'jayashan perera'.
//...code
$stmt->execute();
$result = $stmt->fetchAll(PDO::FETCH_ASSOC);
for ($i=0; $i < sizeof($result) ; $i++) {
$tempCnpj = $result[$i]['CNPJ'];
$tempFornecedor = json_encode(html_entity_decode($result[$i]['Nome_fornecedor']),true) ;
$tempData = $result[$i]['efetivado_data'];
$tempNota = $result[$i]['valor_nota'];
$arrResposta[$i] = ["Status"=>"true", "Cnpj"=>"$tempCnpj", "Fornecedor"=>$tempFornecedor, "Data"=>"$tempData", "Nota"=>"$tempNota" ];
}
echo json_encode($arrResposta);
And no .js i have use
obj = JSON.parse(msg);
I have some data I collect with DomCrawler and store in an array, but it looks like he fails when it comes to special characters like è,à,ï,etc.
As an example I get è instead of è when I echo the result.
When I store my results in a .json file I get this: \u00c3\u00a8
My goal is to save the special character in the .json file.
I've tried encoding it but doesn't seem to have the result I want.
$html = file_get_contents($url);
$crawler = new Crawler($html);
$h1 = $crawler->filter('h1');
$title = $h1->text();
$title = mb_convert_encoding($title, "HTML-ENTITIES", "UTF-8");
Is there anyway I can have my special characters shown?
Thanks a lot!
By using the constructor to add the HTML, the crawler assume that it is in ISO-8859-1. You have to explicitly tell it that your DOM is in UTF-8 with the addHTMLContent method:
$html = file_get_contents($url);
$crawler = new Crawler;
$crawler->addHTMLContent($html, 'UTF-8');
I am trying to save data to a XML file.
if (isset( $_POST['submit'])) {
$name = mysql_real_escape_string($_POST['name']);
$xml = new DOMDocument("1.0", "ISO-8859-1");
$xml->preserveWhiteSpace = false;
$xml->load('/var/www/Report/file.xml');
$element = $xml->getElementsByTagName('entries');
$newItem = $xml->createElement('reports');
$newItem->appendChild($xml->createElement('timestamp', date("F j, Y, g:i a",time())));
$newItem->appendChild($xml->createElement('name', $name));
$element -> item(0) -> appendChild($newItem);
$xml->formatOutput = true; // this adds spaces, new lines and makes the XML more readable format.
$xmlString = $xml->saveXML(); // $xmlString contains the entire String
$xml->save('/var/www/Report/file.xml');
}
Anytime I use mysql_real_escape_string() to escapes special characters in my string or try to to sanitize my data, my XML file looks like something in the image below.
I don't understand why the name start tag is missing in my XML file and why my data $name isn't saved into the XML file either. How could I to fix this. Any help would be appreciated. Thanks in advance
1) Firefox does not show the xml byte-by-byte, rather it formats it itself. Therefore empty elements appear with only one self closing tag, no matter how they appear in the xml source. If you want to see the exact xml output, then open it in a text editor.
2) mysql_real_escape_string escapes characters that are problematic for mysql, not for XML: for example it does not escape '<'. Instead it, you should use DOMDocument::createTextNode. So instead of
$newItem->appendChild($xml->createElement('name', $name));
use
$nameElement = $xml->createElement('name');
$nameElement->appendChild( $xml->createTextNode($name) );
$newItem->appendChild($nameElement);
3) As you see, the $name is empty, but I don't know what can be the problem with your POST data seeing only this code, maybe there is no <input name='name' /> in your form: I sometimes accidentally set an id attribute to inputs instead of the name attribute.
After a webhook (XML) trigger, I have a PHP code doing the following treatment:
$xmlData = fopen('php://input' , 'rb');
while (!feof($xmlData)) { $xmlString .= fread($xmlData, 4096); }
fclose($xmlData);
file_put_contents('orders/order' . date('m-d-y') . '-' . time() . '.xml', $xmlString, FILE_APPEND);
And I also transfer this info to a database:
$xml = new SimpleXMLElement($xmlString);
$address1 = trim($xml->{'billing-address'}->address1);
$sql="INSERT INTO `Customers`(`address1`)
VALUES
('$address1')";
My problem is that character is not properly transported for the xml file and the database.
Original statement:
São Paulo
XML file saved on the server:
<?xml version="1.0" encoding="UTF-8"?>
<address1>São Paulo</address1>
Information on database (utf8_general_ci):
São Paulo
Everything seems to be proper set to UTF-8 but I still have this character problems.
When you establish database connection run the query:
SET NAMES 'utf8
That should help. Also make sure the data coming from your "web hook" is in that format.
I think the php functions you are using are not intended to manage multibytes string. Try putting
mb_internal_encoding("UTF-8");
at the beginning of your php code and have a look to the Multibyte String Functions page on php.net to understand if you need to change some function in favour of its corresponding multibyte version.
As a last chance you might try to iconv the string before inserting it to the database:
$address1 = trim($xml->{'billing-address'}->address1);
$address1 = iconv(iconv_get_encoding($string), "UTF-8", $address1);
I generate a lot of posts in Wordpress from an XML file. The worry: accented characters.
The header of the stream is:
<? Xml version = "1.0" encoding = "ISO-8859-15"?>
Here is the complete flux : http://flux.netaffiliation.com/rsscp.php?maff=177053821BA2E13E910D54
My site is in utf8.
So I use the function utf8_encode ... but that does not solve the problem, the accents are always misunderstood.
Does anyone have an idea?
EDIT 04-10-2011 18:02 (french hour) :
Here is the complete flux : http://flux.netaffiliation.com/rsscp.php?maff=177053821BA2E13E910D54
Here is my code :
/**
* parse an rss flux from netaffiliation and convert each item to posts
* #var $flux = external link
* #return bool
*/
private function parseFluxNetAffiliation($flux)
{
$content = file_get_contents($flux);
$content = iconv("iso-8859-15", "utf-8", $content);
$xml = new DOMDocument;
$xml->loadXML($content);
//get the first link : http://www.netaffiliation.com
$link = $xml->getElementsByTagName('link')->item(0);
//echo $link->textContent;
//we get all items and create a multidimentionnal array
$items = $xml->getElementsByTagName('item');
$offers = array();
//we walk items
foreach($items as $item)
{
$childs = $item->childNodes;
//we walk childs
foreach($childs as $child)
{
$offers[$child->nodeName][] = $child->nodeValue;
}
}
unset($offers['#text']);
//we create one article foreach offer
$nbrPosts = count($offers['title']);
if($nbrPosts <= 0)
{
echo self::getFeedback("Le flux ne continent aucune offre",'error');
return false;
}
$i = 0;
while($i < $nbrPosts)
{
// Create post object
$description = '<p>'.$offers['description'][$i].'</p><p>'.$offers['link'][$i].'</p>';
$my_post = array(
'post_title' => $offers['title'][$i],
'post_content' => $description,
'post_status' => 'publish',
'post_author' => 1,
'post_category' => array(self::getCatAffiliation())
);
// Insert the post into the database
if(!wp_insert_post($my_post));;
$i++;
}
echo self::getFeedback("Le flux a généré {$nbrPosts} article(s) depuis le flux NetAffiliation dans la catégorie affiliation",'updated');
return false;
}
All the posts are generated but... the accented chars are ugly. You can see the result here: http://monsieur-mode.com/test/
There are plenty difficulties which you have to master when swapping between different encodings. Also, encodings which use more than one byte to encode characters (so-called multibyte-encodings) like UTF-8, which is used by WordPress, deserve special attention in PHP.
First, make sure that all the files you create are saved with the same encoding as they will be served. For example, make sure you set the same encoding as in the "Save as..."-dialog as you use in the HTTP Content-Type header.
Second, you need to verify that the input has the same encoding as the file you want to deliver. In your case, the input file has the encoding ISO-8859-15, so you'll need to convert it to UTF-8 using iconv().
Third, you must know that PHP doesn't natively support multibyte-encodings such as UTF-8. Functions such as htmlentities() will produce strange characters. For many of these functions, there are multibyte-alternatives, which are prefixed with mb_. If your encoding is UTF-8, check your files for such functions and replace them if necessary.
For more information about these topics, see Wikipedia about variable-width encodings, and the page in the PHP-Manual.
By default, most application work with UTF-8 data and output UTF-8 content. Wordpress should definitely not be apart and surely works on a UTF-8 basis.
I would simply not convert at all any information when printing, but instead change your header to UTF-8 instead of ISO-8859-15.
If your incoming XML data is ISO-8859-15, use iconv() to convert it:
$stream = file_get_contents("stream.xml");
$stream = iconv("iso-8859-15", "utf-8", $stream);
mb_convert_encoding()saves my life.
Here is my solution :
$content = preg_replace('/ encoding="ISO-8859-15"/is','',$content);
$content = mb_convert_encoding($content,"UTF-8");