SimpleXML will convert all text into UTF-8, if the source XML declaration has another encoding. So, all the text in the resulting SimpleXMLElement will be in UTF-8 automatically.
In my case the source has the following XML decl:
<?xml version="1.0" encoding="windows-1251" ?>
What should I do so as to get normal output? Because, as you can imagine, for now I get stange symbols.
Thanks.
Maybe a stupid answer, but just don't use SimpleXML. Just use DOM.
Try using the iconv to convert the encoding.
Using the iconv() function you can convert from one encodign to another, the TRANSLIT option might work.
$xml = {STRING CONTAINING YOUR XML FILE DATA};
<?php
// convert string from utf-8 to iso8859-1
//$xml = iconv( "UTF-8", "ISO-8859-1//TRANSLIT", $xml);
$xml = iconv( "YOUR_ENCODING", "UTF-8//TRANSLIT", $xml);
?>
My advice is to use UTF-8 as source .php files encoding and (if possible) output encoding too. With gzip compression difference between size of windows-1251 and UTF-8 replies (even for mostly Cyrillic text) is minimal and UTF-8 is better in many ways.
As you said, simplexml will convert windows-1251 to UTF-8 on xml import and then you don't have to worry about any encodings.
If you have to use windows-1251 for output then use something like:
iconv_set_encoding("internal_encoding", "UTF-8");
iconv_set_encoding("output_encoding", "windows-1251");
ob_start("ob_iconv_handler");
One catchup for UTF-8 in PHP source files are char classes in regexps: /[ю]/ won't work as you might have expected, /(ю)/ will.
Related
I have a string that contains a right single quotation mark:
$str = "David’s Spade";
I am sending the string via XML and need to encode it. I have read that I should encode string using htmlspecialchars, but I have found that XML request still fails whereas htmlentities works.
When I error_log $str:
$str; // David\xe2\x80\x99s Spade
htmlspecialchars($str); // David\xe2\x80\x99s Spade
htmlspecialchars($str, ENT_QUOTES, 'UTF-8'); // David\xe2\x80\x99s Spade
htmlentities($str); // David’s Spade
Would it be better to str_replace ’ and then use htmlentities? Are there any other chars htmlentities may miss?
I am sending the string via XML and need to encode it.
No, you don't. If the XML is UTF-8 encoded (it is by default) and as your $str is UTF-8 encoded (as you show by the binary sequences in your question), you do not need to encode it.
This is by the book. So given on the technical information of the data you collaborate with, this is clear and fine.
You then write that some things work and others won't. Whatever you do there, there problem lies within the things you've hidden from your question.
To make this more explicit:
$str = "David’s Spade"; // "David\xE2\x80\x99s Spade"
is a perfectly valid string, for example to use it with an XML library like Simplexml to add it to an XML document:
$xml = new SimpleXMLElement('<?xml version="1.0" encoding="UTF-8"?><doc/>');
$xml->element = $str;
$xml->asXML('php://output');
Output:
<?xml version="1.0" encoding="UTF-8"?>
<doc><element>David’s Spade</element></doc>
As you can see, the XML has been encoded by not changing the byte-sequence of the string here because it's UTF-8.
Let's take some ASCII:
$xml = new SimpleXMLElement('<doc/>');
$xml->element = $str;
$xml->asXML('php://output');
Output:
<?xml version="1.0"?>
<doc><element>David’s Spade</element></doc>
As this example shows, it depends on the document encoding then. This second example is a fall-back of Simplexml to make the output more robust, but actually this wouldn't be necessary as UTF-8 would be the default encoding.
In any case you should not be too concerned about the encoding yourself by using a library that has specialized on creating XML documents. PHP has some few for exactly that. Take one of them.
I just faced that HTML document encoded with UTF-8 becomes incorrect after loadHTML().
There are plenty of such QnA's on the Stack:
PHP DomDocument failing to handle utf-8 characters
PHP DOMDocument loadHTML not encoding UTF-8 correctly
PHP DOM UTF-8 problem
However as I understand most of the answers taking assumption that the source encoding is UTF-8. So it is recommended to use mb_convert_encoding() function in this way:
$dom->loadHTML(mb_convert_encoding($document_in_utf_8, 'HTML-ENTITIES', 'UTF-8'));
I suppose that this will work only if the source document is in UTF-8. Unfortunately in my world the source document can has any available codding: Windows-1251, UTF-8, KOI8-R and so on...
So what is the best practice to handle this problem for any encoding?
UPDATE 1: Just find mb_detect_encoding() function. Is it a good practice to use one to detect the encoding? In the way like this:
$encoding = mb_detect_encoding($doc);
$doc = mb_convert_encoding($doc, 'HTML-ENTITIES', $encoding);
$dom->loadHTML($doc);
Tested this on several documents: looks like it working, but can I be sure that this will work for all reasonable cases?
I'm parsing an xml document using php.
When I see the result in my browser I get the following characters:
ñ instead of spanish ñ
à instead of í
á instead of á
ó instead of ó
é instead of é
I was going to use a str_replace and replace every odd character for the good ones, but sadly the pattern before happens only sometimes and in general I have a wide collection of odd characters :(
The xml heading is:
<?xml version="1.0" encoding="iso-8859-1"?>
But if I change it to utf-8 it simply won't be printed ..
I load the xml as a string with simplexml_load_string (comes from database like that)
Can you please give me any ideas on how to solve this?
Thanks a lot
You have 2 options:
a) include a header('Content-Type: text/html; charset=iso-8859-1'); before any output in your php file.
b) convert the output to utf-8 with $str = mb_convert_encoding($str, 'UTF-8', 'ISO-8859-1');
Both should do the trick.
SimpleXML uses UTF-8 to encode stored strings. You can use an XML-File with iso-8859-1, but if you want to print XML values with this encoding, you have to use utf8_decode before.
$string = preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $string);
// new xml
$xml = new SimpleXMLElement('new.xml');
// Displaying XML in textual form
echo $xml->asXML();
I'm not really sure if this is an encoding problem or what, but I have a problem using simple xml with some of the characters in the text
$xml = <<<HOHOHO
<?xml version="1.0" encoding="iso-8859-2" standalone="yes"?>
<videos>
<video>
<ContentProvider>bl abla</ContentProvider>
<ArtistName>T-Boz</ArtistName>
<CopyrightLine>(C)2009 SME España, S.</CopyrightLine>
</video>
</videos>
HOHOHO;
$a = simplexml_load_string ($xml);
foreach ( $a->video as $new )
die($new->CopyrightLine);
The thing is that the ñ character gets all messed up and becomes something like Ăą, when it should be a ñ.
I find it strange simplexml changes this to a character anyway instead of just keeping it as it is...
I know that this has to do something with hex codes but I haven't found a solution yet
Things I've tried so far:
converting the string to iso-8859-2 with mb_convert_string,
converting the string to utf-8 with mb_convert_string,
converting with html_entity_decode,
converting with html_special chars
all of above attempts either failed to parse xml or just didn't fix the character
Help would me very appreciated!
The problem you have is not the input string, but the output string. SimpleXML uses UTF-8 internally, and if you request a string from the SimpleXMLElement, you will get the string encoded as UTF-8.
$output = (string) $new->CopyrightLine; # will always be UTF-8 encoded
So you need to the re-encoding with the output, not the input.
Compare with this code example and output, that is displayed as UTF-8 while the input is your input.
There is no way around this btw, because SimpleXML will always give you UTF-8 encoded strings.
I have a large file that contains world countries/regions that I'm seperating into smaller files based on individual countries/regions. The original file contains entries like:
EE.04 Järvamaa
EE.05 Jõgevamaa
EE.07 Läänemaa
However when I extract that and write it to a new file, the text becomes:
EE.04 Järvamaa
EE.05 Jõgevamaa
EE.07 Läänemaa
To save my files I'm using the following code:
mb_detect_encoding($text, "UTF-8") == "UTF-8" ? : $text = utf8_encode($text);
$fp = fopen(MY_LOCATION,'wb');
fwrite($fp,$text);
fclose($fp);
I tried saving the files with and without utf8_encode() and neither seems to work. How would I go about saving the original encoding (which is UTF8)?
Thank you!
First off, don't depend on mb_detect_encoding. It's not great at figuring out what the encoding is unless there's a bunch of encoding specific entities (meaning entities that are invalid in other encodings).
Try just getting rid of the mb_detect_encoding line all together.
Oh, and utf8_encode turns a Latin-1 string into a UTF-8 string (not from an arbitrary charset to UTF-8, which is what you really want)... You want iconv, but you need to know the source encoding (and since you can't really trust mb_detect_encoding, you'll need to figure it out some other way).
Or you can try using iconv with a empty input encoding $str = iconv('', 'UTF-8', $str); (which may or may not work)...
It doesn't work like that. Even if you utf8_encode($theString) you will not CREATE a UTF8 file.
The correct answer has something to do with the UTF-8 byte-order mark.
This to understand the issue:
- http://en.wikipedia.org/wiki/Byte_order_mark
- http://unicode.org/faq/utf_bom.html
The solution is the following:
As the UTF-8 byte-order mark is '\xef\xbb\xbf' we should add it to the document's header.
<?php
function writeStringToFile($file, $string){
$f=fopen($file, "wb");
$file="\xEF\xBB\xBF".$string; // utf8 bom
fputs($f, $string);
fclose($f);
}
?>
The $file could be anything text or xml...
The $string is your UTF8 encoded string.
Try it now and it will write a UTF8 encoded file with your UTF8 content (string).
writeStringToFile('test.xml', 'éèàç');
Maybe you want to call htmlentities($text) before writing it into file and html_entity_decode($fetchedData) before output. It'll work with Scandinavian letters.
It appears that your source file is not, in fact, in UTF-8. You might want to try using the same approach you've been using, but with a different encoding, such as UTF-16 perhaps.
You can do it as follows:
<?php
$s = "This is a string éèàç and it is in utf-8";
$f = fopen('myFile',"w");
fwrite($f, utf8_encode($s));
fclose($f);
?>