We are getting base64 encoded (XML) data from a third party. If the XML data is in English, everything works fine, I am able do base64 decode, and parse the XML. If the XML is all lower case French characters, everything works fine. But if the xml data contains upper case French characters (like À), if I do base64 decode and try to parse it, the parser fails. Any suggestions on how to fix this problem?
Thanks.
Base64 is a method to encode 8-bit binary data using 7-bits/US-ASCII charachters. After the Base64 decode you should have a standard XML file.
Probably this XML file contains illegal characters, or does not correctly specify the character encoding it uses.
You mention À, an HTML-specific (not-XML) representation of À. If the XML contains the HTML encoded string À, there should also be a reference in the XML to an entity table specifying how to decode that string.
Alternatively, if your XML contains the À character directly, encoded using (for example) the ISO-8859-1 character set, either your XML should specify this encoding (<?xml version="1.0" encoding="ISO-8859-1"?>), or you should specify it yourself when decoding it.
Failing that, the parser may assume (e.g) UTF-8 encoding is used, and will fail when trying to decode the À.
The exact error message should tell you what the problem is.
[update: À directly]:
Sounds like the XML is invalid then; that they say UTF-8 but are actually using a different encoding. Check the XML bytes (after the base 64 decode) for this; if the À is encoded as one byte, it is definitely not UTF-8.
[update: how to fix?] If they incorrectly specify it in the XML header, they should really replace the false header (<?xml version="1.0" encoding="UTF-8"?>) with the correct one (<?xml version="1.0" encoding="windows-1252"?>).
If they don't specify anything, it looks like the iconv function may be your best bet. I haven't really needed it, so I'm not 100 % sure about this, but looks like you could use: $data = iconv("ISO-8859-1", "UTF-8", $data) after the base64_decode and before the simplexml_load_string. I don't know of a way to specify the encoding directly while decoding the XML.
I'm not really experienced with the PHP specifics of character encoding, so I'm not giving any guarantees...
What's the XML character encoding? Maybe it's not UTF-8 and your parser is trying to parse the XML string as UTF-8.
Related
I have read a XML file with the simplexml_load_file() function. I suppose this function is well written and supports XML encodings correctly. My XML file is in UTF-8 format, i.e. it contains normal ANSI characters along with national characters with multibyte encoding.
So, now I want to write the XML file back with fopen() and fwrite(), also in UTF-8 format.
Should I perform some conversions to do that correctly?
Suppose variable $a contains some UTF-8 encodings. Will it be written correctly?
if your xml is really encoded in utf8 already, then the strings produced by simplexml will be utf8 also. meaning- just write them to the file as-is.
php's interfacing with the underlying libxml is a bit funky though. Even though the xml may be utf8 encoded, make sure that the xml starts with a proper encoding declaration or it may get misinterpreted.
<?xml version="1.0" encoding="UTF-8"?>
I'm not really sure if this is an encoding problem or what, but I have a problem using simple xml with some of the characters in the text
$xml = <<<HOHOHO
<?xml version="1.0" encoding="iso-8859-2" standalone="yes"?>
<videos>
<video>
<ContentProvider>bl abla</ContentProvider>
<ArtistName>T-Boz</ArtistName>
<CopyrightLine>(C)2009 SME España, S.</CopyrightLine>
</video>
</videos>
HOHOHO;
$a = simplexml_load_string ($xml);
foreach ( $a->video as $new )
die($new->CopyrightLine);
The thing is that the ñ character gets all messed up and becomes something like Ăą, when it should be a ñ.
I find it strange simplexml changes this to a character anyway instead of just keeping it as it is...
I know that this has to do something with hex codes but I haven't found a solution yet
Things I've tried so far:
converting the string to iso-8859-2 with mb_convert_string,
converting the string to utf-8 with mb_convert_string,
converting with html_entity_decode,
converting with html_special chars
all of above attempts either failed to parse xml or just didn't fix the character
Help would me very appreciated!
The problem you have is not the input string, but the output string. SimpleXML uses UTF-8 internally, and if you request a string from the SimpleXMLElement, you will get the string encoded as UTF-8.
$output = (string) $new->CopyrightLine; # will always be UTF-8 encoded
So you need to the re-encoding with the output, not the input.
Compare with this code example and output, that is displayed as UTF-8 while the input is your input.
There is no way around this btw, because SimpleXML will always give you UTF-8 encoded strings.
I am receiving an XML response which comes through perfectly.
Words such as "português" and "españa" are correctly formatted.
However, once I have parsed the XML through the php function simplexml_load_string(), the words are transformed as follows: "portugu�s" and "espa�a".
Simple XML always treats text internally as UTF-8 encoded, converting to and from this character set if necessary. To solve your issue either make sure that all output from your app is UTF-8 encoded or convert it to another character set (possibly using utf8decode()).
Im having dificulty getting the letter
Ú
to render through PHP 5.3 and XSL. Its part of a string in a database and that is loaded into an XML node within a tags. However it causes the XSL/XML transformation to not render. Removing the character from the string fixes the problem instantly.
Any ideas?
What character encoding are you using? From the sounds of it you have some sort of character encoding mismatch.
If your XSL is using ISO-8559-1 (or ASCII equivalent) and you are trying to output to a page that is UTF-8 encoded then the character output will be off. It also works vice-versa.
Actually I don't know right answer but I have a solution like below :
"&".htmlentities("Ú");
Your XSL transformation engine probably interprets your document as non-well-formed XML because of encoding issues. If that text containing Ú is stored using some 8-bit encoding (like ISO-8859 variants), then this character will not produce a valid UTF-8 octet if it is used as such without any character conversion. Invalid characters in an XML document will mean it is not well formed XML and processing it as XML is forbidden.
There are many points where that encoding error might happen:
it could be stored in the database incorrectly
it could be read from the database incorrectly
you might produce your XML by concatenating strings that use different encodings
you might manipulate the text with a tool or method that can't handle your encoding or is not aware of it
your XSLT engine might not be aware of the correct encoding of the input stream resulting a rejected file even though it has no encoding error
My random guesses for the probable causes of that are points 3 and 5.
I'm starting out with some XML that looks like this (simplified):
<?xml version="1.0" encoding="UTF-8"?>
<alldata>
<data name="Forsetì" />
</alldata>
</xml>
But after I've parsed it with simplexml_load_string the special character (the i) becomes: ì which is obviously pretty mangled.
Is there a way to prevent this from happening?
I know for a fact the XML is fine, when saved as .txt and viewed in the browser the characters are fine. When I use simplexml_load_string on the XML and then save values as a text file, or to the database, its mangled.
This looks SimpleXML is creating a UTF-8 string, which is then rendered in ISO-8859-1 (latin-1) or something close like CP-1252.
When you save the result to a file and serve that file via a web server, the browser will use the encoding declared in the file.
Including in a web page
Since your web page encoding is not UTF-8, you need to convert the string to whatever encoding you are using, eg ISO-8859-1 (latin-1).
This is easily done with iconv():
$xmlout = iconv('UTF-8', 'ISO-8859-1//TRANSLIT', $xmlout);
Saving to database
You database column is not using UTF-8 collation, so you should use iconv to convert the string to the charset that your database uses.
Assuming your database collation is the same as the encoding that you render in, you will not have to do anything when reading from the database.
Explanation
In UTF-8, a 0xc2 prefix byte is used to access the top half of the "Latin-1 Supplement" block which includes characters such as accented letters, currency symbols, fractions, superscript 2 and 3, the copyright and registered trademark symbols, and the non-breaking space.
However in ISO-8859-1, the byte 0xC2 represents an Â. So when your UTF-8 string is misinterpreted as one of those, then you get  followed by some other nonsense character.
It's very likely that the XML is fine, but the character gets mangled when stored or output.
If you're outputting data on a HTML page: Make sure it's encoded in UTF-8 as well. If your HTML page is in ISO-8859-1, you can use utf8_decode as a quick fix; using UTF-8 is the better option in the long run.
If you're storing the data in a mySQL, you need to have UTF8 selected as the encoding all the way through: As the connection's encoding, in the table, and in the column(s) you insert the data into.
I've also had some problems with this, and it came from the PHP script encoding. Make sure it's set to UTF-8.
If it's still not good, try printing the variable using uft8_encode or utf8_decode.
XML is strict when it comes to entities, like & should be & and ì should ì
So you will need a translation table.
function xml_entity_decode($_string) {
// Set up XML translation table
$_xml=array();
$_xl8=get_html_translation_table(HTML_ENTITIES,ENT_COMPAT);
while (list($_key,)=each($_xl8))
$_xml['&#'.ord($_key).';']=$_key;
return strtr($_string,$_xml);
}
Late to the party... But I've faced this and solved like below.
You have declared encoding in XML so if you load xml file using DOMDocument it won't cause any issue.
But in case it happens in other use case, you can use html_entity_decode like below:
html_entity_decode($xml->saveXML());