PHP encoding Windows-1257 to UTF-8 error - php

Question

I encountered a problem with converting the Windows-1257 file to UTF-8. The original file has
<?xml version="1.0" encoding="windows-1257"?>
on top and I try to convert it using this code:
iconv_set_encoding("internal_encoding", "UTF-8");
iconv_set_encoding("output_encoding", "ISO-8859-1");
$baltic_xml = file_get_contents($remote_file);
$unicode_xml = iconv("UTF-8", "UTF-8//IGNORE", $baltic_xml);
file_put_contents('data/rmtools/import/utf8/'.$files_single, $unicode_xml);
It saves the file as UTF-8, but when I open this file I still get the error:
XML parsing error: Input is not proper UTF-8, indicate encoding ! Bytes: 0x04 0x50 0x72 0x65
Is there any proper way I could convert it to readable UTF-8, or it means that there is still some symbols in the file which is NOT on UTF-8?

Answer 1

You're trying to convert UTF8 to UTF8//IGNORE, and that's why you're receiving that error. The first parameter is the in_charset. iconv on PHP.net Please change
$unicode_xml = iconv("UTF-8", "UTF-8//IGNORE", $baltic_xml);
to
$unicode_xml = iconv("CP1257", "UTF-8//IGNORE", $baltic_xml);
However I'd personally recommend you to use mb_* as iconv relies heavily on your OS's implementation of iconv and can show differences in between OS, mb_* on the other hand is pure php extension and is consistent. Making your code use mb_* changes whole to
ini_set('mbstring.substitute_character','none'); //to remove the unknown characters, in place of //IGNORE in iconv
$baltic_xml = file_get_contents($remote_file);
$unicode_xml = iconv("UTF-8", "UTF-8//IGNORE", $baltic_xml);
$unicode_xml = utf8_encode($unicode_xml); //to correct utf-8 bytes
$unicode_xml = preg_replace('/[^\PC\s]/u', '', $unicode_xml); //to remove control chars in case it has
file_put_contents('data/rmtools/import/utf8/' . $files_single, $unicode_xml);
According to mb supported encodings CP-1257 is not one of them, you may use ISO-8859-13 instead, however please note that there are some inconsistencies between them in some graphical characters (language characters however seem to be consistent according to wikipedia )