Enable Extract and Display Foreign Language Content in browsers using fgetcsv - php

Question

iD;English [en];Chinese [zh];German [de];Hindi [hi];Hindi (TOGO) [hi_TG];Japanese [ja]
Source[local].AlarmGroup[AlarmText_02].ID[1310:90];Unwinder: Accu position difference too big. Check for laminate break;拆卷器： 蓄存器位置差过大。 检查复合片材是否中断;Laminatspeicher: Zu grosse Positionsänderung - Auf Laminatriss prüfen;290;;巻出装置: アキュムレーター位置の差が大きすぎます。 ラミネートが壊れていないか確認してください
Source[local].AlarmGroup[AlarmText_02].ID[1311:91];Unwinder: Accu level too small for auto splice;拆卷器： 自动拼接的蓄存器级别过小;Abwickler: Akku Füllstand zu klein für Autospleiss;291;;巻出装置: 自動紙継を行うにはアキュムレーターのレベルが小さすぎます
I am trying to fetch csv content as mentioned above :
The csv file is saved as Unicode Text. It has Chinese, German, Japanese Language.
I am unable to fetch foreign language in correct format.
CSV reader Code
header('Content-Type: text/html; charset=utf-8');
$row = 1;
$up_file = 'text_SHOT_S.csv';
setlocale(LC_ALL, 'en_US.UTF-8');
if (($handle = fopen($up_file, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$num = count($data);
$row++;
for ($c=0; $c < $num; $c++) {
echo $data[$c].'<br>';
}
}
fclose($handle);}
Output of the following Code:
iD戼㹲䔀渀最氀椀猀栀 嬀攀渀崀㰀牢>Chinese [zh]戼㹲䜀攀爀洀愀渀 嬀搀攀崀㰀牢>Hindi [hi]戼㹲䠀椀渀搀椀 ⠀吀伀䜀伀⤀ 嬀栀椀开吀䜀崀㰀牢>Japanese [ja] 戼㹲匀漀甀爀挀攀嬀氀漀挀愀氀崀⸀䄀氀愀爀洀䜀爀漀甀瀀嬀䄀氀愀爀洀吀攀砀琀开　㈀崀⸀䤀䐀嬀㄀㌀㄀　㨀㤀　崀㰀牢>Unwinder: Accu position difference too big. Check for laminate break戼㹲였睢桓ᩖ⃿쐀墄桛䵖湏읝➏ə‰쀀൧࡙䝔偲⽧♦ⵔ굎㱥牢>Laminatspeicher: Zu grosse Positionsänderung - Auf Laminatriss prüfen戼㹲㈀㤀　㰀牢>戼㹲ff艹앑溈㩿  ꈀ괰ﰰ뼰ﰰ䴰湏湿䱝✰䵙夰丰縰夰Ȱ‰�촰ﰰ젰䰰쨰豘昰䐰樰䐰䬰먰赸垊昰估怰唰䐰ര㰀牢>Source[local].AlarmGroup[AlarmText_02].ID[1311:91]戼㹲唀渀眀椀渀搀攀爀㨀 䄀挀挀甀 氀攀瘀攀氀 琀漀漀 猀洀愀氀氀 昀漀爀 愀甀琀漀 猀瀀氀椀挀攀㰀牢>拆卷器： 自动拼接的蓄存器级别过小戼㹲䄀戀眀椀挀欀氀攀爀㨀 䄀欀欀甀 䘀ﰀ氀氀猀琀愀渀搀 稀甀 欀氀攀椀渀 昀ﰀ爀 䄀甀琀漀猀瀀氀攀椀猀猀㰀牢>291戼㹲㰀牢>巻出装置: 自動紙継を行うにはアキュムレーターのレベルが小さすぎます 戼㹲㰀牢
I either check garbage character or most of the content converted to Chinese.
Also tried the header('Content-Type: text/html; charset=iso-8859-1') and setlocale(LC_CTYPE, 'zh_CN.UTF-8','zh_ZH.big5');
I want the output same as CSV content.
Thanks in advance .

Answer 1

For reading CSV content I used PHPExcel and converted UTF-16 file into UTF-8 then it will fetch Chinese content properly.
Please refer below link for converting UTF-16 File to an UTF-8.
How to Convert an UTF-16 File to an UTF-8 file using PHP
To convert a file simply call the convert_file_to_utf8() function
and pass to it the file path of the file you wish to convert. The
function then uses the PHP function file_get_contents() to pack the
input file’s contents into a string variable which is then passed to
the main converter function which converts the string from UTF-16 to
UTF-8 encoding if necessary. Finally, it uses file_put_contents() to
stuff the resulting string back into the original file, overwriting
the original file contents.
function utf16_to_utf8($str) {
$c0 = ord($str[0]);
$c1 = ord($str[1]);
if ($c0 == 0xFE && $c1 == 0xFF) {
$be = true;
} else if ($c0 == 0xFF && $c1 == 0xFE) {
$be = false;
} else {
return $str;
}
$str = substr($str, 2);
$len = strlen($str);
$dec = '';
for ($i = 0; $i < $len; $i += 2) {
$c = ($be) ? ord($str[$i]) << 8 | ord($str[$i + 1]) :
ord($str[$i + 1]) << 8 | ord($str[$i]);
if ($c >= 0x0001 && $c <= 0x007F) {
$dec .= chr($c);
} else if ($c > 0x07FF) {
$dec .= chr(0xE0 | (($c >> 12) & 0x0F));
$dec .= chr(0x80 | (($c >> 6) & 0x3F));
$dec .= chr(0x80 | (($c >> 0) & 0x3F));
} else {
$dec .= chr(0xC0 | (($c >> 6) & 0x1F));
$dec .= chr(0x80 | (($c >> 0) & 0x3F));
}
}
return $dec;
}
function convert_file_to_utf8($csvfile) {
$utfcheck = file_get_contents($csvfile);
$utfcheck = utf16_to_utf8($utfcheck);
file_put_contents($csvfile, $utfcheck);
}

Answer 2

Please before read this answer, read the different coment.
Mudassir, you can see the exact charset with tortoise, with comparator of file (see img)
Your soft use not utf-8 but utf-16 encoding. If you cant change this, you can use http://php.net/manual/en/function.mb-convert-encoding.php
http://php.net/manual/fr/mbstring.supported-encodings.php
I've try with your file and this function, and it's work correctly. See the code :
header('Content-Type: text/html; charset=utf-8');
$row = 1;
$up_file = 'text_SHOT_S.csv';
setlocale(LC_ALL, 'en_US.UTF-8');
if (($handle = fopen($up_file, "r")) !== FALSE) {
while (($data = fgetcsv($handle, 1000, ";")) !== FALSE) {
$num = count($data);
$row++;
for ($c=0; $c < $num; $c++) {
// echo $data[$c].'<br>';
echo mb_convert_encoding($data[$c],'utf8','utf-16').'<br>';
}
}
fclose($handle);}