Dreaded ???? instead of characters when displaying chinese text. Simplified Chinese Character issue - php

Question

I am trying to produce a multilingual using Drupal 6. My site is hosted on a Windows IIS 7.5 Server.
I have gotten my .po files uploaded and everything works great. Im now at the stage where I am going through the site piece by piece to try and weed out anything that escaped my notice when I first translated teh site.
The problem I am having is in trying to display dates in locale formats.
Here is the php code I use to display a local date:
$format = "%A, %b %#d, %Y : %H:%M%p"
if($language->language == 'zh-hans')
{
$loc=setlocale(LC_TIME, 'chs');
}else{
$loc=setlocale(LC_TIME, 'de');
}
$mytime = strftime($format, $time);
$mytime = iconv(mb_detect_encoding($mytime), 'utf-8', $mytime);
echo $mytime;
The above code should display the date in the local format of the current language (chinese of german)
The german displays as expected:
Freitag, Jun 21, 2013 : 16:48
but the chinese output looks like this:
???, ?? 21, 2013 : 16:48??
Any ideas on how to fix this?

Answer 1

The first problem is that setting the LC_TIME category is not enough by itself. The strftime function will attempt to format the time in the appropriate language, but it seems to use the 1252 codepage by default, so it will be incapable of producing the necessary characters for something like Chinese.
I would have expected that setting LC_TIME and maybe LC_CTYPE would be enough, but the only way I could get it to work was with LC_ALL. So your setlocale calls should look like this:
if($language->language == 'zh-hans')
{
$loc=setlocale(LC_ALL, 'chs');
}else{
$loc=setlocale(LC_ALL, 'de');
}
The next problem is that mb_detect_encoding won't always detect the correct encoding. What works for me on Windows is to use setlocale(LC_CTYPE, 0) to get back the current code page.
$codepage = explode('.',setlocale(LC_CTYPE, 0));
$codepage = 'cp'.$codepage[1];
The value returned from the call to setlocale(LC_CTYPE, 0) is a combination of the language name and the codepage number separated by a dot. We split the two parts with explode and then prefix the number with cp to get the code page in a form that iconv can understand.
We can then simply convert the string to utf-8 like this:
$mytime = iconv($codepage, 'utf-8', $mytime);