When encoding the language "UTF-8" to "windows-1256" files I notice letters to lose I do not know why
Example of the text
<?php
$JSON = file_get_contents("http://gdata.youtube.com/feeds/api/videos?q=sUAvgNwQsaM&alt=json");
$JSON_Data = json_decode($JSON);
$title = $JSON_Data->{'feed'}->{'entry'}[0]->{'title'}->{'$t'};
$title= iconv("UTF-8","windows-1256","$title");
echo $youtube;
?>
If it has been deleted
iconv("UTF-8","windows-1256","$title");
Full text will appear and I did not encode windows-1256
How to solve this problem
A text input to databases windows-1256 encoding
Greetings
Try this: (not my code)
$f[]="\xc2\xac"; $t[]="\x80";
$f[]="\xd9\xbe"; $t[]="\x81";
$f[]="\xc0\x9a"; $t[]="\x82";
$f[]="\xc6\x92"; $t[]="\x83";
$f[]="\xc0\x9e"; $t[]="\x84";
$f[]="\xc0\xa6"; $t[]="\x85";
$f[]="\xc0\xa0"; $t[]="\x86";
$f[]="\xc0\xa1"; $t[]="\x87";
$f[]="\xcb\x86"; $t[]="\x88";
$f[]="\xc0\xb0"; $t[]="\x89";
$f[]="\xd9\xb9"; $t[]="\x8a";
$f[]="\xc0\xb9"; $t[]="\x8b";
$f[]="\xc5\x92"; $t[]="\x8c";
$f[]="\xda\x86"; $t[]="\x8d";
$f[]="\xda\x98"; $t[]="\x8e";
$f[]="\xda\x88"; $t[]="\x8f";
$f[]="\xda\xaf"; $t[]="\x90";
$f[]="\xc0\x98"; $t[]="\x91";
$f[]="\xc0\x99"; $t[]="\x92";
$f[]="\xc0\x9c"; $t[]="\x93";
$f[]="\xc0\x9d"; $t[]="\x94";
$f[]="\xc0\xa2"; $t[]="\x95";
$f[]="\xc0\x93"; $t[]="\x96";
$f[]="\xc0\x94"; $t[]="\x97";
$f[]="\xda\xa9"; $t[]="\x98";
$f[]="\xc4\xa2"; $t[]="\x99";
$f[]="\xda\x91"; $t[]="\x9a";
$f[]="\xc0\xba"; $t[]="\x9b";
$f[]="\xc5\x93"; $t[]="\x9c";
$f[]="\xc0\x8c"; $t[]="\x9d";
$f[]="\xc0\x8d"; $t[]="\x9e";
$f[]="\xda\xba"; $t[]="\x9f";
$f[]="\xd8\x8c"; $t[]="\xa1";
$f[]="\xda\xbe"; $t[]="\xaa";
$f[]="\xd8\x9b"; $t[]="\xba";
$f[]="\xd8\x9f"; $t[]="\xbf";
$f[]="\xdb\x81"; $t[]="\xc0";
$f[]="\xd8\xa1"; $t[]="\xc1";
$f[]="\xd8\xa2"; $t[]="\xc2";
$f[]="\xd8\xa3"; $t[]="\xc3";
$f[]="\xd8\xa4"; $t[]="\xc4";
$f[]="\xd8\xa5"; $t[]="\xc5";
$f[]="\xd8\xa6"; $t[]="\xc6";
$f[]="\xd8\xa7"; $t[]="\xc7";
$f[]="\xd8\xa8"; $t[]="\xc8";
$f[]="\xd8\xa9"; $t[]="\xc9";
$f[]="\xd8\xaa"; $t[]="\xca";
$f[]="\xd8\xab"; $t[]="\xcb";
$f[]="\xd8\xac"; $t[]="\xcc";
$f[]="\xd8\xad"; $t[]="\xcd";
$f[]="\xd8\xae"; $t[]="\xce";
$f[]="\xd8\xaf"; $t[]="\xcf";
$f[]="\xd8\xb0"; $t[]="\xd0";
$f[]="\xd8\xb1"; $t[]="\xd1";
$f[]="\xd8\xb2"; $t[]="\xd2";
$f[]="\xd8\xb3"; $t[]="\xd3";
$f[]="\xd8\xb4"; $t[]="\xd4";
$f[]="\xd8\xb5"; $t[]="\xd5";
$f[]="\xd8\xb6"; $t[]="\xd6";
$f[]="\xd8\xb7"; $t[]="\xd8";
$f[]="\xd8\xb8"; $t[]="\xd9";
$f[]="\xd8\xb9"; $t[]="\xda";
$f[]="\xd8\xba"; $t[]="\xdb";
$f[]="\xd9\x80"; $t[]="\xdc";
$f[]="\xd9\x81"; $t[]="\xdd";
$f[]="\xd9\x82"; $t[]="\xde";
$f[]="\xd9\x83"; $t[]="\xdf";
$f[]="\xd9\x84"; $t[]="\xe1";
$f[]="\xd9\x85"; $t[]="\xe3";
$f[]="\xd9\x86"; $t[]="\xe4";
$f[]="\xd9\x87"; $t[]="\xe5";
$f[]="\xd9\x88"; $t[]="\xe6";
$f[]="\xd9\x89"; $t[]="\xec";
$f[]="\xd9\x8a"; $t[]="\xed";
$f[]="\xd9\x8b"; $t[]="\xf0";
$f[]="\xd9\x8c"; $t[]="\xf1";
$f[]="\xd9\x8d"; $t[]="\xf2";
$f[]="\xd9\x8e"; $t[]="\xf3";
$f[]="\xd9\x8f"; $t[]="\xf5";
$f[]="\xd9\x90"; $t[]="\xf6";
$f[]="\xd9\x91"; $t[]="\xf8";
$f[]="\xd9\x92"; $t[]="\xfa";
$f[]="\xc0\x8e"; $t[]="\xfd";
$f[]="\xc0\x8f"; $t[]="\xfe";
$f[]="\xdb\x92"; $t[]="\xff";
function utf8_to_win($str) {
global $f, $t;
return str_replace($f, $t, $str);
}
Related
I got a json which is generated in a python program and looks like this:
{"0": {"ausschreiber": "Beispiel; Zeitarbeit GmbH", "beschreibung": "\r\nF\u00fcr unseren Kunden suchen wir motivierte studentische Aushilfen auf flexibler Stundenbasis (450\u0080-Basis)", "datum": "17.11.2016", "name": "Studentische Hilfskr\u00e4fte gesucht", "email": "info#hindi.de"}}
now i am decoding the json in my php program to get an associative array and display this on the website.
The Problem is that the special characters like the € char are not displayed but special chars like ö ä ü are displayed.
Here is the php program:
<?php
header('Content-Type: text/html; charset=utf-8');
function compare($old_data, $new_data){
$old_result = json_decode($old_data, true);
$new_result = json_decode($new_data, true);
echo $new_result[0]['beschreibung'];
}
function go4it(){
$db_data=json_content(); //creates the json from the Database
$crawler_data = file_get_contents('http://localhost/phppath/python_program.cgi'); //calls the cgi which returns the json
compare($db_data, $crawler_data);
}
go4it();
What i tried:
set the header to utf-8
$new_result = json_decode(utf8_encode($new data), true);
iconv_set_encoding("internal_encoding", "UTF-8");
iconv_set_encoding("input_encoding", "UTF-8");
iconv_set_encoding("output_encoding", "UTF-8");
Thanks for your help!
EDIT 1
so it seems like the issue is located in the python program, thanks to #FranzGleichmann . I think the problem is with the encoding of the page where i get the content from. The page says it is ISO-8859-1 so i tried this:
url = 'https://www.example.com'
source_code = requests.get(url)
plain_text = source_code.text
plain_text.decode('iso-8859-1', 'ignore').encode('utf8', 'ignore')
print(plain_text.encoding)
but then i get the error: "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 8496: ordinal not in range(128)"
it was a problem with the python script
So after a whole day of googling and debugging I end up here.
MySQL
set to the following encoding:
db: utf8_general_ci
table: utf8_general_ci
column: utf8_general_ci, TEXT
I put in some euro symbols and some other weird characters
acentuação €€€€€
PHP (codeigniter)
config
$config['charset'] = 'UTF-8';
dsn
char_set=utf8,dbcollat=utf8_general_ci
I made some queries to compare
model
$query = $this->db->query("SET NAMES latin1");
$query = $this->db->query("SELECT shortdesc,HEX(shortdesc) FROM `contracttypes` WHERE id = 4");
$ret['latin1'] = $query->row();
$query = $this->db->query("SET NAMES utf8");
$query = $this->db->query("SELECT shortdesc,HEX(shortdesc) FROM `contracttypes` WHERE id = 4");
$ret['utf8'] = $query->row();
return $ret;;
controller
public function utfhell() {
var_dump($this->campagne_model->utfhell());
}
This outputs
array (size=2)
'latin1' =>
object(stdClass)[34]
public 'shortdesc' => string 'acentua��o �����' (length=16)
public 'HEX(shortdesc)' => string '6163656E747561C3A7C3A36F20E282ACE282ACE282ACE282ACE282AC' (length=56)
'utf8' =>
object(stdClass)[33]
public 'shortdesc' => string 'acentuação €€€€€' (length=28)
public 'HEX(shortdesc)' => string '6163656E747561C3A7C3A36F20E282ACE282ACE282ACE282ACE282AC' (length=56)
So far so good, on to a
view
<?php header('Content-Type: text/html; charset="utf-8"', true); ?>
<!doctype html>
<html>
<head>
<title>UTFhell</title>
<link rel="stylesheet" href="../assets/css/style.css"/>
<meta charset="utf-8">
<meta http-equiv="Content-Type" content="text/html;charset=utf-8">
...
<?php
echo 'Original : ', $campagne_info->contractName->shortdesc."<br />";
echo 'UTF8 Encode : ', utf8_encode($campagne_info->contractName->shortdesc)."<br />";
echo 'UTF8 Decode : ', utf8_decode($campagne_info->contractName->shortdesc)."<br />";
echo 'TRANSLIT : ', iconv("ISO-8859-1", "UTF-8//TRANSLIT", $campagne_info->contractName->shortdesc)."<br />";
echo 'IGNORE TRANSLIT : ', iconv("ISO-8859-1", "UTF-8//IGNORE//TRANSLIT", $campagne_info->contractName->shortdesc)."<br />";
echo 'IGNORE : ', iconv("ISO-8859-1", "UTF-8//IGNORE", $campagne_info->contractName->shortdesc)."<br />";
echo 'Plain: ', iconv("ISO-8859-1", "UTF-8", $campagne_info->contractName->shortdesc)."<br />";
echo '€€€€€€€€€€<br>';
?>
None of these now show me a normal euro symbol except the final echo statement, they all give me questionmark diamonds for the eurosymbols
The HEX is the utf8 encoding for that string. So the data is in the table 'correctly'.
The black diamond (�) is the browser's way of saying wtf. It comes from having latin1 characters, but telling the browser
to display utf8 characters.
You could tell the browser to display "Western", that is avoiding the underlying problems.
Remember, the goal is to really use utf8.
Sometimes this occurs together with Question Marks, in which case you must start over.
The cause (probably):
The bytes you had were encoded latin1. You acquired them from somewhere -- file dump online input, etc.
The connection parameters said latin1.
The column/table is declared to be CHARACTER SET said utf8, so during INSERT, they were correctly converted.
When SELECTing, the seting in step 2 was again latin1, so they were converted back to latin1.
When displaying text in a web page, the page's header said that the bytes were utf8.
Solution, Plan A: (Sloppy, but probably workable)
Change #5 so say the appropriate equivalent of latin1.
Solution, Plan B:
Fix the source to be utf8-encoded
query("SET NAMES utf8") (unless there is a way to set it at connect time)
Leave the table/column at CHARACTER SET utf8
Step 2 cover this.
Leave <meta ... UTF-*>.
I have a problem where I thought it was easy to Google but it seems to be not that easy. Ok, here is my problem:
I have to read a CSV file which has a bad and different encoding. I'm not able to correct the CSV file before hand, so I have to handle it in my application. So the CSV file could have the following-encodings:
'Ü5' and 'Möbelmarkt' in the same file.
If I decode (utf8_decode) the right one is correct and the left (which was correct) is wrong. When I try to find out the encoding (mb_detect_encoding) i always get the answer that this is UTF-8.
I still tried the following solutions:
public function convert( $str ) {
return iconv( "Windows-1252", "UTF-8", $str );
}
and
private function getUmlauteArray() {
return array( 'ü'=>'ü', 'ä'=>'ä', 'ö'=>'ö', 'Ö'=>'Ö', 'ß'=>'ß', 'à '=>'à', 'á'=>'á', 'â'=>'â', 'ã'=>'ã', 'ù'=>'ù', 'ú'=>'ú', 'û'=>'û', 'Ù'=>'Ù', 'Ú'=>'Ú', 'Û'=>'Û', 'Ãœ'=>'Ü', 'ò'=>'ò', 'ó'=>'ó', 'ô'=>'ô', 'è'=>'è', 'é'=>'é', 'ê'=>'ê', 'ë'=>'ë', 'À'=>'À', 'Ã'=>'Á', 'Â'=>'Â', 'Ã'=>'Ã', 'Ä'=>'Ä', 'Ã…'=>'Å', 'Ç'=>'Ç', 'È'=>'È', 'É'=>'É', 'Ê'=>'Ê', 'Ë'=>'Ë', 'ÃŒ'=>'Ì', 'Ã'=>'Í', 'ÃŽ'=>'Î', 'Ã'=>'Ï', 'Ñ'=>'Ñ', 'Ã’'=>'Ò', 'Ó'=>'Ó', 'Ô'=>'Ô', 'Õ'=>'Õ', 'Ø'=>'Ø', 'Ã¥'=>'å', 'æ'=>'æ', 'ç'=>'ç', 'ì'=>'ì', 'Ã'=>'í', 'î'=>'î', 'ï'=>'ï', 'ð'=>'ð', 'ñ'=>'ñ', 'õ'=>'õ', 'ø'=>'ø', 'ý'=>'ý', 'ÿ'=>'ÿ', '€'=>'€' );
}
public function fixeUmlaute($string) {
$umlaute = $this->getUmlauteArray();
foreach ($umlaute as $key => $value){
$value = str_replace($key, $value, $string);
}
return $string;
}
and
function valid_utf8( $string ){
return !((bool)preg_match('~[\xF5\xF6\xF7\xF8\xF9\xFA\xFB\xFC\xFD\xFE\xFF\xC0\xC1]~ms',$string));
}
That are all solutions I found with a Google search to change the encoding...(perhaps this "collection" helps anybody else...) So, how can I really detect the wrong characters or where is it my mistake?
Can anybody give me a hint?
Greetz
V
There is a nice PHP class that can help you with that: https://github.com/neitanod/forceutf8
It will convert any charset to UTF8, and handle the detection for you. Hope it helps.
I am hosting a web service in JSON output by PHP.
I have Hebrew data set in DB and I am posting this as an output to Web service.
When I post the data initially it output the result as follows:
JSON:
{
"tasklist": [
{
"customerID": "9936",
"name": "טר ×רמה ×™×–×•× ×•×‘×™× ×•×™ בע"מ",
"cargo":"×ברר",
"destination":"מכר",
"quantity":"1.000000",
"startdate":"03/01/201300: 00: 00"
}
]
}
But this "×ברר" can be readable by Android/Iphone parser and convert it to original Hebrew. But i faced Error in "name": "טר ×רמה ×™×–×•× ×•×‘×™× ×•×™ בע"מ",. where " is in between the string so the JSON is not valid and shows error!
To Over come this issue I used UTF-8 to convert "×ברר" this to Hebrew "נברר". But in this case too the problem remains same:
PHP:
header('Content-type: text/html; charset=UTF-8');
JSON:
{
"tasklist": [
{
"customerID": "9936",
"name": "טר ארמה יזום ובינוי בע"מ",
"cargo":"נברר",
"destination":"מכר",
"quantity":"1.000000",
"startdate":"03/01/201300: 00: 00"
}
]
}
But still the problem remains:
Also in some case I am getting this � because of using UTF-8
"name":"מחצבות כפר גלעדי-חומרי מ�"
How can I overcome this issue?
Is there any other specific encode I need to use?
Note: The data cannot be changes in Database The solution should be while output to JSON.
How the data stored in DB is shown below:
name
מחצבות כפר גלעדי-חומרי מ×
My PHP Script which output JSON:
<?php
//My DB connection and Query comes here
$jsontext = '{"tasklist":[';
while($row = mysql_fetch_array($queryExe)){
$jsontext .= '{"customerID":"'.$row['AUTO_ID'].'",';
$jsontext .='"name":"'.$row['Customer_Name'].'",';
$jsontext .='"cargo":"'.$row['Type_of_Cargo'].'",';
$jsontext .='"destination":"'.$row['Destination'].'",';
$jsontext .='"quantity":"'.$row['Quantity'].'",';
$jsontext .='"startdate":"'.$row['startdate'].'"},';
}
$jsontext = substr_replace($jsontext, '', -1); // to get rid of extra comma
$jsontext .= "]}";
header('Content-type: text/html; charset=UTF-8');
//Output the final JSON
echo $jsontext;
?>
Thank you for your help in advance!
Was the question clear? to understand my issue.
If your db-field is utf8 you should fist do:
mysql_query("SET NAMES 'utf8'");
You should always do the 'SET NAMES...' before inserting your data, too.
Be sure that you really stored utf8 encoded strings!
then do your query:
mysql_query($your_query);
$array = array("tasklist"=>array());
while($row = mysql_fetch_array($queryExe)){
$a = array();
$a["customerID"] = $row['AUTO_ID'];
$a["name"] = $row['Customer_Name'];
$a["cargo"] = $row['Type_of_Cargo'];
$a["destination"] = $row['Destination'];
$a["quantity"] = $row['Quantity'];
$a["startdate"] = $row['startdate'];
$array["tasklist"][] = $a;
}
header("Content-type: application/json; charset=utf-8");
echo json_encode($array);
exit();
i've made the experience that these is not enough when the servers default charset is for example iso. In that case i need to do the following in my .htaccess:
AddDefaultCharset utf-8
You should change your code to use json_encode. You need to pass it properly utf8 encoded data.
If you are using MySQL you can try running the following before your query to get your data.
SET NAMES 'utf8';
You can also look into using utf8_encode.
From http://www.php.net/manual/en/function.json-encode.php#100565
That said, quotes " will produce invalid JSON, but this is only an issue if you're using json_encode() and just expect PHP to magically escape your quotes. You need to do the escaping yourself.
May be you can replace " with \" , i guess it will solve the issue.
Source : PHP JSON String, escape Double Quotes for JS output
I am getting Windows-1256 encoded text from the web and nee to convert it to utf-8.
I tried using mb_convert_encoding and iconv but they don't seem to work.
none of them seem to be capable of handling windows-1256.
How to do it?
Edit: More details about the errors.
When trying
mb_convert_encoding($text,"utf-8", "windows-1256");
I get
Message: mb_convert_encoding() [function.mb-convert-encoding]: Illegal character encoding specified
And when i try
iconv("windows-1256", "utf-8", $text);
I get no errors but it returns an empty string
Trying
echo iconv('WINDOWS-1256', 'UTF-8', 'testÍÊ');
...on http://writecodeonline.com/php/ seems to work correctly (produces testأچأٹ)
Try this, should work:
iconv("windows-1256", "utf-8//TRANSLIT//IGNORE", $text)
Check this:
http://rayed.com/wordpress/wp-content/upload/lib.utf2win.php.txt
Apparently he also had some problems, because he wrote this script, if you can reverse that, it might work.
I reversed it for you, try that:
$f[]="\xc2\xac"; $t[]="\x80";
$f[]="\xd9\xbe"; $t[]="\x81";
$f[]="\xc0\x9a"; $t[]="\x82";
$f[]="\xc6\x92"; $t[]="\x83";
$f[]="\xc0\x9e"; $t[]="\x84";
$f[]="\xc0\xa6"; $t[]="\x85";
$f[]="\xc0\xa0"; $t[]="\x86";
$f[]="\xc0\xa1"; $t[]="\x87";
$f[]="\xcb\x86"; $t[]="\x88";
$f[]="\xc0\xb0"; $t[]="\x89";
$f[]="\xd9\xb9"; $t[]="\x8a";
$f[]="\xc0\xb9"; $t[]="\x8b";
$f[]="\xc5\x92"; $t[]="\x8c";
$f[]="\xda\x86"; $t[]="\x8d";
$f[]="\xda\x98"; $t[]="\x8e";
$f[]="\xda\x88"; $t[]="\x8f";
$f[]="\xda\xaf"; $t[]="\x90";
$f[]="\xc0\x98"; $t[]="\x91";
$f[]="\xc0\x99"; $t[]="\x92";
$f[]="\xc0\x9c"; $t[]="\x93";
$f[]="\xc0\x9d"; $t[]="\x94";
$f[]="\xc0\xa2"; $t[]="\x95";
$f[]="\xc0\x93"; $t[]="\x96";
$f[]="\xc0\x94"; $t[]="\x97";
$f[]="\xda\xa9"; $t[]="\x98";
$f[]="\xc4\xa2"; $t[]="\x99";
$f[]="\xda\x91"; $t[]="\x9a";
$f[]="\xc0\xba"; $t[]="\x9b";
$f[]="\xc5\x93"; $t[]="\x9c";
$f[]="\xc0\x8c"; $t[]="\x9d";
$f[]="\xc0\x8d"; $t[]="\x9e";
$f[]="\xda\xba"; $t[]="\x9f";
$f[]="\xd8\x8c"; $t[]="\xa1";
$f[]="\xda\xbe"; $t[]="\xaa";
$f[]="\xd8\x9b"; $t[]="\xba";
$f[]="\xd8\x9f"; $t[]="\xbf";
$f[]="\xdb\x81"; $t[]="\xc0";
$f[]="\xd8\xa1"; $t[]="\xc1";
$f[]="\xd8\xa2"; $t[]="\xc2";
$f[]="\xd8\xa3"; $t[]="\xc3";
$f[]="\xd8\xa4"; $t[]="\xc4";
$f[]="\xd8\xa5"; $t[]="\xc5";
$f[]="\xd8\xa6"; $t[]="\xc6";
$f[]="\xd8\xa7"; $t[]="\xc7";
$f[]="\xd8\xa8"; $t[]="\xc8";
$f[]="\xd8\xa9"; $t[]="\xc9";
$f[]="\xd8\xaa"; $t[]="\xca";
$f[]="\xd8\xab"; $t[]="\xcb";
$f[]="\xd8\xac"; $t[]="\xcc";
$f[]="\xd8\xad"; $t[]="\xcd";
$f[]="\xd8\xae"; $t[]="\xce";
$f[]="\xd8\xaf"; $t[]="\xcf";
$f[]="\xd8\xb0"; $t[]="\xd0";
$f[]="\xd8\xb1"; $t[]="\xd1";
$f[]="\xd8\xb2"; $t[]="\xd2";
$f[]="\xd8\xb3"; $t[]="\xd3";
$f[]="\xd8\xb4"; $t[]="\xd4";
$f[]="\xd8\xb5"; $t[]="\xd5";
$f[]="\xd8\xb6"; $t[]="\xd6";
$f[]="\xd8\xb7"; $t[]="\xd8";
$f[]="\xd8\xb8"; $t[]="\xd9";
$f[]="\xd8\xb9"; $t[]="\xda";
$f[]="\xd8\xba"; $t[]="\xdb";
$f[]="\xd9\x80"; $t[]="\xdc";
$f[]="\xd9\x81"; $t[]="\xdd";
$f[]="\xd9\x82"; $t[]="\xde";
$f[]="\xd9\x83"; $t[]="\xdf";
$f[]="\xd9\x84"; $t[]="\xe1";
$f[]="\xd9\x85"; $t[]="\xe3";
$f[]="\xd9\x86"; $t[]="\xe4";
$f[]="\xd9\x87"; $t[]="\xe5";
$f[]="\xd9\x88"; $t[]="\xe6";
$f[]="\xd9\x89"; $t[]="\xec";
$f[]="\xd9\x8a"; $t[]="\xed";
$f[]="\xd9\x8b"; $t[]="\xf0";
$f[]="\xd9\x8c"; $t[]="\xf1";
$f[]="\xd9\x8d"; $t[]="\xf2";
$f[]="\xd9\x8e"; $t[]="\xf3";
$f[]="\xd9\x8f"; $t[]="\xf5";
$f[]="\xd9\x90"; $t[]="\xf6";
$f[]="\xd9\x91"; $t[]="\xf8";
$f[]="\xd9\x92"; $t[]="\xfa";
$f[]="\xc0\x8e"; $t[]="\xfd";
$f[]="\xc0\x8f"; $t[]="\xfe";
$f[]="\xdb\x92"; $t[]="\xff";
function win_to_utf8($str) {
global $f, $t;
return str_replace($t, $f, $str);
}