Arabic text output php mysql utf-8 conversion issue - php

I'm importing data from Oracle database to MySQL tables.
I have set my MySQL table charset as utf8_general_ci and database and table name with field column value set as utf-8 as well.
Now, When I fetch the result, it prints like, which is with ? sign:
مرحبا العال� - 5
I have my utf value in column is مرحبا العالÙ
When I compare this string with Oracle string, it shows proper value - exact copy of Oracle database and there it shows perfect string in Arabic.
I have set my html meta with utf-8 as well
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" >
If I set mysql query as below, it shows junk characters:
mysql_query("SET NAMES utf8;");
mysql_query("SET CHARACTER_SET utf8;");
Followed everything possible found over stack and other sites, and still getting an error.
Please help !

Did you save the php-file without BOM? If not, try it. Potential issues with the UTF-8 BOM
Further try with 'utf-8' using single quotes and without SET CHARACTER_SET
mysql_query("SET NAMES 'utf8'");
and with charset utf-8 in the html-document header:
header("content-type: text/html; charset=utf-8");

with pdo you should have this
$_dbhandler = new PDO(PDO_DSN, DB_USERNAME, DB_PASSWORD,array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES UTF8"));

Related

Can't figure out whether MySQL or PHP badly encodes UTF8 strings

I have a form on a PHP page which inserts data into a MySQL database.
Some input fields may contain UTF8 characters as é, è, â, etc. When they are actually inserted into the database, everything gets messed up. For example, a column shows Qréon instead of Qréon.
I used setLocale(LC_CTYPE, 'FR_fr.UTF-8'); at the top of my page for PHP and this <meta charset="utf-8"> is in my HTML header.
My database is run by MySQL, the storage engine is InnoDB, and the collation is utf8_general_mysql500_ci. I also tried utf8_general_ci and utf8_bin but I had no luck.
How do I know if this comes either from PHP or MySQL processing, and how can I fix it ?
Thank you for your time.
I think this can help you:
If your using mysql
mysql_query("SET NAMES 'utf8'");
If your using PDO use this:
$dbh->exec("set names utf8");
Otherwise i could be one of these which helps you specific:
//At the Top of you files
ini_set("default_charset", "UTF-8");
header('Content-type: text/html; charset=UTF-8');
//Before your queries
mysql_query("SET CHARACTER SET utf8 ");
mysql_set_charset('utf8');

MySQL cannot recognize Korean characters

I need to insert korean text to my database which is coming by url get request. However, inserted values are not recognized in MySQL. Please, need quick instruction. Thanks.
setlocale(LC_CTYPE, 'ko_KR.utf8');
mb_internal_encoding("UTF-8");
mb_http_output('utf-8');
$p_text = rawurldecode($_GET["text"]);
Right after your mysql connection make this query
$mysqli->query("SET NAMES 'UTF8'");
or
mysql_query("SET NAMES 'UTF8'");
also make sure that your fields have utf8_general_ci collation

Get UTF-8 string from MySQL via PHP returned "????????"

In MySQL I have a string, which is displayed properly in phpmyadmin, with the collation set to utf8_general_ci .
In the PHP file, I have set the header using plain HTML:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
The php files are hosted on Ubuntu, Apache 2.2, and the files are saved in UTF-8 without BOM (using Notepad++)
I have set the connection to use utf8:
#mysql_query("set names 'utf8'", $this->link);
$r = #mysql_query($query, $this->link);
it returns the string ( varchar 255 ) and using PHP echo , it is displayed to the browser.
However, I am still getting ????? instead of the actual words.
I can't seem to be able to searched for what I might have missed, your help is much appreciated.
===== Edit =====
Database, table and column can have three different collations, I have set them all to utf8_general_ci , and calling mysql_set_charset('utf8') immediately after the connection is made, have solved my problem.
I hope this will act as a useful checklist for future people bumping into the same problem. I for one may very well return in a few months to check if I have missed anything :)
Thanks all for the great input.
Be sure your table is in utf8_general_ci.
Database and tables can have a different charset
(can happen when you change it by yourself).
And be careful with some functions, like htmlentities,
which have a default charset parameter set to ISO-8859-1 for PHP <5.4.0
(set to UTF-8 for PHP >5.4.0)
http://php.net/manual/en/function.htmlentities.php
I think the quotes around 'utf8' are wrong, but either way, you can just use:
mysql_set_charset('utf8');
And suppressing errors with the # operator when you are looking for them, is not a good idea.
Check something like this before the query
mysql_query('SET character_set_results=utf8');
mysql_query('SET names=utf8');
mysql_query('SET character_set_client=utf8');
mysql_query('SET character_set_connection=utf8');
mysql_query('SET character_set_results=utf8');
mysql_query('SET collation_connection=utf8_general_ci');
you can use this to encode the strings to UTF-8 so it will not get ???, but will change the non urf-8 characters like this /u003. so you can decode it later to present it on a text
while($var=mysql_fetch_assoc($r)){
$e['columnI']=utf8_encode($e['columnI']);

MySQL db question marks instead of hebrew characters..?

I'm trying to build a shopping cart using PHP & MySQL.
my db in MySQL is utf8 and my table in the db is utf8,
How can I use Hebrew characters?
I was able to solve this by doing the following:
the db collation has to be utf8_general_ci
the collation of the table with hebrew has to be utf8_general_ci
in your php connection script put header('Content-Type: text/html; charset=utf-8');
in xhtml head tag put <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
after selecting the db in the connection script put mysql_query("SET NAMES 'utf8'");
After a lot of work I found a solution that always works..:
without SET_NAMES.
In the conn.inc.php file, after you selected a database and connected to it, do this:
if(!mysqli_set_charset($conn, 'utf8')) {
echo 'the connection is not in utf8';
exit();
}
...and in the html always use charset utf-8;
That solved it for me. No need to use set_names(), which is ok but it annoyed the hell out of me.
You can use the PDO in your code like this:
$db = new PDO($config['DSN'], $config['dbUserName'], $config['dbPassword']);
$db->exec("SET NAMES 'utf8'");
Make sure you include the single quotation marks around the 'utf8'.
If it's an encoding problem (and it sounds like it is), this query will help:
SET NAMES utf8
Execute this query (e.g. mysql_query("SET NAMES utf8")) right after you connect and before you move any data.
More info
$conn->set_charset("utf8"); Use this for your dbconnect
Where are the question marks showing up? It may be an encoding problem somewhere other than in the data base.
To store non-ASCII characters in a database column, you need to define that column with a specific character set. You can specify a character set at 3 levels: database, table, and column. For example:
CREATE DATABASE db_name CHARACTER SET utf8
CREATE TABLE tbl_name (...) CHARACTER SET utf8
CREATE TABLE tbl_name (col_name CHAR(80) CHARACTER SET utf8, ...)
http://www.herongyang.com/PHP/Non-ASCII-MySQL-Store-Non-ASCII-Character-in-Database.html
In my case I created the DB from a dump, and this command solve it:
ALTER DATABASE <database_name> CHARACTER SET utf8 COLLATE utf8_unicode_ci;
mysql_query('SET NAMES utf8') ..... Put this in Php

UTF-8 problems PHP/MySQL

I've always used ISO-8859-1 encoding, but I'm now going over to UTF-8.
Unfortunately I can't get it to work.
My MySQL DB is UTF-8, my PHP document is encoded in UTF-8, I set a UTF-8 charset, but it still doesn't work.
(it is special characters like æ/ø/å that doesn't work)
Hope you guys can help!
Make sure the connection to your database is also using this character set:
$conn = mysql_connect($server, $username, $password);
mysql_set_charset("UTF8", $conn);
According to the documentation of mysql_set_charset at php.net:
Note:
This is the preferred way to change the charset. Using mysql_query() to execute
SET NAMES .. is not recommended.
See also: http://nl3.php.net/manual/en/function.mysql-set-charset.php
Check the character set of your current connection with:
echo mysql_client_encoding($conn);
See also: http://nl3.php.net/manual/en/function.mysql-client-encoding.php
If you have done these things and add weird characters to your table, you will see it is displayed correct.
Remember to set connection encoding to utf8 as well.
In ext\mysqli do
$mysqli->set_charset("utf8")
In ext\mysql do
mysql_set_charset("utf8")
With other db extensions you might have to run query like
SET NAMES 'utf8'
Some more details about connection encoding in MySQL
As others point out, making sure your source code is utf-8 encoded also helps. Pay special attention to not having BOM (Byte Order Mark) - it would be sent to browser before any code is executed, so using headers or sessions would become impossible.
After connecting to db, run query SET NAMES UTF8
$db = new db(...);
$db->query('set name utf8');
and add this tag to header
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Are you having this error? MySql SELECT UNION Illegal mix of collations Error? Just set you entire mysql to utf 8 then
SET character_set_connection = utf8;
Try this after connecting to mysql:
mysql_query("SET NAMES 'utf8'");
And encode PHP document in UTF-8 without BOM.
I had the same problem but now its resolved. Here is the solution:
1st: update ur table
ALTER TABLE tbl_name
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci;
2nd:
add this in the head section of the HTML code:
Regards
Saleha A.Latif
Nowadays PDO is the recommended way to use mysql. With that you should use the connection string to set encoding. For example: "mysql:host=$host;dbname=$db;charset=utf8"

Categories