php file character encoding, mysql database character encoding, special characters - php

I've a mysql db with some special characters, an input form, some php pages.
In my first php page (requestPage) I have the input form.
This page sends (through GET) a parameter to an other php page (ResultsPage).
At last this php page (ResultsPage) send a query with the parameter to the dbms and shows results.
RequestPage is encoded as utf-8 through
meta http-equiv="Content-Type" content="text/html; charset=utf-8
ResultsPage is encoded as utf-8 through
meta http-equiv="Content-Type" content="text/html; charset=utf-8
header('Content-type: text/html; charset=utf-8');
The database and its tables are encoded as utf8_general_ci.
Now, for instance:
if I put in the form of RequestPage the word "Cantu", the resultsPage executes the query and show me every entry of the DB with the word "Cantù" and for me its OK.
If I put in the form of RequestPage the word "Cantù", the resultsPage executes the query and show me no rows but I want to see the rows with the word "Cantù"!
Any suggestion?

I've solved the problem with this string added in the php resultsPage file:
mysql_query("SET character_set_results = 'utf8', character_set_client='utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'", $connection);
Obviously it has to be inserted after this string:
$connection = mysql_connect($servername,$username,$dbpassword);

Related

Get Chinese characters from MySQL

I am trying to display chinese characters that I have saved in MySQL database, which is saved under utf8_unicode_ci type. I have seen several solutions on the web, but nothing works.
Below is my connection file:
$conn = mysql_connect("localhost","root","password");
mysql_set_charset('utf8',$conn);
mysql_query("SET CHARACTER SET utf8 ");
mysql_select_db("database");
Below is my query:
mysql_query("SET character_set_results=utf8", $conn);
$sql = mysql_query("select * from webdata",$conn);
But it still shows ????. Any ideas?
How to resolve...
When I had a similar issue I firstly displayed the encoding of my text in php using
echo mb_detect_encoding ( $text );
It shows the encoding of the text coming from my query. This showed me that I was getting ASCII from mysql_query when Chinese or Russian characters were in my database.
The change I made was with the following addition after the mysql_connect
mysql_set_charset('UTF8');
My database is utf8_unicode_ci collation and I can see chinese characters.
So, if mb_detect_encoding is now giving you UTF8 for your text then you would be able to show chinese characters.
The next step is to make sure what you pass to the browser has the correct header...
header('Content-Type: text/html; charset=UTF-8');
Put the above at the top of your code in php to make sure the browser is expecting your encoding.
Now that should the question, however ideally you should be using PDO or mysqli rather than mysql_connect method. In this case the equivalent procedural style commands are..
$link = mysqli_connect('localhost', 'my_user', 'my_password', 'test');
mysqli_set_charset($link, "utf8");
Where $link is the equivalent to your connection to the database.
where it show "???", when you print the output to HTML ?
if so, try to add to <head> element the line
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
hope it helped a bit.
EDIT
it seems that you need to declare UTF8 on:
character_set_results = 'utf8',
character_set_client = 'utf8',
character_set_connection = 'utf8',
character_set_database = 'utf8',
character_set_server = 'utf8'"
checkout
PHP UTF8 not displaying chinese characters properly
That should be all you need. Both for Traditional and Simplified Chinese characters
1. Make sure your table is set to COLLATION utf8_general_ci
2. $con = new mysqli("localhost",$username,$password,$database) or die("Error " . mysqli_error($con)); $con->query("SET NAMES 'utf8'");
3. <meta http-equiv="Content-type" content="text/html; charset=utf-8" />

Can't figure out whether MySQL or PHP badly encodes UTF8 strings

I have a form on a PHP page which inserts data into a MySQL database.
Some input fields may contain UTF8 characters as é, è, â, etc. When they are actually inserted into the database, everything gets messed up. For example, a column shows Qréon instead of Qréon.
I used setLocale(LC_CTYPE, 'FR_fr.UTF-8'); at the top of my page for PHP and this <meta charset="utf-8"> is in my HTML header.
My database is run by MySQL, the storage engine is InnoDB, and the collation is utf8_general_mysql500_ci. I also tried utf8_general_ci and utf8_bin but I had no luck.
How do I know if this comes either from PHP or MySQL processing, and how can I fix it ?
Thank you for your time.
I think this can help you:
If your using mysql
mysql_query("SET NAMES 'utf8'");
If your using PDO use this:
$dbh->exec("set names utf8");
Otherwise i could be one of these which helps you specific:
//At the Top of you files
ini_set("default_charset", "UTF-8");
header('Content-type: text/html; charset=UTF-8');
//Before your queries
mysql_query("SET CHARACTER SET utf8 ");
mysql_set_charset('utf8');

Arabic text output php mysql utf-8 conversion issue

I'm importing data from Oracle database to MySQL tables.
I have set my MySQL table charset as utf8_general_ci and database and table name with field column value set as utf-8 as well.
Now, When I fetch the result, it prints like, which is with ? sign:
مرحبا العال� - 5
I have my utf value in column is مرحبا العالÙ
When I compare this string with Oracle string, it shows proper value - exact copy of Oracle database and there it shows perfect string in Arabic.
I have set my html meta with utf-8 as well
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" >
If I set mysql query as below, it shows junk characters:
mysql_query("SET NAMES utf8;");
mysql_query("SET CHARACTER_SET utf8;");
Followed everything possible found over stack and other sites, and still getting an error.
Please help !
Did you save the php-file without BOM? If not, try it. Potential issues with the UTF-8 BOM
Further try with 'utf-8' using single quotes and without SET CHARACTER_SET
mysql_query("SET NAMES 'utf8'");
and with charset utf-8 in the html-document header:
header("content-type: text/html; charset=utf-8");
with pdo you should have this
$_dbhandler = new PDO(PDO_DSN, DB_USERNAME, DB_PASSWORD,array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES UTF8"));

How to handle character sets properly in conjunction with a database, text areas etc etc

For once and for all I want to have clear how to handle ü etc in my application.
I understand using UTF-8 is the best way to store/represent these strange characters.
So I declare in my HTML a meta-charset:
<meta charset="utf-8">
Storing the content in my database, it also needs the right set so I set the collation to utf8_general_ci
In my understanding there isn't an other place where I can define the character set. But what is the problem?
If I go into phpMyAdmin and add jülich in the content field, and I subtract the content from the database through a mysql query and PHP. The ü gets displayed properly.
But if I view it in a Textarea it displays a black diamond shaped figure with a white ? in it.
If I put in the text in a textarea in my application and submit it, it displays correct in the text area. But on the website it displays ü
These are the charset settings in the startup screen of phpMyAdmin:
As an answer on questions off Daan:
character%:
character_set_client = utf8
character_set_connection = utf8
character_set_database = utf8
character_set_filesystem = binary
character_set_results = utf8
character_set_server = latin1
character_set_system = utf8
character_sets_dir = /usr/share/mysql/charsets/
collation%
collation_connection = utf8_general_ci
collation_database = utf8_general_ci
collation_server = latin1_swedish_ci
How can I resolve this?
Things I've done so far:
added: header('Content-type: text/html; charset=utf-8');
added: $this->db->setAttribute(PDO::MYSQL_ATTR_INIT_COMMAND, "SET NAMES 'utf8'");
added: <meta http-equiv="content-type" content="text/html; charset=utf-8" />
added: $this->db = new PDO($dsn, $username, $password,
array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
Solution
All the above is working like a charm. But if you use a function to replace certain special routines and return the replaced string through utf8_decode() it obviously doesn't display the ü ;(
You should really read the MySQL specific PDO documentation as it clearly states that the PDO::MYSQL_ATTR_INIT_COMMAND constant can only be used in the driver_options array when constructing a new database handle.
So instead of using setAttribute you should specify the MYSQL_ATTR_INIT_COMMAND when you create the PDO handle, like this :
$this->db = new PDO($dsn, $username, $password,
array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
My guess is it's probably a browser side thing. A couple more things to check into:
What is your web server sending for the Content-Type header? does it use charset=utf-8?
Do you have a different character set specified in the DOCTYPE of your HTML page?
Make sure phpMyAdmin itself is properly configured to use UTF8. It sounds like it is not.

PHP MySQL database strange characters

I'm trying to output product information stored in a MySQL database, but it's writing out some strange characters, like a diamond with a question mark inside of it.
I think it may be an encoding/UTF8 issue, but I've specified the encoding I want:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Is this right? What should I check for?
If only the data that's coming from database has strange characters in it, be sure that the MySQL connection is also in UTF8 by using:
mysql_query("SET NAMES UTF8");
before any other queries. Otherwise, if the characters appear also in 'handwritten' files, make sure that the files are saved as UTF-8 in your editor. You can also try setting the charset header through PHP:
header('Content-type: text/html; charset=UTF-8');
Also make sure that all fields in the tables you are querying are set as some UTF-8 variant, for example utf8_general_ci.
I assume you want the result to be in utf8
save you php script utf8 encoded
make sure your http header (or some meta tags) tells that output is utf8
all tables in MySql should to be utf8
last but not least, the connection between client and server should be utf8. (This could be handled somewhere in the php.ini setting or by making the following query against the db: SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'
If you follow all 4 point you should never ever have any problem with broken encodings.
The last time I had that trouble, the solution was similar to what Tatu Ulmanen said, but slightly different...
So if his solution does not work, try replacing
mysql_query("SET NAMES UTF8");
with
mysql_query("SET NAMES latin1");
I say this because the default characterset in MySql is latin1, and that is what is used most of the time....
hope that helps...
Seconding what Tatu says.
This is good background reading on encoding: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

Categories