I'm trying to output product information stored in a MySQL database, but it's writing out some strange characters, like a diamond with a question mark inside of it.
I think it may be an encoding/UTF8 issue, but I've specified the encoding I want:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Is this right? What should I check for?
If only the data that's coming from database has strange characters in it, be sure that the MySQL connection is also in UTF8 by using:
mysql_query("SET NAMES UTF8");
before any other queries. Otherwise, if the characters appear also in 'handwritten' files, make sure that the files are saved as UTF-8 in your editor. You can also try setting the charset header through PHP:
header('Content-type: text/html; charset=UTF-8');
Also make sure that all fields in the tables you are querying are set as some UTF-8 variant, for example utf8_general_ci.
I assume you want the result to be in utf8
save you php script utf8 encoded
make sure your http header (or some meta tags) tells that output is utf8
all tables in MySql should to be utf8
last but not least, the connection between client and server should be utf8. (This could be handled somewhere in the php.ini setting or by making the following query against the db: SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'
If you follow all 4 point you should never ever have any problem with broken encodings.
The last time I had that trouble, the solution was similar to what Tatu Ulmanen said, but slightly different...
So if his solution does not work, try replacing
mysql_query("SET NAMES UTF8");
with
mysql_query("SET NAMES latin1");
I say this because the default characterset in MySql is latin1, and that is what is used most of the time....
hope that helps...
Seconding what Tatu says.
This is good background reading on encoding: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
Related
I have a form on a PHP page which inserts data into a MySQL database.
Some input fields may contain UTF8 characters as é, è, â, etc. When they are actually inserted into the database, everything gets messed up. For example, a column shows Qréon instead of Qréon.
I used setLocale(LC_CTYPE, 'FR_fr.UTF-8'); at the top of my page for PHP and this <meta charset="utf-8"> is in my HTML header.
My database is run by MySQL, the storage engine is InnoDB, and the collation is utf8_general_mysql500_ci. I also tried utf8_general_ci and utf8_bin but I had no luck.
How do I know if this comes either from PHP or MySQL processing, and how can I fix it ?
Thank you for your time.
I think this can help you:
If your using mysql
mysql_query("SET NAMES 'utf8'");
If your using PDO use this:
$dbh->exec("set names utf8");
Otherwise i could be one of these which helps you specific:
//At the Top of you files
ini_set("default_charset", "UTF-8");
header('Content-type: text/html; charset=UTF-8');
//Before your queries
mysql_query("SET CHARACTER SET utf8 ");
mysql_set_charset('utf8');
I'm importing data from Oracle database to MySQL tables.
I have set my MySQL table charset as utf8_general_ci and database and table name with field column value set as utf-8 as well.
Now, When I fetch the result, it prints like, which is with ? sign:
مرحبا العال� - 5
I have my utf value in column is مرØبا العالÙ
When I compare this string with Oracle string, it shows proper value - exact copy of Oracle database and there it shows perfect string in Arabic.
I have set my html meta with utf-8 as well
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" >
If I set mysql query as below, it shows junk characters:
mysql_query("SET NAMES utf8;");
mysql_query("SET CHARACTER_SET utf8;");
Followed everything possible found over stack and other sites, and still getting an error.
Please help !
Did you save the php-file without BOM? If not, try it. Potential issues with the UTF-8 BOM
Further try with 'utf-8' using single quotes and without SET CHARACTER_SET
mysql_query("SET NAMES 'utf8'");
and with charset utf-8 in the html-document header:
header("content-type: text/html; charset=utf-8");
with pdo you should have this
$_dbhandler = new PDO(PDO_DSN, DB_USERNAME, DB_PASSWORD,array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES UTF8"));
I'm trying to insert an hebrew value into my mysql db, instead of hebrew the values looks like that.
שדגשדכעשד
The collation of the table is latin1_swedish_ci by default, I tried also to change to utf-8_general_ci, hebrew_bin, hebrew_general_ci but the result is still the same.
In my code I'm using of course the meta tag to configure the charset:
<meta charset="UTF-8">
And before my php query I added this line:
mysql_query("SET NAMES utf8");
I'm viewing the result in the phpmyadmin.
I have solved my Hebrew language problem. It was a database and table row/field encoding issue. Here is the solution I used. I took help from another answer and the link is given below, in case anyone needs it.
The database collation has to be utf8_general_ci.
The collation of the table with Hebrew has to be utf8_general_ci.
In the PHP connection script put
header('Content-Type: text/html; charset=utf-8');
In the xhtml head tag put
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
If you are using MySQLi put this code in the connection script after selecting the database:
mysql_query("SET NAMES 'utf8'");
If you are using PDO, put
$conn->query("SET NAMES 'utf8'");
The first answer helped me and I took it from there
Set charset to achieve the solution
While creating the database
CREATE DATABASE db_name
CHARACTER SET utf8
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci
DEFAULT COLLATE utf8_general_ci
;
Or if the database is already created
CREATE TABLE table_name(
...
)
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci;
OR while writing query
mysql_query("SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'", $conn);
$re = mysql_query('SHOW VARIABLES LIKE "%character_set%";')or die(mysql_error());
while ($r = mysql_fetch_assoc($re)) {var_dump ($r); echo "<br />";}
Check the collation_connection:
show variables like '%collation%'
you should make sure that:
you set utf-8 in php
you use utf-8 in the connection
your table is defined as utf-8_general_ci
the specific field is defined as utf-8_general_ci
Then you should be able to view Hebrew, or any other language, correctly in phpadmin
what finally helped me is to add the charset to the connection:
{"mysql:host=$host;dbname=$db;charset=utf8"}
I would say, in order to make sure that the values are passed well to the database I would add a die statment before inserting to the database and print the value, example:
die($_POST['thevalue']);
//insert to database.
//...
If it goes well, then the problem is on the database side, on the database I would try with this collation
| hebrew | ISO 8859-8 Hebrew | hebrew_general_ci | 1 |
as per http://dev.mysql.com/doc/refman/5.0/en/charset-mysql.html suggest.
But if it fail on the php side, reason can be, the server does not support Hebrew, make sure that on the html output document you use the correct metatag with
...
<meta charset="ISO 8859-8">
...
Let us knows how it proceed, good luck :)
For future use, if you have this issue and you are using PDO and not mysqli, you will need to do it like this:
The Database (all of it) collation has to be utf8_general_ci.
Set the collation of the table with Hebrew also to utf8_general_ci.
In the HTML "head" tag, add this: < meta charset="utf-8" >
In your PHP connection file, add after the connection Query this line: conn->exec("set names utf8");
if you are using classes, then you will probably have the "conn" as a variable.
in that case, you will have to use it like this:
$this->conn->exec("set names utf8");
Hope this helps to future people that have this problem and using PDO.
Best of luck.
I think I figure it out:
here's my code:
$conn = mysqli_connect($dbserver,$dbuser,$dbpwd,$dbname);
if (mysqli_connect_errno()){
printf("Connection failed: %s\n" , mysqli_connect_errno());
exit();
}
printf("Initial character set: %s\n", mysqli_character_set_name($conn));
/* change character set to utf8 */
if (!mysqli_set_charset($conn, "hebrew")) {
printf("Error loading character set hebrew: %s\n", mysqli_error($conn));
exit();
} else {
printf("Current character set: %s\n", mysqli_character_set_name($conn));
}
I've always used ISO-8859-1 encoding, but I'm now going over to UTF-8.
Unfortunately I can't get it to work.
My MySQL DB is UTF-8, my PHP document is encoded in UTF-8, I set a UTF-8 charset, but it still doesn't work.
(it is special characters like æ/ø/å that doesn't work)
Hope you guys can help!
Make sure the connection to your database is also using this character set:
$conn = mysql_connect($server, $username, $password);
mysql_set_charset("UTF8", $conn);
According to the documentation of mysql_set_charset at php.net:
Note:
This is the preferred way to change the charset. Using mysql_query() to execute
SET NAMES .. is not recommended.
See also: http://nl3.php.net/manual/en/function.mysql-set-charset.php
Check the character set of your current connection with:
echo mysql_client_encoding($conn);
See also: http://nl3.php.net/manual/en/function.mysql-client-encoding.php
If you have done these things and add weird characters to your table, you will see it is displayed correct.
Remember to set connection encoding to utf8 as well.
In ext\mysqli do
$mysqli->set_charset("utf8")
In ext\mysql do
mysql_set_charset("utf8")
With other db extensions you might have to run query like
SET NAMES 'utf8'
Some more details about connection encoding in MySQL
As others point out, making sure your source code is utf-8 encoded also helps. Pay special attention to not having BOM (Byte Order Mark) - it would be sent to browser before any code is executed, so using headers or sessions would become impossible.
After connecting to db, run query SET NAMES UTF8
$db = new db(...);
$db->query('set name utf8');
and add this tag to header
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Are you having this error? MySql SELECT UNION Illegal mix of collations Error? Just set you entire mysql to utf 8 then
SET character_set_connection = utf8;
Try this after connecting to mysql:
mysql_query("SET NAMES 'utf8'");
And encode PHP document in UTF-8 without BOM.
I had the same problem but now its resolved. Here is the solution:
1st: update ur table
ALTER TABLE tbl_name
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci;
2nd:
add this in the head section of the HTML code:
Regards
Saleha A.Latif
Nowadays PDO is the recommended way to use mysql. With that you should use the connection string to set encoding. For example: "mysql:host=$host;dbname=$db;charset=utf8"
I used iconv to convert from latin1 to utf8 when I did an mysql dump of a database from mysql v4.0.21, and imported it onto a new server mysql v5.0.45
It was latin1 on the old server, it’s utf8 on the new server, so I ran this on the mysql dump: iconv −f latin1 −t UTF−8 quickwebcms_2010-03-01.sql
It ran successful, then I imported it onto the new server.
Now it displays question (?) marks (example: College?s) and  (example: College’s) when it prints out some of the data in my PHP application.
I exported the table these characters show up in and did a find and replace all within textmate, then imported it back into the new database and it uploads some of the fields as null, so the find and replace may of messed up something in the process. I saved the table csv as utf8 no bom, and just utf8 and it still does the same thing.
Any help as to why this might be happening is appreciated.
If the content of your tables are all OK (and in UTF-8) and you sill have "bad" characters in your Web application, make sure your MySQL connection is using the UTF-8 charset in your PHP script. Even if your databases and tables are in UTF-8, MySQL uses latin1 connections by default (at least in my shared server config). So you have to tell MySQL to send content in UTF-8. Otherwise it will convert it on the fly to latin1 producing "bad" characters in UTF-8 webpages.
Use mysql_set_charset if available otherwise you can set it with a SQL query (always use mysql_set_charset if available):
if (function_exists('mysql_set_charset'))
mysql_set_charset('utf8', $conn);
else
{
if (mysql_query("SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'", $conn) === false)
{
//Error! Do something...
}
}
Also make sure your (X)HTML markup uses UTF-8 too:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
IIRC, mysqldump produces UTF-8 output by default, no matter what the database's encoding is. This user comment in the mySQL manual seems to confirm it:
I am just using default character sets - normally latin1. However, the dump produced by mysqldump is, perhaps surprisingly, in utf8. This seems fine, but leads to trouble with the --skip-opt option to mysqldump, which turns off --set-charset but leaves the dump in utf8.
Perhaps the fact that mysqldump uses utf8 by default, and the importance of the --set-charset option should be more prominently documented (see the documentation for the --default-character-set attribute for the current mention of the use of utf8)
Try skipping the iconv step, might work straight away.
You may be better off loading the data onto the new server as latin1, then using the appropriate ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 DEFAULT COLLATION utf8_unicode_ci on each table (or use a script of some sort to do it for you).
Or you could convert first, then dump.