I have got mysql database in utf8_general_ci and php page with utf-8 charset.
Why in php page i have got some text with bad encoding (which takes from data)?
http://likebox.ru/fbtn/
Use the following query:
ALTER DATABASE db_name_here DEFAULT CHARACTER SET utf8 COLLATE new_char_set_here;
Related
Table was created with:
CREATE TABLE IF NOT EXISTS `mathsqs` (
`questions` varchar(5000) NOT NULL,
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8;
I have inserted data through PHP using mysqli.
To confirm the insertion, I tried SELECT * FROM mathsqs LIMIT 1 on the Windows command line. It shows question marks for non english characters.
How do I see the exact posted data in MySql command line?
Example data I'm trying to handle:
இரு எண்களின் பெருக்கல் பலன் 3375 அவ்வெண்களின் மீ.பெ.வ 15
Assuming that you have already set the table char set to utf8 and it's collation is utf8.
Try adding this line just before you $mysqli command:
$mysqli->set_charset("utf8")
Also, set the utf 8 encoding in your page header where this output is coming.
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
Update:
When running the sql from command line, make sure you have set the default charset property before launching the mysql client. Something like:
Start the client with option --default-character-set=utf8.
mysql --default-character-set=utf8
To set it as a default option to be included automatically each time you run the mysql client add an entry in your my.cnf file, in the [mysql] section as:
[mysql]
default-character-set=utf8
Update #2:
#GopsAB I ran the DML statement to create the table as you specified. Followed the same process and surprisingly, I have having the same problem. I can't figure out why the question marks are displayed even after enforcing the character encoding.
So I digged further and made sure the command prompt was set to use use 'Lucidia Console' font and Active page code chcp 65001, By setting the property of the console to the use 'Lucidia Console' font and then running: chcp 65001 and followed the same process.
But now instead of '?' marks I am getting BOM character boxes....but the surprising thing is when I copy the console text that is displayed for the value of the column, I am getting the proper text: போக்குவரத்து (this is pasted directly from the console). Strange hah!
Important!
Turns out MySQL’s utf8 charset only partially implements proper UTF-8 encoding. It can only store UTF-8-encoded symbols that consist of one to three bytes; encoded symbols that take up four bytes aren’t supported.
In your case, the characters are stored perfectly and they are retrieved in php page and in mysql editors like sql workbench or Toad for SQL properly. Only the command line interface is unable to display them for some weird reason even after setting proper encoding and page type as discussed above. The text displayed in the console when copy/pasted displays correctly in notepad or any other place where you can type.
Running SET NAMES 'big5'; and SET NAMES 'utf8'; doesn't have any effect either
and neither did SET collation_connection = utf8_unicode_ci; SET NAMES utf8; did anything new but displayed only boxes which is the actual value when copy/pasted but on the console itself masked in boxes.
So until here you are good! nothing is wrong in your SQL and the values stored in the database are fine and fetched properly.
Something Extra:
MySQL’s utf8mb4
Luckily, MySQL 5.5.3 (released in early 2010) introduced a new encoding called utf8mb4 which maps to proper UTF-8 and thus fully supports Unicode, including astral symbols.
Switching from MySQL’s utf8 to utf8mb4
For each database:
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
For each table:
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
For each column:
ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
(Don’t blindly copy-paste this! The exact statement depends on the column type, maximum length, and other properties. The above line is just an example for a VARCHAR column.)
MySQL’s utf8mb4 Reference: How to support full Unicode in MySQL databases
I'm migrating a project from a CMS called "Freekore" to CodeIgniter, with my old CMS I didn't have this problem but with CI I can't figure out why I have problems with special characters as ñ,á,é,í etc.
I think I've tried everything.
I got this in header
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
also added this
<?php header('Content-Type: text/html; charset=UTF-8'); ?>
on database config
$db['local']['char_set'] = 'utf8';
$db['local']['dbcollat'] = 'utf8_general_ci';
and on config file
$config['charset'] = 'UTF-8';
But I still get this
COMUNICACIÓN Y PRODUCCIÓN EDITORIAL
instead of this
COMUNICACIÓN Y PRODUCCIÓN EDITORIAL
ah and also added this to .htaccess
AddDefaultCharset UTF-8
Edit
Using this I found out that Current character set is latin1 but how? I've checked database and tables and are utf8_general_ci
Edit 2
I did a new database and checked every column collation, now I'm using utf8_unicode_ci on the database, on every table and on ever row, but I still have the same problem Current character set is latin1
Edit 3
I decided to use utf8_decode and seemed to work but I still have problems with uppercases
After hours searching for something that help me resolve the problem I found an answer so easy in this post so before execute the insertion I used this mysql_query("SET NAMES utf8"); and now the characters are shown as they should. Thank you so much for your help.
did you try to encode your data in UTF-8 before you put them into your new db using this function: http://www.w3schools.com/php/func_xml_utf8_encode.asp
i think you need to change the database collection to UTF 8
execute the following queries .
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
hope it will solve your issue .
The collation of the database table itself is not UTF-8 unicode. Look here for example in phpmyadmin:
You need to update the collation. If you do not have phpmyadmin, you can use the following command:
ALTER TABLE `test_table` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
Your problem is likely with utf8_general_ci (MySQL 5.x default) as your collation. Most people are surprised that not all UTF8 is the same. For the most part you have 3 general "flavors"
utf8_general_ci
utf8_unicode_ci
utf8mb4
The difference is in how many bytes are being used for UTF8. utf8mb4 is the complete UTF8 character set, while utf8_general_ci is a small subset that covers most latin character sets, but probably doesn't cover the characters you mentioned. If you switch to utf8_unicode_ci it might cover them. The catch is that the more bytes you use, the slower your database will run in processing that data.
This thread discusses it in more detail.
What's the difference between utf8_general_ci and utf8_unicode_ci
Echo utf8_encode($row['$your_field']);
I am trying to import data from an XML file into a MYSQL DB using PHP. I am able to get the code to work just fine but when I look at the data in the DB there are special characters. For example, when I look at the XML in my browser it shows up as "outdoors in good weater..." but in the DB it appears to as "outdoors in good weather…".
I've cycled through all the different types of collation for that field in my DB but it does not seem to help much. Sometimes it shows up with the characters mentioned above and others as ???.
I have also tried to sync up the data with the following code in my PHP
$mysqli->query("SET NAMES 'utf8' COLLATE 'utf8_general_ci'");
But, again I have had no luck.
Thank you for reading this and for your help!
Akshay
You need to change the character set to UTF-8, along with your collation:
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
What you are seeing is a Unicode ellipsis character (…) being converted into another character set, which is probably Latin1. That is why it looks garbled.
I need to convert data from old database to new one. Old database was in latin1_swedish_ci collation and have content in cyrilic language like this
<p>ÐрхиепиÑкоп охридÑки и ми...
This content with utf-8 enconding on page looks like this
<p>Архиепископ охридски и митрополит скопски ...
Which is fine. Now I need to convert all of this data into native UTF-8 content. No expirience with these, any sugg.
Thanks
You can try this
ALTER TABLE <tablename> CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci
And note that , this will affect existing column collations also. If you want to change default collasion to utf8 , must change database collation. After that all new table will be utf8
From the manual,
ALTER TABLE t MODIFY col1 CHAR(50) CHARACTER SET utf8;
However, if you have characters that cannot be converted then you will lose that data. First make a backup and try it there.
I am writing a PHP script to import some data from a Microsoft SQL Server database to my MySQL database.
Collation in the SQL Server is Modern_Spanish_CI_AI (so I guess the encoding is ASCII), and in my MySQL database I'm using UTF-8 encoding and utf8_unicode_ci as collation.
When I import the data I get wrong characters corresponding to the Spanish letter Ñ. I've tried to use the PHP functions utf8_encode($string) and mb_convert_encoding($string, "UTF-8") without any success, (I got different wrong characters but still wrong).
Perhaps this helps:
Table's definition in SQL Server:
CREATE TABLE [dbo].[myTable] (
[myField] varchar(2) COLLATE Modern_Spanish_CI_AS NOT NULL,
Table's definition in MySQL
CREATE TABLE `myTable` (
`myField` varchar(2) COLLATE utf8_unicode_ci NOT NULL,
I have no experience with SQL server, but from a cursory glance at this collation chart, Modern_Spanish_CI_AI seems to be essentially a Windows-1252 character set, probably with some Spanish sorting / comparison rules on top.
You should be able to convert the data like this:
$utf8 = iconv("windows-1252", "utf-8", $input);
try it out. However, when using PHP for this, the character set of the connection to each database also plays a role - if it still doesn't work, you need to show some more code.