I'm migrating a project from a CMS called "Freekore" to CodeIgniter, with my old CMS I didn't have this problem but with CI I can't figure out why I have problems with special characters as ñ,á,é,í etc.
I think I've tried everything.
I got this in header
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
also added this
<?php header('Content-Type: text/html; charset=UTF-8'); ?>
on database config
$db['local']['char_set'] = 'utf8';
$db['local']['dbcollat'] = 'utf8_general_ci';
and on config file
$config['charset'] = 'UTF-8';
But I still get this
COMUNICACIÓN Y PRODUCCIÓN EDITORIAL
instead of this
COMUNICACIÓN Y PRODUCCIÓN EDITORIAL
ah and also added this to .htaccess
AddDefaultCharset UTF-8
Edit
Using this I found out that Current character set is latin1 but how? I've checked database and tables and are utf8_general_ci
Edit 2
I did a new database and checked every column collation, now I'm using utf8_unicode_ci on the database, on every table and on ever row, but I still have the same problem Current character set is latin1
Edit 3
I decided to use utf8_decode and seemed to work but I still have problems with uppercases
After hours searching for something that help me resolve the problem I found an answer so easy in this post so before execute the insertion I used this mysql_query("SET NAMES utf8"); and now the characters are shown as they should. Thank you so much for your help.
did you try to encode your data in UTF-8 before you put them into your new db using this function: http://www.w3schools.com/php/func_xml_utf8_encode.asp
i think you need to change the database collection to UTF 8
execute the following queries .
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
hope it will solve your issue .
The collation of the database table itself is not UTF-8 unicode. Look here for example in phpmyadmin:
You need to update the collation. If you do not have phpmyadmin, you can use the following command:
ALTER TABLE `test_table` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
Your problem is likely with utf8_general_ci (MySQL 5.x default) as your collation. Most people are surprised that not all UTF8 is the same. For the most part you have 3 general "flavors"
utf8_general_ci
utf8_unicode_ci
utf8mb4
The difference is in how many bytes are being used for UTF8. utf8mb4 is the complete UTF8 character set, while utf8_general_ci is a small subset that covers most latin character sets, but probably doesn't cover the characters you mentioned. If you switch to utf8_unicode_ci it might cover them. The catch is that the more bytes you use, the slower your database will run in processing that data.
This thread discusses it in more detail.
What's the difference between utf8_general_ci and utf8_unicode_ci
Echo utf8_encode($row['$your_field']);
Related
Table was created with:
CREATE TABLE IF NOT EXISTS `mathsqs` (
`questions` varchar(5000) NOT NULL,
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8;
I have inserted data through PHP using mysqli.
To confirm the insertion, I tried SELECT * FROM mathsqs LIMIT 1 on the Windows command line. It shows question marks for non english characters.
How do I see the exact posted data in MySql command line?
Example data I'm trying to handle:
இரு எண்களின் பெருக்கல் பலன் 3375 அவ்வெண்களின் மீ.பெ.வ 15
Assuming that you have already set the table char set to utf8 and it's collation is utf8.
Try adding this line just before you $mysqli command:
$mysqli->set_charset("utf8")
Also, set the utf 8 encoding in your page header where this output is coming.
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
Update:
When running the sql from command line, make sure you have set the default charset property before launching the mysql client. Something like:
Start the client with option --default-character-set=utf8.
mysql --default-character-set=utf8
To set it as a default option to be included automatically each time you run the mysql client add an entry in your my.cnf file, in the [mysql] section as:
[mysql]
default-character-set=utf8
Update #2:
#GopsAB I ran the DML statement to create the table as you specified. Followed the same process and surprisingly, I have having the same problem. I can't figure out why the question marks are displayed even after enforcing the character encoding.
So I digged further and made sure the command prompt was set to use use 'Lucidia Console' font and Active page code chcp 65001, By setting the property of the console to the use 'Lucidia Console' font and then running: chcp 65001 and followed the same process.
But now instead of '?' marks I am getting BOM character boxes....but the surprising thing is when I copy the console text that is displayed for the value of the column, I am getting the proper text: போக்குவரத்து (this is pasted directly from the console). Strange hah!
Important!
Turns out MySQL’s utf8 charset only partially implements proper UTF-8 encoding. It can only store UTF-8-encoded symbols that consist of one to three bytes; encoded symbols that take up four bytes aren’t supported.
In your case, the characters are stored perfectly and they are retrieved in php page and in mysql editors like sql workbench or Toad for SQL properly. Only the command line interface is unable to display them for some weird reason even after setting proper encoding and page type as discussed above. The text displayed in the console when copy/pasted displays correctly in notepad or any other place where you can type.
Running SET NAMES 'big5'; and SET NAMES 'utf8'; doesn't have any effect either
and neither did SET collation_connection = utf8_unicode_ci; SET NAMES utf8; did anything new but displayed only boxes which is the actual value when copy/pasted but on the console itself masked in boxes.
So until here you are good! nothing is wrong in your SQL and the values stored in the database are fine and fetched properly.
Something Extra:
MySQL’s utf8mb4
Luckily, MySQL 5.5.3 (released in early 2010) introduced a new encoding called utf8mb4 which maps to proper UTF-8 and thus fully supports Unicode, including astral symbols.
Switching from MySQL’s utf8 to utf8mb4
For each database:
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
For each table:
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
For each column:
ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
(Don’t blindly copy-paste this! The exact statement depends on the column type, maximum length, and other properties. The above line is just an example for a VARCHAR column.)
MySQL’s utf8mb4 Reference: How to support full Unicode in MySQL databases
I have a laravel 4.2 project and I have a utf8 fields in database, but the way the data is stored in this filed is like را characters.
In any php file (other than laravel) after selecting the data from database those characters rendered in correct way after using (SET NAMES 'utf8'). I want to do same in laravel. (even if not a database solution)
Here is what i have tried:
make sure all files are utf8
make sure that in config files charset and collation are utf8
use PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8" in config files
I also tried to use Blade::setEchoFormat('e(utf8_encode(%s))'); but did not know how to use it in correct way.
Any help would be appreciated.
Do not use iconv. Do not use utf8_encode.
You have Mojibake.
Were you expecting را instead of را? Is this the HEX that is in the table: D8B1D8A7?
This is the classic case of
The bytes you have in the client are correctly encoded in utf8 (good).
You connected with SET NAMES latin1 (or set_charset('latin1') or ...), probably by default. (It should have been utf8.)
The column in the tables may or may not have been CHARACTER SET utf8, but it should have been that.
If you need to fix the data it takes a "2-step ALTER", something like
ALTER TABLE Tbl MODIFY COLUMN col VARBINARY(...) ...;
ALTER TABLE Tbl MODIFY COLUMN col VARCHAR(...) ... CHARACTER SET utf8 ...;
where the lengths are big enough and the other "..." have whatever else (NOT NULL, etc) was already on the column.
ok here is how i get it to work :
my problem was the opposite of what i thought .
all what i did is removing utf8 settings from database.php config file , and comment
line $connection->prepare($names)->execute(); in MysqlConnector.php file
and the right words shown
thanks for all and if any one know a better solution please share with me.
Set CHARACTER SET to utf8 and COLLATE to utf8_general_ci of your database, table, and column.
ALTER DATABASE db_name CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE table_name MODIFY column_name column_name VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_general_ci;
I am trying to import data from an XML file into a MYSQL DB using PHP. I am able to get the code to work just fine but when I look at the data in the DB there are special characters. For example, when I look at the XML in my browser it shows up as "outdoors in good weater..." but in the DB it appears to as "outdoors in good weather…".
I've cycled through all the different types of collation for that field in my DB but it does not seem to help much. Sometimes it shows up with the characters mentioned above and others as ???.
I have also tried to sync up the data with the following code in my PHP
$mysqli->query("SET NAMES 'utf8' COLLATE 'utf8_general_ci'");
But, again I have had no luck.
Thank you for reading this and for your help!
Akshay
You need to change the character set to UTF-8, along with your collation:
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
What you are seeing is a Unicode ellipsis character (…) being converted into another character set, which is probably Latin1. That is why it looks garbled.
I need to convert data from old database to new one. Old database was in latin1_swedish_ci collation and have content in cyrilic language like this
<p>ÐрхиепиÑкоп охридÑки и ми...
This content with utf-8 enconding on page looks like this
<p>Архиепископ охридски и митрополит скопски ...
Which is fine. Now I need to convert all of this data into native UTF-8 content. No expirience with these, any sugg.
Thanks
You can try this
ALTER TABLE <tablename> CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci
And note that , this will affect existing column collations also. If you want to change default collasion to utf8 , must change database collation. After that all new table will be utf8
From the manual,
ALTER TABLE t MODIFY col1 CHAR(50) CHARACTER SET utf8;
However, if you have characters that cannot be converted then you will lose that data. First make a backup and try it there.
I have got mysql database in utf8_general_ci and php page with utf-8 charset.
Why in php page i have got some text with bad encoding (which takes from data)?
http://likebox.ru/fbtn/
Use the following query:
ALTER DATABASE db_name_here DEFAULT CHARACTER SET utf8 COLLATE new_char_set_here;