I need to convert data from old database to new one. Old database was in latin1_swedish_ci collation and have content in cyrilic language like this
<p>ÐрхиепиÑкоп охридÑки и ми...
This content with utf-8 enconding on page looks like this
<p>Архиепископ охридски и митрополит скопски ...
Which is fine. Now I need to convert all of this data into native UTF-8 content. No expirience with these, any sugg.
Thanks
You can try this
ALTER TABLE <tablename> CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci
And note that , this will affect existing column collations also. If you want to change default collasion to utf8 , must change database collation. After that all new table will be utf8
From the manual,
ALTER TABLE t MODIFY col1 CHAR(50) CHARACTER SET utf8;
However, if you have characters that cannot be converted then you will lose that data. First make a backup and try it there.
Related
I have a laravel 4.2 project and I have a utf8 fields in database, but the way the data is stored in this filed is like را characters.
In any php file (other than laravel) after selecting the data from database those characters rendered in correct way after using (SET NAMES 'utf8'). I want to do same in laravel. (even if not a database solution)
Here is what i have tried:
make sure all files are utf8
make sure that in config files charset and collation are utf8
use PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8" in config files
I also tried to use Blade::setEchoFormat('e(utf8_encode(%s))'); but did not know how to use it in correct way.
Any help would be appreciated.
Do not use iconv. Do not use utf8_encode.
You have Mojibake.
Were you expecting را instead of را? Is this the HEX that is in the table: D8B1D8A7?
This is the classic case of
The bytes you have in the client are correctly encoded in utf8 (good).
You connected with SET NAMES latin1 (or set_charset('latin1') or ...), probably by default. (It should have been utf8.)
The column in the tables may or may not have been CHARACTER SET utf8, but it should have been that.
If you need to fix the data it takes a "2-step ALTER", something like
ALTER TABLE Tbl MODIFY COLUMN col VARBINARY(...) ...;
ALTER TABLE Tbl MODIFY COLUMN col VARCHAR(...) ... CHARACTER SET utf8 ...;
where the lengths are big enough and the other "..." have whatever else (NOT NULL, etc) was already on the column.
ok here is how i get it to work :
my problem was the opposite of what i thought .
all what i did is removing utf8 settings from database.php config file , and comment
line $connection->prepare($names)->execute(); in MysqlConnector.php file
and the right words shown
thanks for all and if any one know a better solution please share with me.
Set CHARACTER SET to utf8 and COLLATE to utf8_general_ci of your database, table, and column.
ALTER DATABASE db_name CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE table_name MODIFY column_name column_name VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_general_ci;
I have this statement which works in the first run
UPDATE accounts SET username = CONVERT(CAST(CONVERT(username USING latin1) AS BINARY) USING utf8)
This would convert latin1 characters into UTF8 chinese character in mysql.
Running it the 2nd time, the character will become weird.
How do I add in a where condition that only update username to utf8 from latin1 if the word is currently in latin1
What is the CHARACTER SET for the column? If it is still latin1, you are asking for a lot of trouble.
A table declared to be latin1, and containing latin1 bytes can be converted to utf8 via
ALTER TABLE tbl CONVERT TO CHARACTER SET utf8;
After that (and assuming you have SET NAMES set correctly), text will be converted to utf8 as it is inserted. No need to a flag; no need for second pass; etc.
I'm migrating a project from a CMS called "Freekore" to CodeIgniter, with my old CMS I didn't have this problem but with CI I can't figure out why I have problems with special characters as ñ,á,é,í etc.
I think I've tried everything.
I got this in header
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
also added this
<?php header('Content-Type: text/html; charset=UTF-8'); ?>
on database config
$db['local']['char_set'] = 'utf8';
$db['local']['dbcollat'] = 'utf8_general_ci';
and on config file
$config['charset'] = 'UTF-8';
But I still get this
COMUNICACIÓN Y PRODUCCIÓN EDITORIAL
instead of this
COMUNICACIÓN Y PRODUCCIÓN EDITORIAL
ah and also added this to .htaccess
AddDefaultCharset UTF-8
Edit
Using this I found out that Current character set is latin1 but how? I've checked database and tables and are utf8_general_ci
Edit 2
I did a new database and checked every column collation, now I'm using utf8_unicode_ci on the database, on every table and on ever row, but I still have the same problem Current character set is latin1
Edit 3
I decided to use utf8_decode and seemed to work but I still have problems with uppercases
After hours searching for something that help me resolve the problem I found an answer so easy in this post so before execute the insertion I used this mysql_query("SET NAMES utf8"); and now the characters are shown as they should. Thank you so much for your help.
did you try to encode your data in UTF-8 before you put them into your new db using this function: http://www.w3schools.com/php/func_xml_utf8_encode.asp
i think you need to change the database collection to UTF 8
execute the following queries .
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
hope it will solve your issue .
The collation of the database table itself is not UTF-8 unicode. Look here for example in phpmyadmin:
You need to update the collation. If you do not have phpmyadmin, you can use the following command:
ALTER TABLE `test_table` CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
Your problem is likely with utf8_general_ci (MySQL 5.x default) as your collation. Most people are surprised that not all UTF8 is the same. For the most part you have 3 general "flavors"
utf8_general_ci
utf8_unicode_ci
utf8mb4
The difference is in how many bytes are being used for UTF8. utf8mb4 is the complete UTF8 character set, while utf8_general_ci is a small subset that covers most latin character sets, but probably doesn't cover the characters you mentioned. If you switch to utf8_unicode_ci it might cover them. The catch is that the more bytes you use, the slower your database will run in processing that data.
This thread discusses it in more detail.
What's the difference between utf8_general_ci and utf8_unicode_ci
Echo utf8_encode($row['$your_field']);
I run the following query in mysql
UPDATE `gamequestions` SET a2 = '≠' WHERE id = 564
It runs successfully but the '?' is inserted in a2 field in place of '≠'
The datatype of a2 is text and also tried with varchar
Any Help greatly appreciated.
you need to change Collation to UTF-8 to store special characters
insert ≠ (not equal to ) in mysql field
The goal in these conversions is always to decide on what charset/collation combination you want to use (UTF8 being the best choice in almost all scenarios) then to convert all tables/columns in your database to use that charset. At that point you can set DB_COLLATE and DB_CHARSET` to the desired charset and collation to match.
Note:
In most cases if a collation is not defined MySQL will assume the default collation for the CHARSET which is specified. For UTF8 the default is utf8_general_ci, which is usually the right choice.
Changing the default charset of the database
ALTER DATABASE MyDb CHARACTER SET utf8;
Changing the default charset of individual tables
ALTER TABLE MyTable CHARACTER SET utf8;
https://dev.mysql.com/doc/refman/5.1/en/charset-unicode-utf8.html
You can add that option in the /mysql/my.cnf. In the [mysqld] section add ’’character-set-server=UTF8"; in the [client] section add “default-character-set=UTF8”.
You can find more information in these links:
http://dev.mysql.com/doc/refman/5.1/en/charset-… http://dev.mysql.com/doc/refman/5.0/en/server-o…
If you need to conver existing data, you can execute:
ALTER TABLE tbl_name CONVERT TO CHARACTER SET charset_name;
You need to check following things
use set names utf8 before you query/insert into the database
using Default CHARSET=utf8 when creating new tables
I have got mysql database in utf8_general_ci and php page with utf-8 charset.
Why in php page i have got some text with bad encoding (which takes from data)?
http://likebox.ru/fbtn/
Use the following query:
ALTER DATABASE db_name_here DEFAULT CHARACTER SET utf8 COLLATE new_char_set_here;