Hey guys I am having trouble trying to convert my web app to support unicode characters. I have the following script which tries to insert russian characters into my mysql database but just outputs ?????? in my mysql database field. I have changed default charset to UTF-8 in my php.ini and have modified my table fields to collation: utf8_unicode_ci. Anyone have any ideas?
mb_language('uni'); mb_internal_encoding('UTF-8');
$sql = 'SET NAMES utf8';
$stmt = $conn->prepare($sql);
$result=$stmt->execute();
$sql = 'SET CHARACTER SET utf8';
$stmt = $conn->prepare($sql);
$result=$stmt->execute();
$sql = 'INSERT INTO topic (topic_id,topic_title) VALUES (?,?)';
$stmt6 = $conn->prepare($sql);
$result=$stmt6->execute(array(0,"дравствуйте"));
?>
show create table edit
CREATE TABLE `topic` (
`id` mediumint(8) NOT NULL AUTO_INCREMENT,
`topic_title` varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
`description` mediumtext CHARACTER SET utf8 COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`),
FULLTEXT KEY `description` (`description`),
FULLTEXT KEY `topic_title` (`topic_title`),
FULLTEXT KEY `topic_title_2` (`topic_title`,`description`),
FULLTEXT KEY `description_2` (`description`)
) ENGINE=MyISAM AUTO_INCREMENT=39 DEFAULT CHARSET=latin1
I got it to work by using the following sql:
ALTER DATABASE mydatabase charset=utf8;
If you are seeing those ?????? on mysql browser or on mysql cli, make sure you tune up the charset settings on the programs, even if the entries are correctly added to the database, displaying them correctly also depends on the client charset settings.
I'm inclined to think the problem isn't with MySQL.
You are entering an non ASCII string as a literal into a PHP script. Are you sure the "дравствуйте" string will be sent to MySQL server as UTF-8 encoded?
I've seen some problems with nos ASCII characters with PHP and MySQL with incompatible charset configurations amongst the operating system, the PHP interpreter (it inherits the default from the operating system), the database and the development tools (the encoding of .php files). If all four are configured to the same encoding, it should work.
You said that your php.ini is configured to UTF-8. If your source php file isn't in UTF-8, it will not work! So, check if your php files are encoded as UTF-8 and if the mysql client is configured to UTF-8 too.
A common mistake is to forget to configure the MySQL client to UTF-8 too and fail to check the data on the database because of charset problems with MySQL client application.
Home it helps!
Normally, (if this you're quoting file is also in utf-8) the PHP & MySQL side is OK, but the HTTP / HTML side of things isn't.
Do you send a Content-Type: text/html;charset=utf-8 header?
Is there a <meta> element on the HTML page that claims another charset then utf-8?
So, after more infomation: in 'normal' PHP scripts it works I gather (?), in PHPMyAdmin it doesn't (which should make it more of a candidate for superuser.com b.t.w.). A version of PHPMyAdmin used would be nice, but normally, a $cfg['DefaultCharset'] = 'utf-8'; in PHPMyAdmin config should work if the rest of non-PHPMyAdmin script works. If not: update to the latest PHPMyAdmin release, and if it still doesn't work file a bug report.
From PHPMYADMIN please change the Collation to utf32_general_ci, they will support and some cases it will make existing "????" to correct one as well.
PHP Myadmin illustration purpose
DB Activity for Storing Emojis
1) default-character-set = utf8mb4 under [client]
2) default-character-set = utf8mb4 under [mysql]
3) Following under [mysqld]
character-set-client-handshake = FALSE
character-set-server = `utf8mb4`
collation-server = `utf8mb4_unicode_ci`
4) Restart Mysql server using below command
`sudo service mysql stop`
`sudo service mysql start`
5) ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
6) ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
7) SET NAMES utf8mb4;
Related
Table was created with:
CREATE TABLE IF NOT EXISTS `mathsqs` (
`questions` varchar(5000) NOT NULL,
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=utf8;
I have inserted data through PHP using mysqli.
To confirm the insertion, I tried SELECT * FROM mathsqs LIMIT 1 on the Windows command line. It shows question marks for non english characters.
How do I see the exact posted data in MySql command line?
Example data I'm trying to handle:
இரு எண்களின் பெருக்கல் பலன் 3375 அவ்வெண்களின் மீ.பெ.வ 15
Assuming that you have already set the table char set to utf8 and it's collation is utf8.
Try adding this line just before you $mysqli command:
$mysqli->set_charset("utf8")
Also, set the utf 8 encoding in your page header where this output is coming.
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
Update:
When running the sql from command line, make sure you have set the default charset property before launching the mysql client. Something like:
Start the client with option --default-character-set=utf8.
mysql --default-character-set=utf8
To set it as a default option to be included automatically each time you run the mysql client add an entry in your my.cnf file, in the [mysql] section as:
[mysql]
default-character-set=utf8
Update #2:
#GopsAB I ran the DML statement to create the table as you specified. Followed the same process and surprisingly, I have having the same problem. I can't figure out why the question marks are displayed even after enforcing the character encoding.
So I digged further and made sure the command prompt was set to use use 'Lucidia Console' font and Active page code chcp 65001, By setting the property of the console to the use 'Lucidia Console' font and then running: chcp 65001 and followed the same process.
But now instead of '?' marks I am getting BOM character boxes....but the surprising thing is when I copy the console text that is displayed for the value of the column, I am getting the proper text: போக்குவரத்து (this is pasted directly from the console). Strange hah!
Important!
Turns out MySQL’s utf8 charset only partially implements proper UTF-8 encoding. It can only store UTF-8-encoded symbols that consist of one to three bytes; encoded symbols that take up four bytes aren’t supported.
In your case, the characters are stored perfectly and they are retrieved in php page and in mysql editors like sql workbench or Toad for SQL properly. Only the command line interface is unable to display them for some weird reason even after setting proper encoding and page type as discussed above. The text displayed in the console when copy/pasted displays correctly in notepad or any other place where you can type.
Running SET NAMES 'big5'; and SET NAMES 'utf8'; doesn't have any effect either
and neither did SET collation_connection = utf8_unicode_ci; SET NAMES utf8; did anything new but displayed only boxes which is the actual value when copy/pasted but on the console itself masked in boxes.
So until here you are good! nothing is wrong in your SQL and the values stored in the database are fine and fetched properly.
Something Extra:
MySQL’s utf8mb4
Luckily, MySQL 5.5.3 (released in early 2010) introduced a new encoding called utf8mb4 which maps to proper UTF-8 and thus fully supports Unicode, including astral symbols.
Switching from MySQL’s utf8 to utf8mb4
For each database:
ALTER DATABASE database_name CHARACTER SET = utf8mb4 COLLATE = utf8mb4_unicode_ci;
For each table:
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
For each column:
ALTER TABLE table_name CHANGE column_name column_name VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
(Don’t blindly copy-paste this! The exact statement depends on the column type, maximum length, and other properties. The above line is just an example for a VARCHAR column.)
MySQL’s utf8mb4 Reference: How to support full Unicode in MySQL databases
I have a laravel 4.2 project and I have a utf8 fields in database, but the way the data is stored in this filed is like را characters.
In any php file (other than laravel) after selecting the data from database those characters rendered in correct way after using (SET NAMES 'utf8'). I want to do same in laravel. (even if not a database solution)
Here is what i have tried:
make sure all files are utf8
make sure that in config files charset and collation are utf8
use PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8" in config files
I also tried to use Blade::setEchoFormat('e(utf8_encode(%s))'); but did not know how to use it in correct way.
Any help would be appreciated.
Do not use iconv. Do not use utf8_encode.
You have Mojibake.
Were you expecting را instead of را? Is this the HEX that is in the table: D8B1D8A7?
This is the classic case of
The bytes you have in the client are correctly encoded in utf8 (good).
You connected with SET NAMES latin1 (or set_charset('latin1') or ...), probably by default. (It should have been utf8.)
The column in the tables may or may not have been CHARACTER SET utf8, but it should have been that.
If you need to fix the data it takes a "2-step ALTER", something like
ALTER TABLE Tbl MODIFY COLUMN col VARBINARY(...) ...;
ALTER TABLE Tbl MODIFY COLUMN col VARCHAR(...) ... CHARACTER SET utf8 ...;
where the lengths are big enough and the other "..." have whatever else (NOT NULL, etc) was already on the column.
ok here is how i get it to work :
my problem was the opposite of what i thought .
all what i did is removing utf8 settings from database.php config file , and comment
line $connection->prepare($names)->execute(); in MysqlConnector.php file
and the right words shown
thanks for all and if any one know a better solution please share with me.
Set CHARACTER SET to utf8 and COLLATE to utf8_general_ci of your database, table, and column.
ALTER DATABASE db_name CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE table_name CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE table_name MODIFY column_name column_name VARCHAR(255) CHARACTER SET utf8 COLLATE utf8_general_ci;
i have this chinese character which i want to insert into database and browser output in correct but when data is inserted into mysql it is in different format something like this.
my chinese characters
图片库,高清图片大全,图库
and inserted into database is
图片库,高清图片大全,图库
i have tried several things like setting utf8
mysql_set_charset("utf8", $connection);
then chenaged the collation from swidish to utf general
then checked the details with this command which proved that my charcter is correctly set to utf8
SHOW CREATE TABLE tablea;
CREATE TABLE `tablea` (
`id` int(12) NOT NULL AUTO_INCREMENT,
`tiya` text CHARACTER SET utf8 NOT NULL,
PRIMARY KEY (`id`),
) ENGINE=MyISAM AUTO_INCREMENT=10 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
can anyone suggest where to solve chinese charracter issue .my insert is this
mysqli_query ($con,"INSERT INTO tablea (tiya) VALUES ('$tit')");
All of your entire environment must be utf-8-capable. The file encoding of you php-file must be utf-8. The web server (apache?) must serve utf-8. Your mysql table must have utf-8 as charset. You connection(s) must be utf-8.
If one in the chain is missing, you get the above results. Additionaly: if you tried to fiddle around with character conversion inbetween, this is probably mixed up entirely. Once the above chain is set up properly, no char conversion is needed at all.
I am writing a PHP script to import some data from a Microsoft SQL Server database to my MySQL database.
Collation in the SQL Server is Modern_Spanish_CI_AI (so I guess the encoding is ASCII), and in my MySQL database I'm using UTF-8 encoding and utf8_unicode_ci as collation.
When I import the data I get wrong characters corresponding to the Spanish letter Ñ. I've tried to use the PHP functions utf8_encode($string) and mb_convert_encoding($string, "UTF-8") without any success, (I got different wrong characters but still wrong).
Perhaps this helps:
Table's definition in SQL Server:
CREATE TABLE [dbo].[myTable] (
[myField] varchar(2) COLLATE Modern_Spanish_CI_AS NOT NULL,
Table's definition in MySQL
CREATE TABLE `myTable` (
`myField` varchar(2) COLLATE utf8_unicode_ci NOT NULL,
I have no experience with SQL server, but from a cursory glance at this collation chart, Modern_Spanish_CI_AI seems to be essentially a Windows-1252 character set, probably with some Spanish sorting / comparison rules on top.
You should be able to convert the data like this:
$utf8 = iconv("windows-1252", "utf-8", $input);
try it out. However, when using PHP for this, the character set of the connection to each database also plays a role - if it still doesn't work, you need to show some more code.
I have a MySQL database table populated with non English data. When I view the data in Navicat MySQL browser the data appears fine. However when I run a php script to select and display the data on a web page it displays question marks instead. The page encoding is set to utf8 and even the MySQL collation is set to utf8 - something gets wrong in selecting and displaying... please help.
MySQL connection settings could be at fault here. Run this MySQL command when you connect to the database from PHP, before you run any other SQL commands:
SET names 'utf8';
This should set the connection's encoding to UTF-8. As you're saying, the page and the database is already in UTF-8 (that should also mean the page sends Content-Type: text/html; charset=utf-8); the connection itself can accidentaly have a different encoding by default :(
In addition, in HTML, inside we must write:
<meta charset="utf-8">
When we create a MySQL database, we must use something like this:
CREATE DATABASE `base` DEFAULT CHARACTER SET=utf8;
When we create tables, use:
CREATE TABLE `base`.`table` (
`key` int(5) unsigned NOT NULL,
`name` varchar(100),
...
PRIMARY KEY (`key`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
In PHP code, after mysql_connect() and mysql_select_db(), use:
mysql_query('SET NAMES UTF8');