I am using PHP for executing a MySQL update sentence with Spanish, English and Japanese characters.
But i'm not able to save Japanese characters into the database. How should I proceed?
Database has utf8_general_ci collation.
$strSQL = "UPDATE table SET value = '" . addslashes($strValue) . "'";
$strSQL = utf8_decode($strSQL);
mysqli_query($cnn, $strSQL);
With addslashes I can get apostrophes to be saved into the database.
With utf8_decode I can get Spanish characters to be saved into the database.
Why you proccess utf8_decode() the sql query?
Look at http://php.net/manual/en/function.utf8-decode.php for details about utf8_decode()
You have to check a few things.
Whether there is a problem in your database and tables Or in your PHP script.
Database :
CREATE DATABASE test DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Table Charset :
CREATE TABLE test.table
(
id INT NOT NULL AUTO_INCREMENT,
text VARCHAR(100) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE = MyISAM;
Connection :
SET NAMES utf8;
E.g :
$mysqli = new mysqli("localhost","my_user","my_password","test_db");
$mysqli->set_charset("utf8");
OR
$link = mysqli_connect("localhost","my_user","my_password","test_db");
mysqli_set_charset($link,"utf8");
Configure the encoding charset of server :
With PHPMyAdmin , Choose UTF-8 when you login.
PHP Script :
header('Content-Type: text/html;charset=UTF-8');
Apache Config (/etc/httpd/conf/httpd.conf) :
AddDefaultCharset UTF-8
Apache .htaccess file :
AddCharset UTF-8 .htm
AddCharset UTF-8 .html
AddCharset UTF-8 .php
PHP Config (/etc/php.ini) :
default_charset = "utf-8"
MySQL Config (/etc/my.cnf ) :
[client]
default-character-set=utf8
[mysqld]
default-collation=utf8_unicode_ci
character-set-server=utf8
default-character-set=utf8
init-connect='SET NAMES utf8'
character-set-client = utf8
User values :
e.g : $_GET , $_POST
You can use mb_convert_encoding() for convert your strings to UTF-8.
Useful Links :
PHP inserting Japanese string to utf8 table as something else, but still reads it successfully
http://es2.php.net/manual/en/mysqli.set-charset.php
Select MySQL rows with Japanese characters
http://php.net/manual/en/function.mysql-set-charset.php
https://www.w3schools.com/php/func_mysqli_set_charset.asp
PHP mysql charset utf8 problems
https://medium.com/#adamhooper/in-mysql-never-use-utf8-use-utf8mb4-11761243e434
https://dev.mysql.com/doc/refman/8.0/en/faqs-cjk.html
https://www.experts-exchange.com/questions/24742133/How-to-insert-a-japanese-character-into-mysql-database.html
Related
Well i got a php script that takes nicknames from a the Steam web-api and insert them into a mysql db. Many of them got rare russian and greek characters. I set php to utf-8 in the php.ini and in all the php files with
mb_internal_encoding('utf-8');
My PDO connector is configured to handle utf8
$connection = new PDO('mysql:host=localhost;dbname=d2bd;mysql:charset=utf8mb4', 'root', '');
$connection->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$connection->setAttribute(PDO::ATTR_PERSISTENT, true);
$connection->setAttribute(PDO::MYSQL_ATTR_INIT_COMMAND, "SET NAMES 'utf8mb4' COLLATE 'utf8mb4_unicode_ci'");
my mysql db is properly configured with utf8mb4
character_set_client utf8mb4
character_set_connection utf8mb4
character_set_database utf8mb4
character_set_filesystem binary
character_set_results utf8mb4
character_set_server utf8mb4
character_set_system utf8
character_sets_dir C:\xampp\mysql\share\charsets\
collation_connection utf8mb4_unicode_ci
collation_database utf8mb4_unicode_ci
collation_server utf8mb4_unicode_ci
completion_type NO_CHAIN
concurrent_insert AUTO
connect_timeout 10
core_file OFF
In few words i take the input of the web-api and encode it with uft8_encode(). Then i insert it into the db. The problem is that some characters are not well encoded and when i recall them from the database they are all bugged.
Example 1:
1.Input -> Перуанский чертовски
2.Encode -> ÐеÑÑанÑкий ÑеÑÑовÑки
3.Insert into DB
4.Select from DB -> Ð?еÑ?Ñ?анÑкий Ñ?еÑ?Ñ?овÑкÐ
5.Decode
6.Output -> �?е�?�?анский �?е�?�?овск�
Example 2:
1.Input -> $ |/| 1 ↓_ € ♥ J
2.Encode -> $ |/| 1 â_ ⬠⥠J
3.Insert into DB
4.Select from DB -> 1 â??_ â?¬ â?¥ J
5.Decode
6.Output -> 1 �??_ �?� �?� J
Checklist for Problems with character/charset/collation
Including mysql, mysqli, PDO
Content
DISCLAIMER
My insert's in my DB doesn't work properly! What can i do?
Change Charset and Collation of a Database or Table
Set the encoding of your skript files
Set the charset of your page with php or meta tag
What's the difference between UTF8 and UTF8mb4?
Answer to this specific Question
Further Information/Additional Links
Side Notes
1. DISCLAIMER
This Answer should not only answer this question, also should the answer be a bit more extensive, so more people find faster a bundled and good answer!
!Important Notice!
If you change something in your Database always make sur you have a backup of your database! Check it 2 times, or 3!
I'm open for improvements and comments, such as error corrections.
In addition I apologize if the grammar is not perfect: D
If you get stuck on a question like this:
Php + Mysql (UTF-8, utf8mb4) some characters are still bug
How to convert an entire MySQL database characterset and collation to UTF-8?
“Incorrect string value” when trying to insert UTF-8 into MySQL
Change MySQL default character set to UTF-8 in my.cnf?
Using utf8mb4 with php and mysql
PDO + MySQL and broken UTF-8 encoding
Error in insertion data in php Mysql
PHP PDO: charset, set names?
SET NAMES utf8 in MySQL?
PHP mysql charset utf8 problems
UTF-8 all the way through
Manipulating utf8mb4 data from MySQL with PHP
ERROR 1115 (42000) : Unknown character set: 'utf8mb4' in mysql
...then my answer maybe helps you!
2. My insert's in my DB doesn't work properly! What can i do?
If your insert's doesn't work properly an your inserted data looks something like this in your database then this could have various reasons!
Examples:
??????????
𫗮𫗮𫗮𫗮
�??_ �?�
â_ ⬠⥠J
Here is a little checklist you can go trought and check if everything is how it should be!
(After the checklist there a few extra informations for mysql, mysqli and PDO)
Checklist:
Make sure default character sets is set on tables, client, server & text fields
If NOT See Point 3
Make sure your database connections character sets
IF NOT See Point mysql/PDO
Make sure if your displaying data that the charset of the document is set!
IF NOT See Point 5
Make sure your skript files are saved with the right charset!
IF NOT See Point 4
Make sure you set your character and your charset!
IF NOT See Point mysql/PDO
Make sure you forms accept utf8!
IF NOT See Point 5
Make sure you have set the connection encoding
IF NOT See Point mysql/pdo
Make sure you have set the servercharacter encoding right
IF NOT See Point mysql/pdo
...
You have to be sure your using utf8/ utf8mb4 everywhere!
mysql:
-mysql_query("SET NAMES 'utf8'"); Run SET NAMES before every query you use. Because if a mysql driver don't provied mechanismus to charset then you have to use SET NAMES!
-mysql_query("SET CHARACTER SET utf8 "); Set character to utf8
-mysql_set_charset('utf8'); Set your charset to utf8
-mysql API driver doesn't support utf8mb4 (ERROR 1115 (42000))
-character_set_server=utf8 to set server character
PDO:
-$dbh->exec("set names utf8"); If your using PDO you can use this line to SET NAMES
-$dbh = new PDO("mysql:host=$host;dbname=$db;charset=utf8"); This line set the charset but you have to have PHP 5.3.6 or higher
-$dbh->setAttribute(PDO::MYSQL_ATTR_INIT_COMMAND, "SET NAMES 'utf8mb4' COLLATE 'utf8mb4_unicode_ci' "); You can also set SET NAMES with this line
-mb_internal_encoding('UTF-8'); to set the encoding when you use PDO
3. Change Charset and Collation of a Database or Table
If you have to change the charset or collation of a database or table you can use these lines of code:
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
4. Set the encoding of your skript files
You may have to check that your skript(php) files are saved with the right charset!
For this i would recommend you Notpad++!
If you have opened your file in notpad go to the menupoint 'Encoding' and change the charset
5. Set the charset of your page with php or meta tag
For displaying data in utf8/utf8mb4 you have to be sure you site is set with the right charset!
You can set the charset in 3 ways like this:
//PHP
ini_set("default_charset", "UTF-8");
header('Content-type: text/html; charset=UTF-8');
//HTML
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Also to accept utf8 in your form use:
<form accept-charset="UTF-8">
6. What's the difference between UTF8 and UTF8mb4?
UTF8:
-utf8 does only support symbols with 3 bytes
-...(many more)
UTF8MB4:
-utf8mb3 does support symbols with 4 bytes
-...(many more)
7. Answer to this specific Question
I think this should work since your using PDO:
(After you created a PDO object! If your using a PHP version less then 5.3.6)
$dbh->exec("set names utf8");
Otherwise try one of these:
ini_set("default_charset", "UTF-8");
header('Content-type: text/html; charset=UTF-8');
UPDATE:
To change the collation or charset of a database or table use this:
ALTER DATABASE databasename CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
8. Further Information/Additional Links
default character set
character set
mysql_set_charset
error_reporting
pdo
mysql
mysqli
9. Side Notes
9.1 Error Reporting
If Error's not get displayed use this code snippet:
<?php
error_reporting(E_ALL);
ini_set("display_errors", 1);
?>
9.2 Unicode
So that you don't make any mistake you have to really understand utf8!
9.3 One word to mysql, mysqli and PDO
My Personal ranking is:
PDO
mysqli
mysql
I would recommend you to use PDO or mysqli, because the have many benefits against mysql!
I changed the collation of the tables from SQLyog, but it seems that it's broken. When i changed them directly from a sql query it worked.
This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 8 years ago.
I am using PHP 5.5 and when I attempt to insert a UTF-8 character in the MySQL database PDO cuts it off at the first non-ASCII character.
I have set my connection to be:
(DB_TYPE.':host='.DB_HOST.';dbname='.DB_NAME.';charset=utf8', DB_USER, DB_PASS, array(PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC, PDO::ATTR_ERRMODE => PDO::ERRMODE_WARNING))
I have tried the SET NAMES that everyone posts, but that doesn’t work either because the problem is NOT on the MySQL side of things.
When I do an insert through phpMyAdmin and directly from the MySQL console, it works!
When I select the accented string with PDO, it works!
The problem is only on INSERT and UPDATE using PDO specifically!
Here is the SQL of the table. It is all in UTF-8 but maybe someone knows of a conflict between a setting and PDO
CREATE TABLE IF NOT EXISTS `mytable` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_lang` int(11) NOT NULL DEFAULT '2',
`id_tgroup_cat` int(11) NOT NULL,
`fieldfor` int(11) NOT NULL,
`colors` varchar(100) NOT NULL,
`text` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=34 ;
I have already tried to make text a varchar field and that did not change anything.
The actual insert in PHP:
$query = $this->db->prepare("UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
$query->execute(array($text, $colors, $id));
Where $text = "référence" (only saves the letter R in the database but without accents it saves everything) and $colors is an empty string for test purposes and $id is 2.
This is the key clue to me:
Where $text = "référence" (only saves the letter R in the database but
without accents it saves everything) and $colors is an empty string
for test purposes and $id is 2.
Sounds like it is a UTF-8 encoding issue. While the database is UTF-8 the whole chain from the code to the database—including the connection—should be UTF-8 clean.
How exactly does $this->db->prepare relate to the PHP connection to MySQL? A bit unclear from the code you have shown. But based on what you are showing, perhaps adjusting your query like this would help:
$query = $this->db->prepare("SET collation_connection = utf8_bin;
SET NAMES utf8;
UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
Or maybe this:
$this->db->exec("SET collation_connection = utf8_bin; SET NAMES utf8;");
$query = $this->db->prepare("UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
Note my forced-in addition of SET collation_connection = utf8_bin; as well as SET NAMES utf8;
In general you need to make sure your entire chain from the connection, to the database, to the tables is all UTF8 clean. I have a detailed answer to a similar question here.
But in your case, check the actual MySQL server my.cnf file. The following would set the whole chain to UTF-8:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
EDIT: And since the original poster indicates the data is coming from an HTML5 form, I also think checking the BOM (byte order mark) for the actual HTML5 file itself would help as well. It should be set to UTF8. More details on what a BOM is are over here. Specifically the accepted answer from Martin Code which explains:
The UTF-8 BOM is a sequence of bytes (EF BB BF) that allows the reader
to identify the file as an UTF-8 file.
This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 9 years ago.
I set all about encoding to utf8. but browser show me '???????'. My settings are below.
MySQL "show variables like 'c%'"
character_set_client | utf8
character_set_connection | utf8
character_set_database | utf8
.... (all utf8)
MySQL my.cnf
[client]
default-character-set = utf8
[mysqld]
init_connect="SET collation_connection = utf8_general_ci"
init_connect="SET names utf8"
character-set-server = utf8
collation-server = utf8_general_ci
[mysql]
default-character-set = utf8
In all my PHP file included:
<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />
Apache2 httpd.conf
AddDefaultCharset Off (if utf-8, show me '???' ,too)
In MySQL console, (database is dumped)
SELECT name from campagin LIMIT 1
=> print in working order, not '????'
I don't know what is problem.
For reference, I test this case.. in test.php
<?php
mysql_connect('localhost','root',ROOT_PW);
mysql_select_db(TEST_DB_NAME);
mysql_query("INSERT INTO a SET msg='안녕하세요'"); // char is KOREAN, meaning 'Hi'
$res = mysql_query("SELECT msg FROM a LIMIT 1");
$row = mysql_fetch_assoc($res);
echo $row['msg'];
?>
=> this print '안녕하세요' in working order not '????'
but in MySQL console
"SELECT msg FROM a LIMIT 1"
=> print not '안녕하세요' but abnormally '안녕하세요'
ADDED :
in mysql console
"SHOW CREATE TABLE a"
=> Table | Create Table
a | CREATE TABLE `a` ( 'msg' varchar(255) DEFAULT NULL ) ENGINE=InnoDB DEFAULT CHARSET=utf8
Using the base mysql_ connector you will need to declare
mysql_query( "SET NAMES utf8" );
mysql_query( "SET CHARACTER SET utf8" );
These tell MySQL that your communications with it are going to be via UTF8, otherwise it may not work.
Further more you need to make certain your tables and their columns are defined UTF-8, MySQL handles encoding at a column level. if you SHOW CREATE TABLE campagin what do you get for encoding on the table and columns? If a column doesn't spit out an encoding, it will default to the table level encoding.
Having an issue with strange characters showing up when inserting into a database, have tried tirelessly to figure out the issue but I am out of ideas...
Basically if I insert this data like so (this is just testing):
$valy = "…industry's favorite </em><strong><em>party of the year</em></strong><em>, </em><a href='http://www.unitingagainstlungcancer.org/getinvolved/strolling-supper-with-blues-news'><span class='s1'><em>Joan's…";
$valy = mysql_real_escape_string($valy);
$query = "INSERT INTO test_table (data) VALUES ('".$valy."')";
mysql_query($query,$dbhandle);
this will end up in the database (notice the A characters):
"...industry's favorite party of the year, http://www.unitingagainstlungcancer.org/getinvolved/strolling-supper-with-blues-news'>Joan's..."
I have tried to line up all the character settings:
php default_charset = utf-8
mysql table & row charset = utf-8
Mysql instance variables:
character set client utf8
(Global value) latin1
character set connection utf8
(Global value) latin1
character set database latin1
character set filesystem binary
character set results utf8
(Global value) latin1
character set server latin1
character set system utf8
What could this issue be?
One thing you may be missing is when you setup the connection. There you should also set the encoding to utf8.
Example:
$link = mysql_connect('localhost', 'user', 'password');
mysql_set_charset('utf8',$link);
However, don't use the mysql extension, it's deprecated: http://php.net/manual/en/function.mysql-set-charset.php
Try to set the mysql connection to UTF-8 as well:
mysql_query("SET NAMES 'utf8'");
I have a table in mysql DB which contains special character like Ø,Æ,etc. I cannot find these fields when i run a search with php. but when i run the same sql in phpmyadmin, i get results.
this is the table structure:
CREATE TABLE IF NOT EXISTS `clientinfo` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`adresse` varchar(160) NOT NULL,
`gatenavn` varchar(20) NOT NULL,
`husnr` varchar(20) NOT NULL,
`bokstav` varchar(2) NOT NULL,
`postnr` varchar(20) NOT NULL,
`poststed` varchar(60) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=398 ;
This is a sample query:
SELECT * FROM clientinfo WHERE gatenavn = 'EKRAVEIEN' AND husnr = '1' AND postnr = '2010' AND poststed = 'STRØMMEN'
when i run this query in phpmyadmin, i get result; but don't get when i run with php. I am using mysqli. need some help.
Tell your connection instance, to deliver UTF-8, before making queries. In MySqli you can call the function set_charset(), afterwards the connection object will deliver UTF-8.
Calling this function makes you independent of the database configuration, if necessary the connection will convert the returned data. Of course it is fastest if no conversion is necessary, so adjusting the configuration is a good thing too.
// tells the mysqli connection to deliver UTF-8 encoded strings.
$db = new mysqli($dbHost, $dbUser, $dbPassword, $dbName);
$db->set_charset('utf8');
Try setting :
SET NAMES utf8;
before your query in the same mysql session
If you have access to your MySQL configuration file, set these settings to my.cfg:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
Try using htmlspecialchars php function.
string htmlspecialchars ( string $string , int $flags = ENT_COMPAT | ENT_HTML401
, string $encoding = 'UTF-8' , bool $double_encode = true)
You can change the encoding info of your string. Eg KOI8-R for Russian symbols
This could be a issue with your php.ini file, your Apache Webserver, or charset of your HTML Document or your MySQL
Couple of things to to:
Always ensure that your charset is set to utf8 in your html meta tags
Set
default_charset = "utf-8";
in your php.ini file
And add
AddDefaultCharset UTF-8
to your httpd.conf if you are outputting unicode chars