How to insert an hebrew value into a mysql db in php - php

I'm trying to insert an hebrew value into my mysql db, instead of hebrew the values looks like that.
שדגשדכעשד
The collation of the table is latin1_swedish_ci by default, I tried also to change to utf-8_general_ci, hebrew_bin, hebrew_general_ci but the result is still the same.
In my code I'm using of course the meta tag to configure the charset:
<meta charset="UTF-8">
And before my php query I added this line:
mysql_query("SET NAMES utf8");
I'm viewing the result in the phpmyadmin.

I have solved my Hebrew language problem. It was a database and table row/field encoding issue. Here is the solution I used. I took help from another answer and the link is given below, in case anyone needs it.
The database collation has to be utf8_general_ci.
The collation of the table with Hebrew has to be utf8_general_ci.
In the PHP connection script put
header('Content-Type: text/html; charset=utf-8');
In the xhtml head tag put
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
If you are using MySQLi put this code in the connection script after selecting the database:
mysql_query("SET NAMES 'utf8'");
If you are using PDO, put
$conn->query("SET NAMES 'utf8'");
The first answer helped me and I took it from there

Set charset to achieve the solution
While creating the database
CREATE DATABASE db_name
CHARACTER SET utf8
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci
DEFAULT COLLATE utf8_general_ci
;
Or if the database is already created
CREATE TABLE table_name(
...
)
DEFAULT CHARACTER SET utf8
COLLATE utf8_general_ci;
OR while writing query
mysql_query("SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'", $conn);
$re = mysql_query('SHOW VARIABLES LIKE "%character_set%";')or die(mysql_error());
while ($r = mysql_fetch_assoc($re)) {var_dump ($r); echo "<br />";}

Check the collation_connection:
show variables like '%collation%'

you should make sure that:
you set utf-8 in php
you use utf-8 in the connection
your table is defined as utf-8_general_ci
the specific field is defined as utf-8_general_ci
Then you should be able to view Hebrew, or any other language, correctly in phpadmin

what finally helped me is to add the charset to the connection:
{"mysql:host=$host;dbname=$db;charset=utf8"}

I would say, in order to make sure that the values are passed well to the database I would add a die statment before inserting to the database and print the value, example:
die($_POST['thevalue']);
//insert to database.
//...
If it goes well, then the problem is on the database side, on the database I would try with this collation
| hebrew | ISO 8859-8 Hebrew | hebrew_general_ci | 1 |
as per http://dev.mysql.com/doc/refman/5.0/en/charset-mysql.html suggest.
But if it fail on the php side, reason can be, the server does not support Hebrew, make sure that on the html output document you use the correct metatag with
...
<meta charset="ISO 8859-8">
...
Let us knows how it proceed, good luck :)

For future use, if you have this issue and you are using PDO and not mysqli, you will need to do it like this:
The Database (all of it) collation has to be utf8_general_ci.
Set the collation of the table with Hebrew also to utf8_general_ci.
In the HTML "head" tag, add this: < meta charset="utf-8" >
In your PHP connection file, add after the connection Query this line: conn->exec("set names utf8");
if you are using classes, then you will probably have the "conn" as a variable.
in that case, you will have to use it like this:
$this->conn->exec("set names utf8");
Hope this helps to future people that have this problem and using PDO.
Best of luck.

I think I figure it out:
here's my code:
$conn = mysqli_connect($dbserver,$dbuser,$dbpwd,$dbname);
if (mysqli_connect_errno()){
printf("Connection failed: %s\n" , mysqli_connect_errno());
exit();
}
printf("Initial character set: %s\n", mysqli_character_set_name($conn));
/* change character set to utf8 */
if (!mysqli_set_charset($conn, "hebrew")) {
printf("Error loading character set hebrew: %s\n", mysqli_error($conn));
exit();
} else {
printf("Current character set: %s\n", mysqli_character_set_name($conn));
}

Related

How to solve Mysql Database collation? [duplicate]

I have my database properly set to UTF-8 and am dealing with a database containing Japanese characters. If I do SELECT *... from the mysql command line, I properly see the Japanese characters. When pulling data out of the database and displaying it on a webpage, I see it properly.
However, when viewing the table data in phpMyAdmin, I just see garbage text. ie.
ç§ã¯æ—¥æœ¬æ–™ç†ãŒå¥½ãã§ã™ã€‚日本料ç†ã‚...
How can I get phpMyAdmin to display the characters in Japanese?
The character encoding on the HTML page is set to UTF-8.
Edit:
I have tried an export of my database and opened up the .sql file in geany. The characters are still garbled even though the encoding is set to UTF-8. (However, doing a mysqldump of the database also shows garbled characters).
The character set is set correctly for the database and all tables ('latin' is not found anywhere in the file)
CREATE DATABASE `japanese` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
I have added the lines to my.cnf and restarted mysql but there is no change. I am using Zend Framework to insert data into the database.
I am going to open a bounty for this question as I really want to figure this out.
Unfortunately, phpMyAdmin is one of the first php application that talk to MySQL about charset correctly. Your problem is most likely due to the fact that the database does not store the correct UTF-8 strings at first place.
In order to correctly display the characters correctly in phpMyAdmin, the data must be correctly stored in the database. However, convert the database into correct charset often breaks web apps that does not aware charset-related feature provided by MySQL.
May I ask: is MySQL > version 4.1? What web app is the database for? phpBB? Was the database migrated from an older version of the web app, or an older version of MySQL?
My suggestion is not to brother if the web app you are using is too old and not supported. Only convert database to real UTF-8 if you are sure the web app can read them correctly.
Edit:
Your MySQL is > 4.1, that means it's charset-aware. What's the charset collation settings for you database? I am pretty sure you are using latin1, which is MySQL name for ASCII, to store the UTF-8 text in 'bytes', into the database.
For charset-insensitive clients (i.e. mysql-cli and php-mod-mysql), characters get displayed correctly since they are being transfer to/from database as bytes. In phpMyAdmin, bytes get read and displayed as ASCII characters, that's the garbage text you seem.
Countless hours had been spend years ago (2005?) when MySQL 4.0 went obsolete, in many parts of Asia. There is a standard way to deal with your problem and gobbled data:
Back up your database as .sql
Open it up in UTF-8 capable text editor, make sure they look correct.
Look for charset collation latin1_general_ci, replace latin1 to utf8.
Save as a new sql file, do not overwrite your backup
Import the new file, they will now look correctly in phpMyAdmin, and Japanese on your web app will become question marks. That's normal.
For your php web app that rely on php-mod-mysql, insert mysql_query("SET NAMES UTF8"); after mysql_connect(), now the question marks will be gone.
Add the following configuration my.ini for mysql-cli:
# CLIENT SECTION
[mysql]
default-character-set=utf8
# SERVER SECTION
[mysqld]
default-character-set=utf8
For more information about charset on MySQL, please refer to manual:
http://dev.mysql.com/doc/refman/5.0/en/charset-server.html
Note that I assume your web app is using php-mod-mysql to connect to the database (hence the mysql_connect() function), since php-mod-mysql is the only extension I can think of that still trigger the problem TO THIS DAY.
phpMyAdmin use php-mod-mysqli to connect to MySQL. I never learned how to use it because switch to frameworks* to develop my php projects. I strongly encourage you do that too.
Many frameworks, e.g. CodeIgniter, Zend, use mysqli or pdo to connect to databases. mod-mysql functions are considered obsolete cause performance and scalability issue. Also, you do not want to tie your project to a specific type of database.
If you're using PDO don't forget to initiate it with UTF8:
$con = new PDO('mysql:host=' . $server . ';dbname=' . $db . ';charset=UTF8', $user, $pass, array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
(just spent 5 hours to figure this out, hope it will save someone precious time...)
I did a little more googling and came across this page
The command doesn't seem to make sense but I tried it anyway:
In the file /usr/share/phpmyadmin/libraries/dbi/mysqli.dbi.lib.php at the end of function PMA_DBI_connect() just before the return statement I added:
mysqli_query($link, "SET SESSION CHARACTER_SET_RESULTS =latin1;");
mysqli_query($link, "SET SESSION CHARACTER_SET_CLIENT =latin1;");
And it works! I now see Japanese characters in phpMyAdmin. WTF? Why does this work?
I had the same problem,
Set all text/varchar collations in phpMyAdmin to utf-8 and in php files add this:
mysql_set_charset("utf8", $your_connection_name);
This solved it for me.
the solution for this can be as easy as :
find the phpmysqladmin connection function/method
add this after database is conncted $db_conect->set_charset('utf8');
phpmyadmin doesn't follow the MySQL connection because it defines its proper collation in phpmyadmin config file.
So if we don't want or if we can't access server parameters, we should just force it to send results in a different format (encoding) compatible with client i.e. phpmyadmin
for example if both the MySQL connection collation and the MySQL charset are utf8 but phpmyadmin is ISO, we should just add this one before any select query sent to the MYSQL via phpmyadmin :
SET SESSION CHARACTER_SET_RESULTS =latin1;
Here is my way how do I restore the data without looseness from latin1 to utf8:
/**
* Fixes the data in the database that was inserted into latin1 table using utf8 encoding.
*
* DO NOT execute "SET NAMES UTF8" after mysql_connect.
* Your encoding should be the same as when you firstly inserted the data.
* In my case I inserted all my utf8 data into LATIN1 tables.
* The data in tables was like ДЕТСКИÐ.
* But my page presented the data correctly, without "SET NAMES UTF8" query.
* But phpmyadmin did not present it correctly.
* So this is hack how to convert your data to the correct UTF8 format.
* Execute this code just ONCE!
* Don't forget to make backup first!
*/
public function fixIncorrectUtf8DataInsertedByLatinEncoding() {
// mysql_query("SET NAMES LATIN1") or die(mysql_error()); #uncomment this if you already set UTF8 names somewhere
// get all tables in the database
$tables = array();
$query = mysql_query("SHOW TABLES");
while ($t = mysql_fetch_row($query)) {
$tables[] = $t[0];
}
// you need to set explicit tables if not all tables in your database are latin1 charset
// $tables = array('mytable1', 'mytable2', 'mytable3'); # uncomment this if you want to set explicit tables
// duplicate tables, and copy all data from the original tables to the new tables with correct encoding
// the hack is that data retrieved in correct format using latin1 names and inserted again utf8
foreach ($tables as $table) {
$temptable = $table . '_temp';
mysql_query("CREATE TABLE $temptable LIKE $table") or die(mysql_error());
mysql_query("ALTER TABLE $temptable CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci") or die(mysql_error());
$query = mysql_query("SELECT * FROM `$table`") or die(mysql_error());
mysql_query("SET NAMES UTF8") or die(mysql_error());
while ($row = mysql_fetch_row($query)) {
$values = implode("', '", $row);
mysql_query("INSERT INTO `$temptable` VALUES('$values')") or die(mysql_error());
}
mysql_query("SET NAMES LATIN1") or die(mysql_error());
}
// drop old tables and rename temporary tables
// this actually should work, but it not, then
// comment out this lines if this would not work for you and try to rename tables manually with phpmyadmin
foreach ($tables as $table) {
$temptable = $table . '_temp';
mysql_query("DROP TABLE `$table`") or die(mysql_error());
mysql_query("ALTER TABLE `$temptable` RENAME `$table`") or die(mysql_error());
}
// now you data should be correct
// change the database character set
mysql_query("ALTER DATABASE DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci") or die(mysql_error());
// now you can use "SET NAMES UTF8" in your project and mysql will use corrected data
}
Change latin1_swedish_ci to utf8_general_ci in phpmyadmin->table_name->field_name
This is where you find it on the screen:
First, from the client do
mysql> SHOW VARIABLES LIKE 'character_set%';
This will give you something like
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | latin1 |
| character_set_connection | latin1 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | latin1 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
where you can inspect the general settings for the client, connection, database
Then you should also inspect the columns from which you are retrieving data with
SHOW CREATE TABLE TableName
and inspecting the charset and collation of CHAR fields (though usually people do not set them explicitly, but it is possible to give CHAR[(length)] [CHARACTER SET charset_name] [COLLATE collation_name] in CREATE TABLE foo ADD COLUMN foo CHAR ...)
I believe that I have listed all relevant settings on the side of mysql.
If still getting lost read fine docs and perhaps this question which might shed some light (especially how I though I got it right by looking only at mysql client in the first go).
1- Open file:
C:\wamp\bin\mysql\mysql5.5.24\my.ini
2- Look for [mysqld] entry and append:
character-set-server = utf8
skip-character-set-client-handshake
The whole view should look like:
[mysqld]
port=3306
character-set-server = utf8
skip-character-set-client-handshake
3- Restart MySQL service!
Its realy simple to add multilanguage in myphpadmin if you got garbdata showing in myphpadmin, just go to myphpadmin click your database go to operations tab in operation tab page see collation section set it to utf8_general_ci, after that all your garbdata will show correctly. a simple and easy trick
The function and file names don't match those in newer versions of phpMyAdmin. Here is how to fix in the newer PHPMyAdmins:
Find file:
phpmyadmin/libraries/DatabaseInterface.php
In function: public function query
Right after the opening { add this:
if($link != null){
mysqli_query($link, "SET SESSION CHARACTER_SET_RESULTS =latin1;");
mysqli_query($link, "SET SESSION CHARACTER_SET_CLIENT =latin1;");
}
That's it. Works like a charm.
I had exactly the same problem. Database charset is utf-8 and collation is utf8_unicode_ci. I was able to see Unicode text in my webapp but the phpMyAdmin and sqldump results were garbled.
It turned out that the problem was in the way my web application was connecting to MySQL. I was missing the encoding flag.
After I fixed it, I was able to see Greek characters correctly in both phpMyAdmin and sqldump but lost all my previous entries.
just uncomment this lines in libraries/database_interface.lib.php
if (! empty($GLOBALS['collation_connection'])) {
// PMA_DBI_query("SET CHARACTER SET 'utf8';", $link, PMA_DBI_QUERY_STORE);
//PMA_DBI_query("SET collation_connection = '" .
//PMA_sqlAddslashes($GLOBALS['collation_connection']) . "';", $link, PMA_DBI_QUERY_STORE);
} else {
//PMA_DBI_query("SET NAMES 'utf8' COLLATE 'utf8_general_ci';", $link, PMA_DBI_QUERY_STORE);
}
if you store data in utf8 without storing charset you do not need phpmyadmin to re-convert again the connection. This will work.
Easier solution for wamp is:
go to phpMyAdmin,
click localhost,
select latin1_bin for Server connection collation,
then start to create database and table
Add:
mysql_query("SET NAMES UTF8");
below:
mysql_select_db(/*your_database_name*/);
It works for me,
mysqli_query($con, "SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'");
ALTER TABLE table_name CONVERT to CHARACTER SET utf8;
*IMPORTANT: Back-up first, execute after

Get Chinese characters from MySQL

I am trying to display chinese characters that I have saved in MySQL database, which is saved under utf8_unicode_ci type. I have seen several solutions on the web, but nothing works.
Below is my connection file:
$conn = mysql_connect("localhost","root","password");
mysql_set_charset('utf8',$conn);
mysql_query("SET CHARACTER SET utf8 ");
mysql_select_db("database");
Below is my query:
mysql_query("SET character_set_results=utf8", $conn);
$sql = mysql_query("select * from webdata",$conn);
But it still shows ????. Any ideas?
How to resolve...
When I had a similar issue I firstly displayed the encoding of my text in php using
echo mb_detect_encoding ( $text );
It shows the encoding of the text coming from my query. This showed me that I was getting ASCII from mysql_query when Chinese or Russian characters were in my database.
The change I made was with the following addition after the mysql_connect
mysql_set_charset('UTF8');
My database is utf8_unicode_ci collation and I can see chinese characters.
So, if mb_detect_encoding is now giving you UTF8 for your text then you would be able to show chinese characters.
The next step is to make sure what you pass to the browser has the correct header...
header('Content-Type: text/html; charset=UTF-8');
Put the above at the top of your code in php to make sure the browser is expecting your encoding.
Now that should the question, however ideally you should be using PDO or mysqli rather than mysql_connect method. In this case the equivalent procedural style commands are..
$link = mysqli_connect('localhost', 'my_user', 'my_password', 'test');
mysqli_set_charset($link, "utf8");
Where $link is the equivalent to your connection to the database.
where it show "???", when you print the output to HTML ?
if so, try to add to <head> element the line
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
hope it helped a bit.
EDIT
it seems that you need to declare UTF8 on:
character_set_results = 'utf8',
character_set_client = 'utf8',
character_set_connection = 'utf8',
character_set_database = 'utf8',
character_set_server = 'utf8'"
checkout
PHP UTF8 not displaying chinese characters properly
That should be all you need. Both for Traditional and Simplified Chinese characters
1. Make sure your table is set to COLLATION utf8_general_ci
2. $con = new mysqli("localhost",$username,$password,$database) or die("Error " . mysqli_error($con)); $con->query("SET NAMES 'utf8'");
3. <meta http-equiv="Content-type" content="text/html; charset=utf-8" />

How to handle character sets properly in conjunction with a database, text areas etc etc

For once and for all I want to have clear how to handle ü etc in my application.
I understand using UTF-8 is the best way to store/represent these strange characters.
So I declare in my HTML a meta-charset:
<meta charset="utf-8">
Storing the content in my database, it also needs the right set so I set the collation to utf8_general_ci
In my understanding there isn't an other place where I can define the character set. But what is the problem?
If I go into phpMyAdmin and add jülich in the content field, and I subtract the content from the database through a mysql query and PHP. The ü gets displayed properly.
But if I view it in a Textarea it displays a black diamond shaped figure with a white ? in it.
If I put in the text in a textarea in my application and submit it, it displays correct in the text area. But on the website it displays ü
These are the charset settings in the startup screen of phpMyAdmin:
As an answer on questions off Daan:
character%:
character_set_client = utf8
character_set_connection = utf8
character_set_database = utf8
character_set_filesystem = binary
character_set_results = utf8
character_set_server = latin1
character_set_system = utf8
character_sets_dir = /usr/share/mysql/charsets/
collation%
collation_connection = utf8_general_ci
collation_database = utf8_general_ci
collation_server = latin1_swedish_ci
How can I resolve this?
Things I've done so far:
added: header('Content-type: text/html; charset=utf-8');
added: $this->db->setAttribute(PDO::MYSQL_ATTR_INIT_COMMAND, "SET NAMES 'utf8'");
added: <meta http-equiv="content-type" content="text/html; charset=utf-8" />
added: $this->db = new PDO($dsn, $username, $password,
array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
Solution
All the above is working like a charm. But if you use a function to replace certain special routines and return the replaced string through utf8_decode() it obviously doesn't display the ü ;(
You should really read the MySQL specific PDO documentation as it clearly states that the PDO::MYSQL_ATTR_INIT_COMMAND constant can only be used in the driver_options array when constructing a new database handle.
So instead of using setAttribute you should specify the MYSQL_ATTR_INIT_COMMAND when you create the PDO handle, like this :
$this->db = new PDO($dsn, $username, $password,
array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
My guess is it's probably a browser side thing. A couple more things to check into:
What is your web server sending for the Content-Type header? does it use charset=utf-8?
Do you have a different character set specified in the DOCTYPE of your HTML page?
Make sure phpMyAdmin itself is properly configured to use UTF8. It sounds like it is not.

MySQL db question marks instead of hebrew characters..?

I'm trying to build a shopping cart using PHP & MySQL.
my db in MySQL is utf8 and my table in the db is utf8,
How can I use Hebrew characters?
I was able to solve this by doing the following:
the db collation has to be utf8_general_ci
the collation of the table with hebrew has to be utf8_general_ci
in your php connection script put header('Content-Type: text/html; charset=utf-8');
in xhtml head tag put <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
after selecting the db in the connection script put mysql_query("SET NAMES 'utf8'");
After a lot of work I found a solution that always works..:
without SET_NAMES.
In the conn.inc.php file, after you selected a database and connected to it, do this:
if(!mysqli_set_charset($conn, 'utf8')) {
echo 'the connection is not in utf8';
exit();
}
...and in the html always use charset utf-8;
That solved it for me. No need to use set_names(), which is ok but it annoyed the hell out of me.
You can use the PDO in your code like this:
$db = new PDO($config['DSN'], $config['dbUserName'], $config['dbPassword']);
$db->exec("SET NAMES 'utf8'");
Make sure you include the single quotation marks around the 'utf8'.
If it's an encoding problem (and it sounds like it is), this query will help:
SET NAMES utf8
Execute this query (e.g. mysql_query("SET NAMES utf8")) right after you connect and before you move any data.
More info
$conn->set_charset("utf8"); Use this for your dbconnect
Where are the question marks showing up? It may be an encoding problem somewhere other than in the data base.
To store non-ASCII characters in a database column, you need to define that column with a specific character set. You can specify a character set at 3 levels: database, table, and column. For example:
CREATE DATABASE db_name CHARACTER SET utf8
CREATE TABLE tbl_name (...) CHARACTER SET utf8
CREATE TABLE tbl_name (col_name CHAR(80) CHARACTER SET utf8, ...)
http://www.herongyang.com/PHP/Non-ASCII-MySQL-Store-Non-ASCII-Character-in-Database.html
In my case I created the DB from a dump, and this command solve it:
ALTER DATABASE <database_name> CHARACTER SET utf8 COLLATE utf8_unicode_ci;
mysql_query('SET NAMES utf8') ..... Put this in Php

PHP MySQL database strange characters

I'm trying to output product information stored in a MySQL database, but it's writing out some strange characters, like a diamond with a question mark inside of it.
I think it may be an encoding/UTF8 issue, but I've specified the encoding I want:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
Is this right? What should I check for?
If only the data that's coming from database has strange characters in it, be sure that the MySQL connection is also in UTF8 by using:
mysql_query("SET NAMES UTF8");
before any other queries. Otherwise, if the characters appear also in 'handwritten' files, make sure that the files are saved as UTF-8 in your editor. You can also try setting the charset header through PHP:
header('Content-type: text/html; charset=UTF-8');
Also make sure that all fields in the tables you are querying are set as some UTF-8 variant, for example utf8_general_ci.
I assume you want the result to be in utf8
save you php script utf8 encoded
make sure your http header (or some meta tags) tells that output is utf8
all tables in MySql should to be utf8
last but not least, the connection between client and server should be utf8. (This could be handled somewhere in the php.ini setting or by making the following query against the db: SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'
If you follow all 4 point you should never ever have any problem with broken encodings.
The last time I had that trouble, the solution was similar to what Tatu Ulmanen said, but slightly different...
So if his solution does not work, try replacing
mysql_query("SET NAMES UTF8");
with
mysql_query("SET NAMES latin1");
I say this because the default characterset in MySql is latin1, and that is what is used most of the time....
hope that helps...
Seconding what Tatu says.
This is good background reading on encoding: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

Categories