Storing and displaying unicode string (हिन्दी) using PHP and MySQL - php

I have to store hindi text in a MySQL database, fetch it using a PHP script and display it on a webpage. I did the following:
I created a database and set its encoding to UTF-8 and also the collation to utf8_bin.
I added a varchar field in the table and set it to accept UTF-8 text in the charset property.
Then I set about adding data to it. Here I had to copy data from an existing site.
The hindi text looks like this: सूर्योदय:05:30
I directly copied this text into my database and used the PHP code echo(utf8_encode($string)) to display the data. Upon doing so the browser showed me "??????".
When I inserted the UTF equivalent of the text by going to "view source" in the browser, however, सूर्योदय translates into सूर्योदय.
If I enter and store सूर्योदय in the database, it converts perfectly.
So what I want to know is how I can directly store सूर्योदय into my database and fetch it and display it in my webpage using PHP.
Also, can anyone help me understand if there's a script which when I type in सूर्योदय, gives me सूर्योदय?
Solution Found
I wrote the following sample script which worked for me. Hope it helps someone else too
<html>
<head>
<title>Hindi</title></head>
<body>
<?php
include("connection.php"); //simple connection setting
$result = mysql_query("SET NAMES utf8"); //the main trick
$cmd = "select * from hindi";
$result = mysql_query($cmd);
while ($myrow = mysql_fetch_row($result))
{
echo ($myrow[0]);
}
?>
</body>
</html>
The dump for my database storing hindi utf strings is
CREATE TABLE `hindi` (
`data` varchar(1000) character set utf8 collate utf8_bin default NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `hindi` VALUES ('सूर्योदय');
Now my question is, how did it work without specifying "META" or header info?
Thanks!


Did you set proper charset in the HTML Head section?
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
or you can set content type in your php script using -
header( 'Content-Type: text/html; charset=utf-8' );
There are already some discussions here on StackOverflow - please have a look
How to make MySQL handle UTF-8 properly
setting utf8 with mysql through php
PHP/MySQL with encoding problems
So what i want to know is how can i
directly store सूर्योदय into my
database and fetch it and display in
my webpage using PHP.
I am not sure what you mean by "directly storing in the database" .. did you mean entering data using PhpMyAdmin or any other similar tool? If yes, I have tried using PhpMyAdmin to input unicode data, so it has worked fine for me - You could try inputting data using phpmyadmin and retrieve it using a php script to confirm. If you need to submit data via a Php script just set the NAMES and CHARACTER SET when you create mysql connection, before execute insert queries, and when you select data. Have a look at the above posts to find the syntax. Hope it helps.
** UPDATE **
Just fixed some typos etc


<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
<?php
$con = mysql_connect("localhost","root","");
if (!$con)
{
die('Could not connect: ' . mysql_error());
}
mysql_query('SET character_set_results=utf8');
mysql_query('SET names=utf8');
mysql_query('SET character_set_client=utf8');
mysql_query('SET character_set_connection=utf8');
mysql_query('SET character_set_results=utf8');
mysql_query('SET collation_connection=utf8_general_ci');
mysql_select_db('onlinetest',$con);
$nith = "CREATE TABLE IF NOT EXISTS `TAMIL` (
`data` varchar(1000) character set utf8 collate utf8_bin default NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1";
if (!mysql_query($nith,$con))
{
die('Error: ' . mysql_error());
}
$nithi = "INSERT INTO `TAMIL` VALUES ('இந்தியா நாட்டின் பக்கங்கள்')";
if (!mysql_query($nithi,$con))
{
die('Error: ' . mysql_error());
}
$result = mysql_query("SET NAMES utf8");//the main trick
$cmd = "select * from TAMIL";
$result = mysql_query($cmd);
while($myrow = mysql_fetch_row($result))
{
echo ($myrow[0]);
}
?>
</body>
</html>


For those who are looking for PHP ( >5.3.5 ) PDO statement, we can set charset as per below:
$dbh = new PDO('mysql:host=localhost;dbname=testdb;charset=utf8', 'username', 'password');


CREATE DATABASE hindi_test
CHARACTER SET utf8
COLLATE utf8_unicode_ci;
USE hindi_test;
CREATE TABLE `hindi` (`data` varchar(200) COLLATE utf8_unicode_ci NOT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO `hindi` (`data`) VALUES('कंप्यूटर');


For Those who are facing difficulty just got to php admin and change collation to utf8_general_ci
Select Table go to Operations>> table options>> collations should be there

Related

Insert special characters in database from php

I have a joomla database, where the mySQL connection collation is utf8_general_ci.
I have some additional datatables in the database (not related to the joomla installation) that I want to populate with data from a PHP script.
LATER EDIT: check below
When I try inserting special characters (language and region specific characters) I get jibberish in the database, like îăîșț instead of îăîșț.
The collation for all the columns in the joomla database is : utf8mb4_unicode_ci (if this makes any difference)
The weird thing is, that if I show the content of the database in an email / mobile app (browser based) it shows the data correctly. But I can see, that something is not right with inserting the valus from PHP, since if I insert it manually from the phpmyadmin panel, the value will be correctly displayed in the table.
<?php header('Content-type: text/html; charset=UTF-8'); //header is specified here
$value_to_insert = addslashes($_REQUEST['value']); //get the value from the parameter
inserttask($value_to_insert); //insert the value into the datatable
function inserttask($value_to_insert)
{
$con = mysqli_connect("host",username,password,database); //set up my mysqli connection
//mysql_set_charset('utf8'); //this didn't help
if (!$con)
{
die();
}
$sql = "INSERT INTO `table` (`value`) VALUES ('".mysqli_real_escape_string($con, $value_to_insert)."')";
if($result=mysqli_query($con,$sql))
{
print "OK";
}
else
{
print "ERROR";
}
}
?>
Any ideas how this insert should be made, to make it compatible with any region dependend character?
Later edit:
I ran the "show variables like '%collation%'"
collation_connection - utf8_general_ci
collation_database - latin1_swedish_ci
collation_server - latin1_swedish_ci
Could this be the problem?

After setting up the connection as I had to change the connections charset in a specific way.
$con = mysqli_connect("mydb12.surf-town.net","bosteen_licadm","Ugymate92","bosteen_p2p");
$con->set_charset("utf8");
After this, the insert works!

Arabic language not display proper in my home page

mysql_query("SET NAMES 'utf8'");
mysql_query('SET CHARACTER SET utf8');
$sql = "SELECT * FROM mytable";
$data = selectArray($sql);
insert arabic data is = `تظهر البيانات العربي
display in my database = تظهر البيانات العربيØ
display in cakePhp = تظهر البيانات العربية (display proper)
display in corePhp = تظهر البيانات العربيØ
Let me know any one have solution of this issue.

when create table use like this
CREATE DATABASE mydb DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;

Mojibake -- search this forum for that. You hit a common problem; this is a duplicate.

Chinese characters ending up blank in Database [duplicate]

This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 3 years ago.
Im making an API for a company that inserts orders via this api into our DB, but I'm running into a problem:
Chinese characters don't get stored into my tables, how to fix this?
I have:
- My test file which uses cURL to post set to utf8 via header*
- The API code which inserts the data set to utf8 via header*
- Stored my phpfile as an UTF-8 encoded file
- mysqli connection set to utf8 ($connection->set_charset("utf8"))
- mysql-database set to utf8_general_ci
- mysql-table set to utf8_general_ci
- mysql-columns set to utf8_general_ci
- I insert the values into a VARCHAR and into a TEXT column (for test purposes)
- Inserted the values directly into the query -> no result
- Inserted the values via iconv("ISO-8859-1","UTF-8", $text)-> weird result
- Inserted the values via iconv("UTF-8","UTF-8", $text)-> no result
- Tested characters like áéíúó, which work perfectly fine
- Set my postvalues for curl via foreach($post as $k=>$v){ $data.= $k.'='.$v;} to send as data
- Set my postvalues for curl via http_build_query($post) to send as data
- Used a query before inserted doing SET CHARACTER SET utf8
- Used a query before inserted doing SET NAMES 'utf8'
When I echo the query before inserting it, I see the chinese characters
When I check the database, it's blank values
(not unexpected:) When I Select it and output it, it's blank
*PHP charset header:
header('Content-Type: text/html; charset=utf-8');
I'm out of ideas, anyone?

I have same problem and use this query and my problem solved
("SET CHARACTER SET utf8")
$mysqli->query("SET CHARACTER SET utf8")

Verify that the field has utf8_unicode_ci
CREATE TABLE `table` (
`id` int(15) NOT NULL,
`fieldname` varchar(20) NOT NULL,
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
and the charset is utf8
mysqli_set_charset($con,"utf8");

OOP
public function open_connection() {
$this->connection = mysqli_connect(DB_SERVER, DB_USER, DB_PASS, DB_NAME);
if(mysqli_connect_errno()) {
$msg = "Database connection failed: ";
$msg .= mysqli_connect_error();
$msg .= " (" . mysqli_connect_errno() . ")";
exit($msg);
}
// Change character set to utf8
mysqli_set_charset($this->connection,"utf8");
}

How can I get UTF8 to work correctly using mysql and php?

I have a database where the table is encoded as utf8. I have a value in it that's in Korean. The characters display fine in the database. But when they are echoed from the database I get a bunch of question marks.
Here is my code, after the connect and select_db statements:
mysql_query('SET NAMES utf8');
$query = 'SELECT * FROM english WHERE id = ' . $_GET['dealerID'];
$result = mysql_query($query) or die(mysql_error());
$row = mysql_fetch_array($result);
What am I doing wrong here? (Yes, 'english' is the right table). I tried Michael's suggestion below to encode the table as utf8_general_ci and I get a MySQL error. Suggestions? What's the correct name of the character set?
If I run a query in PHPMyAdmin, I get 서울시 서초구 서초1동 1425-10 세중프라자 4층/

Review this guide which talks about UTF8, mysql, and PHP all together.
Summary:
make sure the browser knows the page is in utf8
<?php header("Content-type: text/html; charset=utf-8"); ?>
The table in mysql needs to be set for utf8 as well as the string fields within the db
utf8_general_ci
tell php that when it talks to mysql to talk in utf8
mysqli_set_charset('utf8');
Here are some more details as well.

Can't insert cyrillic text into mysql database

When I'm trying to insert cyrillic text into MySQL database it inserts it like:
г???????????? ?? ????????
Рісѓрїр°ріс‹рї р° с‹рір°рї
So, I have two pages: registration.php and addUser.php. In each of them
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Database consist of 11 tables, each table has collation: utf8_general_ci, type: MyISAM. Each field in every table has Collation: utf8_general_ci.
When I'm writing to database directly in phpMyAdmin and then show this data to web-page. In English and Russian - all OK.
But when I'm full my form with personal data on registration.php and then going to addUser.php - all cyrillic characters displayed like I wrote upper - on page and in database too.
function AddNewUser($Name, $Surname, $FatherName, $Email, $Password, $Phone, $DegreeID, $RankID,
$Organization, $Department, $Country, $City, $Address, $Job)
{
//fetch data from database for dropdown lists
//connect to db or die)
$db = mysql_connect($GLOBALS["gl_kussdbName"], $GLOBALS["gl_kussUserName"], $GLOBALS["gl_kussPassword"] ) or die ("Unable to connect");
//to prevenr ????? symbols in unicode - utf-8 coding
mysql_query("SET NAMES 'UTF8'");
//select database
mysql_select_db($GLOBALS["gl_kussDatabase"], $db);
$sql = "INSERT INTO UserDetails (
UserFirstName,
UserLastName,
UserFatherName,
UserEmail,
UserPassword,
UserPhone,
UserAcadDegreeID,
UserAcadRankID,
UserOrganization,
UserDepartment,
UserCountry,
UserCity,
UserAddress,
UserPosition)
VALUES(
'".$Name."',
'".$Surname."',
'".$FatherName."',
'".$Email."',
'".$Password."',
'".$Phone."',
'".$DegreeID."',
'".$RankID."',
'".$Organization."',
'".$Department."',
'".$Country."',
'".$City."',
'".$Address."',
'".$Job."'
);";
//execute SQL-query
$result = mysql_query($sql, $db);
if (!$result)
{
die('Invalid query: ' . mysql_error());
}
//close database = very inportant
mysql_close($db);
}
?>
There also such information in phpMyAdmin:
auto increment increment 1
auto increment offset 1
autocommit ON
automatic sp privileges ON
back log 50
basedir \usr\local\mysql-5.1\
big tables OFF
binlog cache size 32,768
binlog format STATEMENT
bulk insert buffer size 8,388,608
character set client utf8
(Global value) cp1251
character set connection utf8
(Global value) cp1251
character set database cp1251
character set filesystem binary
character set results utf8
(Global value) cp1251
character set server cp1251
character set system utf8
character sets dir \usr\local\mysql-5.1\share\charsets\
collation connection utf8_general_ci
(Global value) cp1251_general_ci
collation database cp1251_general_ci
collation server cp1251_general_ci
completion type 0
concurrent insert 1
So I need to properly show, save and select russian text from database. Thanx!
connect timeout 10
datadir \usr\local\mysql-5.1\data\

Try calling mysql_set_charset('utf8'); after connecting to the database. I think it's similar to executing a SET NAMES query, but since the PHP manual says using that function over a SET NAMES query is recommended, I'd try it.
Also, when you display your content, you could try echo htmlentities($string, ENT_COMPAT, 'UTF-8');

I store greek in tables created like ths:
CREATE TABLE `test` (
`test_id` SMALLINT UNSIGNED NOT NULL AUTO_INCREMENT,
`test_name` VARCHAR(30) NOT NULL,
PRIMARY KEY (`test_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT;
Or if table already created I guess you can change the charset it in the phpmyadmin interface. Maybe this helps.

Check your MySQL configuration and ensure that your encoding is defined correctly.
Add these lines to my.cnf or my.ini, which ever your installation uses.
These settings did the trick for me:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
character-set-server=utf8

I have tried multiple collations in phpMyAdmin as well as changing the charset of the page, which didn;t make sense but i was willing to try anything after two days of research. this command helped me: mysql_set_charset('utf8');
Collation on the column was set to koi8r_general_ci

Edit your structure field to set collation utf16_general_ci.
After that insert your data.

Categories