Chinese characters ending up blank in Database [duplicate] - php

This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 3 years ago.
Im making an API for a company that inserts orders via this api into our DB, but I'm running into a problem:
Chinese characters don't get stored into my tables, how to fix this?
I have:
- My test file which uses cURL to post set to utf8 via header*
- The API code which inserts the data set to utf8 via header*
- Stored my phpfile as an UTF-8 encoded file
- mysqli connection set to utf8 ($connection->set_charset("utf8"))
- mysql-database set to utf8_general_ci
- mysql-table set to utf8_general_ci
- mysql-columns set to utf8_general_ci
- I insert the values into a VARCHAR and into a TEXT column (for test purposes)
- Inserted the values directly into the query -> no result
- Inserted the values via iconv("ISO-8859-1","UTF-8", $text)-> weird result
- Inserted the values via iconv("UTF-8","UTF-8", $text)-> no result
- Tested characters like áéíúó, which work perfectly fine
- Set my postvalues for curl via foreach($post as $k=>$v){ $data.= $k.'='.$v;} to send as data
- Set my postvalues for curl via http_build_query($post) to send as data
- Used a query before inserted doing SET CHARACTER SET utf8
- Used a query before inserted doing SET NAMES 'utf8'
When I echo the query before inserting it, I see the chinese characters
When I check the database, it's blank values
(not unexpected:) When I Select it and output it, it's blank
*PHP charset header:
header('Content-Type: text/html; charset=utf-8');
I'm out of ideas, anyone?

I have same problem and use this query and my problem solved
("SET CHARACTER SET utf8")
$mysqli->query("SET CHARACTER SET utf8")

Verify that the field has utf8_unicode_ci
CREATE TABLE `table` (
`id` int(15) NOT NULL,
`fieldname` varchar(20) NOT NULL,
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
and the charset is utf8
mysqli_set_charset($con,"utf8");

OOP
public function open_connection() {
$this->connection = mysqli_connect(DB_SERVER, DB_USER, DB_PASS, DB_NAME);
if(mysqli_connect_errno()) {
$msg = "Database connection failed: ";
$msg .= mysqli_connect_error();
$msg .= " (" . mysqli_connect_errno() . ")";
exit($msg);
}
// Change character set to utf8
mysqli_set_charset($this->connection,"utf8");
}

Related

Insert special characters in database from php

I have a joomla database, where the mySQL connection collation is utf8_general_ci.
I have some additional datatables in the database (not related to the joomla installation) that I want to populate with data from a PHP script.
LATER EDIT: check below
When I try inserting special characters (language and region specific characters) I get jibberish in the database, like îăîșț instead of îăîșț.
The collation for all the columns in the joomla database is : utf8mb4_unicode_ci (if this makes any difference)
The weird thing is, that if I show the content of the database in an email / mobile app (browser based) it shows the data correctly. But I can see, that something is not right with inserting the valus from PHP, since if I insert it manually from the phpmyadmin panel, the value will be correctly displayed in the table.
<?php header('Content-type: text/html; charset=UTF-8'); //header is specified here
$value_to_insert = addslashes($_REQUEST['value']); //get the value from the parameter
inserttask($value_to_insert); //insert the value into the datatable
function inserttask($value_to_insert)
{
$con = mysqli_connect("host",username,password,database); //set up my mysqli connection
//mysql_set_charset('utf8'); //this didn't help
if (!$con)
{
die();
}
$sql = "INSERT INTO `table` (`value`) VALUES ('".mysqli_real_escape_string($con, $value_to_insert)."')";
if($result=mysqli_query($con,$sql))
{
print "OK";
}
else
{
print "ERROR";
}
}
?>
Any ideas how this insert should be made, to make it compatible with any region dependend character?
Later edit:
I ran the "show variables like '%collation%'"
collation_connection - utf8_general_ci
collation_database - latin1_swedish_ci
collation_server - latin1_swedish_ci
Could this be the problem?
After setting up the connection as I had to change the connections charset in a specific way.
$con = mysqli_connect("mydb12.surf-town.net","bosteen_licadm","Ugymate92","bosteen_p2p");
$con->set_charset("utf8");
After this, the insert works!

How to convert non-readable utf8 chars in MySQL to readable [duplicate]

This question already has answers here:
strange character encoding of stored data , old script is showing them fine new one doesn't
(2 answers)
How to convert an entire MySQL database characterset and collation to UTF-8?
(20 answers)
Closed 7 years ago.
I have Arabic text stored in mysql like this format
شقق جديده غبر مسكونه 220 متر..
.150متر..طبربور ..ااسكانات قموم
والنجار للاستÙسار
it is in this format because I didn't use the below before inserting data into database from HTML textboxes:
mysql_query('SET CHARACTER SET utf8');
My question now, how to convert the text into MySQL columns to readable text like this:
شقق جديده غبر مسكونه 220 متر ..150متر..طبربور ..ااسكانات قموم والنجار للاستفسار
I tried the below for one column cell to check, but it didn't work:
mysql_query('SET CHARACTER SET utf8');
$sql = 'SELECT content FROM messages WHERE id=500';
$qry = mysql_query($sql);
$result =mysql_fetch_object($qry);
$text= $result->content;
mysql_query('SET CHARACTER SET utf8');
$sql ='UPDATE messages SET content= "'.$text.'" where id=500';
mysql_query($sql);
Note: if I use the below
mysql_query('SET CHARACTER SET utf8');
before I insert data into database from HTML textarea, then it works fine for store and read, but I need to convert the already entered data into tables.
Thanks.
The solution in brief can be done in 3 simple steps:
A. Change the Database collation to utf8 via the command below:
ALTER DATABASE <db_name> CHARACTER SET utf8 COLLATE utf8_general_ci;
already the table cells I want to change have collation of utf8_general_ci
B. Implement this query to change the content:
update messages set content=CONVERT(BINARY CONVERT(content USING latin1) USING utf8);
C. add the below before you connect to the DB:
mysql_query('SET CHARACTER SET utf8');
And you are done!!! Thanks for all
Use the ALTER DATABASE and ALTER TABLE commands.
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
Source.

can't get UTF-8 names into a mysql database

I'm having a problem getting UTF-8 names written into a MySQL database... Here's what I have.
PHP page head has....
<meta charset="utf-8">
the MySQL column is: Char (80) with utf8_unicode_ci (these were originally latin1... I've changed them to UTF-8, truncated the database, then rerun the code)
The variable echoes to screen: Germán Mera
but writes it to database as Germán Mera
I tried putting utf8_encode(); around the variable, but then it writes to database as: Germán Mera and screen as Germán Mera (I know that command only works on iso-8859-1.. I think the JSON page is already UTF-8)
Here is an excerpt of the code I am using to get the name (for sake of simplicity, I'm only showing relevant code - I know what's shown below is not secure)
$str = file_get_contents('http://fantasy.mlssoccer.com/web/api/elements/647/');
$jsonarray = json_decode($str, true);
$name = $jsonarray['web_name'];
mysqli_query ($con, "INSERT INTO mlsprices (name) VALUES ('$name')");
Any idea how I can get this to write to the database properly? When I search, I only get quite complicated answers (eg, this) and there's surely an easier way.
Try using SET NAMES 'UTF8' after connecting to MySQL:
$con=mysqli_connect("host", "user", "pw", "db");
if (!$con)
{
die('Failed to connect to mySQL: ' .mysqli_connect_errno());
}
/* change character set to utf8 */
if (!$con->set_charset("utf8")) {
printf("Error loading character set utf8: %s\n", $con->error);
}
As the manual says:
SET NAMES indicates what character set the client will use to send SQL
statements to the server... It also specifies the character set that the server should
use for sending results back to the client.

Can't insert cyrillic text into mysql database

When I'm trying to insert cyrillic text into MySQL database it inserts it like:
г???????????? ?? ????????
Рісѓрїр°ріс‹рї р° с‹рір°рї
So, I have two pages: registration.php and addUser.php. In each of them
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Database consist of 11 tables, each table has collation: utf8_general_ci, type: MyISAM. Each field in every table has Collation: utf8_general_ci.
When I'm writing to database directly in phpMyAdmin and then show this data to web-page. In English and Russian - all OK.
But when I'm full my form with personal data on registration.php and then going to addUser.php - all cyrillic characters displayed like I wrote upper - on page and in database too.
function AddNewUser($Name, $Surname, $FatherName, $Email, $Password, $Phone, $DegreeID, $RankID,
$Organization, $Department, $Country, $City, $Address, $Job)
{
//fetch data from database for dropdown lists
//connect to db or die)
$db = mysql_connect($GLOBALS["gl_kussdbName"], $GLOBALS["gl_kussUserName"], $GLOBALS["gl_kussPassword"] ) or die ("Unable to connect");
//to prevenr ????? symbols in unicode - utf-8 coding
mysql_query("SET NAMES 'UTF8'");
//select database
mysql_select_db($GLOBALS["gl_kussDatabase"], $db);
$sql = "INSERT INTO UserDetails (
UserFirstName,
UserLastName,
UserFatherName,
UserEmail,
UserPassword,
UserPhone,
UserAcadDegreeID,
UserAcadRankID,
UserOrganization,
UserDepartment,
UserCountry,
UserCity,
UserAddress,
UserPosition)
VALUES(
'".$Name."',
'".$Surname."',
'".$FatherName."',
'".$Email."',
'".$Password."',
'".$Phone."',
'".$DegreeID."',
'".$RankID."',
'".$Organization."',
'".$Department."',
'".$Country."',
'".$City."',
'".$Address."',
'".$Job."'
);";
//execute SQL-query
$result = mysql_query($sql, $db);
if (!$result)
{
die('Invalid query: ' . mysql_error());
}
//close database = very inportant
mysql_close($db);
}
?>
There also such information in phpMyAdmin:
auto increment increment 1
auto increment offset 1
autocommit ON
automatic sp privileges ON
back log 50
basedir \usr\local\mysql-5.1\
big tables OFF
binlog cache size 32,768
binlog format STATEMENT
bulk insert buffer size 8,388,608
character set client utf8
(Global value) cp1251
character set connection utf8
(Global value) cp1251
character set database cp1251
character set filesystem binary
character set results utf8
(Global value) cp1251
character set server cp1251
character set system utf8
character sets dir \usr\local\mysql-5.1\share\charsets\
collation connection utf8_general_ci
(Global value) cp1251_general_ci
collation database cp1251_general_ci
collation server cp1251_general_ci
completion type 0
concurrent insert 1
So I need to properly show, save and select russian text from database. Thanx!
connect timeout 10
datadir \usr\local\mysql-5.1\data\
Try calling mysql_set_charset('utf8'); after connecting to the database. I think it's similar to executing a SET NAMES query, but since the PHP manual says using that function over a SET NAMES query is recommended, I'd try it.
Also, when you display your content, you could try echo htmlentities($string, ENT_COMPAT, 'UTF-8');
I store greek in tables created like ths:
CREATE TABLE `test` (
`test_id` SMALLINT UNSIGNED NOT NULL AUTO_INCREMENT,
`test_name` VARCHAR(30) NOT NULL,
PRIMARY KEY (`test_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT;
Or if table already created I guess you can change the charset it in the phpmyadmin interface. Maybe this helps.
Check your MySQL configuration and ensure that your encoding is defined correctly.
Add these lines to my.cnf or my.ini, which ever your installation uses.
These settings did the trick for me:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
character-set-server=utf8
I have tried multiple collations in phpMyAdmin as well as changing the charset of the page, which didn;t make sense but i was willing to try anything after two days of research. this command helped me: mysql_set_charset('utf8');
Collation on the column was set to koi8r_general_ci
Edit your structure field to set collation utf16_general_ci.
After that insert your data.

Storing and displaying unicode string (हिन्दी) using PHP and MySQL

I have to store hindi text in a MySQL database, fetch it using a PHP script and display it on a webpage. I did the following:
I created a database and set its encoding to UTF-8 and also the collation to utf8_bin.
I added a varchar field in the table and set it to accept UTF-8 text in the charset property.
Then I set about adding data to it. Here I had to copy data from an existing site.
The hindi text looks like this: सूर्योदय:05:30
I directly copied this text into my database and used the PHP code echo(utf8_encode($string)) to display the data. Upon doing so the browser showed me "??????".
When I inserted the UTF equivalent of the text by going to "view source" in the browser, however, सूर्योदय translates into सूर्योदय.
If I enter and store सूर्योदय in the database, it converts perfectly.
So what I want to know is how I can directly store सूर्योदय into my database and fetch it and display it in my webpage using PHP.
Also, can anyone help me understand if there's a script which when I type in सूर्योदय, gives me सूर्योदय?
Solution Found
I wrote the following sample script which worked for me. Hope it helps someone else too
<html>
<head>
<title>Hindi</title></head>
<body>
<?php
include("connection.php"); //simple connection setting
$result = mysql_query("SET NAMES utf8"); //the main trick
$cmd = "select * from hindi";
$result = mysql_query($cmd);
while ($myrow = mysql_fetch_row($result))
{
echo ($myrow[0]);
}
?>
</body>
</html>
The dump for my database storing hindi utf strings is
CREATE TABLE `hindi` (
`data` varchar(1000) character set utf8 collate utf8_bin default NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `hindi` VALUES ('सूर्योदय');
Now my question is, how did it work without specifying "META" or header info?
Thanks!
Did you set proper charset in the HTML Head section?
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
or you can set content type in your php script using -
header( 'Content-Type: text/html; charset=utf-8' );
There are already some discussions here on StackOverflow - please have a look
How to make MySQL handle UTF-8 properly
setting utf8 with mysql through php
PHP/MySQL with encoding problems
So what i want to know is how can i
directly store सूर्योदय into my
database and fetch it and display in
my webpage using PHP.
I am not sure what you mean by "directly storing in the database" .. did you mean entering data using PhpMyAdmin or any other similar tool? If yes, I have tried using PhpMyAdmin to input unicode data, so it has worked fine for me - You could try inputting data using phpmyadmin and retrieve it using a php script to confirm. If you need to submit data via a Php script just set the NAMES and CHARACTER SET when you create mysql connection, before execute insert queries, and when you select data. Have a look at the above posts to find the syntax. Hope it helps.
** UPDATE **
Just fixed some typos etc
<meta http-equiv="Content-Type" content="text/html;charset=UTF-8">
<?php
$con = mysql_connect("localhost","root","");
if (!$con)
{
die('Could not connect: ' . mysql_error());
}
mysql_query('SET character_set_results=utf8');
mysql_query('SET names=utf8');
mysql_query('SET character_set_client=utf8');
mysql_query('SET character_set_connection=utf8');
mysql_query('SET character_set_results=utf8');
mysql_query('SET collation_connection=utf8_general_ci');
mysql_select_db('onlinetest',$con);
$nith = "CREATE TABLE IF NOT EXISTS `TAMIL` (
`data` varchar(1000) character set utf8 collate utf8_bin default NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1";
if (!mysql_query($nith,$con))
{
die('Error: ' . mysql_error());
}
$nithi = "INSERT INTO `TAMIL` VALUES ('இந்தியா நாட்டின் பக்கங்கள்')";
if (!mysql_query($nithi,$con))
{
die('Error: ' . mysql_error());
}
$result = mysql_query("SET NAMES utf8");//the main trick
$cmd = "select * from TAMIL";
$result = mysql_query($cmd);
while($myrow = mysql_fetch_row($result))
{
echo ($myrow[0]);
}
?>
</body>
</html>
For those who are looking for PHP ( >5.3.5 ) PDO statement, we can set charset as per below:
$dbh = new PDO('mysql:host=localhost;dbname=testdb;charset=utf8', 'username', 'password');
CREATE DATABASE hindi_test
CHARACTER SET utf8
COLLATE utf8_unicode_ci;
USE hindi_test;
CREATE TABLE `hindi` (`data` varchar(200) COLLATE utf8_unicode_ci NOT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
INSERT INTO `hindi` (`data`) VALUES('कंप्यूटर');
For Those who are facing difficulty just got to php admin and change collation to utf8_general_ci
Select Table go to Operations>> table options>> collations should be there

Categories