What could break charsets between database and display? - php

I am using a MySQL database to store some strings which contain German umlauts (äüö). The table "testtable" and the column "text" are utf8_bin collated, and PHPMyAdmin tells me the "MySQL connection" is also utf8_bin.
I then use a PHP script to read the strings and display them:
$sql = "SELECT `text` FROM `testtable` WHERE `id`=$id";
$db = mysql_connect(MYSQL_HOST, MYSQL_USER, MYSQL_PASS, false);
mysql_select_db(MYSQL_DATABASE,$db);
$result = mysql_query($sql,$db);
if (!$result or mysql_errno()) die("Something was wrong with the query: $sql");
$rows = array();
while(($rows[] = mysql_fetch_assoc($result)) || array_pop($rows));
foreach ($rows as $r) {echo $r['text']}
The file itself is encoded in utf-8 according to my editor, and the page is declared as utf-8 in the head:
<meta charset="utf-8">
Yet, the text "This is a testmessage äüöß" in the database is displayed as "This is a testmessage ����". When I write umlauts directly into the HTML of the PHP file, or echo umlauts directly, they are displayed correctly, so I figure the encoding mistake must be somewhere between database and PHP server.
What factors that I've overlooked could be messing up the encoding here, or what could I try to figure out where exactly the problem lies?

PHPMyAdmin tells you only about its own connection charset and not about connection between database and your application. Try to run query
SET NAMES utf8
right after you made a connection

Related

PHP - insert Unicode data into MYSQL formatted properly

I am inserting some data from a sjson file into mysql database. Some of the data is formatted using unicode.
When I echo the data on the browser is showing fine however, it is inserted in the database not formatted properly.
In the database I am using as Collation utf8_unicode_ci.
JSON data:
anch\u2019io, sar\u00e0
Showing in the Browser:
anch'io, sarà
Showing in the mysql database:
anch’io, sarÃ
How can I inserted in the database the text properly formatted?
PHP
$getUrl = "https://example.com/79809000.json";
$json_level = file_get_contents($getUrl);
$data_level = json_decode($json_level);
$text = $data_level->{"text"};
mysqli_query($conn, "INSERT INTO `70_level`(`text`) VALUES ('$text')");
I have tried to use addslashes, htmlentities but it does not work.
You are probably not using utf8 as your character set connection/collation connection.
The easiest way to fix this will be to use
$conn = mysqli_connect(...);
mysqli_query($conn, "SET NAMES 'utf8'");
The SET NAMES query should be the first query you run, so make sure you put it right after your mysqli_connect function.
Note that this change will affect all the data, so if you already have data in the database - you will need to re-insert it using the new (utf8) charset.

Urdu / Arabic font data from MySQL is displaying as ????? in JSON

I am developing an Android app with Urdu/Arabic data store in MySQL database on my web server and using JSON_Encoding to generate the JSON string. The JSON string is then being used in Android app to perform various functions (populating RecyclerView and other view objects with data). I am able to store Urdu / Arabic data in MySQL database, but when I use PHP script to generate JSON, all the fields containing Urdu characters is displaying data as ??????
I was using the utf8mb4_unicode_ci as I read the this is easy for storing non-English data and performing multiple functions, but after this encoding problem, I have changed that to utf8_general_ci for all the tables and fields in MySQL database. Below is the PHP script I am using to generate the JSON string from MySQL:
<?php
require "conn.php";
mysqli_query("SET NAMES 'utf8'");
mysqli_query('SET CHARACTER SET utf8');
$sql_qry = "SELECT * FROM countrybasic;";
$result = mysqli_query($conn, $sql_qry);
$response = array();
while($row = mysqli_fetch_array($result)){
array_push($response, array("id"=>$row[0],"name"=>$row[1],"capital"=>$row[2],"continent"=>$row[3],"population"=>$row[4],"gdp"=>$row[5]));
}
echo json_encode(array("server_response"=>$response));
mysqli_close($conn);
?>
The Name and Capital fields are the ones I store my Urdu data in.
Please help me out to resolve this issue.
Thanks.
Create your table [countrybasic] with collation utf8mb4_unicode_ci and make the name column with the same colation.
Now insert some sample data in different languages.
Get the data using MySQLi query result.
Note: If you save the data when the collation is different and get that one after changing the collation then that data will not fetch correctly.
I hope this will work.
You just have to change the Charset to UTF8, and you can use these lines for PHP to do it:
$statSQL= 'SET CHARACTER SET utf8';
mysqli_query($your_db,$sSQL)
or die ('charset in DB didn\'t change');
I hope this help :)
$CONNECTION = mysqli_connect($host,$user,$password,$database);
// Check connection
if (mysqli_connect_errno())
{
echo "Failed to connect to MySQL: " . mysqli_connect_error();
}
mysqli_query ($CONNECTION ,"set character_set_results='utf8'");
$queryutf8 = "select * from yourtable";
$res_utf8 = mysqli_query($CONNECTION ,$queryutf8 );
You need to change the default utf8 in the wamp server
check the below link for more detail.
Arabic characters doesn't show in phpMyAdmin

can't get UTF-8 names into a mysql database

I'm having a problem getting UTF-8 names written into a MySQL database... Here's what I have.
PHP page head has....
<meta charset="utf-8">
the MySQL column is: Char (80) with utf8_unicode_ci (these were originally latin1... I've changed them to UTF-8, truncated the database, then rerun the code)
The variable echoes to screen: Germán Mera
but writes it to database as Germán Mera
I tried putting utf8_encode(); around the variable, but then it writes to database as: Germán Mera and screen as Germán Mera (I know that command only works on iso-8859-1.. I think the JSON page is already UTF-8)
Here is an excerpt of the code I am using to get the name (for sake of simplicity, I'm only showing relevant code - I know what's shown below is not secure)
$str = file_get_contents('http://fantasy.mlssoccer.com/web/api/elements/647/');
$jsonarray = json_decode($str, true);
$name = $jsonarray['web_name'];
mysqli_query ($con, "INSERT INTO mlsprices (name) VALUES ('$name')");
Any idea how I can get this to write to the database properly? When I search, I only get quite complicated answers (eg, this) and there's surely an easier way.
Try using SET NAMES 'UTF8' after connecting to MySQL:
$con=mysqli_connect("host", "user", "pw", "db");
if (!$con)
{
die('Failed to connect to mySQL: ' .mysqli_connect_errno());
}
/* change character set to utf8 */
if (!$con->set_charset("utf8")) {
printf("Error loading character set utf8: %s\n", $con->error);
}
As the manual says:
SET NAMES indicates what character set the client will use to send SQL
statements to the server... It also specifies the character set that the server should
use for sending results back to the client.

UTF8 reading Persian string as ? in MySQL 5.0

I have a table in MySQL 5.0, which I put city names in it in Persian and using a page I try to read a specific city name!
It used to word, but suddenly from today, the city names in my page are all '?????' like!
I go to the phpMyAdmin, change all the collation settings to "utf_persian_ci" and nothing happens!
The interesting part of it is that the "Browse" option of phpMyAdmin shows everything ok (all city names are ok!) but when I try to get them using this kind of query from a page the thing happens:
$result = dbquery("SELECT * FROM ".DB_CITIES." WHERE cty_company_id = ".$_GET['cmp']." AND EXISTS (SELECT cu_cmp_id FROM ".DB_COMPANY_USERS." WHERE cu_cmp_id = ".$_GET['cmp']." AND cu_usr_id = $user_id) AND EXISTS (SELECT ctus_user_id FROM scada_city_users WHERE ctus_user_id = $user_id AND ctus_city_id = cty_id)");
Thanks in advance!
There is a simple way to tell the database it should deliver UTF-8 encoded strings. Just tell your connection object, which character-set you expect, the database does the rest for you.
$db = new mysqli($dbHost, $dbUser, $dbPw, $dbName);
// tell the db to deliver UTF-8 encoded strings.
$db->set_charset("utf8");
The collation only defines how two entries should be compared, it's not the same as defining the charset. Your HTML page shoud also be UTF-8 encoded, some more informations you can find here here.

How can I get UTF8 to work correctly using mysql and php?

I have a database where the table is encoded as utf8. I have a value in it that's in Korean. The characters display fine in the database. But when they are echoed from the database I get a bunch of question marks.
Here is my code, after the connect and select_db statements:
mysql_query('SET NAMES utf8');
$query = 'SELECT * FROM english WHERE id = ' . $_GET['dealerID'];
$result = mysql_query($query) or die(mysql_error());
$row = mysql_fetch_array($result);
What am I doing wrong here? (Yes, 'english' is the right table). I tried Michael's suggestion below to encode the table as utf8_general_ci and I get a MySQL error. Suggestions? What's the correct name of the character set?
If I run a query in PHPMyAdmin, I get 서울시 서초구 서초1동 1425-10 세중프라자 4층/
Review this guide which talks about UTF8, mysql, and PHP all together.
Summary:
make sure the browser knows the page is in utf8
<?php header("Content-type: text/html; charset=utf-8"); ?>
The table in mysql needs to be set for utf8 as well as the string fields within the db
utf8_general_ci
tell php that when it talks to mysql to talk in utf8
mysqli_set_charset('utf8');
Here are some more details as well.

Categories