Characters not displaying correctly on a UTF-8 website - php

I've done everything I can think of, but special characters are not displaying correctly on this webpage.
For example, in the database it's:
But on the site it's:
Nouveaux R�alistes
Here's everything I've checked...
The database is set to UTF-8:
The page was written in NetBeans, with the document encoding set to UTF-8:
The page header declares UTF-8:
The meta charset is set to UTF-8:
I've even added the following line to my .htacess:
But there characters are still not displaying correctly, and I get the following error from the W3C Validator:
I feel like I've attempted everything, but it still won't work. (I've even tried htmlspecialchars and htmlentities in PHP, but the page doesn't even render!)
UPDATE
As requested, here is the code I'm using:
class Exhibition {
public $exhibitionDetails;
public function __construct(Database $db, $exhibitionID){
$this->_db = $db;
$params['ExhibitionID'] = $exhibitionID;
$STH = $this->_db->prepare("SELECT *
FROM Exhibition
INNER JOIN Schedule
ON Exhibition.ExhibitionID = Schedule.ExhibitionID
WHERE Schedule.Visible = 1
AND Exhibition.ExhibitionID = :ExhibitionID;");
$STH->execute($params);
$this->exhibitionDetails = $STH->fetchAll(PDO::FETCH_ASSOC);
}
}
And...
try {
$db = new Database(SITE_ROOT."/../");
$db->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$exhibition = new Exhibition($db,$_GET['id']);
} catch (PDOException $e) {
echo "<p class='error'>ERROR: ".$e->getMessage()."</p>";
}
And finally...
<p><?php echo $exhibition->exhibitionDetails[0]["Desc"]; ?></p>

If you are using mysql_* functions:
mysql_query("SET NAMES 'utf8'");
If you are using PDO
$dsn = 'mysql:host=localhost;dbname=testdb';
$username = 'username';
$password = 'password';
$options = array(
PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8',
);
$dbh = new PDO($dsn, $username, $password, $options);
It sets connection encoding.

It's been a few years since I've used PHP but back then it didn't natively support Unicode and a quick search of google tells me it still doesn't. You can still make it work though.
Here's a great link:
Encodings And Character Sets To Work With Text

Related

Character encoding issue with PHP/MySQL

I have a web application I have been developing locally on my computer, I have recently launched it on my web server with the exact same settings for MySQL, but for some reason I am encountering a character encoding issue where "smart quotes" (’) are being displayed as black triangles with question marks (�), but only on the server.
This is on the HTML:
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
My local MySQL database (without character issues) is MySQL 5.7.26 with an InnoDB table with UTF8MB4 encoding and a table collation of utf8mb4-unicode-ci
My server MySQL database (with character issues) is MySQL 5.5.5-10.3.21 with an InnoDB table with UTF8MB4 encoding and a table collation of utf8mb4-unicode-ci
Perhaps there is a PHP setting somewhere I am missing?
Edit:
if I run this script on my computer is works perfectly, but if I run the same script on my server with the same data in the database, it give me the character encoding issue:
error_reporting(E_ALL);
include 'dbconfig.php';
$id = 19;
try {
$pdo = new PDO("mysql:host=$servername;dbname=$dbname", $username, $password);
$pdo->setAttribute(PDO::ATTR_ORACLE_NULLS, PDO::NULL_TO_STRING);
$pdo->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$sql = $pdo->prepare("SELECT * FROM products WHERE id=:id");
$sql->execute(['id' => $id]);
var_dump ($sql->fetch());
while ($row = $sql->fetch()) {
$data[] = $row;
}
echo json_encode($data);
} catch(PDOException $e) {
//echo "Error: " . $e->getMessage();
echo "Database Error";
}
$conn = null;
(Since this clearly shows a PDO omission, I am reopening)
One-line change:
$pdo = new
PDO("mysql:host=$servername;dbname=$dbname;dbname=db;charset=utf8mb4'",
$username, $password);
References:
http://mysql.rjweb.org/doc.php/charcoll#php
Trouble with UTF-8 characters; what I see is not what I stored (Look especially for "black diamond".)

How to insert data which contain non english letter in php slimframwork

I am working with PHP slim framework to develop API. I can GET and DELETE but with the inserting (POST) and updating (PUT), I am facing problems.
class db {
//Propertise
private $dbhost = 'localhost';
private $dbuser = 'root';
private $dbpass = '';
private $dbname = 'bralivnara-api';
//Connect
public function connect(){
try {
$options = array(PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8', 'CHARSET utf8');
$dbConnection = new PDO("mysql:host=$this->dbhost;dbname=$this->dbname;", $this->dbuser, $this->dbpass, $options);
// set the PDO error mode to exception
$dbConnection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
//echo "Connected successfully";
}
catch(PDOException $e)
{
echo "Connection failed: " . $e->getMessage();
}
return $dbConnection;
}
}
When I use rest easy I receive the 500 error.
Response Status: 500 (Internal Server Error)
When I passed the URL and press enter in browser, it shows the error
Connection failed: SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'add' at line 1
Before I had an issue with Swedish letters also, it was not displaying in browser correctly. If anyone has same issue you can fix this issue with that header. It will work for all non-English letters
return $response->withStatus(200)
->withHeader('Content-Type', 'application/json;charset=utf-8;')
->write(json_encode($tranasVcDatainsamling, JSON_UNESCAPED_UNICODE));
If you want to see complete get code you can join our discussion.
Now the last problem which still remains is inserting to database. Here is the code for insertion to the database.
$app->post('/api/tranas-vc/add', function (Request $request, Response $response){
$besöksdatum = $request->getParam('besöksdatum');
$kön = $request->getParam('kön');
$ålder_år = $request->getParam('ålder_år');
$sökorsak = $request->getParam('sökorsak');
$vp_objektiv_prio = $request->getParam('vp_objektiv_prio');
$vp_objektiv_variabel_med_högst_prio = $request->getParam('vp_objektiv_variabel_med_högst_prio');
$vp_mätning_prio = $request->getParam('vp_mätning_prio');
$vp_mätning_variabel_med_högst_prio = $request->getParam('vp_mätning_variabel_med_högst_prio');
$ess_val = $request->getParam('ess_val');
$prio_ess = $request->getParam('prio_ess ');
$prio_total = $request->getParam('prio-total');
$utskrivning_hänvisning_återbesök = $request->getParam('utskrivning_hänvisning_återbesök');
$åtgärd_remiss = $request->getParam('åtgärd_remiss');
$om_lab_vilka_prover = $request->getParam('om-lab-vilka-prover');
$sql = "INSERT INTO
`tranasvcdatainsamling`
(
`besöksdatum`,
`kön`,
`ålder_år`,
`sökorsak`,
`vp_objektiv_prio`,
`vp_objektiv_variabel_med_högst_prio`,
`vp_mätning_prio`,
`vp_mätning_variabel_med_högst_prio`,
`ess_val`,
`prio_ess`,
`prio_total`,
`utskrivning_hänvisning_återbesök`,
`åtgärd_remiss`,
`om_lab_vilka_prover`
)
VALUES
(
:`besöksdatum`,
:`kön`,
:`ålder_år`,
:`sökorsak`,
:`vp_objektiv_prio`,
:`vp_objektiv_variabel_med_högst_prio`,
:`vp_mätning_prio`,
:`vp_mätning_variabel_med_högst_prio`,
:`ess_val`,
:`prio_ess`,
:`prio_total`,
:`utskrivning_hänvisning_återbesök`,
:`åtgärd_remiss`,
:`om_lab_vilka_prover`
)";
try {
// Get DB Objects
$db = new db();
// Connect
$db = $db->connect();
// Execute Query
$stmt = $db->prepare($sql);
$stmt->bindParam(':besöksdatum', $besöksdatum);
$stmt->bindParam(':kön', $kön);
$stmt->bindParam(':ålder_år', $ålder_år);
$stmt->bindParam(':sökorsak', $sökorsak);
$stmt->bindParam(':vp_objektiv_prio', $vp_objektiv_prio);
$stmt->bindParam(':vp_objektiv_variabel_med_högst_prio', $vp_objektiv_variabel_med_högst_prio);
$stmt->bindParam(':vp_mätning_prio', $vp_mätning_prio);
$stmt->bindParam(':vp_mätning_variabel_med_högst_prio', $vp_mätning_variabel_med_högst_prio);
$stmt->bindParam(':ess_val', $ess_val);
$stmt->bindParam(':prio_ess', $prio_ess);
$stmt->bindParam(':prio_total', $prio_total);
$stmt->bindParam(':utskrivning_hänvisning_återbesök', $utskrivning_hänvisning_återbesök);
$stmt->bindParam(':åtgärd_remiss', $åtgärd_remiss);
$stmt->bindParam(':om_lab_vilka_prover', $om_lab_vilka_prover);
$stmt->execute();
echo '{"notice": {"text": "Tranås VC Entry Added"}';
} catch(PDOEception $e) {
echo '{"error": {"text": '.$e->getMessage().'}';
}
});
We did all possible experiments on that code, you can see those suggestions in comments, but we still had the problem in POST and PUT.
I commented the $stmt->execute(); to stop the execution of the query and check, I got 200 success in rest easy, but data was not inserted. I browser the error was same. It shows that the problem is in execution. I checked that code on English letters kind of data, and it was working.
I am quite sure that I had the problem because of Swedish letter, which is non-English.
I database those columns which contain Swedish letters look like that
kön varchar(125) latin1_swedish_ci
sökorsak varchar(125) latin1_swedish_ci
Which show latin1_swedish_ci, that I want you to consider also.
I will appreciate your help, and please join this discussion. If we fix this issue it will be very good research for non-English letters issue. I google almost whole day but did not got solution.
You already use SET NAMES which is the right approach, but you need to wrap the utf8 inside a single quotation mark, as defined in the MySQL documentation
SET NAMES 'charset_name' [COLLATE 'collation_name']
$options = array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES 'utf8'", 'CHARSET utf8');
^ ^

Encoding problems when importing sql statement from file

I have encountered a problem, when i try to execute a sql statement I have stored in a file. The problem is that special characters like æøå becomes gibberish when they are imported to the database.
I use the code below:
$dsn = 'mysql:host='.DBHOST.';dbname='.DBBASE;
$db = new PDO($dsn, DBUSER, DBPASS);
//$db->exec("set names 'utf8'");
$sql = "TRUNCATE TABLE wtm_meta_language";
$db->exec($sql);
$sql = file_get_contents('language.sql');
$db->exec($sql);
The language.sql file is encoded in UTF-8 and so is the database.
I have tried to force the database connection to use utf-8 by adding "set names 'utf8'" to my code (row 3, that is now a comment), but when I add this row nothing gets imported at all.
I hope someone has an idea to how to solve this.
Your results are not getting retrieved because you did not add charset=utf8 .
$db = new PDO('mysql:host='.DBHOST.';dbname='.DBBASE.';charset=utf8', 'username', 'password');
and then it should allow you to get the required result.
Could be that PHP does not detect that the file is UTF-8 since, according to your comment, the BOM is missing.
I found this little function here which should force it to read UTF-8.
function file_get_contents_utf8($fn) {
$opts = array(
'http' => array(
'method'=>"GET",
'header'=>"Content-Type: text/html; charset=utf-8"
)
);
$context = stream_context_create($opts);
$result = #file_get_contents($fn,false,$context);
return $result;
}
Also, if I were you I'd use mysql_set_charset to set the character set on the mysql connection. Also I'd use SHOW VARIABLES LIKE 'character_set%'; so see what the current connection character set is.

Alternative to manual "SET NAMES utf8" on every database call [duplicate]

This question already has answers here:
Whether to use "SET NAMES"
(3 answers)
Closed 9 years ago.
I have Japanese on my site coming from a mysql database.
Other Japanese text on the site displays fine, but the stuff coming out of the database looks like:
????????
The database is set to UTF8, so is the web page.
To fix the issue, before I make any calls to the database I use:
$db->query("SET NAMES utf8");
I was wondering if there was a better/easier way to do the above without having to set names before every query?
You don't have to use SET NAMES utf8 before every query, you just have to run it once the connection is established. Here's an example with PDO :
try {
$dns = "mysql:host={HOST};dbname={DB}";
$user = "{USER}";
$pass = "{PASS}";
$con = new PDO($dns, $user, $pass);
} catch ( Exception $e ) {
echo "Connexion error ", $e->getMessage();
die();
}
$con->query("SET NAMES utf8;");
//and now, as many queries as you want
If you're using PHP >= 5.3.6, you can define the charset in the connection string (prior PHP versions ignore it) :
try {
$dns = "mysql:host={HOST};dbname={DB};charset=utf8";
$user = "{USER}";
$pass = "{PASS}";
$con = new PDO($dns, $user, $pass);
} catch ( Exception $e ) {
echo "Connexion error ", $e->getMessage();
die();
}
//and now, as many queries as you want

Polish characters in mysql response

I have a problem with polish characters. I can't get correctly written words, like '?ukasz' instead of "Łukasz" or even "null", when it supposed to be "Kraków". I tried "mysql_set_charset('utf-8'/'iso-8859-1')" after mysql_connect or iconv(on json_encode($output)) and it's still the same (except now there is "Krak\u00f3" instead of "null"). I'll appreciate any help.
This is a php file for my Android app:
$id_client = $_REQUEST['id_klienta'];
$con=mysql_connect('localhost', 'root', '') or die(mysql_error());
mysql_select_db('courier_helper') or die(mysql_error());
$sql=mysql_query("SELECT * FROM `clients` WHERE id_klienta='$id_client'");
while($r=mysql_fetch_assoc($sql))
$output[]=$r;
print(json_encode($output));
mysql_close($con);
?>
You have to make sure, that you are using UTF-8 everywhere:
script file encoding (UTF-8 instead of ANSI) - you can set encoding it in Notepad++
html code (meta charset)
database table charset (when you are creating table or database)
database mysql_set_charset('utf8', $connection_obj);
database SET NAMES utf8 - run that SQL command after connecting
And one more thing - get familiar with PDO. This is my PDO connect function I use:
function DbConnect()
{
$db_host = "localhost";
$db_name = "database_name";
$db_user = "your_username";
$db_pass = "your_passwd";
$link = new PDO("mysql:host=$db_host;dbname=$db_name; charset=UTF-8", $db_user, $db_pass);
$link->exec("set names utf8;");
return $link;
}
You can use that function like this (this is PDO example):
$link = DbConnect();
$query = $link->prepare("SELECT id FROM wp_users");
$query->execute();
$result = $query->fetchAll(PDO::FETCH_ASSOC);
You should have your database storing data as UTF8, which means converting your existing tables.
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
You also need to make sure your connection to the database is UTF8. You can make sure of that by running a SET NAMES query right after your connect.
SET NAMES UTF8
As others mentioned, you should start using PDO.

Categories