I am using PHP for executing a MySQL update sentence with Spanish, English and Japanese characters.
But i'm not able to save Japanese characters into the database. How should I proceed?
Database has utf8_general_ci collation.
$strSQL = "UPDATE table SET value = '" . addslashes($strValue) . "'";
$strSQL = utf8_decode($strSQL);
mysqli_query($cnn, $strSQL);
With addslashes I can get apostrophes to be saved into the database.
With utf8_decode I can get Spanish characters to be saved into the database.
Why you proccess utf8_decode() the sql query?
Look at http://php.net/manual/en/function.utf8-decode.php for details about utf8_decode()
You have to check a few things.
Whether there is a problem in your database and tables Or in your PHP script.
Database :
CREATE DATABASE test DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
Table Charset :
CREATE TABLE test.table
(
id INT NOT NULL AUTO_INCREMENT,
text VARCHAR(100) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
PRIMARY KEY (`id`)
) ENGINE = MyISAM;
Connection :
SET NAMES utf8;
E.g :
$mysqli = new mysqli("localhost","my_user","my_password","test_db");
$mysqli->set_charset("utf8");
OR
$link = mysqli_connect("localhost","my_user","my_password","test_db");
mysqli_set_charset($link,"utf8");
Configure the encoding charset of server :
With PHPMyAdmin , Choose UTF-8 when you login.
PHP Script :
header('Content-Type: text/html;charset=UTF-8');
Apache Config (/etc/httpd/conf/httpd.conf) :
AddDefaultCharset UTF-8
Apache .htaccess file :
AddCharset UTF-8 .htm
AddCharset UTF-8 .html
AddCharset UTF-8 .php
PHP Config (/etc/php.ini) :
default_charset = "utf-8"
MySQL Config (/etc/my.cnf ) :
[client]
default-character-set=utf8
[mysqld]
default-collation=utf8_unicode_ci
character-set-server=utf8
default-character-set=utf8
init-connect='SET NAMES utf8'
character-set-client = utf8
User values :
e.g : $_GET , $_POST
You can use mb_convert_encoding() for convert your strings to UTF-8.
Useful Links :
PHP inserting Japanese string to utf8 table as something else, but still reads it successfully
http://es2.php.net/manual/en/mysqli.set-charset.php
Select MySQL rows with Japanese characters
http://php.net/manual/en/function.mysql-set-charset.php
https://www.w3schools.com/php/func_mysqli_set_charset.asp
PHP mysql charset utf8 problems
https://medium.com/#adamhooper/in-mysql-never-use-utf8-use-utf8mb4-11761243e434
https://dev.mysql.com/doc/refman/8.0/en/faqs-cjk.html
https://www.experts-exchange.com/questions/24742133/How-to-insert-a-japanese-character-into-mysql-database.html
I have my database properly set to UTF-8 and am dealing with a database containing Japanese characters. If I do SELECT *... from the mysql command line, I properly see the Japanese characters. When pulling data out of the database and displaying it on a webpage, I see it properly.
However, when viewing the table data in phpMyAdmin, I just see garbage text. ie.
ç§ã¯æ—¥æœ¬æ–™ç†ãŒå¥½ãã§ã™ã€‚日本料ç†ã‚...
How can I get phpMyAdmin to display the characters in Japanese?
The character encoding on the HTML page is set to UTF-8.
Edit:
I have tried an export of my database and opened up the .sql file in geany. The characters are still garbled even though the encoding is set to UTF-8. (However, doing a mysqldump of the database also shows garbled characters).
The character set is set correctly for the database and all tables ('latin' is not found anywhere in the file)
CREATE DATABASE `japanese` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
I have added the lines to my.cnf and restarted mysql but there is no change. I am using Zend Framework to insert data into the database.
I am going to open a bounty for this question as I really want to figure this out.
Unfortunately, phpMyAdmin is one of the first php application that talk to MySQL about charset correctly. Your problem is most likely due to the fact that the database does not store the correct UTF-8 strings at first place.
In order to correctly display the characters correctly in phpMyAdmin, the data must be correctly stored in the database. However, convert the database into correct charset often breaks web apps that does not aware charset-related feature provided by MySQL.
May I ask: is MySQL > version 4.1? What web app is the database for? phpBB? Was the database migrated from an older version of the web app, or an older version of MySQL?
My suggestion is not to brother if the web app you are using is too old and not supported. Only convert database to real UTF-8 if you are sure the web app can read them correctly.
Edit:
Your MySQL is > 4.1, that means it's charset-aware. What's the charset collation settings for you database? I am pretty sure you are using latin1, which is MySQL name for ASCII, to store the UTF-8 text in 'bytes', into the database.
For charset-insensitive clients (i.e. mysql-cli and php-mod-mysql), characters get displayed correctly since they are being transfer to/from database as bytes. In phpMyAdmin, bytes get read and displayed as ASCII characters, that's the garbage text you seem.
Countless hours had been spend years ago (2005?) when MySQL 4.0 went obsolete, in many parts of Asia. There is a standard way to deal with your problem and gobbled data:
Back up your database as .sql
Open it up in UTF-8 capable text editor, make sure they look correct.
Look for charset collation latin1_general_ci, replace latin1 to utf8.
Save as a new sql file, do not overwrite your backup
Import the new file, they will now look correctly in phpMyAdmin, and Japanese on your web app will become question marks. That's normal.
For your php web app that rely on php-mod-mysql, insert mysql_query("SET NAMES UTF8"); after mysql_connect(), now the question marks will be gone.
Add the following configuration my.ini for mysql-cli:
# CLIENT SECTION
[mysql]
default-character-set=utf8
# SERVER SECTION
[mysqld]
default-character-set=utf8
For more information about charset on MySQL, please refer to manual:
http://dev.mysql.com/doc/refman/5.0/en/charset-server.html
Note that I assume your web app is using php-mod-mysql to connect to the database (hence the mysql_connect() function), since php-mod-mysql is the only extension I can think of that still trigger the problem TO THIS DAY.
phpMyAdmin use php-mod-mysqli to connect to MySQL. I never learned how to use it because switch to frameworks* to develop my php projects. I strongly encourage you do that too.
Many frameworks, e.g. CodeIgniter, Zend, use mysqli or pdo to connect to databases. mod-mysql functions are considered obsolete cause performance and scalability issue. Also, you do not want to tie your project to a specific type of database.
If you're using PDO don't forget to initiate it with UTF8:
$con = new PDO('mysql:host=' . $server . ';dbname=' . $db . ';charset=UTF8', $user, $pass, array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"));
(just spent 5 hours to figure this out, hope it will save someone precious time...)
I did a little more googling and came across this page
The command doesn't seem to make sense but I tried it anyway:
In the file /usr/share/phpmyadmin/libraries/dbi/mysqli.dbi.lib.php at the end of function PMA_DBI_connect() just before the return statement I added:
mysqli_query($link, "SET SESSION CHARACTER_SET_RESULTS =latin1;");
mysqli_query($link, "SET SESSION CHARACTER_SET_CLIENT =latin1;");
And it works! I now see Japanese characters in phpMyAdmin. WTF? Why does this work?
I had the same problem,
Set all text/varchar collations in phpMyAdmin to utf-8 and in php files add this:
mysql_set_charset("utf8", $your_connection_name);
This solved it for me.
the solution for this can be as easy as :
find the phpmysqladmin connection function/method
add this after database is conncted $db_conect->set_charset('utf8');
phpmyadmin doesn't follow the MySQL connection because it defines its proper collation in phpmyadmin config file.
So if we don't want or if we can't access server parameters, we should just force it to send results in a different format (encoding) compatible with client i.e. phpmyadmin
for example if both the MySQL connection collation and the MySQL charset are utf8 but phpmyadmin is ISO, we should just add this one before any select query sent to the MYSQL via phpmyadmin :
SET SESSION CHARACTER_SET_RESULTS =latin1;
Here is my way how do I restore the data without looseness from latin1 to utf8:
/**
* Fixes the data in the database that was inserted into latin1 table using utf8 encoding.
*
* DO NOT execute "SET NAMES UTF8" after mysql_connect.
* Your encoding should be the same as when you firstly inserted the data.
* In my case I inserted all my utf8 data into LATIN1 tables.
* The data in tables was like ДЕТСКИÐ.
* But my page presented the data correctly, without "SET NAMES UTF8" query.
* But phpmyadmin did not present it correctly.
* So this is hack how to convert your data to the correct UTF8 format.
* Execute this code just ONCE!
* Don't forget to make backup first!
*/
public function fixIncorrectUtf8DataInsertedByLatinEncoding() {
// mysql_query("SET NAMES LATIN1") or die(mysql_error()); #uncomment this if you already set UTF8 names somewhere
// get all tables in the database
$tables = array();
$query = mysql_query("SHOW TABLES");
while ($t = mysql_fetch_row($query)) {
$tables[] = $t[0];
}
// you need to set explicit tables if not all tables in your database are latin1 charset
// $tables = array('mytable1', 'mytable2', 'mytable3'); # uncomment this if you want to set explicit tables
// duplicate tables, and copy all data from the original tables to the new tables with correct encoding
// the hack is that data retrieved in correct format using latin1 names and inserted again utf8
foreach ($tables as $table) {
$temptable = $table . '_temp';
mysql_query("CREATE TABLE $temptable LIKE $table") or die(mysql_error());
mysql_query("ALTER TABLE $temptable CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci") or die(mysql_error());
$query = mysql_query("SELECT * FROM `$table`") or die(mysql_error());
mysql_query("SET NAMES UTF8") or die(mysql_error());
while ($row = mysql_fetch_row($query)) {
$values = implode("', '", $row);
mysql_query("INSERT INTO `$temptable` VALUES('$values')") or die(mysql_error());
}
mysql_query("SET NAMES LATIN1") or die(mysql_error());
}
// drop old tables and rename temporary tables
// this actually should work, but it not, then
// comment out this lines if this would not work for you and try to rename tables manually with phpmyadmin
foreach ($tables as $table) {
$temptable = $table . '_temp';
mysql_query("DROP TABLE `$table`") or die(mysql_error());
mysql_query("ALTER TABLE `$temptable` RENAME `$table`") or die(mysql_error());
}
// now you data should be correct
// change the database character set
mysql_query("ALTER DATABASE DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci") or die(mysql_error());
// now you can use "SET NAMES UTF8" in your project and mysql will use corrected data
}
Change latin1_swedish_ci to utf8_general_ci in phpmyadmin->table_name->field_name
This is where you find it on the screen:
First, from the client do
mysql> SHOW VARIABLES LIKE 'character_set%';
This will give you something like
+--------------------------+----------------------------+
| Variable_name | Value |
+--------------------------+----------------------------+
| character_set_client | latin1 |
| character_set_connection | latin1 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | latin1 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/share/mysql/charsets/ |
+--------------------------+----------------------------+
where you can inspect the general settings for the client, connection, database
Then you should also inspect the columns from which you are retrieving data with
SHOW CREATE TABLE TableName
and inspecting the charset and collation of CHAR fields (though usually people do not set them explicitly, but it is possible to give CHAR[(length)] [CHARACTER SET charset_name] [COLLATE collation_name] in CREATE TABLE foo ADD COLUMN foo CHAR ...)
I believe that I have listed all relevant settings on the side of mysql.
If still getting lost read fine docs and perhaps this question which might shed some light (especially how I though I got it right by looking only at mysql client in the first go).
1- Open file:
C:\wamp\bin\mysql\mysql5.5.24\my.ini
2- Look for [mysqld] entry and append:
character-set-server = utf8
skip-character-set-client-handshake
The whole view should look like:
[mysqld]
port=3306
character-set-server = utf8
skip-character-set-client-handshake
3- Restart MySQL service!
Its realy simple to add multilanguage in myphpadmin if you got garbdata showing in myphpadmin, just go to myphpadmin click your database go to operations tab in operation tab page see collation section set it to utf8_general_ci, after that all your garbdata will show correctly. a simple and easy trick
The function and file names don't match those in newer versions of phpMyAdmin. Here is how to fix in the newer PHPMyAdmins:
Find file:
phpmyadmin/libraries/DatabaseInterface.php
In function: public function query
Right after the opening { add this:
if($link != null){
mysqli_query($link, "SET SESSION CHARACTER_SET_RESULTS =latin1;");
mysqli_query($link, "SET SESSION CHARACTER_SET_CLIENT =latin1;");
}
That's it. Works like a charm.
I had exactly the same problem. Database charset is utf-8 and collation is utf8_unicode_ci. I was able to see Unicode text in my webapp but the phpMyAdmin and sqldump results were garbled.
It turned out that the problem was in the way my web application was connecting to MySQL. I was missing the encoding flag.
After I fixed it, I was able to see Greek characters correctly in both phpMyAdmin and sqldump but lost all my previous entries.
just uncomment this lines in libraries/database_interface.lib.php
if (! empty($GLOBALS['collation_connection'])) {
// PMA_DBI_query("SET CHARACTER SET 'utf8';", $link, PMA_DBI_QUERY_STORE);
//PMA_DBI_query("SET collation_connection = '" .
//PMA_sqlAddslashes($GLOBALS['collation_connection']) . "';", $link, PMA_DBI_QUERY_STORE);
} else {
//PMA_DBI_query("SET NAMES 'utf8' COLLATE 'utf8_general_ci';", $link, PMA_DBI_QUERY_STORE);
}
if you store data in utf8 without storing charset you do not need phpmyadmin to re-convert again the connection. This will work.
Easier solution for wamp is:
go to phpMyAdmin,
click localhost,
select latin1_bin for Server connection collation,
then start to create database and table
Add:
mysql_query("SET NAMES UTF8");
below:
mysql_select_db(/*your_database_name*/);
It works for me,
mysqli_query($con, "SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'");
ALTER TABLE table_name CONVERT to CHARACTER SET utf8;
*IMPORTANT: Back-up first, execute after
This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 3 years ago.
I can't understand whats problem.
I write data to mysql with php and utf-8 issues
Write in HTML form Binəqədi r. Çiçək qəs
Writed database Bin?q?di r. ÃÃÂ
When I use mysqli_set_charset($conn,"utf8"); in insert and update query
Writed database Bin?q?di r. Çiç?k q?s
Write data manually direct to database and use mysqli_set_charset($conn,"utf8"); in select works normal.
How can I fix it? INSERT and UPDATE with UTF-8.
This is my insert code
public function insert($sql) {
$conn = new mysqli(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, DB_DATABASE);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
mysqli_set_charset($conn,"utf8");
// $conn->set_charset("utf8"); use same this.
if ($conn->query($sql) === TRUE) {
return 1;
} else {
return 0;
}
$conn->close();
}
NOTE: This issues on Linux and OSX. In windows work goods. How can I fix it for linux hosting?
try
AddDefaultCharset utf-8
in you apache configuration
or in you php file use
header( 'content-type: text/html; charset=utf-8' );
and also try to
default_charset = "utf-8";
in you php.ini
also you can try with
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
character-set-client-handshake = false #force encoding to uft8
character-set-server=utf8
collation-server=utf8_general_ci
[mysqld_safe]
default-character-set=utf8
in your my.ini and restart mysql demon
Please consider all of these:
Before your queries, do this:
$conn->query('SET NAMES utf8');
Be sure your file encoding is utf8 (not the html encoding). I mean the file encoding which your IDE is using.
Be sure about your table charset
This is clearly a duplicate of a well-known issue, but cannot be flagged for closure due to the bounty. So, the checklist:
Ensure database columns, tables, and database are set to character set utf8mb4
Ensure the connection is set to the same character set with $conn->set_charset("utf8mb4");
Ensure your server is configured to inform web browsers that it will be sending Unicode text. In PHP, do header("Content-Type: text/html;charset=utf8"); or set it up in your web server configuration.
Try using set names
public function insert($sql) {
$conn = new mysqli(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, DB_DATABASE);
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
// $conn->set_charset("utf8");
$conn->query("SET NAMES 'utf8'");
if ($conn->query($sql) === TRUE) {
return 1;
} else {
return 0;
}
$conn->close();
}
From MySQL (http://dev.mysql.com/doc/refman/5.7/en/charset-connection.html):
SET NAMES 'charset_name' [COLLATE 'collation_name']
SET NAMES indicates what character set the client will use to send SQL statements to the server. Thus, SET NAMES 'cp1251' tells the server, “future incoming messages from this client are in character set cp1251.” It also specifies the character set that the server should use for sending results back to the client. (For example, it indicates what character set to use for column values if you use a SELECT statement.)
A SET NAMES 'charset_name' statement is equivalent to these three statements:
SET character_set_client = charset_name;
SET character_set_results = charset_name;
SET character_set_connection = charset_name;
Setting character_set_connection to charset_name also implicitly sets collation_connection to the default collation for charset_name. It is unnecessary to set that collation explicitly. To specify a particular collation, use the optional COLLATE clause:
SET CHARACTER SET charset_name
SET CHARACTER SET is similar to SET NAMES but sets character_set_connection and collation_connection to character_set_database and collation_database. A SET CHARACTER SET charset_name statement is equivalent to these three statements:
SET character_set_client = charset_name;
SET character_set_results = charset_name;
SET collation_connection = ##collation_database;
Setting collation_connection also implicitly sets character_set_connection to the character set associated with the collation (equivalent to executing SET character_set_connection = ##character_set_database). It is unnecessary to set character_set_connection explicitly.
My opinion is that in some point the proper character set for the field description is not specified when using stored procedure. Do check again that your database/table/field have as character set utf8_general_ci
From MySQL (http://dev.mysql.com/doc/refman/5.7/en/charset-connection.html):
Example: Suppose that column1 is defined as CHAR(5) CHARACTER SET latin2. If you do not say SET NAMES or SET CHARACTER SET, then for SELECT column1 FROM t, the server sends back all the values for column1 using the character set that the client specified when it connected. On the other hand, if you say SET NAMES 'latin1' or SET CHARACTER SET latin1 before issuing the SELECT statement, the server converts the latin2 values to latin1 just before sending results back. Conversion may be lossy if there are characters that are not in both character sets.
The gibberish you displayed seems to involve more than one error. Çiç, when Mojibaked to latin1 gives Çiç.
Please provide SELECT col, HEX(col) FROM ... for something that is 'bad'.
Meanwhile, your reply about the stored procedure -- Do SHOW CREATE PROCEDURE ... and check what character set was in effect when the procedure was created. That could be part of the problem. (I noticed that that decade-old bug report you mentioned failed to consider this aspect.)
This question already has answers here:
strange character encoding of stored data , old script is showing them fine new one doesn't
(2 answers)
How to convert an entire MySQL database characterset and collation to UTF-8?
(20 answers)
Closed 7 years ago.
I have Arabic text stored in mysql like this format
شقق جديده غبر مسكونه 220 متر..
.150متر..طبربور ..ااسكانات قموم
والنجار للاستÙسار
it is in this format because I didn't use the below before inserting data into database from HTML textboxes:
mysql_query('SET CHARACTER SET utf8');
My question now, how to convert the text into MySQL columns to readable text like this:
شقق جديده غبر مسكونه 220 متر ..150متر..طبربور ..ااسكانات قموم والنجار للاستفسار
I tried the below for one column cell to check, but it didn't work:
mysql_query('SET CHARACTER SET utf8');
$sql = 'SELECT content FROM messages WHERE id=500';
$qry = mysql_query($sql);
$result =mysql_fetch_object($qry);
$text= $result->content;
mysql_query('SET CHARACTER SET utf8');
$sql ='UPDATE messages SET content= "'.$text.'" where id=500';
mysql_query($sql);
Note: if I use the below
mysql_query('SET CHARACTER SET utf8');
before I insert data into database from HTML textarea, then it works fine for store and read, but I need to convert the already entered data into tables.
Thanks.
The solution in brief can be done in 3 simple steps:
A. Change the Database collation to utf8 via the command below:
ALTER DATABASE <db_name> CHARACTER SET utf8 COLLATE utf8_general_ci;
already the table cells I want to change have collation of utf8_general_ci
B. Implement this query to change the content:
update messages set content=CONVERT(BINARY CONVERT(content USING latin1) USING utf8);
C. add the below before you connect to the DB:
mysql_query('SET CHARACTER SET utf8');
And you are done!!! Thanks for all
Use the ALTER DATABASE and ALTER TABLE commands.
ALTER DATABASE databasename CHARACTER SET utf8 COLLATE utf8_unicode_ci;
ALTER TABLE tablename CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;
Source.
This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 8 years ago.
I am using PHP 5.5 and when I attempt to insert a UTF-8 character in the MySQL database PDO cuts it off at the first non-ASCII character.
I have set my connection to be:
(DB_TYPE.':host='.DB_HOST.';dbname='.DB_NAME.';charset=utf8', DB_USER, DB_PASS, array(PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC, PDO::ATTR_ERRMODE => PDO::ERRMODE_WARNING))
I have tried the SET NAMES that everyone posts, but that doesn’t work either because the problem is NOT on the MySQL side of things.
When I do an insert through phpMyAdmin and directly from the MySQL console, it works!
When I select the accented string with PDO, it works!
The problem is only on INSERT and UPDATE using PDO specifically!
Here is the SQL of the table. It is all in UTF-8 but maybe someone knows of a conflict between a setting and PDO
CREATE TABLE IF NOT EXISTS `mytable` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_lang` int(11) NOT NULL DEFAULT '2',
`id_tgroup_cat` int(11) NOT NULL,
`fieldfor` int(11) NOT NULL,
`colors` varchar(100) NOT NULL,
`text` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=34 ;
I have already tried to make text a varchar field and that did not change anything.
The actual insert in PHP:
$query = $this->db->prepare("UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
$query->execute(array($text, $colors, $id));
Where $text = "référence" (only saves the letter R in the database but without accents it saves everything) and $colors is an empty string for test purposes and $id is 2.
This is the key clue to me:
Where $text = "référence" (only saves the letter R in the database but
without accents it saves everything) and $colors is an empty string
for test purposes and $id is 2.
Sounds like it is a UTF-8 encoding issue. While the database is UTF-8 the whole chain from the code to the database—including the connection—should be UTF-8 clean.
How exactly does $this->db->prepare relate to the PHP connection to MySQL? A bit unclear from the code you have shown. But based on what you are showing, perhaps adjusting your query like this would help:
$query = $this->db->prepare("SET collation_connection = utf8_bin;
SET NAMES utf8;
UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
Or maybe this:
$this->db->exec("SET collation_connection = utf8_bin; SET NAMES utf8;");
$query = $this->db->prepare("UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
Note my forced-in addition of SET collation_connection = utf8_bin; as well as SET NAMES utf8;
In general you need to make sure your entire chain from the connection, to the database, to the tables is all UTF8 clean. I have a detailed answer to a similar question here.
But in your case, check the actual MySQL server my.cnf file. The following would set the whole chain to UTF-8:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
EDIT: And since the original poster indicates the data is coming from an HTML5 form, I also think checking the BOM (byte order mark) for the actual HTML5 file itself would help as well. It should be set to UTF8. More details on what a BOM is are over here. Specifically the accepted answer from Martin Code which explains:
The UTF-8 BOM is a sequence of bytes (EF BB BF) that allows the reader
to identify the file as an UTF-8 file.