I have a strange character encoding issue.
I have a page where text is pulled from my database and displayed once on the page, once in the title, and once in a jquery ui modal dialog.
The character is an n with a tilde over it. It is from the same field/column/record in the database.
On the page it is fine. In the title it appears as the diamond/question mark. In the dialog it appears as the diamond/question mark. On a previous page, (it is a CMS, so all the headers are the same) it displays, in the page, as the diamond/question mark.
So what am I missing?
Charset and collation set as UTF8 in MYSQL.
Page header is UTF8 <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
I even did this in php:
ini_set('default_charset', 'utf-8');
header('Content-Type: text/html; charset=utf-8');
header('Accept-Charset: utf-8');
header('Accept: text/html');
mb_language('uni');
mb_internal_encoding('UTF-8');
iconv_set_encoding("input_encoding", "utf-8");
iconv_set_encoding("internal_encoding", "utf-8");
iconv_set_encoding("output_encoding", "utf-8");
So, I could understand if it wasn't showing right in all places, but differently on one page is messing with my mind.
Thanks for any help.
So, I'm sort of not so bright. I being by saying that first off, adding SET NAMES fixed many problems across the board. So thank you for that. I will be putting this bit into all of my connection scripts from now on:
mysql_query("SET NAMES UTF8");
I found my error was at this line:
echo htmlentities($person_name);
So, once everything was properly dispalying as utf-8, I still got the
Muñoz
because it seems that you can't convert 3 and 4 byte utf8 characters using htmlentities(). See links below.
http://www.php.net/manual/en/function.htmlentities.php#96648
This issue is also discussed at:
http://www.php.net/manual/en/function.htmlentities.php#92105
Thanks for the help all.
Related
I have these Chinese characters:
汉字/漢字''test
If I do
echo utf8_encode($chinesevar);
it displays
??/??''test
Or even if I just do a simple
echo $chinesevar
it still displays some weird characters...
So how am I going to display these Chinese characters without using the <meta> tag with the UTF-8 thingy .. or the ini_set UTF-8 thing or even the header() thing with UTF-8?
Simple:
save your source code in UTF-8
output an HTTP header to specify to your browser that it should interpret the page using UTF-8:
header('Content-Type: text/html; charset=utf-8');
Done.
utf8_encode is for converting Latin-1 encoded strings to UTF-8. You don't need it.
For more details, see Handling Unicode Front To Back In A Web App.
Look that your file is in UTF8 without BOM and that your webserver deliver your site in UTF-8
HTML:
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
in PHP:
header('Content-Type: text/html; charset=utf-8');
And if you work with a database look that your database is in UTF-8 if you read the text from your database.
$chinesevarOK = mb_convert_encoding($chinesevar, 'HTML-ENTITIES', 'UTF-8');
Perhaps take a look at the following solutions:
Your database, table and field COLLATE should be utf8_unicode_ci
Check if your records are showing the correct characters within the database...
Set your html to utf8
Add the following line to your php after connecting to the database
mysqli_set_charset($con,"utf8");
http://www.w3schools.com/php/func_mysqli_set_charset.asp
save your source code in UTF-8 No BOM
I have some texts in French (containing accented characters such as "é"), stored in a MySQL table whose collation is utf8_unicode_ci (both the table and the columns), that I want to output on an HTML5 page.
The HTML page charset is UTF-8 (< meta charset="utf-8" />) and the PHP files themselves are encoded as "UTF-8 without BOM" (I use Notepad++ on Windows). I use PHP5 to request the database and generate the HTML.
However, on the output page, the special characters (such as "é") appear garbled and are replaced by "�".
When I browse the database (via phpMyAdmin) those same accented characters display just fine.
What am I missing here?
(Note: changing the page encoding (through Firefox's "web developer" menu) to ISO-8859-1 solves the problem... except for the special characters that appears directly in the PHP files, which become now corrupted. But anyway, I'd rather understand why it doesn't work as UTF-8 than changing the encoding without understanding why it works. ^^;)
I experienced that same problem before, and what I did are the following
1) Use notepad++(can almost adapt on any encoding) or eclipse and be sure in to save or open it in UTF-8 without BOM.
2) set the encoding in PHP header, using header('Content-type: text/html; charset=UTF-8');
3) remove any extra spaces on the start and end of my PHP files.
4) set all my table and columns encoding to utf8mb4_general_ci or utf8mb4_unicode_ci via PhpMyAdmin or any mySQL client you have. A comparison of the two encodings are available here
5) set mysql connection charset to UTF-8 (I use PDO for my database connection )
PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8"
PDO::MYSQL_ATTR_INIT_COMMAND => "SET CHARACTER SET utf8"
or just execute the SQL queries before fetching any data
6) use a meta tag <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
7) use a certain language code for French
<meta http-equiv="Content-language" content="fr" />
8) change the html element lang attribute to the desired language
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="fr" lang="fr">
and will be updating this more because I really had a hard time solving this problem before because I was dealing with Japanese characters in my past projects
9) Some fonts are not available in the client PC, you need to use Google fonts to include it on your CSS
10) Don't end your PHP source file with ?>
NOTE:
but if everything I said above doesn't work, try to adjust your encoding depending on the character-set you really want to display, for me I set everything to SHIFT-JIS to display all my japanese characters and it really works fine. But using UFT-8 must be your priority
This works for me
Make your database utf8_general_ci
Save your files in N++ as UTF-8 without BOM
Put $mysqli->query('SET NAMES utf8'); after the connection to the database in your PHP file
Put < meta charset="utf-8" /> in your HTML-s
Works perfect.
If your php.ini default_charset is not set to UTF-8, you need to use a Content-type to define your data. Apply the following header at the top of your file(s) :
header("Content-type: text/html; charset=utf-8");
If you have still troubles with encoding, the cause may be one of the following:
a database server charset problem (check encoding of your server)
a database client charset problem (check encoding of your connection)
a database table charset problem (check encoding of your table)
a php default encoding problem (check default_encoding parameter in parameters.ini)
a multibyte missconfigured (see mb_string parameters in parameters.ini)
a <form> charset problem (check that it is sent as utf-8)
a <html> charset problem (where no enctype is set in your html file)
a Content-encoding: problem (where the wrong encoding is sent by Apache).
SET NAMES worked for me.
My issue was in one of my editing pages the field with the foreign characters would not display, on the production web pages there was no problem.
I know you already have an answer. That's great. But strangely none of these answers solved my issue. I'd like to share my answer for the benefit of the others who may encounter the same issues.
I also had the same problems as the OP, with regards to French accents in a multi-lingual application.
But I encountered this issue for the first time when I had to pass (French accented) data as segments in AJAX calls.
Yes, we must have the database set to work with UTF8. But the fact that AJAX calls had query strings (in my case segments, since I'm using CodeIgniter), I had to simply encode the French text.
To do this on the client-side, use the Javascript encodeURI() function with your data.
And to reverse it in PHP, just use urldecode($MyStr) where data was received as parameters.
Hope this helps.
Type something full French signs in your (php) file
Save that file as UTF-8
Paste line beneath into your website header
header('Content-Type: text/html; charset=utf-8');
Page (file) should look good.
If looks good go here for mysql behavior after (SET_NAMES).
I have these Chinese characters:
汉字/漢字''test
If I do
echo utf8_encode($chinesevar);
it displays
??/??''test
Or even if I just do a simple
echo $chinesevar
it still displays some weird characters...
So how am I going to display these Chinese characters without using the <meta> tag with the UTF-8 thingy .. or the ini_set UTF-8 thing or even the header() thing with UTF-8?
Simple:
save your source code in UTF-8
output an HTTP header to specify to your browser that it should interpret the page using UTF-8:
header('Content-Type: text/html; charset=utf-8');
Done.
utf8_encode is for converting Latin-1 encoded strings to UTF-8. You don't need it.
For more details, see Handling Unicode Front To Back In A Web App.
Look that your file is in UTF8 without BOM and that your webserver deliver your site in UTF-8
HTML:
<meta http-equiv="content-type" content="text/html; charset=UTF-8" />
in PHP:
header('Content-Type: text/html; charset=utf-8');
And if you work with a database look that your database is in UTF-8 if you read the text from your database.
$chinesevarOK = mb_convert_encoding($chinesevar, 'HTML-ENTITIES', 'UTF-8');
Perhaps take a look at the following solutions:
Your database, table and field COLLATE should be utf8_unicode_ci
Check if your records are showing the correct characters within the database...
Set your html to utf8
Add the following line to your php after connecting to the database
mysqli_set_charset($con,"utf8");
http://www.w3schools.com/php/func_mysqli_set_charset.asp
save your source code in UTF-8 No BOM
In my PHP page, product descriptions are showing up with the infamous £ issue.
Aha! I thought - I must remember to move the entire website over to UTF-8. So I:
Specified in both the mysql_set_charset('utf8'); and the array(PDO::MYSQL_ATTR_INIT_COMMAND => "SET NAMES utf8") command. I'm migrating the website over to PDO - so still in a transitional process. The actual part of the website where the product description is saved and read both use the new PDO method.
Specified ini_set('default_charset', 'utf-8'); at the start of my header.php page.
Specified <meta http-equiv="Content-Type" content="text/html;charset=utf-8" /> in the head tag of the HTML page. I verified that there were no caching issues by checking that it appears.
Changed the collation of the products table to utf8_general_ci.
But the error still occurs. It even occurs on new products and when I edit the product to remove and re-add the pound sign. Is there anything I'm missing?
I found the problem. I was throwing htmlentities() onto the strings for security when I was echoing them to the browser. I should have thrown:
htmlentities($string, ENT_COMPAT | ENT_HTML401, "UTF-8"); to rectify this.
There are 2 more things
set header("Content-Type: text/html; charset=utf-8");
make sure your file encoding is UTF-8 (eg: open file with notepad and save it as UTF-8 ENCODING)
Recently I switched hosting from one provider to the other and I have problems displaying Cyrillic characters. The characters which are read from the database are displayed correctly, but characters which are hardcoded in the php file aren't (they are displayed as question marks).
The files which contain the php source code are saved in utf-8 form. Help anybody?
Try placing a meta tag indicating the encoding in the head section:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
My PHP module was exactly with same problem all text was like "?????????????????"
And my code was written with Notepad++. I found the solution for the problem. The problem wasn't in the header charset or meta tag because the browser actually knows that it is the UTF-8 charset. I tried all encodings from the browser and the result was the same so I knew the problem is somewhere else, not in the browser character encoding at all.
I just opened the PHP module with Notepad++ and selected all code. After that in the Encoding menu I selected "Convert to UTF-8." After uploading to the server, everything worked like a charm.
put this after connecting database:
mysql_query("SET NAMES UTF8");
for mysqli
mysqli_query($connecDB,"SET NAMES UTF8");
and in the header of the page:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
The problem seems quite strange.
What's the form of these question marks? Is it black diamonds with questions or just plain question marks?
First of all double check if your files are really utf-8 encoded.
Try to add this header to your code (above all output)
header('Content-Type: text/html; charset=utf-8');
But I doubt it would help, as your database text already looks good.
Do you have any SET NAMES queries in your code? What charset it is set?
It had something to do with the encoding of the php files. The files were created using Windows Notepad and saved with utf-8 encoding.
When I used Notepad2 to open the files, the encoding of the files was "utf-8 with signature". When I changed encoding to "utf-8", the text displayed correctly.
The reason for your problem is often accidental re-encoding the script files by a programmer's editor. It isn't a good practice to hardcode strings which rely on encoding in your php files.
Try switching your browser's encoding to find what encoding is used for hardcoded text, it might help you address the issue. Also make sure to send proper http headers for each page:
header('Content-Type: text/html; charset=utf-8');
Optionaly you can insert meta tag in you HTML:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
For me, the line that made the difference is the following:
$mysqli->set_charset("utf8")
Straight from the PHP documentation page.
The server was returning latin, after setting the charset to utf8 now works fine.
I have been fighting with this exact same problem as I'm trying to add a bit of french/german internationalization to a few controls on a widget.
Characters with accents that are stored in my db print fine as UTF-8. However, characters that are hardcoded into arrays in PHP files either display as the black diamond with a question mark inside or the little square box.
I've tried encoding/decoding the hardcoded strings from my php file every which way, but couldn't get the characters to display properly.
Since I have such a finite set of characters and am working strictly with HTML, I just added a bit of functionality to my intl class to substitute the characters for html entities.
I have these properties.
static $accentEntities = array('á' => 'á',
'É' => 'É',
'é' => 'é',
'í' => 'í',
'û' => 'û',
'ü' => 'ü');
static $accents = array();
static $entities = array();
I setup some my replacement arrays in my constructor...
foreach (self::$accentEntities as $char => $entity) {
self::$accents[] = $char;
self::$entities[] = $entity;
}
And then when I need one of my hardcoded strings in my class I just return it like so...
return str_replace(self::$accents,self::$entities,$str);
It's a totally ghetto solution... but for now, it works. I'd definitely like to hear the correct way to display accents/special characters that are hardcoded into a PHP file.