I have reworked a website and now it is xhtml valid etc and using UTF8. Everything is fine, but if anywhere in the Database is a Euro-char it is just displayed as a questionmark.
What would be the right way to fix this?
As output is done by Typo3 i cant change much about that.
Try executing these queries before the queries that fetch data:
SET NAMES utf8
SET CHARACTER SET utf8
This might be due to wrong database connection encoding
Lookup SET NAMES sql statement
$db_link = mysql_connect($host,$user,$passwd);
mysql_query("SET CHARACTER SET 'utf8';", $db_link);
mysql_query("SET NAMES 'utf8';", $db_link);
DON'T issue both statements!
Don't issue
SET NAMES utf8
SET CHARACTER SET utf8
one after another. It can cause trouble. I already had bad experience with SET CHARACTER SET utf8 right after SET NAMES utf8.
I recommend to issue only SET NAMES ...
MySQL docs has explanations why: http://dev.mysql.com/doc/refman/5.0/en/charset-connection.html
In short: SET NAMES ... sets connection's charset to the same as client ans result charset. while SET CHARACTER NAME... sets different connection's charset and collation.
Please read the doc and decide whichever it better in your case. Or even better make a test.
What charset (encoding, collation,...) are you using for the database column that contains the € sign?
The problem could be that your data stored in this column ist mixed up because the € sign is a somehow difficult beast when not using UTF-8 character encoding. The problem ist that the € sign is encoded as \xA4 when using ISO-8859-15 and as \x80 when using Windows-1252 (the common Western-European charset on Windows machines).
If your data inside the column is not encoded correctly MySQL won't be able to transcode it into UTF-8 correctly - even if you use SET NAMES utf8.
If you're sure you have the right character data (0x20AC), it could also be the fonts ont he client-side. If the font you are using does not handle that particular character, you'll just see some default question mark.
However, why not use the escape code €, which gives you: €
Cheers,
You could try something like
$value = iconv('ISO-8859-1', 'UTF-8//TRANSLIT', $value);
The "ISO-8859-1" part may be different depending on your MySQL table character encoding.
Related
I am working on a application where I need to store/show data containing many special characters. I have set database collation utf8. I have set collation of table utf8 and character set as utf8_unicode_ci. It is storing all special characters like é, â. But whenever a character ,€ comes it isn't stored as it is. Like whenever there is a word “attributed†it becomes âattributedâ. I am currently using Laravel 5.2 (PHP) .
What I have tried so far
I have set following in my code
iconv_set_encoding('internal_encoding', 'UTF-8');
mb_internal_encoding('UTF-8');
I have also tried
$value = array_map("utf8_encode", $array);
But this special character isn't getting stored as it is. Will any one let me know what should I do to get this special character saved as it is.
try setting your collation to "utf8_general_ci" in your mysql
Normally its no problem to store the € sign to a database field. Check if all your scripts are in a correct coding.
Set your table and all data in that table to utf8_general_ci then try to change your php file to UTF-8.
header('Content-Type: text/html; charset=utf-8');
And if you have incorrect data then use.
utf8_encode("test");
to encode your string to a correct UTF-8 string. If that isn't working i think your data or string your try to convert is not correct.
For being able to use UTF8 characters in your queries you must run a certain query setting this, just like this:
$db->query("SET NAMES UTF8");
But I've seen that you said you're using Laravel. I haven't been working on it, but I guess this's automatically set by the charset parameter written in config, just like in the screenshot of this question:
Laravel UTF-8 To Database
I'm having problems when inserting Norwegian letters (æøå) into a database. The charset coding of my document (using Notepad++) is set to UTF-8. The variable that is being inserted has the proper characters, but when it is inserted a word that should be shown as "spørsmål", is when inserted to the database, shown as "spørsmÃ¥l".
I'm using the following code to insert:
$newContent = htmlspecialchars($_POST['newContent']);
$stmt = $mysqli->prepare("UPDATE info SET frontpageText=?");
$stmt->bind_param('s', $newContent);
$stmt->execute();
$stmt->close();
And when connecting to the databse, I've tried using
$mysqli->set_charset("utf8");
$mysqli->query("SET NAMES 'utf8'");
I've also done the following:
The database collation is set to utf8_danish_ci
Using header('Content-type: text/html; charset=utf-8'); (PHP header)
Using meta charset="utf-8" (in the HTML-head)
The document itself is encoded in utf-8
Tried running SET NAMES utf8;
The worst part about this, is that I've actually had it working earlier, but I've appearantly broken something (which I don't know what).
Anyone has an idea what could be done to fix this issue?
EDIT: Problem has been solved. Apparently the table wasn't properly set to UTF-8. Ran this code in phpMyAdmin
ALTER TABLE table_name CHARSET = 'utf8';
The character set and the collation have separate encodings (one can be latin1 and the other can be utf8).
A character set is a set of symbols and encodings. A collation is a set of rules for comparing characters in a character set. Let's make the distinction clear with an example of an imaginary character set.
To diagnose the encoding of a table this query can be run, where ##TABLENAME## should be the actual table name.
SHOW CREATE TABLE ##TABLENAME##
If the encoding is not utf8 it can be altered with
alter table table_name charset = 'utf8';
Here's a thread on collation vs. character set, What does character set and collation mean exactly?.
You can convert all characters to html chars and save into database that way.
mb_convert_encoding($string, 'HTML-ENTITIES', 'utf-8');
All browsers will display it in the right way.
Make sure the default table charset or field charset is set to utf-8.
I know you've said that database charset is set correctly, but tables may interfere with that if they specify a different charset.
I've inherited a MySQL database which contains a field named Description of type text and collation of latin1_swedish_ci.
The problem with this field is it contains utf-8 data with some Unicode characters, e.g. character 733, etc. Sometimes this character also exists in the field represented as HTML encoded "˝" as well.
I'm trying to read the table and export the data to a CSV file and I need to represent this character as a double quote.
Reading the HTML encoded character is easy enough. However, it appears that the actual Unicode character is converted to utf-8 before I can do anything with it resulting in a "?".
How do I read in the Unicode character 733 (U+02DD), recognize it and convert it?
Here's a simplified (not tested) version of the code.
<?
$testconn=odbc_connect ("TESTLIB", "......", "......");
$query="SELECT Description FROM TestTable";
$rsWeb=mysql_query($query));
$WebRow=mysql_fetch_row($rsWeb));
$Desc = $WebRow[0];
$Desc = str_replace('"','""',$Desc);
fwrite($output,"\"".$Desc."\",\r\n");
%>
Also set charset to utf-8 when connecting to SQL server:
http://php.net/manual/en/mysqli.set-charset.php
$mysqli->set_charset("utf8");
I think your connection charset is not utf8, that's why chars are being converted to '?'.
Read this: http://dev.mysql.com/doc/refman/5.1/en/charset-connection.html
Post result for query:
show variables like 'char%';
You really should put only non-entity (Unicode) version in the database, and entity-decode the rest. However, when you want to use UTF-8 with MySQL, there are a few things to remember:
Your table column's collation should be utf8_bin or similar.
Your table's collation and database collation should also be utf8_bin just in case.
Your connection charset should be UTF8. Do this by executing the "SET NAMES utf8" query.
Also, if you're outputting a HTML page, that should have the UTF8 charset as well. If everything is correct, the UTF8 characters should come out fine.
Good luck!
I already set mySQL Collation as utf8_unicode_ci, if I manual insert chinese character in my database, it's successful work to displaying chinese character, but once i use my code, it was display this
ã€å¼µç‘žæŒ¯ã€æ±Ÿç¥¥ç¶�..
I had add this <meta http-equiv="Content-Type" content="text/html;charset=utf-8" ></meta> in my header, I had try utf8_encode ,but still the same problem happen.
thank you and hope you guy reply me soonest
Have you set the connection character set / collation? Execute this query immediately after creating a connection
SET NAMES 'utf8'
You have probably forgot to execute this query after connecting
SET NAMES utf8
Try it and you'll see
there 2 steps you should do
add a query with UTF8
$query = "SET NAMES 'utf8'";
mysql_query($query);
make sure your file is encoded as UTF8,
open your script with your favourite editor and save as UTF8
To ensure MySQL expects UTF-8 encoding by default from client connections, use the following query:
SET NAMES 'utf8'
In addition, make sure PHP interprets the string as UTF-8 string. Since PHP does not support multibyte characters, you must use a function to allow PHP to work with UTF-8 strings:
utf8_decode()
or something.
I need help with a character encoding problem that I want to sort once and for all. Here is an example of some content which I pull from a XML feed, insert into my database and then pull out.
As you can not see, a lot of special html characters get corrupted/broken.
How can I once and for all stop this? How am I able to support all types of characters, etc.?
I've tried literally every piece of coding I can find, it sometimes corrects it for most but still others are corrupted.
To absolutely once and for all make sure you will never have problems with encoding again:
Use UTF-8 everywhere and on everything!
That is (if you use mysql and php):
Set all the tables in your database to collation "utf8_general_ci" for example.
Once you establish the database connection, run the following SQL query: "SET NAMES 'utf8'"
Always make sure the settings of your editor are set to UTF-8 encoding.
Have the following meta tag in the section of your HTML documents:
<meta http-equiv="content-type" content="text/html; charset=utf-8">
And couple of bonus tips:
When you use PHP for string manipulation, use the multibyte functions.
You might check http://docs.kohanaphp.com/core/utf8 as well at some point.
OR:
You can just use one simple server side configuration file that takes care of all encoding stuff. In this case you wont need header and/or meta tags at all or php.ini file modification. Just add your wanted character set encoding to .htaccess file and put it into your www root. If you want to fiddle with character set strings and use your php code for that - thats another story. Database collation must ofcourse be correct.
Footnote: UTF-8 is not the encoding solution its an a solution. It doesn't matter what character set/encoding one is using as long as the used environment has been taking to consideration.
My favorite article about encodings from JoelOnSoftware: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets
It seems that an UTF-8 encoded text is interpreted with ISO 8859-1.
If you’re processing XML documents, you have to use the encoding given either in the charset parameter in HTTP header field Content-Type or in the encoding attribute in the XML declaration. If none of both is given, the XML specification declares UTF-8 or UTF-16 as the default character encoding and you have to use some detection.
It looks like the link you gave has data that is encoded in utf-8. (Follow that link, then change the encoding of your browser to utf-8).
I sounds like you are having problems with inserting and retrieving from your database. Make sure your database table has utf-8 set as the encoding.
After you connect to the database, but before you do any transactions, execute the following line which makes sure all database communication is in UTF-8:
mysql_query("SET character_set_results = 'utf8', character_set_client = 'utf8', character_set_connection = 'utf8', character_set_database = 'utf8', character_set_server = 'utf8'", $dbconn);
First off, make sure your database's character encoding is set to support UTF-8. Secondly, PHP's ICONV is going to be your friend. Finally, ensure that your response headers are sending the proper character encoding (again, UTF-8).
header('Content-type: text/html; charset=UTF-8') ;
/**
* Encodes HTML safely for UTF-8. Use instead of htmlentities.
*
* #param string $var
* #return string
*/
function html_encode($var)
{
return htmlentities($var, ENT_QUOTES, 'UTF-8');
}
Those two rescued me and I think it is now working. I'll come back if I continue to encounter problems. Should I store it in the DB, eg as "&" or as "&"?
Did you try utf8_encode() and utf8_decode()?
Which one you use will depend entirely on how your data is encoded, which you don't specify, but they are quite useful for this kind of cases.