I have installed Propel 1.6. I can create tables in MySQL with propel commands.
Below is my propel settings in file: runtime-config.xml
<propel>
<datasources default="myProject">
<datasource id="myProject">
<adapter>mysql</adapter>
<connection>
<dsn>mysql:host=localhost;dbname=myDBname</dsn>
<user>myUser</user>
<password>mypass</password>
**<charset>utf8</charset>
<collate>utf8_unicode_ci</collate>**
</connection>
</datasource>
</datasources>
</propel>
MySQL database and table User has collation utf8_unicode_ci (see photo below):
mySql collation screenshot
Ι create a new Patient object to test everything is ok, through the following code:
$pat = new Patient();
$pat->setEmail("tg#gmail.com");
$pat->setAddress("Η διεύθυνσή μου");
$pat->setAmka("555555555");
$pat->setBirthdate("1966-01-01");
$pat->setFirstname("Τοόνομάμου");
$pat->setLastname("τοεπώνυμόμου");
$pat->setPhone("2109999999");
$pat->setSex(1);
$pat->save();
I checked through debug mode in Netbeans and the object $pat contains the values in the correct format so i can read them.
After save(), in mysql the greek values are showing like this:
mySql values saved screenshot
I would like your help to solve this issue.
Thank you in advance.
Τοόνομάμου, when "Mojibaked", becomes Τοόνομάμου. Notice the pattern often has Î and a second character, like your screenshot. Apparently, latin1 was involved at some point.
Trouble with UTF-8 characters; what I see is not what I stored discusses Mojibake and its causes.
It may be that you have "double encoding", which that link also discusses.
If you choose to fix the data rather than start over, see http://mysql.rjweb.org/doc.php/charcoll#fixes_for_various_cases
Finally, i found a solution.
In MySQL, i checked my settings using the following command:
show variables like 'char%';
I had to replace character_set utf8 with utf8mb4.
Everything works perfect now!!
For more info, https://mathiasbynens.be/notes/mysql-utf8mb4
You must specify the charset in the propel connection DSN - in your runtime-config.xml file like this:
<dsn>mysql:host=localhost;dbname=myDBname;charset=UTF8</dsn>
https://github.com/propelorm/sfPropelORMPlugin/issues/74#issuecomment-2011350
Related
I always seen in phpmyadmin special characters encoded like:
میثم ابراهیمیجØعشق ØŒ سر ب راه ØŒ نبض ØŒ شبهای ØŒ غنچه ها ØŒ شکوه ØŒ همین امروز ØŒ عادت ت'
I always thought that was just a problem related to phpmyadmin since on my application all of them were displayed correctly.
Now I'm exporting this database and in my mysql dump I see exactly the characters above so seems that they are stored in this way on the database.
There is an easy way to dump the utf8 characters?
I already tried to follow those suggestion: I need help fixing Broken UTF8 encoding
but the only thing that let visualize the characters properly is print them on a web page and add on top:
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
The collation of the fields is utf8_unicode_ci.
The connection exploit:
$options = array(\PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8',);
Edit: #Álvaro-González Attemp to retrieve hexadecimal values
I enter in the database the symbol € then from phpmyadmin I select with: SELECT name, HEX(name) as hex_name FROM `items`
that's the result:
name €
hex_name C3A2E2809AC2AC
I will provide any further information on request
Thanks
You wanted €? But the hex is C3A2E2809AC2AC. That is a case of "double encoding" - when you stored the data.
See Trouble with utf8 characters; what I see is not what I stored , especially in the discussion about "double encoding".
The data is broken. You may be able to fix the data with a messy UPDATE. See http://mysql.rjweb.org/doc.php/charcoll#example_of_double_encoding .
Edit
Your original stuff looks like
CONVERT(BINARY(CONVERT('میثم اب' USING latin1)) USING utf8mb4)میثم اب -- mojibake to ut8,
I have a simple (custom) CMS accepting markdown and displaying it in in a web page. Works fine in php5.6 (using the ondrej/php5 ppa on ubuntu 15.10). Mysql collation set to utf8 everywhere.
Upgrade the server to php7.0 (ondrej/php) and it displays garbage characters. I tried migrating the relevant mysql tables and fields to utf8mb4 / utf8mb4_unicode_ci with no luck.
Downgrade to php5.6 and it all works fine.
I have a hunch it is some strange php setting I don't know about? php.ini default_collation=UTF-8. Couldn't find anything else that worked. phpMyAdmin shows garbage no matter what version of php or server settings, so it is not much help.
What could i try next?
Source text (copied from php5.6 rendered page)
아동 보호 정책에 대한 규정
This Code is part of the
Rendered output (from php7 and phpMyAdmin)
ì•„ë™ ë³´í˜¸ ì •ì±…ì— ëŒ€í•œ ê·œì •
This Code is part of the
Use this to change a table to utf8mb4:
ALTER TABLE tbl CONVERT TO CHARACTER SET utf8mb4 COLLATION utf8mb4_unicode_520_ci;
However, if the table was already messed up, then this won't fix it. Do the following to verify:
SELECT col, HEX(col) FROM tbl WHERE ...
For example, 아동 보호 정책에 대한 규정 will show a hex of EC9584 EB8F99 EBB3B4 ED98B8 ECA095 ECB185 EC9790 EB8C80 ED959C EAB79C ECA095. (Please ignore the spaces.)
For Korean text, you should see (mostly) groups of 3 hex bytes of the form Ewxxyy, where w is A or B or C or D, as shown in the example above. Hex 20 (only 1 byte) represents a space.
ì•„ë™ ë³´í˜¸ ì •ì±…ì— ëŒ€í•œ ê·œì • is the Mojibake for it. This implies that somewhere latin1 was erroneously involved, probably when you INSERTed the text. In that case, you will see something like C3AC E280A2 E2809E C3AB C28F E284A2 C3AB C2B3 C2B4 C3AD CB9C C2B8 ... -- mostly 2-byte Cwxx hex.
If you see that, an UPDATE of something like this will repair the data:
CONVERT(BINARY(CONVERT(CONVERT(col USING utf8mb4) USING latin1)) USING utf8mb4) (Edit: removed call to UNHEX.)
I have a MySQL db in utf8_general_ci.
And my sphinx.conf is like this:
source jobs
{
type = mysql
sql_sock = /var/run/mysqld/mysqld.sock
sql_query_pre = SET NAMES utf8
...
}
When I query "système" I would like sphinx to search for "système" & "systeme" in the DB.
AND when I query "systeme" I would like sphinx to search for "système" & "systeme" too.
What it does now is removing all the characters before the accents (including the accents themselves). So "système" becomes "me" and "dév" becomes "v"...
PS : I'm using the sphinxapi.php - which shouldn't be preferred over SphinxQL, I know, but it should still work with the api. And I use EXTENDED match mode.
You need to setup your charset_table to be able do this
http://sphinxsearch.com/docs/current.html#charsets
Alas there is no 'magic' config option to just magically work with all languages text, need to setup charset_table to deal with the langauge(s) you deal with.
Although this is pretty close:
http://sphinxsearch.com/forum/view.html?id=9312
(ie steals the hard work MySQL had done with collations and mimics it in charset_table)
this might look like a similar issues for utf8 and Arabic language with MySQL database but i searched for result and found none..
my database endocing is set to utf8_general_ci ,
i had my php paging to be encoded as ansi by default
the arabic language in database shows as : ãÌÑÈ
but i changed it to utf8 ,
if i add new input to database , the arabic language in database shows as : زين
i dont care how it show indatabase as long as it shows normally in php page ,
after changing the php page to utf8 , when adding input than retriving it , if show result as it should .
but the old data which was added before converting the page encoding to uft8 show like this : �����
i tried a lot of methods for fixis this like using iconv in ssh and php , utf8_decode() utf8_encode() .. and more but none worked .
so i was hoping that you have a solution for me here ?
update :: Main goal was solved by retrieving data from php page in old encoding ' windows-1256' than update it from ssh .
but one issue left ::
i have some text that was inserted as 'windows-1256' and other that was inserted as 'utf-8' so now the windows encoding was converted to utf-8 and works fine , but the original utf-8 was converted as well to something unreadable , using iconv in php, with old page encoding ..
so is there a way to check what encoding is original in order to convert or not ?
Try run query set name utf8 after create a DB connection, before run any other query.
Such as :
$dbh = new PDO('mysql:dbname='.DB_NAME.';host='.DB_HOST, DB_USER, DB_PASSWORD);
$dbh->exec('set names utf8');
In Cake, I have this issue with Finnish language not displaying properly. I have set utf encoding in config.php, charset output in default.ctp and also config in core.php
Is there a reason why it's not coming out properly?
To give you an idea the link is below:
http://www.likeslomakkeet.net/petitions/add
What if you re-import your data to database after changed your database.php and database collations? Try re-adding any commune with special characters like "Hämeenkyrö" and see how it looks like in database.
edit: You could also filter out all communes with "(lakkautettu)" because they no longer exists.
Did you also set the database connection to UTF-8 in database.php?
For MySQL, that would be:
'encoding' => 'utf8' // no hyphen