Relevent code:
$status = $db->run(
"INSERT INTO user_wall (accountID, fromID, text, datetime) VALUES (:toID, :fromID, :text, '" . time() . "')",
array(":toID" => $toID, ":fromID" => %accountID, ":text" => $text)
);
I am taking input text from javascript, throwing it in an AJAX call to handle it, which calls a function which includes these lines of code.
The text string in question is: "Türkçe Türkçe Türkçe!"
Upon investigating the database, the following value is saved "Türkçe Türkçe Türkçe!", which is double utf8_encode'd.
When viewing the text by SELECTing it from the database, I get "Türkçe Türkçe Türkçe!", which is how they should be saved in the database in the first place.
As far as I know, I am not encoding the data as it is being prepared by PDO...
Encoding is a b*tch. You need to make sure it is as you expect it to be in several places:
The HTML page with the Javascript. Set it to utf-8 with a meta tag like:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
The connection to your database. Execute the query set names 'utf8' after connecting to the database (and before any other queries).
Your database field. In MySQL it's called a collation set it to utf8_general_ci (ci stands for case-insensitive).
If you have these 3 your data should always be, and stay, utf-8 (unless you're doing encoding yourself).
For good measure, make sure your source code files are utf-8 as well. Windows typically defaults to iso.
Related
I always seen in phpmyadmin special characters encoded like:
میثم ابراهیمیجØعشق ØŒ سر ب راه ØŒ نبض ØŒ شبهای ØŒ غنچه ها ØŒ شکوه ØŒ همین امروز ØŒ عادت ت'
I always thought that was just a problem related to phpmyadmin since on my application all of them were displayed correctly.
Now I'm exporting this database and in my mysql dump I see exactly the characters above so seems that they are stored in this way on the database.
There is an easy way to dump the utf8 characters?
I already tried to follow those suggestion: I need help fixing Broken UTF8 encoding
but the only thing that let visualize the characters properly is print them on a web page and add on top:
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
The collation of the fields is utf8_unicode_ci.
The connection exploit:
$options = array(\PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8',);
Edit: #Álvaro-González Attemp to retrieve hexadecimal values
I enter in the database the symbol € then from phpmyadmin I select with: SELECT name, HEX(name) as hex_name FROM `items`
that's the result:
name €
hex_name C3A2E2809AC2AC
I will provide any further information on request
Thanks
You wanted €? But the hex is C3A2E2809AC2AC. That is a case of "double encoding" - when you stored the data.
See Trouble with utf8 characters; what I see is not what I stored , especially in the discussion about "double encoding".
The data is broken. You may be able to fix the data with a messy UPDATE. See http://mysql.rjweb.org/doc.php/charcoll#example_of_double_encoding .
Edit
Your original stuff looks like
CONVERT(BINARY(CONVERT('میثم اب' USING latin1)) USING utf8mb4)میثم اب -- mojibake to ut8,
I have a database issue, that I'm unable to understand. I'm from Denmark and have made a sign-up system in PHP and MySQL. Now... I have made two tables seperately.
One of the tables (let's call it table1) displays my beloved danish letters (æøå) just fine, when I'm querying them from the database through PHP. But when I go to phpMyAdmin, then the letters are displayed wierdly... For instance: It looks like this in phpMyAdmin:
Bjørn (which is Bjørn)
But the again, when I get them from the database with a mysql_query('SELECT * FROM $tablename'), then it is displayed as 'Bjørn' (as it should).
Now to the problem...
In the other table (let's call it table 2), then in phpMyAdmin 'Bjørn' is displayed as 'Bjørn' (what seems correct). But when I pull it into PHP with mysql_query('SELECT * FROM $tablename') then it is displayed as 'Bj?rn'. All of the letters 'æøå' is displayed as a '?'.
I tried doing a SHOW TABLE STATUS, and it shows that the Collation is the same.
In table1, then the variables are VARCHAR(255), while in table2, the variables are TEXT.
Both tables are created like this:
CREATE TABLE >>tablename<< ( bla bla bla ) CHARSET=UTF8
You should connecto mysql like this
$link = mysql_connect('localhost', 'user', 'password');
mysql_set_charset('utf8',$link);
And then try executing the query it fetches properly
The problem here is , you should specify the charset while you are connecting to DB also .
Even while storing also your inserts gets garballed if you dont set the charset in your connection to utf8 so do verify once whether you are setting like this while connecting to DB or not .
Hope this helps
Also last but not least , while displaying in the browser also you should set html headers while dumping the data from DB to browser like below
<?php
header('Content-type: text/html; charset=utf-8');
?>
I don't know what's wrong with phpmyadmin, but in my own programming I do the following things:
PHP (before anything else):
header("Content-Type: text/html; charset=utf-8");
HTML:
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
DB query:
SET CHARACTER SET utf8;
Or if using PDO my dsn looks like:
mysql:dbname=mydb;host=localhost;charset=utf8
I have two .php files, a form with a text field where you put the names you want to search in the database, and a results file that processes the post...
The names in the database will be searched using this query:
SELECT * FROM acw_papers_web web
INNER JOIN acw_papers_web_autores aut
ON web.id_paper_web = aut.id_paper_web
WHERE aut.nombre_autor_pw LIKE '%autorname%'
ORDER BY web.probabilidad DESC
The problem is that when I send the post, insted of sending lópez, it sends lópez...
How can I fix it... both .php files are utf-8 encoded...
Since you did not provide much information, I cannot exactly pinpoint the problem. But here are two possible solutions:
Set the correct charset in the <head> of both files:
<meta charset="UTF-8" /> for HTML5 or <meta http-eqiv="Content-Type" content="text/html; charset=UTF-8" /> for everything else
Set the character set of the database tables to UTF-8. In MySQL:
ALTER TABLE acw_papers_web CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE acw_papers_web_autores CHARACTER SET utf8 COLLATE utf8_general_ci;
That's all I could think of, for now.
You need to use
urlencode() - URL-encodes string
urldecode() - URL-decode
Functions to send special characters in the Post Request.
I have this code in drupal 6 to retrieve arabic values from Oracle databse:
<?php
session_start();
$conn=oci_connect('localhost','pass','IP....');
$stid=oci_parse($conn,"select arabic_name from arabic_names_table");
oci_execute($stid);
if($row-oci_fetch_array($stid,OCI_ASSOC+OCI_RETURNS_NULLS))
{
$name_ar=$row['arabic_name'];
}
?>
When values are retrieved from the DB or inserted to the DB they appears like this ???
Please note:
My Oracle database reads normal Arabic characters. From PL/SQL I can insert arabic values
I have installed the mbstring
I have the utf-8 encoding enabled.
How can I solve this problem?
From the oracle database, when you try to fetch data, normally you will get the character encoding will be the encoding type of the client installed in the system (the machine that you installed the php). This encoding will be the charset of the windows registry for the oracle client. (see HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\KEY_OraClient11g_home1), and the key is NLS_LANG. If you search the value of the above key, you will get something like ARABIC_UNITED ARAB EMIRATES.AR8MSWIN1256. Please note that the encoding type is AR8MSWIN1256. In the character map array this is mapped to windows-1256 ( windows-1256 => AR8MSWIN1256 ).
See this link http://websvn.projects.ez.no/wsvn/ezoracle/?op=comp&compare[]=%2Fstable#385&compare[]=%2Fstable#386.
That is, after you fetch the data from the database the char encoding will be windows-1256. Now if your web page is using utf-8 charset, you need to convert the string to utf-8. For this you can use iconv().
$win1256 = iconv('windows-1256', 'utf-8', $my_string); //$my_string -> windows-1256
echo $win1256; // Results the utf-8 format .
If you are still facing problem you check the charset in the page, it must be utf-8.
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
I think this will solve your problem.
Background:
There is a table, events; this table is formatted latin1. Individual columns in this table are set to utf8. The column we will cherry pick to discuss is 'title' which is one of the utf8 columns. The website is set for utf8 both via apache and the meta tag.
As a test, if I save décor or © into the title field and perform
select title, LENGTH(title) as len, CHAR_LENGTH(title) as chlen
from events where length(title) != char_length(title)
I will get décor or ©, 12, 10 back as a result; which is expected showing that the data has indeed been properly saved into my utf8 column.
However, upon echoing the title out to a page, it's mangeld into d�cor or � which makes no sense to me since, as mentioned before, the character encoding is set to utf-8 on the page.
Not sure if this final detail makes a difference but if I edit the page and resubmit the mangled text it turns into d%uFFFDcor or %uFFFD both in the database and when displayed to the page. Further submits cause no change.
Actual Question:
Does anyone have an idea as to what I may be doing wrong? :-P
Well, there's likely one of three problems.
1. Mysql's connection is not using UTF-8
This means that it's converted to another charset (likely Latin-1) before it hits PHP. I've found the best solution is to run the following queries:
SET CHARACTER SET = "utf8";
SET character_set_database = "utf8";
SET character_set_connection = "utf8";
SET character_set_server = "utf8";
2. The page rendered is not really set to UTF-8
Set both the Content-type header and the <meta> tag content types to UTF-8. Some browsers don't respect one or the other...
header ('Content-Type: text/html; charset=UTF-8');
echo '<meta http-equiv="content-type" content="text/html; charset=utf-8" />';
As noted in the comments, that's not the problem...
3. You're doing something to the string before echoing it
Most of PHP's string functions will not do well with UTF-8. If you're calling a normal function that doesn't accept a $charset parameter, the chances are that it won't work with utf-8 strings (such as str_replace). If it does have a $charset parameter (like htmlspecialchars, make sure that you set it.
echo htmlspecialchars($content, ENT_COMPAT, 'UTF-8');