Updated Question :
I had one old script with mysql 5.5.46 . i was storing my text in one columnt with blob type. my text was persian like this : سلام خوبی جه خبر .
after some month's i changed blob column to longblob column with phpmyadmin with gui(without any converting query).
everything work's correctly but i get backup from my mysql and restored this db after 2 years. now it's not show my persian characters correctly.english text is ok but persian text's is going someting like the hex i mentioned in my question.
I need text's stored in page_about and page_contact columns in this table:
Creating Table :
mysql_query("CREATE TABLE `".$prefix."ProFolio_info` (
`id` int(5) NOT NULL auto_increment,
`page_about` blob NOT NULL,
`page_contact` blob NOT NULL,
PRIMARY KEY (`id`)
) TYPE=MyISAM AUTO_INCREMENT=1") or die(mysql_error());
Updating Data :
$info_query = mysql_query("SELECT * FROM ".$prefix."ProFolio_info ORDER BY id DESC LIMIT 0,10");
while($info_row = mysql_fetch_array($info_query)){
$about_page = html_entity_decode($info_row['page_about']);
$contact_page = html_entity_decode($info_row['page_contact']);
if(isset($_POST['change_settings']) && $LOGGEDIN == 'yes'){
$new_aboutpage = clean_page($_POST['about_page']);
$new_contactpage = clean_page($_POST['contact_page']);
if($about_page != $new_aboutpage){
mysql_query("UPDATE ".$prefix."ProFolio_info SET page_about = '$new_aboutpage' WHERE id = '$info_id'");
}
if($contact_page != $new_contactpage){
mysql_query("UPDATE ".$prefix."ProFolio_info SET page_contact = '$new_contactpage' WHERE id = '$info_id'");
}
get texts from DB :
<textarea name="contact_page"><? echo str_replace('<br />', '', $contact_page); ?></textarea>
</div>
i tested some query like cast and convert and convert column to longtest but result is the same and i have wrong chracters.
I think it stored with latin1 collation but I select column with utf-8
HEX
BC28620264F736C6173683B26756D6C3B264F736C6173683B26736563743B265567726176653BC284264F736C6173683B26736563743B20265561637574653B266D6163723B265567726176653BC281264F736C6173683B266F7264663B265567726176653BC28520264F736C6173683B26736563743B2655636972633BC28C265567726176653BC286264F736C6173683B266E6F743B264F736C6173683B26736563743B20265567726176653BC286265567726176653BC2852655636972633BC28C265561637574653B266D6163723B265567726176653BC286264F736C6173683B266E6F743B265567726176653BC28720264F736C6173683B26736563743B265567726176653BC285264F736C6173683B26736563743B20265567726176653BC285264F736C6173683B26737570333B265567726176653BC286265561637574653B26636F70793B265567726176653BC28720265561637574653B266D6163723B265567726176653B
i uploaded my script in github
also uploaded script in my server and sending text from script but data going to db correctly ! i think it's just because of my backup of database.
you can check my ready script from here and view my text's from kave note's menu;
There are 4 places to "say" utf8:
The data in the client must be utf8-encoded. (It probably was.)
SET NAMES utf8 or equivalent. (You took care of that in new PDO.)
CHARACTER SET utf8 on the table or column declaration. Please provide `SHOW CREATE TABLE, but do not try to fix it if it says latin1.
On the html page, <meta ... charset=UTF-8 ...>
Please provide SELECT col, HEX(col) FROM tbl WHERE ... so we can see whether the data was messed up in the table. There two possible fixes for the data; we need to see the hex to know which fix to apply.
I have too got the same problem the main reason is windows uses different kind of character encoding for different characterset. I have found the package convert character set. It has solved my problem and it may solve your too.
Related
I feel like this should be a simple fix, but the answer eludes me. Ive tried googling, and searching here... to no avail.
i have a long text stored in my database.
The collation is latin1_swedish_ci"
and when I see it in the database, it is stored correctly. For example:
string= Sally was walking one day and saw Tom. Tom said "Hi, Sally!" Sally's response was "Hi, Tom."
every " or ' shows up as a white question mark on a black diamond.
I want to
$result=mysqli_query($db,"SELECT string FROM Table WHERE 1")
while($row = mysqli_fetch_assoc($result)){
echo $row['string'];
}
and have all of the characters show up.
can anyone help?
You may be able to fix yours with this example.
SQL:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`string` longtext,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
INSERT INTO `test` VALUES ('1', 'Sally was walking one day and saw Tom. Tom said \"Hi, Sally!\" Sally\'s response was \"Hi, Tom.\"');
PHP:
$query = "SELECT id, string FROM test WHERE id = 1";
if ($result = mysqli_query($link, $query)) {
while ($row = mysqli_fetch_assoc($result)) {
echo $row['string'];
}
mysqli_free_result($result);
}
mysqli_close($link);
A single question mark is one kind of symptom; a string of question marks is another; and black diamond with a question mark is yet another problem. Sweep the slate clean... Get rid of all encoding functions and...
Have the data in the client be utf8-encoded, and
Establish that the connection is CHARACTER SET utf8, and
Have the tables/columns be CHARACTER SET utf8, and
On html pages, use <meta charset=UTF-8>.
After doing some more research, I found a post that didn't come up before.
This solution to a similar problem was offered by Emil H:
MySQL performs character set conversions on the fly to something
called the connection charset. You can specify this charset using the
sql statement
SET NAMES utf8 or use a specific API function such as
mysql_set_charset():
mysql_set_charset("utf8", $conn); If this is done correctly there's no
need to use functions such as utf8_encode() and utf8_decode().
You also have to make sure that the browser uses the same encoding.
This is usually done using a simple header:
header('Content-type: text/html;charset=utf-8'); (Note that the
charset is called utf-8 in the browser but utf8 in MySQL.)
In most cases the connection charset and web charset are the only
things that you need to keep track of, so if it still doesn't work
there's probably something else your doing wrong. Try experimenting
with it a bit, it usually takes a while to fully understand.
shareedit edited Mar 25 '09 at 12:17 answered Mar 25 '09 at 11:52
Emil H 28k75778
after reading that, I looked up the php code she/he was talking about and found that there is an sqli equivilant, with different syntax.
(PHP 5 >= 5.0.5, PHP 7)
mysqli::set_charset -- mysqli_set_charset — Sets the default client character set
Description ¶
Object oriented style
bool mysqli::set_charset ( string $charset )
Procedural style
bool mysqli_set_charset ( mysqli $link , string $charset )
Sets the default character set to be used when sending data from and to the database server.
I hope this helps anyone having the problem I had.
I have a question for a simple query. Lets say I have the table colors(color, code)
CREATE TABLE IF NOT EXISTS `colors` (
`color` varchar(20) NOT NULL,
`code` varchar(20) NOT NULL,
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
with data:
INSERT INTO colors VALUES ('blue', 'Conventional'), ('green', 'Realistic');
When I make a simple select query
SELECT * FROM colors WHERE code = 'Realistic';
I get the corresponding row.
Then, I translated the code column into greek. So I updated the colors table.
UPDATE colors set code = 'Συμβατικός' WHERE color = 'blue';
UPDATE colors set code = 'Ρεαλιστικός' WHERE color = 'green';
The table is succesfully updated. However, if I now make the query
SELECT * FROM colors WHERE code = 'Ρεαλιστικός';
I get no results!
Why could this happen? Is there any idea out there, because I can't figure it out.
Thanks in advance.
You've stepped into the weird and wonderful world of UTF-8.
First thing to know is this:
Setting your database collation to UTF-8 isn't enough.
You need to pay attention to a few more things. My bet is that you forgot or didn't know about some of these things to remember. Try them and see if it helps.
Modify your php.ini file to use UTF-8 as the default character set: default_charset = "utf-8";
Set UTF-8 as the default character set for all MySQL connections (yes, your connections use a charset as well)
On the MySQL side of things, modifications to the my.ini file may be required as follows:
[client]
default-character-set=UTF-8
[mysql]
default-character-set=UTF-8
[mysqld]
character-set-client-handshake = false #force encoding to uft8
character-set-server=UTF-8
collation-server=UTF-8_general_ci
[mysqld_safe]
default-character-set=UTF-8
Restart your MySQL after you've made these changes.
If the connecting client has no way to specify the encoding for its communication with MySQL, after the connection is established you may have to run the following command/query: set names UTF-8;
Once again I'm having problems with saving special characters into a database. After lots of searchs I still could not find solution so I am starting a new thread.
I have MySQL DB using UTF-8 character set and PHP application that reads data from XML files into DB. Earlier I had problems with estonian characters, which I managed to solve. For example & scaron; (š) is in XML as html entity & eth; and it is converted in PHP to & #353;. Earlier in PHP script I run mysql query "SET NAMES utf8". š saves into DB correctly.
Now I'm fighting with lithuanian characters, for example ų (& #371), which is as numeric entity, & #371;, in XML file. I am not doing any conversion for this in PHP since I assume that when & eth; converted to & #353; works with scaron, shouldn't & #371; save into DB as ų without PHP conversion? After save that appears in DB as question mark and if I try to use mb_convert_encoding() or html_entity_decode() result is ų.
Any advice?
You simple should make sure your table has correct encoding and run SET names just after connection.
I've prepared simple test. Try to run it to make sure everything works fine.
1) Create database testencoding and import the following code to it
CREATE TABLE IF NOT EXISTS `sample` (
`id` int(11) NOT NULL,
`value` varchar(255) COLLATE utf8_unicode_ci NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1 ;
--
ALTER TABLE `sample`
ADD PRIMARY KEY (`id`);
2) Create simple PHP script with following content and run it:
<?php
header('Content-Type: text/html; charset=utf-8');
mb_internal_encoding('utf-8');
$subjectvalue='ų ų';
$link = mysqli_connect("localhost","root","","testencoding");
mysqli_query($link,"SET NAMES 'utf8'");
mysqli_query($link,"INSERT INTO sample(`value`) VALUES('".mysqli_real_escape_string($link,$subjectvalue)."')");
$result = mysqli_query($link, "SELECT * FROM sample");
echo "<br /><br />Data from database<br /><br />";
while ($data = mysqli_fetch_assoc($result)) {
echo $data['id'].' '.$data['value']."<br />";
}
3) On my PC all results are as expected:
As output from PHP file I have:
Data from database
1 ų ų
In phpMyadmin I have:
ų ų
So everything works fine. Try it and compare with my results
I tried to insert Hebrew a text value into a column,
But it changes the value to Gibberish.
An example of that:
mssql_query ("UPDATE TABLE SET COLUMON = N'בדיקה'");
As you can assume, It changes the value of the column, But the value changed to ????? and if I try to do it from Query Analyser it works fine.
My column's collation is HEBREW_CI_AS. How can I fix this?
You need to specify collation preperty for the string in the INSERT statement you are using. Also the string you are inserting should be of UNICODE datatype - use N prefix for that.
INSERT INTO MEMB_INFO (User, Pass, Name) VALUES ('Joni', '123456', N'גוני דף' COLLATE HEBREW_CI_AS)
Check that PHP variable can handle unicode characters. Otherwise it will be PHP that turns your string into question marks.
You may check out SQL Server drivers for PHP.
And Unicode Character Properties from PHP doicumentation.
Some resources on PHP and unicode:
http://www.sitepoint.com/bringing-unicode-to-php-with-portable-utf8/
http://php.net/manual/en/function.utf8-encode.php
http://allseeing-i.com/How-to-setup-your-PHP-site-to-use-UTF8
http://www.yiiframework.com/wiki/16/how-to-set-up-unicode/
http://pageconfig.com/post/portable-utf8
I solve this problem if someone else has this problem here is my way to fix that:
Create a new database for this specific table or else tables for your web.
Set Hebrew_CI_AS as collation (everyone to what he created).
In your PHP code use mb_convert_encoding() function for SELECT and INSERT.
All I want is to SELECT an utf8_bin string stored in an table with 2 rows
1 id and 2 continut first is an int(20) NOT NULL AUTO INCREMENT and second is VARCHAR(2000) utf8_bin NOT NULL used for Romanian language inserts.
The data is stored correct whe i acces it from phpmyadmin, but on echo it returns strange characters instead romanian diacritics.
$sqlis = "SELECT continut FROM cantari WHERE id = {$id}";
$dbh->query($sqlis);
foreach ($dbh->query($sqlis) as $liniie);
$continut = $liniie['continut'];
This is the result: Cau?i r�uri ?i izvoare
$continut is my data from sql. I've setted meta on file as utf8 <meta charset="utf-8"> in header's content.
Can htaccess help me or css? or how to replace that elements with the normal Romanian diacritics?
Have you set the character set to UTF-8 on the Mysql connection?
For example:
$dbh->set_charset("utf8")
Without it, the data returned from MySQL will be translated to Cp1252. Any chars that don't fit in Cp1252 will be set to "?".
$sqlis = "SELECT continut FROM cantari WHERE id = {$id}; SET NAMES 'utf8' ";
So all we need is to specify in SQL server side that we will use utf8 characters.