I have data with accents in my database. like image below
when I want to display the data with my controller it gives me this
Here is the code
header('Content-Type: text/html; charset=utf-8');
$icozim=$this->Icozim->find('all');
debug($icozim,0,0);
when I run my function I have this
How can I solve this problem?
sql for my table is
CREATE TABLE `icozims` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`synonymes` MEDIUMTEXT NOT NULL,
PRIMARY KEY (`id`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB;
In your APP/Config/database.php look for the line:
// 'encoding' => 'utf8',
and uncomment it.
There are several things that you have to cover. All of these Encode has to be the same (assigned properly). then you can retrieve symbols or special characters properly.
Encode that you are using for Database
At the same time Encode you are using for Table
Encode of your HTML header
Encode of php code when ever you are retrieving/ printing / saving your data.
Also there are some functions that you can play around which does that for you.
e.g. mb_convert_encoding($value, 'UTF-8', 'HTML-ENTITIES')
If you are using any kind of frameworks you might need to set the Encode in core class
If your data has been saved with this specific symbol then you will need to edit all of them or write a function to convert them to a symbol or character you want to show. I remember I had the same problem with one of my project with wordpress which we removed all manually but after that I find an article that there was a plugin which sort out that problem automatically for you.
Related
I have the following serialized array stored in a MySQL longblob data field:
a:1:{s:10:"attributes";a:1:{s:13:"Ticket Holder";a:1:{i:0;s:8:"Joe Blow";}}}
In PHP, when I query the field, unserialize it, and print it out, the following empty array is printed:
Array
(
)
This is the table create statement:
CREATE TABLE `order` (
`state` varchar(255) CHARACTER SET ascii DEFAULT NULL,
`data` longblob,
`created` int(11) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
Your serialized data is invalid. You must never mainpulate serialized data manually.
Strings are serialized as:
s:<i>:"<s>";
where <i> is an integer representing the string length of <s>, and <s> is the string value.
So in this case valid data is :
a:1:{s:10:"attributes";a:1:{s:13:"Ticket Holder";a:1:{i:0;s:8:"Joe Blow";}}}
Joe Blow string length is 8 but in your serialized strings is defined 13.
See : Structure of a Serialized PHP string
Use base64_encode
The core problem comes from the binary encoding done by the database and the extra bytes it adds to the value stored. This "corrupts" the data that is being fetched from the database and causes unserialize to fail. To help mitigate this problem, you can use base64_encode and base64_decode which will allow you to insert and extract the data painlessly.
// data is encoded before insert
$dataToInsert = base64_encode(serialize($myArray));
// decode the data before unserializing it
$dataToRead = unserialize(base64_decode($longblob));
Converting the column to UTF8 in the SELECT statement and then unserializing it.
CONVERT (data USING utf8)
I'm not sure why this is necessary. I'm guessing it has something to do with the utf8mb4_general_ci collation.
Thank you for all of your help guys!
I feel like this should be a simple fix, but the answer eludes me. Ive tried googling, and searching here... to no avail.
i have a long text stored in my database.
The collation is latin1_swedish_ci"
and when I see it in the database, it is stored correctly. For example:
string= Sally was walking one day and saw Tom. Tom said "Hi, Sally!" Sally's response was "Hi, Tom."
every " or ' shows up as a white question mark on a black diamond.
I want to
$result=mysqli_query($db,"SELECT string FROM Table WHERE 1")
while($row = mysqli_fetch_assoc($result)){
echo $row['string'];
}
and have all of the characters show up.
can anyone help?
You may be able to fix yours with this example.
SQL:
CREATE TABLE `test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`string` longtext,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1;
INSERT INTO `test` VALUES ('1', 'Sally was walking one day and saw Tom. Tom said \"Hi, Sally!\" Sally\'s response was \"Hi, Tom.\"');
PHP:
$query = "SELECT id, string FROM test WHERE id = 1";
if ($result = mysqli_query($link, $query)) {
while ($row = mysqli_fetch_assoc($result)) {
echo $row['string'];
}
mysqli_free_result($result);
}
mysqli_close($link);
A single question mark is one kind of symptom; a string of question marks is another; and black diamond with a question mark is yet another problem. Sweep the slate clean... Get rid of all encoding functions and...
Have the data in the client be utf8-encoded, and
Establish that the connection is CHARACTER SET utf8, and
Have the tables/columns be CHARACTER SET utf8, and
On html pages, use <meta charset=UTF-8>.
After doing some more research, I found a post that didn't come up before.
This solution to a similar problem was offered by Emil H:
MySQL performs character set conversions on the fly to something
called the connection charset. You can specify this charset using the
sql statement
SET NAMES utf8 or use a specific API function such as
mysql_set_charset():
mysql_set_charset("utf8", $conn); If this is done correctly there's no
need to use functions such as utf8_encode() and utf8_decode().
You also have to make sure that the browser uses the same encoding.
This is usually done using a simple header:
header('Content-type: text/html;charset=utf-8'); (Note that the
charset is called utf-8 in the browser but utf8 in MySQL.)
In most cases the connection charset and web charset are the only
things that you need to keep track of, so if it still doesn't work
there's probably something else your doing wrong. Try experimenting
with it a bit, it usually takes a while to fully understand.
shareedit edited Mar 25 '09 at 12:17 answered Mar 25 '09 at 11:52
Emil H 28k75778
after reading that, I looked up the php code she/he was talking about and found that there is an sqli equivilant, with different syntax.
(PHP 5 >= 5.0.5, PHP 7)
mysqli::set_charset -- mysqli_set_charset — Sets the default client character set
Description ¶
Object oriented style
bool mysqli::set_charset ( string $charset )
Procedural style
bool mysqli_set_charset ( mysqli $link , string $charset )
Sets the default character set to be used when sending data from and to the database server.
I hope this helps anyone having the problem I had.
Updated Question :
I had one old script with mysql 5.5.46 . i was storing my text in one columnt with blob type. my text was persian like this : سلام خوبی جه خبر .
after some month's i changed blob column to longblob column with phpmyadmin with gui(without any converting query).
everything work's correctly but i get backup from my mysql and restored this db after 2 years. now it's not show my persian characters correctly.english text is ok but persian text's is going someting like the hex i mentioned in my question.
I need text's stored in page_about and page_contact columns in this table:
Creating Table :
mysql_query("CREATE TABLE `".$prefix."ProFolio_info` (
`id` int(5) NOT NULL auto_increment,
`page_about` blob NOT NULL,
`page_contact` blob NOT NULL,
PRIMARY KEY (`id`)
) TYPE=MyISAM AUTO_INCREMENT=1") or die(mysql_error());
Updating Data :
$info_query = mysql_query("SELECT * FROM ".$prefix."ProFolio_info ORDER BY id DESC LIMIT 0,10");
while($info_row = mysql_fetch_array($info_query)){
$about_page = html_entity_decode($info_row['page_about']);
$contact_page = html_entity_decode($info_row['page_contact']);
if(isset($_POST['change_settings']) && $LOGGEDIN == 'yes'){
$new_aboutpage = clean_page($_POST['about_page']);
$new_contactpage = clean_page($_POST['contact_page']);
if($about_page != $new_aboutpage){
mysql_query("UPDATE ".$prefix."ProFolio_info SET page_about = '$new_aboutpage' WHERE id = '$info_id'");
}
if($contact_page != $new_contactpage){
mysql_query("UPDATE ".$prefix."ProFolio_info SET page_contact = '$new_contactpage' WHERE id = '$info_id'");
}
get texts from DB :
<textarea name="contact_page"><? echo str_replace('<br />', '', $contact_page); ?></textarea>
</div>
i tested some query like cast and convert and convert column to longtest but result is the same and i have wrong chracters.
I think it stored with latin1 collation but I select column with utf-8
HEX
BC28620264F736C6173683B26756D6C3B264F736C6173683B26736563743B265567726176653BC284264F736C6173683B26736563743B20265561637574653B266D6163723B265567726176653BC281264F736C6173683B266F7264663B265567726176653BC28520264F736C6173683B26736563743B2655636972633BC28C265567726176653BC286264F736C6173683B266E6F743B264F736C6173683B26736563743B20265567726176653BC286265567726176653BC2852655636972633BC28C265561637574653B266D6163723B265567726176653BC286264F736C6173683B266E6F743B265567726176653BC28720264F736C6173683B26736563743B265567726176653BC285264F736C6173683B26736563743B20265567726176653BC285264F736C6173683B26737570333B265567726176653BC286265561637574653B26636F70793B265567726176653BC28720265561637574653B266D6163723B265567726176653B
i uploaded my script in github
also uploaded script in my server and sending text from script but data going to db correctly ! i think it's just because of my backup of database.
you can check my ready script from here and view my text's from kave note's menu;
There are 4 places to "say" utf8:
The data in the client must be utf8-encoded. (It probably was.)
SET NAMES utf8 or equivalent. (You took care of that in new PDO.)
CHARACTER SET utf8 on the table or column declaration. Please provide `SHOW CREATE TABLE, but do not try to fix it if it says latin1.
On the html page, <meta ... charset=UTF-8 ...>
Please provide SELECT col, HEX(col) FROM tbl WHERE ... so we can see whether the data was messed up in the table. There two possible fixes for the data; we need to see the hex to know which fix to apply.
I have too got the same problem the main reason is windows uses different kind of character encoding for different characterset. I have found the package convert character set. It has solved my problem and it may solve your too.
Once again I'm having problems with saving special characters into a database. After lots of searchs I still could not find solution so I am starting a new thread.
I have MySQL DB using UTF-8 character set and PHP application that reads data from XML files into DB. Earlier I had problems with estonian characters, which I managed to solve. For example & scaron; (š) is in XML as html entity & eth; and it is converted in PHP to & #353;. Earlier in PHP script I run mysql query "SET NAMES utf8". š saves into DB correctly.
Now I'm fighting with lithuanian characters, for example ų (& #371), which is as numeric entity, & #371;, in XML file. I am not doing any conversion for this in PHP since I assume that when & eth; converted to & #353; works with scaron, shouldn't & #371; save into DB as ų without PHP conversion? After save that appears in DB as question mark and if I try to use mb_convert_encoding() or html_entity_decode() result is ų.
Any advice?
You simple should make sure your table has correct encoding and run SET names just after connection.
I've prepared simple test. Try to run it to make sure everything works fine.
1) Create database testencoding and import the following code to it
CREATE TABLE IF NOT EXISTS `sample` (
`id` int(11) NOT NULL,
`value` varchar(255) COLLATE utf8_unicode_ci NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1 ;
--
ALTER TABLE `sample`
ADD PRIMARY KEY (`id`);
2) Create simple PHP script with following content and run it:
<?php
header('Content-Type: text/html; charset=utf-8');
mb_internal_encoding('utf-8');
$subjectvalue='ų ų';
$link = mysqli_connect("localhost","root","","testencoding");
mysqli_query($link,"SET NAMES 'utf8'");
mysqli_query($link,"INSERT INTO sample(`value`) VALUES('".mysqli_real_escape_string($link,$subjectvalue)."')");
$result = mysqli_query($link, "SELECT * FROM sample");
echo "<br /><br />Data from database<br /><br />";
while ($data = mysqli_fetch_assoc($result)) {
echo $data['id'].' '.$data['value']."<br />";
}
3) On my PC all results are as expected:
As output from PHP file I have:
Data from database
1 ų ų
In phpMyadmin I have:
ų ų
So everything works fine. Try it and compare with my results
I'm doing a project with zend framework and I'm pulling data from a utf-8 database. The project is utf-8 as well.
In a form, I have a select element displaying a list of countries. The problem is:
In french or spanish, some countries are not displayed.
After doing a var_dump() of my country list, I saw that those were the countries with special characters. Accented ones.
in the var_dump I could see the character represented as a ? in a diamond. I tried changing the encoding to iso-8859-1 and I could see the var_dump result with the special characters just fine.
How come data coming from a utf-8 database are displaying in iso-8859-1!
Can I store iso-8859-1 character set in a utf-8 table in mysql without problem? Shouldn't it display messed up characters?
confused.
--
delimiter $$
CREATE TABLE `geo_Country` (
`CountryID` int(10) NOT NULL,
`CountryName` varchar(45) NOT NULL,
`CountryCompleteName` varchar(45) NOT NULL,
`Nationality` varchar(45) NOT NULL,
`CreationDate` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`Status` tinyint(1) NOT NULL DEFAULT '1',
`LanguageCode` char(2) NOT NULL,
`ZoneID` int(10) NOT NULL,
PRIMARY KEY (`CountryID`,`LanguageCode`),
KEY `fk_geo_Country_web_Language1` (`LanguageCode`),
KEY `fk_geo_Country_geo_Zone` (`ZoneID`),
KEY `idx_CountryName` (`CountryName`)
CONSTRAINT `fk_geo_Country_geo_Zone` FOREIGN KEY (`ZoneID`) REFERENCES `geo_Zone` (`ZoneID`) ON DELETE NO ACTION ON UPDATE NO ACTION,
CONSTRAINT `fk_geo_Country_web_Language1` FOREIGN KEY (`LanguageCode`) REFERENCES `web_Language` (`LanguageCode`) ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB DEFAULT CHARSET=utf8$$
The thing to remember with UTF-8 is this:
Everything in your entire application needs to be UTF-8!
For a normal PHP/MySQL web application (a form, posting to a database), you need to check if:
Your database connection uses UTF-8 (execute this query right after your connection is set up: SET NAMES UTF8;)
Your PHP code uses UTF-8. That means no using character set translation/encoding functions (no need to when everything is UTF-8).
Your HTML output is UTF-8, by either sending a Content-Type: text/html; charset=utf8 header, of using a <meta charset="utf8"> tag (for HTML5, for other HTML variants, use <meta http-equiv="Content-Type" content="text/html; charset=utf8">)
In your case of var_dump'ing, there is just some plain text that is sent to the browser, without any mention of a character set. Looking at rule #3, this means your browser is displaying this in a different character set, presumably latin1, thus giving you the diamonds/question marks/blocks.
If you need to check if your data is stored properly, use a database client like PHPMyAdmin to view the record. This way you're viewing the content as UTF-8 (NOTE: this is a setting in PMA, so check if it is not set to a different charset!).
On a side note, set the collation of your databases' text columns to utf8_general_ci, this is not used for storing, but for sorting. So this isn't related to your problem, but it's a good practice to do so.
When connecting to database you should set up cleint encoding.
for Zend_Db it seems should be like this (notice 'driver_options'):
$params = array(
'host' => 'localhost',
'username' => 'username',
'password' => 'password',
'dbname' => 'dbname',
'driver_options' => array(PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES UTF8;');
);
for the application.ini
resources.db.params.charset = utf8
as a last resort you could just run this query SET NAMES UTF8 manually just like any other query.