I have to store the phonetics of words in a database in this format:
blow = \ˈblō\
But when I see in the database, it's stored like this:
blow = \?bl?\
There are ? instead of symbols you can see.
Please correct me as to what I am doing wrong.
Thanks
your database does not support all characters i would recommend to change it to something like this:
character set: utf8
collation: utf8_general_ci
right now you have a standard character set and thats why it changes to ?????
Related
I have one php form where i used to enter data to database(phpmyadmin), and i used SELECT query to display all values in database to view in php form.
Also i have another PHP file which i used to create JSON from the same db table.
Here when i enter foreign languages like "Experiența personală:" the value getting saved in DB is "ExperienÈ›a personală: " but when i use select query to display this in same php form it coming correctly "Experiența personală:". So the db is correct and now am using following php code to create JSON
<?php
$servername = "localhost";
$username = "root";
$password = "root";
$dbname = "aaps";
// Create connection
$con=mysqli_connect($servername,$username,$password,$dbname);
// Check connection
mysqli_set_charset($con, 'utf8');
//echo "connected";
$rslt=mysqli_query($con,"SELECT * FROM offers");
while($row=mysqli_fetch_assoc($rslt))
{
$taxi[] = array('code'=> $row["code"], 'name'=> $row["name"],'contact'=> $row["contact"], 'url'=> $row["url"], 'details'=> $row["details"]);
}
header("Content-type: application/json; charset=utf-8");
echo json_encode($taxi);
?>
and JSON looks like
[{"code":"CT1","name":"Experien\u00c8\u203aa personal\u00c4\u0192: ","contact":"4535623643","url":"images\/offers\/event-logo-8.jpg","details":"Experien\u00c8\u203aa personal\u00c4\u0192: jerhbehwgrh 234234 hjfhjerg#$%$#%#4"},{"code":"ewrw","name":"Experien\u00c8\u203aa personal\u00c4\u0192: ","contact":"ewfew","url":"","details":"eExperien\u00c8\u203aa personal\u00c4\u0192: Experien\u00c8\u203aa personal\u00c4\u0192: Experien\u00c8\u203aa personal\u00c4\u0192: "},{"code":"Experien\u00c8\u203aa personal\u00c4\u0192: ","name":"Experien\u00c8\u203aa personal\u00c4\u0192: ","contact":"","url":"","details":"Experien\u00c8\u203aa personal\u00c4\u0192: "}]
In this "\u00c8\u203aa" this is wrong it supposed to be "\u021b" (t).
So pho used to creating JSON making this issue.
But am unable to find exactly why its coming like this . please help
Avoid Unicode -- note the extra argument:
json_encode($s, JSON_UNESCAPED_UNICODE)
Don't use utf8_encode/decode.
ă turning into ă is Mojibake. It probably means that
The bytes you have in the client are correctly encoded in utf8 (good).
You connected with SET NAMES latin1 (or set_charset('latin1') or ...), probably by default. (It should have been utf8.)
The column in the tables may or may not have been CHARACTER SET utf8, but it should have been that.
If you need to fix for the data it takes a "2-step ALTER", something like
ALTER TABLE Tbl MODIFY COLUMN col VARBINARY(...) ...;
ALTER TABLE Tbl MODIFY COLUMN col VARCHAR(...) ... CHARACTER SET utf8 ...;
Before making any changes, do
SELECT col, HEX(col) FROM tbl WHERE ...
With that, ă should show hex of C483. If you see C384C692, you have "double-encoding", which is messier to fix.
Depending on the version of MySql in the database, it may not be using the full utf-8 set, as stated in the documentation:
The ucs2 and utf8 character sets do not support supplementary characters that lie outside the BMP. Characters outside the BMP compare as REPLACEMENT CHARACTER and convert to '?' when converted to a Unicode character set.
This, however, is not likely to be related to your problem. I would try a couple of different things and see if it solves your problem.
use SET NAMES utf-8
You can read more about that here
use utf8_encode() when inserting data to the database, and utf8_decode() when extracting. That way, you don't have to worry about MySql manipulating the unicode characters. Documentation
I am trying to save a string in database and get something like this
Период д

The string that I want to save : Период действия S...
The table encoding is: cp1251_general_ci
I don't know in which encoding the string is - I am getting it from an excel document.
I tried this, but it didn' help.
$nomer = iconv('UTF-8','Windows-1251', $str );
Is there a solution for this?
You haven't mentioned the database engine you're using, but try changing the table encoding to something like utf8_general_ci or utf8_unicode_ci.
I tried to insert Hebrew a text value into a column,
But it changes the value to Gibberish.
An example of that:
mssql_query ("UPDATE TABLE SET COLUMON = N'בדיקה'");
As you can assume, It changes the value of the column, But the value changed to ????? and if I try to do it from Query Analyser it works fine.
My column's collation is HEBREW_CI_AS. How can I fix this?
You need to specify collation preperty for the string in the INSERT statement you are using. Also the string you are inserting should be of UNICODE datatype - use N prefix for that.
INSERT INTO MEMB_INFO (User, Pass, Name) VALUES ('Joni', '123456', N'גוני דף' COLLATE HEBREW_CI_AS)
Check that PHP variable can handle unicode characters. Otherwise it will be PHP that turns your string into question marks.
You may check out SQL Server drivers for PHP.
And Unicode Character Properties from PHP doicumentation.
Some resources on PHP and unicode:
http://www.sitepoint.com/bringing-unicode-to-php-with-portable-utf8/
http://php.net/manual/en/function.utf8-encode.php
http://allseeing-i.com/How-to-setup-your-PHP-site-to-use-UTF8
http://www.yiiframework.com/wiki/16/how-to-set-up-unicode/
http://pageconfig.com/post/portable-utf8
I solve this problem if someone else has this problem here is my way to fix that:
Create a new database for this specific table or else tables for your web.
Set Hebrew_CI_AS as collation (everyone to what he created).
In your PHP code use mb_convert_encoding() function for SELECT and INSERT.
I generate a mysql query from a form with a free text search field.
Something like:
SELECT ... FROM ... WHERE 'something' LIKE '%SEARCH%'
All this works fine and returns the valid rows when the search does not contain any special characters, like the danish characters ÆØÅ.
When these letters ARE used, the query returns no results, all though when i take the generated query string and plug it into phpMyAdmin i get exacly the result i want.
Thanks
add this line of code in your connection file...
mysql_set_charset("utf8", $db);
it is better for you to encode your data to UTF-8 before you pass it into query...
I'm not using Danish but Czech but I think there are the same (at least by UTF8 implementation) - you must keep in mind used encoding (original server script, data tables self and also your database connection handler).
I think you have an encoding problem, maybe phpMyAdmin is using a different client encoding than your other client. SET NAMES 'encoding' should just do what you need, I think.
Also we can use PHP variable and convert it before the select operation (supposing data base is ISOO-8859-2) .
Example:
// word with special characters
$search='kötészeti';
// conversion to ISO
$search=iconv("UTF-8","ISO-8859-2", $search);
// create search condition
$condition="SELECT ... FROM ... WHERE 'something' LIKE '%$search%'";
// apply query
mysql_query($condition);
I have a DB - with the table articles.
I want to convert the title, and content field to utf8
now - all data looks like this: פורטל רעל × ×¤×ª×— רשמית!
I want it to become normal hebrew characters.
Thanks
The following MySQL function will return the correct utf8 string after double-encoding:
CONVERT(CAST(CONVERT(field USING latin1) AS BINARY) USING utf8)
It can be used with an UPDATE statement to correct the fields:
UPDATE tablename SET field = CONVERT(CAST(CONVERT(field USING latin1) AS BINARY) USING utf8);
if you need to convert the whole database , you can back it as databaseback.sql file then form your command line
iconv -f latain -t utf-8 < databaseback.sql > databaseback.utf8.sql
you can use the http://www.php.net/manual/en/function.iconv.php
to convert each row in php in case you don't have command line access
and lastly don't forget to convert the collation of each field in phpmyadmin , then you can resotre the utf8 back easily
update
if you got iconv is not recognized , it means that you don't have iconv installed
much more easier solution is :
Migrating MySQL Data to Unicode
http://daveyshafik.com/archives/166-migrating-mysql-data-to-unicode.html
You can make mysqldump from this database. Then download something like Notepad++, open dump file, convert it to UTF8, then replace through the file all encodings to utf-8 including the first SET NAMES operator.
If you make dump to file via phpMyAdmin (with default settings) use output file encoding ISO-8859-1 instead of UTF-8 as you can see by default.
You can write a little php script which does the conversion. See http://www.php.net/manual/en/function.mb-detect-encoding.php and http://php.net/manual/en/function.mb-convert-encoding.php This is how I did this.
And remember to use strict mode! http://www.php.net/manual/en/function.mb-detect-encoding.php#102510
In pseudocode it would be sth. like this:
str = getDataAsString()
if(!isUTF8(str)) {
str = convert2UTF8(str)
}
saveStr2DB()
try
ALTER TABLE `tablename` CHANGE `field_name` `field_name` VARCHAR( 200 ) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL