Can't figure out why is it happening, there is definitely something wrong with the enviroment.
I have db declared as
CREATE DATABASE `mydb` /*!40100 DEFAULT CHARACTER SET latin1 */
Then I have a table declared as
CREATE TABLE `myabstract_table` (
`key_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`varchar_field` varchar(128) NOT NULL
PRIMARY KEY (`key_id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
Then I have usual for this case php code:
function execSQL($conn, $sql, $values = false) {
try {
$stmt = $conn->prepare($sql);
if ($values)
foreach($values as $param=>$value)
$stmt->bindValue($param, $value);
if ( ! $stmt->execute() ) {
error_log ("PDO Error: ".json_encode($stmt->errorInfo()));
return false;
}
} catch (PDOException $e) {
error_log ("Exception: " . $e->getMessage());
return false;
}
return $stmt;
}
and after that the strings are saving (via INSERT statement) to db with question marks instead of non-ASCII symbols
PS. I perform set names 'utf8' before any sql request, all php stuff is cofigured for UTF-8, html document header contains meta tag with UTF-8 secified as charset;
the only thing I didn't try is to change default charset of schema itself (it's currently latin1) since I'm pretty sure it's not at issue. Or is it?
OK, I've found the reason. There was another bit I forgot to mention. It's stored procedure. This procedure was inserting rows not usual INSERT right from php.
I think, default schema charset affects procedure IN varchars parameter declared without explicit charset. I've finally done the following:
CREATE PROCEDURE `sp_myabstract_proc`(
`prm_key_id` int(10),
`prm_varchar_param` varchar(512) CHARACTER SET utf8 COLLATE utf8_general_ci
)
BEGIN
-- some insert logic here
END;
(CHARACTER SET utf8 option made the trick)
So in my case mysql engine was dealing with prm_varchar_param as with latin1 inherited from schema attribute.
You need to change the character set of the database to UTF-8 or a character set for Cyrillic characters.
Latin-1 can't encode Cyrillic characters and that's why they're appearing as question marks in the database.
edit: I just saw you're declaring the table with UTF-8 character set, so I'm stumped.
Related
$db_name = 'myDbName';
CREATE DATABASE IF NOT EXISTS `$db_name` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
The moment these 2 instructions are executed, the database is created, but its name is 'myDbName', quotes included.
If i remove the quotes in the first line, i'll get a PHP error, and if i remove the from the second one, i'll get a mySql error.
Is there a way to remove them/create a database with the correct name without directly accessing the DBMS?
I'd really prefer not to hardcode the name in the second line
Thanks in advance for the answers
You can use string concat and remove backticks
"CREATE DATABASE IF NOT EXISTS " . $db_name . " DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci";
you can also avoid concat and using var inside quote (this could be useful if you need backticks for allow reserved word)
"CREATE DATABASE IF NOT EXISTS '$db_name' DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci";
This question already has answers here:
UTF-8 all the way through
(13 answers)
Closed 8 years ago.
I am using PHP 5.5 and when I attempt to insert a UTF-8 character in the MySQL database PDO cuts it off at the first non-ASCII character.
I have set my connection to be:
(DB_TYPE.':host='.DB_HOST.';dbname='.DB_NAME.';charset=utf8', DB_USER, DB_PASS, array(PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC, PDO::ATTR_ERRMODE => PDO::ERRMODE_WARNING))
I have tried the SET NAMES that everyone posts, but that doesn’t work either because the problem is NOT on the MySQL side of things.
When I do an insert through phpMyAdmin and directly from the MySQL console, it works!
When I select the accented string with PDO, it works!
The problem is only on INSERT and UPDATE using PDO specifically!
Here is the SQL of the table. It is all in UTF-8 but maybe someone knows of a conflict between a setting and PDO
CREATE TABLE IF NOT EXISTS `mytable` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_lang` int(11) NOT NULL DEFAULT '2',
`id_tgroup_cat` int(11) NOT NULL,
`fieldfor` int(11) NOT NULL,
`colors` varchar(100) NOT NULL,
`text` text NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=34 ;
I have already tried to make text a varchar field and that did not change anything.
The actual insert in PHP:
$query = $this->db->prepare("UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
$query->execute(array($text, $colors, $id));
Where $text = "référence" (only saves the letter R in the database but without accents it saves everything) and $colors is an empty string for test purposes and $id is 2.
This is the key clue to me:
Where $text = "référence" (only saves the letter R in the database but
without accents it saves everything) and $colors is an empty string
for test purposes and $id is 2.
Sounds like it is a UTF-8 encoding issue. While the database is UTF-8 the whole chain from the code to the database—including the connection—should be UTF-8 clean.
How exactly does $this->db->prepare relate to the PHP connection to MySQL? A bit unclear from the code you have shown. But based on what you are showing, perhaps adjusting your query like this would help:
$query = $this->db->prepare("SET collation_connection = utf8_bin;
SET NAMES utf8;
UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
Or maybe this:
$this->db->exec("SET collation_connection = utf8_bin; SET NAMES utf8;");
$query = $this->db->prepare("UPDATE mytable
SET text = ?,
colors = ?
WHERE id = ?");
Note my forced-in addition of SET collation_connection = utf8_bin; as well as SET NAMES utf8;
In general you need to make sure your entire chain from the connection, to the database, to the tables is all UTF8 clean. I have a detailed answer to a similar question here.
But in your case, check the actual MySQL server my.cnf file. The following would set the whole chain to UTF-8:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
EDIT: And since the original poster indicates the data is coming from an HTML5 form, I also think checking the BOM (byte order mark) for the actual HTML5 file itself would help as well. It should be set to UTF8. More details on what a BOM is are over here. Specifically the accepted answer from Martin Code which explains:
The UTF-8 BOM is a sequence of bytes (EF BB BF) that allows the reader
to identify the file as an UTF-8 file.
I have a table in mysql DB which contains special character like Ø,Æ,etc. I cannot find these fields when i run a search with php. but when i run the same sql in phpmyadmin, i get results.
this is the table structure:
CREATE TABLE IF NOT EXISTS `clientinfo` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`adresse` varchar(160) NOT NULL,
`gatenavn` varchar(20) NOT NULL,
`husnr` varchar(20) NOT NULL,
`bokstav` varchar(2) NOT NULL,
`postnr` varchar(20) NOT NULL,
`poststed` varchar(60) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=398 ;
This is a sample query:
SELECT * FROM clientinfo WHERE gatenavn = 'EKRAVEIEN' AND husnr = '1' AND postnr = '2010' AND poststed = 'STRØMMEN'
when i run this query in phpmyadmin, i get result; but don't get when i run with php. I am using mysqli. need some help.
Tell your connection instance, to deliver UTF-8, before making queries. In MySqli you can call the function set_charset(), afterwards the connection object will deliver UTF-8.
Calling this function makes you independent of the database configuration, if necessary the connection will convert the returned data. Of course it is fastest if no conversion is necessary, so adjusting the configuration is a good thing too.
// tells the mysqli connection to deliver UTF-8 encoded strings.
$db = new mysqli($dbHost, $dbUser, $dbPassword, $dbName);
$db->set_charset('utf8');
Try setting :
SET NAMES utf8;
before your query in the same mysql session
If you have access to your MySQL configuration file, set these settings to my.cfg:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
Try using htmlspecialchars php function.
string htmlspecialchars ( string $string , int $flags = ENT_COMPAT | ENT_HTML401
, string $encoding = 'UTF-8' , bool $double_encode = true)
You can change the encoding info of your string. Eg KOI8-R for Russian symbols
This could be a issue with your php.ini file, your Apache Webserver, or charset of your HTML Document or your MySQL
Couple of things to to:
Always ensure that your charset is set to utf8 in your html meta tags
Set
default_charset = "utf-8";
in your php.ini file
And add
AddDefaultCharset UTF-8
to your httpd.conf if you are outputting unicode chars
This question already has an answer here:
json_encode problems with utf8 [closed]
(1 answer)
Closed 6 years ago.
I am writing to the database in the form of data from a form with jQuery json_encode.
However, data from the database will corrupt.
$db->query("SET NAMES utf8");
$kelime = array("Merhaba","Dünya");
$bilgi = json_encode($kelime);
$incelemeEkle = "
INSERT INTO incelemeRapor SET
bigData = '".$bilgi."'
";
$db->query($incelemeEkle);
Database Table Schema;
CREATE TABLE `incelemeRapor` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`bigData` text COLLATE utf8_unicode_ci,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=2 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
MySQL Inserted Example Data;
["Merhaba","Du00fcnya"]
Always escape your data before puting it in a SQL query:
$incelemeEkle = "
INSERT INTO incelemeRapor SET
bigData = '".mysql_real_escape_string($bilgi)."'
";
(added mysql_real_escape_string() call)
json_encode() encodes non-ascii characters with the \u<code-point> notation; so json_encode(array("Merhaba","Dünya")); returns ["Merhaba","D\u00fcnya"].
Then this string is embeded in a SQL query:
INSERT INTO incelemeRapor SET
bigData = '["Merhaba","D\u00fcnya"]'
There is no special meaning for the escape sequence \u, so MySQL just removes the \; and this results in ["Merhaba","Du00fcnya"] being stored in database.
So if you escape the string, the query becomes:
$incelemeEkle = "
INSERT INTO incelemeRapor SET
bigData = '["Merhaba","D\\u00fcnya"]'
";
And ["Merhaba","D\u00fcnya"] is stored in the database.
I tried with mysql_real_escape_string() but not worked for me (result to empty field in database).
So I looked here : http://php.net/manual/fr/json.constants.php and the flag JSON_UNESCAPED_UNICODE worked for me fine :
$json_data = json_encode($data,JSON_UNESCAPED_UNICODE);
JSON_UNESCAPED_UNICODE is available only since PHP 5.4.0 !
So in addition to ensuring that your database is using utf8_unicode_ci, you also want to make sure PHP is using the proper encoding. Typically I run the following two commands at the top of any function which is going to potentially have foreign characters within them. Even better is to run it as one of the first commands when your app starts:
mb_language('uni');
mb_internal_encoding('UTF-8');
Those two lines have saved me a ton of headaches!
Like user576875 says, you just need to correctly treat your string before inserting it into the database. mysql_real_escape_string() is one way to do that. Prepared statements are another way. This will also save you from the SQL injection security issue that you might be susceptible to if you write user input directly into SQL. Always use one of the above two methods.
Also, note that this has little to do with UTF8. JSON is ASCII safe, so as long as you use an ASCII like character set (utf8, iso-8859-1), the data will be inserted and stored correctly.
I would apply BASE64 encoding to the JSON string. This should work with nearly every php setting, database, database version and setting:
$values = array("Test" => 1, "the" => 2, "West" => 3);
$encoded = base64_encode(json_encode($values));
$decoded = json_decode(base64_decode($encoded), true);
When I'm trying to insert cyrillic text into MySQL database it inserts it like:
г???????????? ?? ????????
Рісѓрїр°ріс‹рї р° с‹рір°рї
So, I have two pages: registration.php and addUser.php. In each of them
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
Database consist of 11 tables, each table has collation: utf8_general_ci, type: MyISAM. Each field in every table has Collation: utf8_general_ci.
When I'm writing to database directly in phpMyAdmin and then show this data to web-page. In English and Russian - all OK.
But when I'm full my form with personal data on registration.php and then going to addUser.php - all cyrillic characters displayed like I wrote upper - on page and in database too.
function AddNewUser($Name, $Surname, $FatherName, $Email, $Password, $Phone, $DegreeID, $RankID,
$Organization, $Department, $Country, $City, $Address, $Job)
{
//fetch data from database for dropdown lists
//connect to db or die)
$db = mysql_connect($GLOBALS["gl_kussdbName"], $GLOBALS["gl_kussUserName"], $GLOBALS["gl_kussPassword"] ) or die ("Unable to connect");
//to prevenr ????? symbols in unicode - utf-8 coding
mysql_query("SET NAMES 'UTF8'");
//select database
mysql_select_db($GLOBALS["gl_kussDatabase"], $db);
$sql = "INSERT INTO UserDetails (
UserFirstName,
UserLastName,
UserFatherName,
UserEmail,
UserPassword,
UserPhone,
UserAcadDegreeID,
UserAcadRankID,
UserOrganization,
UserDepartment,
UserCountry,
UserCity,
UserAddress,
UserPosition)
VALUES(
'".$Name."',
'".$Surname."',
'".$FatherName."',
'".$Email."',
'".$Password."',
'".$Phone."',
'".$DegreeID."',
'".$RankID."',
'".$Organization."',
'".$Department."',
'".$Country."',
'".$City."',
'".$Address."',
'".$Job."'
);";
//execute SQL-query
$result = mysql_query($sql, $db);
if (!$result)
{
die('Invalid query: ' . mysql_error());
}
//close database = very inportant
mysql_close($db);
}
?>
There also such information in phpMyAdmin:
auto increment increment 1
auto increment offset 1
autocommit ON
automatic sp privileges ON
back log 50
basedir \usr\local\mysql-5.1\
big tables OFF
binlog cache size 32,768
binlog format STATEMENT
bulk insert buffer size 8,388,608
character set client utf8
(Global value) cp1251
character set connection utf8
(Global value) cp1251
character set database cp1251
character set filesystem binary
character set results utf8
(Global value) cp1251
character set server cp1251
character set system utf8
character sets dir \usr\local\mysql-5.1\share\charsets\
collation connection utf8_general_ci
(Global value) cp1251_general_ci
collation database cp1251_general_ci
collation server cp1251_general_ci
completion type 0
concurrent insert 1
So I need to properly show, save and select russian text from database. Thanx!
connect timeout 10
datadir \usr\local\mysql-5.1\data\
Try calling mysql_set_charset('utf8'); after connecting to the database. I think it's similar to executing a SET NAMES query, but since the PHP manual says using that function over a SET NAMES query is recommended, I'd try it.
Also, when you display your content, you could try echo htmlentities($string, ENT_COMPAT, 'UTF-8');
I store greek in tables created like ths:
CREATE TABLE `test` (
`test_id` SMALLINT UNSIGNED NOT NULL AUTO_INCREMENT,
`test_name` VARCHAR(30) NOT NULL,
PRIMARY KEY (`test_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT;
Or if table already created I guess you can change the charset it in the phpmyadmin interface. Maybe this helps.
Check your MySQL configuration and ensure that your encoding is defined correctly.
Add these lines to my.cnf or my.ini, which ever your installation uses.
These settings did the trick for me:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
character-set-server=utf8
I have tried multiple collations in phpMyAdmin as well as changing the charset of the page, which didn;t make sense but i was willing to try anything after two days of research. this command helped me: mysql_set_charset('utf8');
Collation on the column was set to koi8r_general_ci
Edit your structure field to set collation utf16_general_ci.
After that insert your data.