When I try to insert some rare special characters (∨ ∧ → ↔ ∴), they get stored as question marks. But when I try to insert some more common special characters (© ® ¬ á) everything goes fine.
I've set every variable, database, table and connection I could find to UTF-8, but no luck yet. What am I missing? Thanks in advance!
Here is a minimal example:
<?php
header('content-type:text/html; charset=utf-8;');
$connection = mysql_connect('localhost', 'root', '');
mysql_select_db('test', $connection);
mysql_set_charset('utf8', $connection);
mysql_query('SET NAMES UTF8');
$special_character = '∴';
echo 'Encoding of the special character before insert: '.base64_encode($special_character).'<br />';
mysql_query('INSERT INTO table (column) VALUES ("'.$special_character.'")');
$query = mysql_query('SELECT column FROM table');
while ($ROW = mysql_fetch_assoc($query))
{
echo 'Encoding of the special character after retrieval: '.base64_encode($ROW['column']).'<br />';
echo 'Output: '.$ROW['column'].'<br />';
}
mysql_close($connection);
?>
The output of this script is:
Encoding of the special character before insert: 4oi0
Encoding of the special character after retrieval: Pw==
Output: ?
Could it be because I have the standard MySQL installation for Mac OSX, which doesn't have a my.cnf file? Maybe the defaults that come with this installation are not UTF-8? Anyone knows?
UPDATE: I have determined that the problem is in my local installation of MySQL, because it does not appear when I run the code in my web host. I still want to solve it though.
I had a similar issue in the eclipse java console in the past. The information in the database in my case was stored properly. What happened is that the console is not meant to display those characters.
To see them I had my application generate a file with the special characters in them and opened them in notepad++ with which you can change the character set to apply to the file. I had this issue with Russian characters.
What you storage in database it must content in charset table that you use
for example for UTF-8:
http://www.utf8-chartable.de/
You missed one. Your connection to mysql also has to be set to UTF8. After initially connecting to mysql, your first "query" should be "SET NAMES UTF8". If you are using a terminal program, run that "query" also.
It seems like the problem was in my local installation of MySQL. I was using an outdated version that came bundled with my Mac OSX 10.5. I downloaded and installed the more recent version of MySQL, and problem solved.
And that's why, boys and girls, it is always good to have the most recent version of the software you use.
Related
I have Sql Server version 10.50. And PHP version, 5.5.28.
I have a DB with collation Turkish_CI_AS and there is some tables also Turkish collaction.
My problem is, When i SELECT some records with PHP sqlsrv function then put them to while() loop, the returning results contains Turkish characters turning it self to English Characters, for example: The record in the table is İnsan but when i echo my while loop its gives me Insan.
I did some changes in php.ini file and set default encodings as Turkish:,
mssql.charset = "ISO-8859-9" and default_charset = "ISO-8859-9". But nothing changed.
I don't know is it sql related or php related problem, i hope you can give me some advices.
By the way, there is no problem on Sql server management studio. It gaves me right results when i run same query.
You can use utf-8 encoding as a standard if you work with any language, I always use Arabic with English and utf-8 give me full support. You should make your database encoding utf8_general_ci, then when you insert your Turkish data you must use this query in the main insertion data function in PHP:
mysql_query(" SET NAMES 'utf8'");
This will support any data with any language.
Ok, here is how i resolve my problem in my IIS server which is connecting to Mssql:
I go to
Control Panel > Region > Administrative > Change system locale...
And i changed my Current system locale... as Turkish
I use these codes for adding PHP/Mysqli Turkish Language Support.
// Türkçe Dil Desteği
date_default_timezone_set("Europe/Istanbul"); // For Default Date
setlocale(LC_ALL, 'tr_TR.UTF-8'); // For UTF-8 Characters
$db->query("SET NAMES 'utf8'"); // For All Sql Queries Set Default UTF-8
$db->set_charset("utf8"); // For All Sql Queries Set Default UTF-8
i have a database that contains both chinese characters and english. when i queried the database to display them on my browser using php, all i get for the chinese characters are gibberish, none readable characters.
what i have tried:
ini_set('mssql.charset', 'UTF-8')
on the client side i made sure i included the meta tag specifying UTF-8.
am using mssql server
language is php
any help will be great. thanks
I worked on a Chinese migration project last year and can point you at a few things to look at:
Check you are using NVARCHAR and NCHAR as your DB data types. Do not use VARCHAR/CHAR
I've not used PHP much, but in the majority of languages I've worked in, you need to make sure the appropriate locale is set/supported.
So a few things to check, try outputing some Chinese text on a sample .php file to confirm it is displaying as expected. Next, update your schema with the correct data types. Finally, point your test script at the DB to see if the data is displayed as expected.
Try setting the character set within the PHP file after you've connected. Then run the query.
<?php
include('db_access.php');
$dbc = mysqli_connect(DB_HOST, DB_USER, DB_PASSWORD, DB_NAME);
mysqli_set_charset($dbc, "utf8"); // Set the character set here!!
$query = "SELECT chinese FROM my_table";
while ($row = mysqli_fetch_array($query)) {
echo $row['chinese'].'<br />';
}
?>
PHP Manual: http://cn2.php.net/manual/zh/mysqli.set-charset.php
I have problem with German characters on my web site,
in html/php part of website i have this code to set utf-8:
<meta charset="utf-8">
in mysql, i have this code to set utf-8
SET CHARSET 'utf8';
Here is some word on German: Gemäß
Here is how that word looks in mysql table:
Gemäß
Here is how that word is shown on the site: Gemäß
What is a problem? Thanks.
I was using this code to get title:
$title = mysql_real_escape_string(htmlentities($_POST['title']));
I just override that to
$title = $_POST['title'];
At first, make sure, that you have UTF-8 characters in your database.
After that, try using SET NAMES 'UTF8' after connecting to MySQL:
$con=mysqli_connect("host", "user", "pw", "db");
if (!$con)
{
die('Failed to connect to mySQL: ' .mysqli_connect_errno());
}
mysqli_query($con, "SET NAMES 'UTF8'") or die("ERROR: ". mysqli_error($con));
As the manual says:
SET NAMES indicates what character set the client will use to send SQL
statements to the server... It also specifies the character set that the server should
use for sending results back to the client.
Try SET NAMES 'utf8' or SET NAMES 'utf-8'. Some of these works fine for portuguese, probably for german too. I just can't remember which one is correct, but if it is not, an error will be produced.
you should make sure that the CONNECTION is also utf-8.
with mysqli this is done with something like this:
$connection = mysqli_connect($host, $user, $pass, $db_name);
$connection->set_charset("utf8");
Now if somehow you ended up with wrong characters in the database there is a way to make it right:
in a PHP script, retrieve the information as you do now, i.e without setting the connection. This way the mistake will be inverted and corrected and in your php file you will have the characters in the correct utf-8 format.
in a PHP script, write back the information with setting the connection to utf-8
at this point you should see the character correct in your database
now change all your read/write functions of your site to use the utf-8 from now on
in HTML5 use
<meta charset="utf-8">
in HTML 4.0.1 use
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
the results are html entity encoded as if they were processed by htmlentities(), I wonder if your variables are ibserted as received from the form or are being processed by say a wysiwg editor for instance?
Anyway, these should print fine on an html template but an html_entity_decode() should do it to.
Hope this helps
Set the data type in your database to use UTF-8 as well, this should solve the problem.
I had the same problem. which I solved by using:
if you have already created your table, you need the modify the character set as:
alter table <table name> convert to character set utf8 collate utf8_general_ci.
your tables character set is set to latin_swedish by default by MySQL.
also, you might face some problems while retrieving the data and displaying it to you page.For that include: mysql_set_charset('utf8') just below the line where you have connected your database.
eg:
mysql_connect('localhost','root','');
mysql_select_db('my db');
mysql_set_charset('utf8');
You will need to do this for php 5.x
$yourNiceLookingString =
htmlspecialchars ($YourStringFromDB, ENT_COMPAT | ENT_HTML401, 'ISO-8859-1');
and for php 4.x
$yourNiceLookingString = htmlspecialchars($YourStringFromDB);
what's up? :-)
I have one problem and i hope you can help me with it.
One friend of mine have a simple solid html website and i implemented little php; CRUD system for articles... problem i came across is placing and getting cyrillic characters from mysql database.
What i want to achive is next:
In the main navigation there are some separated sections, whose names, ids and item's order i want to place in mysql and than to pull names and to put each name as a link. Names are supposed to be cyrillic characters.
The problem comes when i, using php mysql_fetch_assoc function, try to display names which are inserted with cyrillic characters in database row, collation of row is utf8_general_ci, and i end with ????? insted of original characters. If i submit cyrillic characters via submit form to mysql it shows something like this У.
How can i solve this, thanks in advance!? :-)
Make sure you call this after connecting to database.
mysql_query("SET NAMES UTF8");
Also make sure that HTML file has charset meta tag set to UTF-8 or send header before output.
header("Content-Type: text/html; charset=utf-8");
I had the same problem until I encoded the 'Collation' column in my table to 'utf8_bin'.
if its really mysql fetch assoc messing up you should try:
mysql-set-charset
from the docs:
Note:
This is the preferred way to change
the charset. Using mysql_query() to
execute SET NAMES .. is not
recommended.
also make sure your files are saved as utf8 and check iconv_set_encoding / iconv_get_encoding
For anyone having more complex issues with legacy project upgrades from versions before PHP 5.6 and MYSQL 5.1 to PHP 7 & Latest MySQL/Percona/MariaDB etc...
If the project uses utf8_encode($value) you can either try removing the function from the value being prepared and use the accepted answer for setting UTF-8 encoding for all input.
--- OR ---
Try replacing utf8_encode($value) with mb_convert_encoding($value, 'utf-8')
PDO USERS
If you are using PDO here are two ways how to set utf8:
$options = [
\PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8'
];
new \PDO($dsn, $username, $passwd, $options);
--- OR ---
$dsn = 'mysql:host=localhost;charset=utf8;'
new \PDO($dsn, $username, $passwd);
I can confirm that mb_convert_encoding($value, 'utf-8') to SQL table using utf8_unicode_ci works for Cyrillic and Umlaut.
I'm trying to store some data in a SQL Server database through php.
Problem is that special chars aren't converted properly. My app's charset is iso-8859-1
and the one used by the server is windows-1252.
Converting the data manually before inserting doesn't help, there seems to be some
conversion going on.
Running the SQL query 'set char_convert off' doesn't help either.
Anyone have any idea how I can get this to work?
EDIT: I have tried ini_set('mssql.charset', 'windows-1252'); as well, but no result with that one either.
Client charset is necessary but not sufficient:
ini_set('mssql.charset', 'UTF-8');
I searched for two days how to insert UTF-8 data (from web forms) into MSSQL 2008 through PHP. I read everywhere that you can't, you need to convert to UCS2 first (like cypher's solution recommends).
On Windows SQLSRV said to be a good solution, which I couldn't try, since I am developing on Mac OSX.
However, FreeTDS manual (what PHP mssql uses on OSX) says to add a letter "N" before the opening quote:
mssql_query("INSERT INTO table (nvarcharField) VALUES (N'űáúőűá球最大的采购批发平台')", +xon);
According to this discussion, N character tells the server to convert to Unicode.
https://softwareengineering.stackexchange.com/questions/155859/why-do-we-need-to-put-n-before-strings-in-microsoft-sql-server
I had the same problem and ini_set('mssql.charset', 'utf-8') did not work for me.
However, it worked in uppercase:
ini_set('mssql.charset', 'UTF-8');
I suggest looking at the following points:
Ensure that the columns that you're storing the information in are nchar or nvarchar as char and nvarchar don't support UCS-2 (SQLServer doesn't store in UTF-8 format btw)
If you're connecting with the mssql library/extension for PHP, run: ini_set('mssql.charset', 'utf-8'); as there's no function with a charset argument (connect, query etc)
Ensure that your browsers charset is also set to UTF-8
If ini_set('mssql.charset', 'UTF-8'); doesn't help AND you don't have root access to modify the system wide freetds.conf file, here's what you can do:
1. Set up /your/local/freetds.conf file:
[sqlservername]
host=192.168.0.56
port=1433
tds version=7.0
client charset=UTF-8
2. Make sure your connection DSN is using the servername, not the IP:
'dsn' => 'dblib:host=sqlservername;dbname=yourdb
3. Make FreeTDS to use your local freetds.conf file as an unprivileged user from php script via env variables:
putenv('FREETDSCONF=/your/local/freetds.conf');
If you are using TDS protocol version 7 or above, ALL communications over the wire are converted to UCS2. The server will convert from UCS2 into whatever the table or column collation is set to, unless the column is nvarchar or ntext. You can store UTF-8 into regular varchar or text, you just have to use a TDS protocol version lower than 7, like 6.0 or 4.2. The only drawback with this method is that you cannot query any nvarchar, ntext, or sys.* tables (I think you also can't do any CAST()ing) - as the server refuses to send anything that might possibly be converted to UTF-8 to any client using protocol version lower than 7.
It is not possible to avoid converting character sets when using TDS protocol version 7 or higher (roughly equivalent to MSSQL 2005 or newer).
In my case, It worked after I added the "CharacterSet" parameters into sqlsrv_connect() connection's option.
$connectionInfo = array(
"Database"=>$DBNAME,
"ConnectionPooling"=>0,
"CharacterSet"=>"UTF-8"
);
$LAST_CONNECTION = sqlsrv_connect($DBSERVER, $connectionInfo);
See documentation here :
https://learn.microsoft.com/en-us/sql/connect/php/connection-options?view=sql-server-2017
I've had luck in a similar situation (using a PDO ODBD connection) using the following code to convert the encoding before printing output:
$data = mb_convert_encoding($data, 'ISO-8859-1', 'windows-1252');
I had to manually set the source encoding, because it was erroneously being reported as 'ISO-8859-1' by mb_detect_encoding().
My data was also being stored in the database by another application, so I might be in a unique situation, although I hope it helps!
For me editing this file:
/etc/freetds/freetds.conf
...and changing/setting 'tds version' parameter to '7.0' helped. Edit your freetds.conf and try to change this parameter for your server configuration (or global).
It will work even without apache restart.
I did not notice someone to mention another way of converting results from MSSQL database. The good old iconv() function:
iconv (string $in_charset, string $out_charset, string $str): string;
In my case everything else failed to provide meaningful conversion, except this one when getting the results. Of course, this is done inside the loop of parsing the results of the query - from CP1251 to UTF-8:
foreach ($records as $row=>$col) {
$array[$row]['StatusName'] = iconv ('CP1251', 'UTF-8' , $records[$row]['StatusName']);
}
Ugly, but it works.
Can't you just convert your tables to your application encoding? Or use utf-8 in both?
I don't know whether MSSQL supports table-level encodings, though.
Also, try the MB (multibyte) string functions, if the above fails.
You should set the charset with ini_set('mssql.charset', 'windows-1252') before the connection. If you use it after the mssql_connect it has no effect.
Just adding ini_set('mssql.charset', 'UTF-8'); didn't help me in my case. I had to specify the UTF-8 character set on the column:
$age = 30;
$name = utf8_encode("Joe");
$select = sqlsrv_query($conn, "SELECT * FROM Users WHERE Age = ? AND Name = ?",
array(array($age), array($name, SQLSRV_PARAM_IN, SQLSRV_PHPTYPE_STRING('UTF-8')));
You can use the mysql_set_charset function:
http://it2.php.net/manual/en/function.mysql-set-charset.php