I have some of the fields in the database(MS SQL Server) which at times have the Japanese data/characters as well. When I retrieve them using php, the data turn into ???? Such as I got a field Model which stores "BMW and JAPANESE CHARACTER" and when it comes to the page it turn into "BMW and ????????"
Below is my code;
include ('org_dataDSN.php');
//Setting up database virtual connection
echo "Connecting Database <br>";
header( 'Content-Type: text/html; charset=utf-8' );
echo "Successfully connected....";
$subQuery="select model, make from infChnge_CS2002";
$subRes=odbc_exec($connect, $subQuery);
$ix=odbc_num_rows($subRes);
echo "Success.." . $ix;
while(odbc_fetch_row($subRes))
{
$cstate = odbc_result($subRes, 1);
$sname = odbc_result($subRes, 2);
echo $cstate . "<br>";
echo $sname . "";
}
odbc_close($connect);
?>
First cast the data in the MSSQL query using cast function
SELECT CAST(model AS TEXT) as model, CAST(make AS TEXT) as make, FROM r_table;
In php, convert the casted data to utf-8:
$model = iconv('CP1255', 'UTF-8', $model);
$make = iconv('CP1255', 'UTF-8', $make);
i used this for mysql connection to correct ????? in Persian Language
$connect=mysql_connect('localhost','root','');
mysql_query("SET NAMES 'utf8'", $connect);
query this "SET NAMES 'utf8'" right after creating connection to server
Related
Problem
I am getting 0 results when searching for an Arabic word in a MySQL database using the SELECT query with PHP.
However, the same exact query yields results in alternative clients, namely SQLBuddy and the likes. Everything is encoded in UTF-8.
Code
<?php
$host = "localhost";
$username = "hans_wehr_client";
// i know my security is a joke :)
$password = "hans_wehr";
$database = "hans_wehr";
$conn = new mysqli($host, $username, $password, $database);
if ($conn == TRUE){
$search = $_GET["search"];
$encoded_search = utf8_encode($search);
echo $encoded_search."<br>";
header('Content-Type: text/html; charset=utf-8');
$sql = "SELECT * FROM dictionary WHERE ARABIC LIKE '$search'";
echo $sql."<br>";
mysqli_query($conn,"SET NAMES 'utf8'");
mysqli_query($conn,'SET CHARACTER SET utf8');
$result = mysqli_query($conn, $sql);
if (mysqli_num_rows($result) > 0) {
// output data of each row
while($row = mysqli_fetch_assoc($result)) {
header('Content-Type: text/html; charset=utf-8');
echo $row["ARABIC"]. " - Meaning: " . $row["ENGLISH1"]. " " . $row["ENGLISH2"]. "<br>";
}
}else {
echo "0 results";
}
}
?>
Before the mods get the pitchforks, I have to clear up my troubleshooting logic.
Encoding. I set the page encoding to utf-8 using header('Content-Type:
text/html; charset=utf-8'); and ran the queries mysqli_query($conn,"SET NAMES 'utf8'"); and mysqli_query($conn,'SET CHARACTER SET utf8');, this cleared up the ??????? and Ùؤتا
rendered instead of Arabic words issue. That is kind of a different
issue. Source. and Source2.
Database Charset. My database and columns are set to UTF-8.
Other clients work. SQLBuddy/MySQL native client/ PHPMyAdmin appear to be working because running the same exact query yields result. Therefore I appear to be on the same bloody boat with him. The query SELECT * FROM dictionary WHERE ARABIC LIKE 'آخَر، أُخرى' returns a result on SQLbuddy but nada on PHP.
Possible solution:
Running the query SELECT * FROM dictionary WHERE ARABIC LIKE 'آخَر، أُخرى' yields me a result.
However running the query with a UTF-8 encoded version of the Arabic word returns 0 results. SELECT * FROM dictionary WHERE ARABIC LIKE 'آخَر، أُخرى' I think this simulates PHP.
The UTF-8 Arabic word version is obtained by decoding the automatically URL encoded $[_GET] parameter i.e %26%231570%3B%26%231582%3B%26%231614%3B%26%231585%3B%26%231548%3B+%26%231571%3B%26%231615%3B%26%231582%3B%26%231585%3B%26%231609%3B
Could it be that the MySQLi actually queries the UTF-8 version instead of the actual Arabic word? Therefore finding no match since they are different?
If so how can I explicitly tell PHP not to URL encode my search term and therefore pass it as it is?
Since according to my tinfoil theory, http://localhost/hans_wehr/search_ar.php?search=آخَر، أُخرى would work but http://localhost/hans_wehr/search_ar.php?search=%26%231570%3B%26%231582%3B%26%231614%3B%26%231585%3B%26%231548%3B+%26%231571%3B%26%231615%3B%26%231582%3B%26%231585%3B%26%231609%3B
Inputs will be greatly appreciated.
Use html_entity_decode():
Use html_entity_decode() on your $_GET["search"] value
<?php
$host = "localhost";
$username = "hans_wehr_client";
// i know my security is a joke :)
$password = "hans_wehr";
$database = "hans_wehr";
$conn = new mysqli($host, $username, $password, $database);
if ($conn == TRUE){
$search = $_GET["search"];
$encoded_search =html_entity_decode($search, ENT_COMPAT, 'UTF-8');
echo $encoded_search."<br>";
header('Content-Type: text/html; charset=utf-8');
$sql = "SELECT * FROM dictionary WHERE ARABIC LIKE '$encoded_search'";
echo $sql."<br>";
mysqli_query($conn,"SET NAMES 'utf8'");
mysqli_query($conn,'SET CHARACTER SET utf8');
$result = mysqli_query($conn, $sql);
if (mysqli_num_rows($result) > 0) {
// output data of each row
while($row = mysqli_fetch_assoc($result)) {
header('Content-Type: text/html; charset=utf-8');
echo $row["ARABIC"]. " - Meaning: " . $row["ENGLISH1"]. " " . $row["ENGLISH2"]. "<br>";
}
}else {
echo "0 results";
}
}
?>
this is getting me mad.
This is the code I'm using right now:
<?php
//open connection to mysql db
$connection = mysqli_connect( "mysql.kit-series.net", "XXXXX", "XXXXXX", "kit_series_net" ) or die( "Error " . mysqli_error( $connection ) );
$connection->set_charset( "utf8" );
//fetch table rows from mysql db
$nombre = $_POST['nombre'];
$select = "SELECT ID FROM wp_qnnpx4_posts WHERE post_type='tvshows' AND post_title = '" . $nombre . "'";
$result = $connection->query( utf8_encode( $select ) );
$result = $result->fetch_assoc();
$id = $result['ID'];
echo $select;
echo $id;
//close the db connection
mysqli_close( $connection );
The result it's the expected if I use the select on console.
The result it's the expected if I use the select in the PHP and the
$nombre varible has no accents.
The result it's the expected if I use
the select in the console and the $nombre varible has accents.
Only when $nombre has accents and is used on the PHP file, the result isn't the expect, and it just return no results.
I printed the select in that case, put it on console, and it works. I dind't know what more to do.
Any idea?
Don't use utf8_encode unless you are processing non-UTF8 strings. From your question and comments it turns out that you are completely on UTF8 (HTML, database, database connection, PHP file encoding), so calling utf8_encode is not only not necessary, it will have averse effects.
As said in an appreciated contribution to the PHP docs on utf8_encode:
If your text is not encoded in ISO-8859-1, you do not need this function. If your text is already in UTF-8, you do not need this function. In fact, applying this function to text that is not encoded in ISO-8859-1 will most likely simply garble that text.
I'm using utf8_unicode_ci for a Column in MYSQL Table to store TAMIL Language Characters.
I'm Implementing an AngularJS Project, in that I'm calling the PHP Service, the return type is a JSON Data. I Can't able to get the actual Characters, instead of that I'm getting ?????????.
My PHP Sample Source Code:
<?php
header("Access-Control-Allow-Origin: *");
header("Content-Type: application/json; charset=UTF-8");
$outp = "";
$sql_select = "";
$sql_select .= "SELECT * FROM poll_quest WHERE qid = $nqid LIMIT 1 ";
$bQuery = mysql_query($sql_select, $link);
while ($bRow = mysql_fetch_array($bQuery)) {
$qflag = true;
$outp .= '{ "ID": ' . $bRow['qid'] . ',';
$outp .= '"Ans":"' . $bRow['ans_tam'] . '" }';
}
$outp ='{"records":['.$outp.']}';
echo($outp);
?>
My Tamil Words are
மோசம்
மோசமாக இல்லை
நன்று
மிக நன்று
The MySQL Table Structure Snapshot:
The MySQL Table Data Snapshot:
The Output JSON Data Snapshot:
{
"records":[
{
"ID":"1",
"Ans":"??. ????????"
},
{
"ID":"2",
"Ans":"??. ?????????"
},
........
{
"ID":"5",
"Ans":"??. ??????????"
}
]
}
Kindly assist me, how to get the actual characters in the Response JSON...
First things first the MySQL driver i.e. mysql_* has been deprecated in favour of PDO or MySQLi a while back. With that said, lets move on.
The answer to you need to set the character set on the connection i.e. you will need to do the following to make it work -
<?php
...
$link = mysqli_connect(...);
mysql_set_charset('utf8', $link); // This is your answer
...
For future reference, I have reworked the above code to use MySQLi to help you out a bit -
<?php
// NOTE : I prefer using PDO but I have used mysqli for this example
// NOTE : The following code HAS NOT BEEN TESTED!
header('Content-Type: application/json; charset=UTF-8');
// Connection information
$link = new mysqli($host, $user, $password, $dbname);
$link->set_charset('utf8'); // This is important or mysql_set_charset('utf8', $link) if you insist on using mysql_*
// Check connection
if (mysqli_connect_errno()) {
printf('Connect failed: %s\n', mysqli_connect_error());
exit();
}
$outp = '';
// Always use prepared statements!!
if ($stmt = $link->prepare('SELECT qid, ans_tam FROM poll_quest WHERE qid = ? LIMIT 1')) {
$stmt->bind_param('i', $nqid); // user 's' if this is a string for the first parameter
$stmt->execute();
$stmt->bind_result($qid, $ans_tam);
while ($stmt->fetch()) {
$qflag = true;
$outp .= '{ "ID" : ' . $qid . ', "Ans" : "' . $ans_tam . '" }';
}
$stmt->close();
}
echo('{"records":[' . $outp . ']}');
I believe the only thing you miss is
mysql_set_charset('utf8');
even though you have set utf8_unicode_ci for tables or fields, the connection to your database is not necessarily automatically set to use utf8 character encoding.
NB: You should consider using utf8mb4_unicode_ci instead of utf_unicode_ci, and you should really consider to set the caracter set and collation on database (at least table) level not field level (I see you have latin1_swedish_ci as default indicating your entire database is latin1_swedish_ci.
I get a text from a database in UTF-8 using JSON and it is in a file right now. When I print the data it is:
توحید در نگاه امام علی (علیه السلام)
But when I insert it in database I get this error:
Insertion Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '�وحید در نگاه امام علی ' at line 1
This is my code
<?php
header('Content-Type: application/json; charset=utf-8');
$con = mysql_connect("localhost","bbbbbbb","bbbbbbb");
mysql_select_db("db", $con);
if(file_exists('./json/file.json')){
$jsondata = file_get_contents('./json/file.json');
$data = json_decode($jsondata, true);
foreach ($data['nodes'] as $node){
$data_element = $node['node'];
$title = $data_element['title'];
$summary = $data_element['summary'];
$body = $data_element['body'];
$id = $data_element['id'];
print $title."\n";
$insert = "INSERT INTO main(title) VALUES ($title)";
mysql_query($insert) or die("Insertion Error: ". mysql_error());
}
}
else
print "File doesn't exist";
The database and its columns are all utf8_general_ci. Why the printed text is not same as the one that is inserted in Database?
The database-connection needs to know it is utf8 too. mysql_set_charset('utf8').
Instead of the deprecated mysql_con, you should have a look at mysqli.
use this
$mysqli = new mysqli("localhost","root","","your_db_name");
$mysqli->query("SET NAMES 'utf8'");
$mysqli->query("SET CHARACTER SET utf8");
Although #ericwenn and #SahilManchal bring up valid points, the immediate error is the failure to quote the string in ... VALUES ($title) ....
Do not blindly put a string into the query.
This will work but it is not safe: ... VALUES ('$title') .... It is subject to a hack called "sql injection". See real_escape_string.
Also, after connecting with mysqli, use
$mysqli->set_charset('utf8');
That is preferable to SET NAMES.
I have a PHP script that gets a XML, selects some data, and insert them in my database (MySQL). All my fields are utf8_bin. This XML is ISO-8859-1, and can't change this because is another site that sends it to me.
Example: the string "NG Contábil" is set "NG Contábil" in my db. This is my script:
<?php
header('Content-Type: text/html; charset=utf-8');
mysql_query("SET NAMES 'utf8'");
mysql_query('SET character_set_connection=utf8');
mysql_query('SET character_set_client=utf8');
mysql_query('SET character_set_results=utf8');
include '/PagSeguroLibrary/PagSeguroLibrary.php';
include '/PagSeguroLibrary/domain/PagSeguroAccountCredentials.class.php';
include 'conexao.php';
include 'alias_array.php';
include 'retorna_cpf.php';
$conexao = ConectaBD::get_instance();
$conexao->conectar_pronto();
$conexao->BD_pronto();
//(...)
$xml = simplexml_load_file('arquivo.xml');
if($xml === null)
$xml = simplexml_load_string($resposta_transacao);
$status = $xml->status;
$nome = $xml->sender->name;
$email = $xml->sender->email;
$codigo = $xml->code;
$vetor = $xml->items->item;
$cpf = retorna_cpf($email);
foreach($vetor as $v)
{
$nome_produto = $v->description;
if($nome_produto != 'frete')
{
//Retorna o id do produto a partir da descrição
$result = mysql_query('SELECT id_product
FROM ps_product_lang
WHERE name = "'.$nome_produto.'"');
$array = Array();
while($row = mysql_fetch_alias_array($result))
{
foreach($row as $linha)
array_push($array, $linha);
}
mysql_query('INSERT INTO pagamento(Status, Nome, Email, CPF, idproduto, Codigo, Inscrito, id, Enviado, NomeProduto)
VALUES ("'.$status.'", "'.$nome.'", "'.$email.'", "'.$cpf.'", "'.$array[0].'", "'.$codigo.'", 0, null, 0, "'.$nome_produto.'")');
}
}
fclose($arquivo);
unlink('arquivo.xml');
?>
Thanks for any answer!
You have overlooked just one little nuance:
mysql_query("SET NAMES 'utf8'"); is not a magical spell that have to be cast in order to make proper encoding, but actually an SQL query, which needs to be run in the same instance you are actually using to run SQL queries.
So, if you are connecting to your mysql database using ConectaBD::get_instance(); you have to run SET NAMES utf8 query after that call, not before.
I don't know why, but adding utf8_decode() solves the problem
I know.
simplexml's output is always utf-8.
While, as I pointed out above, you don't set your client connection into utf8.
So, it remains default latin1
With (quite useless) utf8_decode() call you're casting your utf-8 data back into latin1 and thus it correctly stored into database.
This should solve your issue:
...
$xml = simplexml_load_file('arquivo.xml');
$xml = iconv('ISO-8859-1', 'UTF-8', $xml);
...