I am trying to fetch our Arabic values from JDE Database using the following connection string:
$dsn = "Driver={SQL Server};Server=10.10.10.27;Database=JDE;charset=utf8";
$username = "username";
$password = "password";
$string = "odbc:".$dsn.";Uid=".$username.";Pwd=".$password.";";
$con = new PDO($string);
As you can see I have the charset=utf8 specified.
I also have my HTML meta present:
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
I also have used arabic characters fetched from MySQL before and it is displayed correctly and I can see Arabic characters, but from JDE database I get the following:
name: ??? ???? ???? ??? ??? ??? ???? ???? ??? ???
I tried adding the following in my php code trying different output:
echo iconv('windows-1256', 'utf-8', $DataFromJDE);
echo utf8_decode($DataFromJDE);
echo utf8_encode($DataFromJDE);
But all failed.
Is there a configuration I need to do on the server?
I am using Apache with PHP 7 on a Windows server.
The JDE is in a separate server.
What am I missing? Is it from PHP or JDE?
Try something like:
echo $DataFromJDE . '<br>';
Note that you don't need to use utf8_decode(...) or anything similar, browsers support UTF-8 as is, without any need for reformatting.
YES!
Finally Cracked the code!
Thank you So much for the amazing explanations and clarifications. your answers helped a lot.
So basically what I had to do is.
Change my query to the following
$con->prepare('SELECT CAST( CAST(JDEColumn AS VARBINARY(MAX)) AS varchar(120)) AS JDEColumn FROM TABLE where COLUMN = :SELECTION');
courtesy of this thread here https://stackoverflow.com/a/29975739/11835221
Using iconv() as follows:
iconv('UTF-16LE', 'utf-8', $DataFromJDE);
got the clue from this answer by reverse engineering a bit https://stackoverflow.com/a/51130953/11835221
AND Voila! I got my Arabic data!
Hoping this helps another hair pulling soul some day.
Cheers!
Related
I am working for a company who bought DBA Manufacturing and I need to retrieve some values from the Pervasive SQL 2000i DB. Unfortunately there are no plans to upgrade the DB so I have to make due with what I have. I am running a 32 bit
I was able to make an odbc_connect call in PHP to the DSN on a windows machine. I am running PHP on Windows with NGINX.
<html>
<head>
<title>
Placeholder Title
</title>
</head>
<body>
<?php
// mb_internal_encoding("UTF-8");
// connect to the dbamfg DSN
$connect=odbc_connect("Driver={Pervasive ODBC Client Interface};ServerName=dbamfg.1583;ServerDSN=dbadefault;","","");
// query the users table for name and surname
$query = "select * from BKARINVL where BKAR_INVL_INVNM = 600100 order by bkar_invl_cntr ";
//Prepare the statement
$result = odbc_prepare ($connect, $query);
// execute the query
odbc_execute($result);
// fetch the data from the database
while(odbc_fetch_row($result)){
$SalesOrder = odbc_result($result, 1);
$InvLine = odbc_result($result, 2);
$EstShpDt = odbc_result($result, 3);
$ProdCode = odbc_result($result, 4);
$ProdDesc = odbc_result($result, 5);
echo ('<br />'.$InvLine.'<br />');
echo ($SalesOrder.'<br />');
echo ($EstShpDt.'<br />');
echo ($ProdCode.'<br />');
echo ($ProdDesc.'<br />');
}
// closing the connection
odbc_close($connect);
?>
</body>
</html>
The query returns numeric values just fine. When it comes to text columns it returns 10 characters and after that it shows characters that are not displaying correctly.
here is an example:
1
600100
2015-05-31
020-0015-1
DIGITAL VI!2S��
2
600100
2015-05-31
020-0012-0
2 X 4 AUDI!5S��
3
600100
2015-05-31
020-0065-0
16-CKT LOW!8S��
I'm excited and shocked that I have gotten this far! It's exceptionally fast to retrieve values from Pervasive through the ODBC connection and I feel like I am very close to being successful. I just need a push in the right direction.
Everything I have tried so far: I have tried to set the internal encoding of the page to several different character sets including UTF-8, Windows, ASCII, etc. I have also tried to select the checkbox under the ODBC driver setup options during DSN setup in Windows, which says "Use OEM/ANSI Conversion". This does nothing but change the characters by a bit.
Another clue to this is that for some reason when I call the connection string using System.Data.odbc in C# it returns the query and text values in full.
I have tried using iconv to convert the values to UTF-8, etc., to no avail. I've tried forcing the header to display the page in different character sets. I have tried mb_detect_encoding() on the odbc_result() and the return value says UTF-8, but I assume it has to do There must be someone out there that knows how to fix this. Please, would anyone point me in the right direction? I will be forever grateful!
If anything more needs to be added to solve this I will include it when requested. Thank you!
I have this code in drupal 6 to retrieve arabic values from Oracle databse:
<?php
session_start();
$conn=oci_connect('localhost','pass','IP....');
$stid=oci_parse($conn,"select arabic_name from arabic_names_table");
oci_execute($stid);
if($row-oci_fetch_array($stid,OCI_ASSOC+OCI_RETURNS_NULLS))
{
$name_ar=$row['arabic_name'];
}
?>
When values are retrieved from the DB or inserted to the DB they appears like this ???
Please note:
My Oracle database reads normal Arabic characters. From PL/SQL I can insert arabic values
I have installed the mbstring
I have the utf-8 encoding enabled.
How can I solve this problem?
From the oracle database, when you try to fetch data, normally you will get the character encoding will be the encoding type of the client installed in the system (the machine that you installed the php). This encoding will be the charset of the windows registry for the oracle client. (see HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\KEY_OraClient11g_home1), and the key is NLS_LANG. If you search the value of the above key, you will get something like ARABIC_UNITED ARAB EMIRATES.AR8MSWIN1256. Please note that the encoding type is AR8MSWIN1256. In the character map array this is mapped to windows-1256 ( windows-1256 => AR8MSWIN1256 ).
See this link http://websvn.projects.ez.no/wsvn/ezoracle/?op=comp&compare[]=%2Fstable#385&compare[]=%2Fstable#386.
That is, after you fetch the data from the database the char encoding will be windows-1256. Now if your web page is using utf-8 charset, you need to convert the string to utf-8. For this you can use iconv().
$win1256 = iconv('windows-1256', 'utf-8', $my_string); //$my_string -> windows-1256
echo $win1256; // Results the utf-8 format .
If you are still facing problem you check the charset in the page, it must be utf-8.
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
I think this will solve your problem.
I've done this before in .Net but I am somewhat new to PHP and could use some help. I'm trying to pull data in PHP from a MSSQL db, some of which is Greek letters. The data type of the MSSQL columns is nvarchar and the Greek characters are displayed correctly in management studio; however, whenever I pull the data with the sqlsrv_query function in php I get a bunch of question marks.
In sqlsrv_connect function I've tried setting CharacterSet"=>"UTF-8 which doesn't work.
I've tried iconv(), but am not sure which sets to use.
I tried mb_detect_encoding() and it returned ASCIIASCII.
Here is some sample code:
public function _getOrderItemDetails($orderItemIds,$productTypeId){
$params = explode(',', $orderItemIds);
$sql = "select od.*, ofs.Sort
from orderDetail od
left join orderFieldSort ofs on ofs.fieldName = od.name and productTypeId in ($productTypeId)
where orderItemid in (".rtrim(str_repeat('?,', count($params)),',').")
order by ofs.sort";
return $this->_compileResults($sql,'OrderDetail',$params);;
}
private function _compileResults($sql,$classType,$params ){
$result = sqlsrv_query($this->conn,$sql,$params);
$arrayResult = array();
while($orderObject = sqlsrv_fetch_object($result,$classType)){
array_push($arrayResult, $orderObject);
}
return $arrayResult;
}
Any suggestions are welcome. Please help!
A couple of things to maybe try.
1) Can you validate exactly what character encoding your DB is in? I recommend UTF8 for everything as it should handle the widest range of characters. "ASCIIASCII" is not a character encoding to my knowledge do you mean "ASCII"?
2)Try using mysql_set_charset('utf8'); changing the char set to match your db encoding. It "Sets the default character set for the current connection." from within the current php script. PHP official docs
3)This is an outlier but where are you viewing the results? Irrelevant of the data returned from your DB the if you are viewing the results in a web browser you will need to make sure the page has the appropriate charter encoding set in the head if this is a miss match from the character set returned from your db it can result in the question marks or diamonds... Typically something like this
<head>
<title>page title</title>
<meta charset="UTF-8" />
</head>
I had exactly the same problem with you so when i was searching to find the solution i saw your question.The answer is very simple.I guess somewhere in your code you connect to the database with code like the following :
$serverName = "YOUR_SERVER";
$connectionInfo = array("Database"=>"YourDatabase");
$link = sqlsrv_connect($serverName,$connectionInfo);
if(!$link ) {
die('Could not connect: ' . mysql_error());
}
To make greek characters show up you need to change line 2, from:
$connectionInfo = array("Database"=>"YourDatabase");
to :
$connectionInfo = array("Database"=>"YourDatabase","CharacterSet"=>"UTF-8");
and the rest code stays the same.
This solution was suggested by #Graham to his own question i think, and it is from here : Greek character insertion in php compared to SQL server management studio
I know there have been a lot of almost the same questions, but I still didn't find the answer to my problem.
I want to place "les Îles Açores" into the db. But I get:
les Îles Açores
I tried usin:
SET Names 'ut8)
$mysqli->set_charset("utf8");
mysql_real_escape_string()
htmlentities (Here I got htmlentities, but I want to know if there's another way)
Code:
$name_fr = $_POST["name_fr"]; $name_nl = $_POST["name_nl"];
$arr_kollommen = array("NAME_FR","NAME_NL");
$arr_waardes = array($naam_nl,$naam_fr);
$obj_db->insert("landen",$arr_kollommen,$arr_waardes);
Does someone has an idea how to solve my litle problem?
Thank you very much!
Make sure the table uses the correct CHARSET, for example:
CREATE TABLE myTable (
one VARCHAR(255),
two VARCHAR(255)
) DEFAULT CHARSET=utf8;
Make sure you actually write in UTF8 (meaning your IDE / editor you write your code must have encoding set to UTF8).
Is the record corrupted both in the DB and on your page after you fetch it or only in DB?
$name_fr = $_POST["name_fr"];
$name_nl = $_POST["name_nl"];
$arr_kollommen = array("NAME_FR","NAME_NL");
$arr_waardes = array($naam_nl,$naam_fr);
$obj_db->insert("landen",$arr_kollommen,$arr_waardes);
Try using instead of encode to utf_8 decode.
like this:
$name_fr = $_POST["name_fr"];
$name_nl = $_POST["name_nl"];
$naam_fr = utf8_decode($naam_fr);
$naam_nl = utf8_decode($naam_nl);
$arr_kollommen = array("NAME_FR","NAME_NL");
$arr_waardes = array($naam_nl,$naam_fr);
$obj_db->insert("landen",$arr_kollommen,$arr_waardes);
2 possible reasons i can see:
1) Your database doesn't feature UTF-8 fields
2) When you read your data from the server, you are not setting the connection as utf-8. If you have to set it utf-8 when writting you also have to set it utf-8 when reading.
Check using PHPMyAdmin if the data is wrecked... If it is, then it means that your SET names'utf-8' is not working...
Do you pass the "UTF-8" parameter into your htmlentities, and html_entity_decode this way ?
html_entity_decode($text,ENT_QUOTES , "UTF-8");
I'm using Codeigniter not for so long but I've some charset problems.. I'm asking around at the CI Forum, but I want to go further, still no global solution: http://codeigniter.com/forums/viewthread/204409/
The problem was a database error 1064. I've got a solution, use iconv! Works fine, but I think it's not necessary. I'm searching a lot on the internet for charset's etc but I'm using CI now, how about charsets and CI...
So I've a lot of question about it, I hope someone can make it clear for me:
What’s the best way to set the charset global? And what to set?
In the head
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
In config/config.php
$config['charset'] = 'UTF-8';
In config/database.php
$db['default']['char_set'] = 'utf8';
$db['default']['dbcollat'] = 'utf8_general_ci';
In .htaccess, my rewrite rules and
php_value magic_quotes_gpc Off
AddDefaultCharset UTF-8
Also need send a header? Where to place? Something like?
header('Content-Type: text/html; charset=UTF-8');
In my editor (Notepad++) save files as UTF-8? Or UTF-8 (without BOM)? Or is ANSI good (this is what I’m using now)?
Use utf8_unicode_ci or utf8_general_ci for the MySQL database? And why?
How about reading RSS feeds, how to handle multiple charsets? Where I’m working on I’ve two feeds, one with UTF-8 encoding and the other with ISO-8859-1. This will be stored in the database and will be compared sometimes to see if there are new items. It fails on special chars.
I'm working with:
- CI 2.0.3
- PHP 5.2.17
- MySQL 5.1.58
More information added:
Model:
function update_favorite($data)
{
$this->db->where('id', $data['id']);
$this->db->where('user_id', $data['user_id']);
$this->db->update('favorites', $data);
return;
}
Controller:
$this->favorites_model->update_favorite(array(
'id' => $id,
'rss_last' => $rss_last,
'user_id' => $this->session->userdata('user_id')
));
When $rss_last is a “normal” value like: “test” (without quotes) it works fine.
When it’s a value with more length like (in Dutch): F-Secure vindt malware met certificaat van Maleisische overheid
I get this error:
Error Number: 1064
You have an error in your SQL syntax; check the manual that
corresponds to your MySQL server version for the right syntax to use
near ‘vindt malware met certificaat van Maleisische overheid,
user_id = ‘1’ WHERE `i’ at line 1
UPDATE favorites SET id = ‘15’, rss_last = F-Secure vindt
malware met certificaat van Maleisische overheid, user_id = ‘1’
WHERE id = ‘15’ AND user_id = ‘1’
Filename:
/home/.../domains/....nl/public_html/new/models/favorites_model.php
Line Number: 35
Someone at the CI forum told me to use this:
'rss_last' => iconv("UTF-8", "UTF-8//TRANSLIT", $rss_last)
This works fine, but I think this is not necessary..
The value $rss_last came out a RSS feed, as told before, sometimes a UTF-8 and other times a ISO-8859-1 encoding:
$rss = file_get_contents('http://www.website.com/rss.xml');
$feed = new SimpleXmlElement($rss);
$rss_last = $feed->channel->item[0]->title;
It looks like this last part is the problem, when $rss_last is set to the value it works fine:
$rss_last = 'F-Secure vindt malware met certificaat van Maleisische overheid';
When the value came out the RSS it give problems...
Some more questions..
Just found this: Detect encoding and make everything UTF-8
Best solution? But.. is iconv not more simple, do something like this:
$encoding = some_function_to_get_encoding_from_feed($feed);
$rss_last = iconv($encoding, "UTF-8//TRANSLIT", $feed->channel->item[0]->title);
But what to use for "some_function_to_get_encoding_from_feed"? mb_detect_encoding?
And mb_convert_encoding vs iconv?
1) There is no global solution.
2)
AddDefaultCharset UTF-8
It's needed for Apache response to client with right encoding. Make it.
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
not necessarily, but recommended by W3C.
$config['charset'] = 'UTF-8';
it's desirable
$db['default']['char_set'] = 'utf8';
$db['default']['dbcollat'] = 'utf8_general_ci';
Encoding for CI connection to database. If encoding of your database is UTF-8 - make it mandatory.
header('Content-Type: text/html; charset=UTF-8');
Do not do this unless necessary. Charset already indicated in HTML code and .htaccess.
Use utf8_unicode_ci or utf8_general_ci for the MySQL database? And why?
For their own language (Russian), I use utf8_general_ci.
In my editor (Notepad++) save files as UTF-8?
Absolutely! All code that Apache will give as UTF8 should be in UTF8.
How about reading RSS feeds, how to handle multiple charsets?
If you have each RSS in each table - you can specify charset for each table and set right encoding with each sql query.
Yes, cyrillic symbols, for example, will fails on non-UTF8.
UTF-8 (without BOM) should give you the best results based on your configuration and there's no need to send separate headers since the encoding is already selected in the head part. Utf8_general_ci should do fine for the MySQL database.
Perhaps the entries in the database are not valid?