PHP output JSON Web Service charset UTF-8 error - php

I am hosting a web service in JSON output by PHP.
I have Hebrew data set in DB and I am posting this as an output to Web service.
When I post the data initially it output the result as follows:
JSON:
{
"tasklist": [
{
"customerID": "9936",
"name": "טר ×רמה ×™×–×•× ×•×‘×™× ×•×™ בע"מ",
"cargo":"×ברר",
"destination":"מכר",
"quantity":"1.000000",
"startdate":"03/01/201300: 00: 00"
}
]
}
But this "×ברר" can be readable by Android/Iphone parser and convert it to original Hebrew. But i faced Error in "name": "טר ×רמה ×™×–×•× ×•×‘×™× ×•×™ בע"מ",. where " is in between the string so the JSON is not valid and shows error!
To Over come this issue I used UTF-8 to convert "×ברר" this to Hebrew "נברר". But in this case too the problem remains same:
PHP:
header('Content-type: text/html; charset=UTF-8');
JSON:
{
"tasklist": [
{
"customerID": "9936",
"name": "טר ארמה יזום ובינוי בע"מ",
"cargo":"נברר",
"destination":"מכר",
"quantity":"1.000000",
"startdate":"03/01/201300: 00: 00"
}
]
}
But still the problem remains:
Also in some case I am getting this � because of using UTF-8
"name":"מחצבות כפר גלעדי-חומרי מ�"
How can I overcome this issue?
Is there any other specific encode I need to use?
Note: The data cannot be changes in Database The solution should be while output to JSON.
How the data stored in DB is shown below:
name
מחצבות כפר גלעדי-חומרי מ×
My PHP Script which output JSON:
<?php
//My DB connection and Query comes here
$jsontext = '{"tasklist":[';
while($row = mysql_fetch_array($queryExe)){
$jsontext .= '{"customerID":"'.$row['AUTO_ID'].'",';
$jsontext .='"name":"'.$row['Customer_Name'].'",';
$jsontext .='"cargo":"'.$row['Type_of_Cargo'].'",';
$jsontext .='"destination":"'.$row['Destination'].'",';
$jsontext .='"quantity":"'.$row['Quantity'].'",';
$jsontext .='"startdate":"'.$row['startdate'].'"},';
}
$jsontext = substr_replace($jsontext, '', -1); // to get rid of extra comma
$jsontext .= "]}";
header('Content-type: text/html; charset=UTF-8');
//Output the final JSON
echo $jsontext;
?>
Thank you for your help in advance!
Was the question clear? to understand my issue.

If your db-field is utf8 you should fist do:
mysql_query("SET NAMES 'utf8'");
You should always do the 'SET NAMES...' before inserting your data, too.
Be sure that you really stored utf8 encoded strings!
then do your query:
mysql_query($your_query);
$array = array("tasklist"=>array());
while($row = mysql_fetch_array($queryExe)){
$a = array();
$a["customerID"] = $row['AUTO_ID'];
$a["name"] = $row['Customer_Name'];
$a["cargo"] = $row['Type_of_Cargo'];
$a["destination"] = $row['Destination'];
$a["quantity"] = $row['Quantity'];
$a["startdate"] = $row['startdate'];
$array["tasklist"][] = $a;
}
header("Content-type: application/json; charset=utf-8");
echo json_encode($array);
exit();
i've made the experience that these is not enough when the servers default charset is for example iso. In that case i need to do the following in my .htaccess:
AddDefaultCharset utf-8

You should change your code to use json_encode. You need to pass it properly utf8 encoded data.
If you are using MySQL you can try running the following before your query to get your data.
SET NAMES 'utf8';
You can also look into using utf8_encode.

From http://www.php.net/manual/en/function.json-encode.php#100565
That said, quotes " will produce invalid JSON, but this is only an issue if you're using json_encode() and just expect PHP to magically escape your quotes. You need to do the escaping yourself.
May be you can replace " with \" , i guess it will solve the issue.
Source : PHP JSON String, escape Double Quotes for JS output

Related

JSON encode not working for MYSQL result Array in PHP 7 [duplicate]

I have a stdClass object called $post that, when dumped via print_r(), returns the following:
stdClass Object (
[ID] => 12981
[post_title] => Alumnus' Dinner Coming Soon
[post_parent] => 0
[post_date] => 2012-01-31 12:00:51
)
Echoing the result from calling json_encode() on this object results in the following:
{
"ID": "12981",
"post_title": null,
"post_parent": "0",
"post_date": "2012-01-31 12:00:51"
}
I'm assuming that something with the single quote is causing json_encode to choke, but I don't know what format is needed to escape that. Any ideas?
EDIT: Fixed mismatch in code examples. I'm running PHP version 5.3.8
EDIT2: Directly after encoding the object, I did this:
echo json_last_error() == JSON_ERROR_UTF8;
This printed 1, which means that the following error occurred: "Malformed UTF-8 characters, possibly incorrectly encoded". json_last_error()
EDIT3: Calling utf8_decode() on the post title resulted in the following: "Alumnus? Dinner Coming Soon". This data is being pulled from a MySQL database - the post title in particular is a text field, UTF-8 encoded. Maybe this single-quote is improperly encoded? The thing is, I have a SQL GUI app, and it appears correctly in that.
You need to set the connection encoding before executing queries. How this is done depends on the API you are using to connect:
call mysql_set_charset("utf8") if you use the old, deprecated API.
call mysqli_set_charset("utf8") if you use mysqli
add the charset parameter to the connection string if you use PDO and PHP >= 5.3.6. In earlier versions you need to execute SET NAMES utf8.
When you obtain data from MySQL any text will be encoded in "client encoding", which is likely windows-1252 if you don't configure it otherwise. The character that is causing your problem is the "curly quote", seen as 92 in the hex dump, which confirms that the mysql client is encoding text in windows-1252.
Another thing you might consider is pass all text through utf8_encode, but in this case it wouldn't produce the correct result. PHP's utf8_encode converts iso-8859-1-encoded text. In this encoding \x92 is a non-printable control character, which would be converted into a non-printable control character in utf-8. You could use str_replace("\x92", "'", $input) to fix the problem for this particular character, but if there's any chance there will be any other non-ascii characters in the database you'll want to have the client use UTF-8.
What I've had to do in the past to json_encode on text with utf8 characters is
json_encode( utf8_encode( $s ) );
and in some cases
json_encode( htmlspecialchars( utf8_encode( $s ) ) );
the utf8_encode() to handle special characters (note, that's Encode, not Decode)
the htmlspecialchars() depending on how you mean to use the JSON string, you may be able to leave this out
and finally, json_encode() to get your JSON packet.
Since you want to json_encode an object, you would need to call utf8_encode() on each text part first, or write a simple recursive utf8_encode(). For your example case, this would do:
function myEncode($o) {
$o->title = utf8_encode($o->title);
return json_encode($o);
}
I would like to refer you about this issue,
on link
I suggest you use a json_encode wrapper like this :
function safe_json_encode($value){
if (version_compare(PHP_VERSION, '5.4.0') >= 0) {
$encoded = json_encode($value, JSON_PRETTY_PRINT);
} else {
$encoded = json_encode($value);
}
switch (json_last_error()) {
case JSON_ERROR_NONE:
return $encoded;
case JSON_ERROR_DEPTH:
return 'Maximum stack depth exceeded'; // or trigger_error() or throw new Exception()
case JSON_ERROR_STATE_MISMATCH:
return 'Underflow or the modes mismatch'; // or trigger_error() or throw new Exception()
case JSON_ERROR_CTRL_CHAR:
return 'Unexpected control character found';
case JSON_ERROR_SYNTAX:
return 'Syntax error, malformed JSON'; // or trigger_error() or throw new Exception()
case JSON_ERROR_UTF8:
$clean = utf8ize($value);
return safe_json_encode($clean);
default:
return 'Unknown error'; // or trigger_error() or throw new Exception()
}
}
function utf8ize($mixed) {
if (is_array($mixed)) {
foreach ($mixed as $key => $value) {
$mixed[$key] = utf8ize($value);
}
} else if (is_string ($mixed)) {
return utf8_encode($mixed);
}
return $mixed;
}
And after define these function you can use it direct,
echo safe_json_encode($response);
i was having the same issue while JSON encoding a php array from an ODBC query results, my Server's OBC is configured with 'en_US.819', is a production server so there is no way i can even touch that!!.
when i tried:
echo json_encode($GLOBALS['response'], true);
Where 'respose' is an array with the results, it works as intended as long no bizarre char is present, if so, json_encode fails returning empty.
The solution?.... UTF encode results while fetching the rows from query:
$result = odbc_exec($conn, $sql_query);
$response = array();
while( $row = odbc_fetch_array($result) ) {
$json['pers_identificador'] = $row['pers_identificador'];
$json['nombre_persona'] = utf8_encode( $row['nombre_persona'] );
$json['nombre_1'] = utf8_encode($row['nombre_1'] );
$json['nombre_2'] = utf8_encode($row['nombre_2'] );
array_push($response, $json);
}
Now json_encode works!!!, the resulting string is something like this:
{"page_id":300,"max_rows":"100","cant_rows":"12897","datos":
[{"pers_identificador":"301","cedula":"15250068","interno_1":"178202","interno_2":"","nombre_persona":"JOSE JUAN PANDOLFO ZAGORODKO","nombre_1":"JOSE","nombre_2":"JUAN",....
That fixed my issue.
You can try to set the charset in the database config:
'charset' => 'utf8',
'driver_options' => array(
PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
)

PHP/Python json_decode doesn´t display special characters like €

I got a json which is generated in a python program and looks like this:
{"0": {"ausschreiber": "Beispiel; Zeitarbeit GmbH", "beschreibung": "\r\nF\u00fcr unseren Kunden suchen wir motivierte studentische Aushilfen auf flexibler Stundenbasis (450\u0080-Basis)", "datum": "17.11.2016", "name": "Studentische Hilfskr\u00e4fte gesucht", "email": "info#hindi.de"}}
now i am decoding the json in my php program to get an associative array and display this on the website.
The Problem is that the special characters like the € char are not displayed but special chars like ö ä ü are displayed.
Here is the php program:
<?php
header('Content-Type: text/html; charset=utf-8');
function compare($old_data, $new_data){
$old_result = json_decode($old_data, true);
$new_result = json_decode($new_data, true);
echo $new_result[0]['beschreibung'];
}
function go4it(){
$db_data=json_content(); //creates the json from the Database
$crawler_data = file_get_contents('http://localhost/phppath/python_program.cgi'); //calls the cgi which returns the json
compare($db_data, $crawler_data);
}
go4it();
What i tried:
set the header to utf-8
$new_result = json_decode(utf8_encode($new data), true);
iconv_set_encoding("internal_encoding", "UTF-8");
iconv_set_encoding("input_encoding", "UTF-8");
iconv_set_encoding("output_encoding", "UTF-8");
Thanks for your help!
EDIT 1
so it seems like the issue is located in the python program, thanks to #FranzGleichmann . I think the problem is with the encoding of the page where i get the content from. The page says it is ISO-8859-1 so i tried this:
url = 'https://www.example.com'
source_code = requests.get(url)
plain_text = source_code.text
plain_text.decode('iso-8859-1', 'ignore').encode('utf8', 'ignore')
print(plain_text.encoding)
but then i get the error: "UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 8496: ordinal not in range(128)"
it was a problem with the python script

php returns invalid json

The following php code returns an invalid json error not sure why
<?php
include("dbConnect.php");
$sql = "SELECT QID, Question, Answer,CatID FROM Questions";
$res = mysqli_query($con,$sql);
$result = array();
while($row = mysqli_fetch_array($res)){
array_push($result,
array('qid'=>$row[0],
'question'=>$row[1],
'answer'=>$row[2],
'catid'=>$row[3]
));
}
echo json_encode(array("result"=>$result));
mysqli_close($con);
when I run the link to the php it returns this data:
{"result":[{"qid":"1","question":"Question 1","answer":"Answer 1","catid":"1"},{"qid":"2","question":"Question 2","answer":"Answer 2","catid":"2"},{"qid":"3","question":"Question 3","answer":"Answer 3","catid":"3"},{"qid":"4","question":"Question 4","answer":"Answer 4","catid":"1"},{"qid":"5","question":"Question 5","answer":"Answer 5","catid":"3"},{"qid":"6","question":"Question 6","answer":"Answer 6","catid":"3"}]}
and I tried running the link to the php that returns the json data using this json formatter website
I get these errors:
Error:Invalid media type, expecting application/json.[Code 28, Structure 0]
Error:Invalid encoding, expecting UTF-8, UTF-16 or UTF-32.[Code 29, Structure 0]
Error:Strings should be wrapped in double quotes.[Code 17, Structure 114]
Error:Invalid characters found.[Code 18, Structure 114]
Error:Strings should be wrapped in double quotes.[Code 17, Structure 116]
Error:Invalid characters found.[Code 18, Structure 116]
If I try copying the resulting json data I get a valid JSON format but when I try running it from the php link that is stored on my server I get the above errors
UPDATE:
link to php
The problem isn't the JSON string, it's the header data.
You must specify the Content-Type header manually, otherwise it will send the output as text/html:
<?php
include("dbConnect.php");
$sql = "SELECT QID, Question, Answer,CatID FROM Questions";
$res = mysqli_query($con,$sql);
$result = array();
while($row = mysqli_fetch_array($res)){
array_push($result,
array('qid'=>$row[0],
'question'=>$row[1],
'answer'=>$row[2],
'catid'=>$row[3]
));
}
header("Content-Type: application/json; charset=utf-8");
echo json_encode(array("result"=>$result));
mysqli_close($con);
Add header('Content-Type: application/json', true); to your code before echo json_encode () to set the correct mime type.
The other error messages from the validator website are curious; i cannot see invalid characters or fields that are not wrapped in double quotes. Maybe, the validator website has some bugs.
add header in php file..
header("Content-Type: application/json; charset=UTF-8");

PHP corrupted when echo data from database to ajax

I have this problem and don't know what is causing it.
When I load data to ajax my ajax script from www.something.tld/index.html they are ok. But when I load data from script.php which firstly retrieve data from db and then echo this data my ajax end with error. But the data output is the same! What's wrong?
example:
index.html output is "XXX"
script.php output is "XXX" but in my script I generate data with:
echo $data[0]->var;
Any idea why this happens? it happens when I retrieve data from db and then echo it out.
Internet Explorer said: Syntax error Line 281, character 163 in ttm/canvas.1.03.jgz
here is link for jgz http://www.freefilehosting.net/canvas103
I found out when I use anz function in php ajax will not parse output correctly!
this is parsed ok:
$from ='05/17/2012 10:15:00';
$to ='05/17/2012 10:30:00';
$day ='5/17/2012';
echo '{ "dtstart": "'.$from.'", "dtend": "'.$to.'", "day": "'.$day.'" },';
this is not parsed correctly:
$from= preg_replace('/[\x00-\x1F\x80-\xFF]/', '',$data[0]->time_from);
$to = preg_replace('/[\x00-\x1F\x80-\xFF]/', '',$data[0]->time_to);
$day = preg_replace('/[\x00-\x1F\x80-\xFF]/', '',$data[0]->day);
$from ='05/17/2012 10:15:00';
$to ='05/17/2012 10:30:00';
$day ='5/17/2012';
echo '{ "dtstart": "'.$from.'", "dtend": "'.$to.'", "day": "'.$day.'" },';
If you use json_encode() (like you're supposed to) then PHP takes care of all the encoding quirks for you. Such as escaping those non-printable characters and things past 0x7F.
echo json_encode(Array(
"dtstart"=>$from,
"dtend"=>$to,
"day"=>$day
));
Depending on the encoding of your files, you may need to run the values through utf8_encode() first, but that's something you have to test first and then try.

json_decode returns NULL after webservice call [duplicate]

This question already has answers here:
PHP json_decode() returns NULL with seemingly valid JSON?
(29 answers)
Closed 4 months ago.
There is a strange behaviour with json_encode and json_decode and I can't find a solution:
My php application calls a php web service. The webservice returns json that looks like this:
var_dump($foo):
string(62) "{"action":"set","user":"123123123123","status":"OK"}"
now I like to decode the json in my application:
$data = json_decode($foo, true)
but it returns NULL:
var_dump($data):
NULL
I use php5.
The Content-Type of the response from the webservice: "text/html; charset=utf-8" (also tried to use "application/json; charset=utf-8")
What could be the reason?
Well, i had a similar issue and the problems was the PHP magic quotes in the server... here is my solution:
if(get_magic_quotes_gpc()){
$param = stripslashes($_POST['param']);
}else{
$param = $_POST['param'];
}
$param = json_decode($param,true);
EDIT:
Just did some quick inspection of the string provided by the OP. The small "character" in front of the curly brace is a UTF-8 B(yte) O(rder) M(ark) 0xEF 0xBB 0xBF. I don't know why this byte sequence is displayed as  here.
Essentially the system you aquire the data from sends it encoded in UTF-8 with a BOM preceding the data. You should remove the first three bytes from the string before you throw it into json_decode() (a substr($string, 3) will do).
string(62) "{"action":"set","user":"123123123123","status":"OK"}"
^
|
This is the UTF-8 BOM
As Kuroki Kaze discovered, this character surely is the reason why json_decode fails. The string in its given form is not correctly a JSON formated structure (see RFC 4627)
Print the last json error when debugging.
json_decode( $so, true, 9 );
$json_errors = array(
JSON_ERROR_NONE => 'No error has occurred',
JSON_ERROR_DEPTH => 'The maximum stack depth has been exceeded',
JSON_ERROR_CTRL_CHAR => 'Control character error, possibly incorrectly encoded',
JSON_ERROR_SYNTAX => 'Syntax error',
);
echo 'Last error : ', $json_errors[json_last_error()], PHP_EOL, PHP_EOL;
Also use the json.stringify() function to double check your JSON syntax.
None of the solutions above worked for me, but html_entity_decode($json_string) did the trick
Try this
$foo = utf8_encode($foo);
$data = json_decode($foo, true);
make sure that if you sent the data by POST / GET, the server has not escape the quotes
$my_array = json_decode(str_replace ('\"','"', $json_string), true);
"{"action":"set","user":"123123123123","status":"OK"}"
This little apostrophe in the beginning - what is it? First symbol after the doublequote.
I had the similar problem in a live site. In my local site it was working fine. For fixing the same I Just have added the below code
json_decode(stripslashes($_GET['arr']));
I just put this
$result = mb_convert_encoding($result,'UTF-8','UTF-8');
$result = json_decode($result);
and it's working
Yesterday I spent 2 hours on checking and fixing that error finally I found that in JSON string that I wanted to decode were '\' slashes. So the logical thing to do is to use stripslashes function or something similiar to different PL.
Of course the best way is sill to print this var out and see what it becomes after json_decode, if it is null you can also use json_last_error() function to determine the error it will return integer but here are those int described:
0 = JSON_ERROR_NONE
1 = JSON_ERROR_DEPTH
2 = JSON_ERROR_STATE_MISMATCH
3 = JSON_ERROR_CTRL_CHAR
4 = JSON_ERROR_SYNTAX
5 = JSON_ERROR_UTF8
In my case I got output of json_last_error() as number 4 so it is JSON_ERROR_SYNTAX. Then I went and take a look into the string it self which I wanted to convert and it had in last line:
'\'title\' error ...'
After that is really just an easy fix.
$json = json_decode(stripslashes($response));
if (json_last_error() == 0) { // you've got an object in $json}
I had such problem with storage json-string in MySQL.
Don't really know why, but using htmlspecialchars_decode berofe json_decode resolved problem.
Non of these solutions worked for me.
What DID eventually work was checking the string encoding by saving it to a local file and opening with Notepad++.
I found out it was 'UTF-16', so I was able to convert it this way:
$str = mb_convert_encoding($str,'UTF-8','UTF-16');
Maybe you use thing as $ ${: these chars should be quoted.
I was having this problem, when I was calling a soap method to obtain my data, and then return a json string, when I tried to do json_decode I just keep getting null.
Since I was using nusoap to do the soap call I tried to just return json string and now I could do a json_decode, since I really neaded to get my data with a SOAP call, what I did was add ob_start() before include nusoap, id did my call genereate json string, and then before returning my json string I did ob_end_clean(), and GOT MY PROBLEM FIXED :)
EXAMPLE
//HRT - SIGNED
//20130116
//verifica se um num assoc deco é valido
ob_start();
require('/nusoap.php');
$aResponse['SimpleIsMemberResult']['IsMember'] = FALSE;
if(!empty($iNumAssociadoTmp))
{
try
{
$client = new soapclientNusoap(PartnerService.svc?wsdl',
array(
// OPTS
'trace' => 0,
'exceptions' => false,
'cache_wsdl' => WSDL_CACHE_NONE
)
);
//MENSAGEM A ENVIAR
$sMensagem1 = '
<SimpleIsMember>
<request>
<CheckDigit>'.$iCheckDigitAssociado.'</CheckDigit>
<Country>Portugal</Country>
<MemberNumber">'.$iNumAssociadoDeco.'</MemberNumber>
</request>
</SimpleIsMember>';
$aResponse = $client->call('SimpleIsMember',$sMensagem1);
$aData = array('dados'=>$aResponse->xpto, 'success'=>$aResponse->example);
}
}
ob_end_clean();
return json_encode($aData);
I don't know Why?
But this work:
$out = curl_exec($curl);
$out = utf8_encode($out);
$out = str_replace("?", "", $out);
if (substr($out,1,1)!='{'){
$out = substr($out,3);
}
$arResult["questions"] = json_decode($out,true);
without utf8_encode() - Don't work
Check the encoding of your file. I was using netbeans and had to use iso windows 1252 encoding for an old project and netbeans was using this encoding since then for every new file. json_decode will then return NULL. Saving the file again with UTF-8 encoding solved the problem for me.
In Notepad++, select Encoding (from the top menu) and then ensure that "Encode in UTF-8" is selected.
This will display any characters that shouldn't be in your json that would cause json_decode to fail.
Try using json_encode on the string prior to using json_decode... idk if will work for you but it did for me... I'm using laravel 4 ajaxing through a route param.
$username = "{username: john}";
public function getAjaxSearchName($username)
{
$username = json_encode($username);
die(var_dump(json_decode($username, true)));
}
You should try out json_last_error_msg(). It will give you the error message and tell you what is wrong. It was introduced in PHP 5.5.
$foo = "{"action":"set","user":"123123123123","status":"OK"}";
$data = json_decode($foo, true);
if($data == null) {
throw new Exception('Decoding JSON failed with the following message: '
. json_last_error_msg());
}
// ... JSON decode was good => Let's use the data
Before applying PHP related solutions, validate your JSON format. That may be the problem. Try below online JSON format validator. If your JSON format is invalid, correct it first, because PHP doesn't decode invalid JSON strings.
https://jsonformatter.org/
Laravel specific answer:
I got the same issue in Laravel. And this did the trick for me
$result = json_decode($result->getContent(), true);
In my case, when I was printing to the screen, json was fine and I copied and decode with json_deocode() function. It was working fine. But, when I was trying to put jsonString directly in the function, it was returning null because quotes were coming like these ". So I used htmlspecialchars_decode() function and now it is working fine.
I am new here, so if I am making any mistakes in writing answer then sorry for that. I hope it'll help somebody.
Sometimes the problem is generated when the content is compressed, so adding the Accept-Encoding: identity header can solve the problem without having to wrangle with the response.
$opts = array(
'http' =>
array(
'header' =>
array(
'Accept-Encoding: identity',
),
),
);
$context = stream_context_create($opts);
$contents = file_get_contents('URL', false, $context);
i had a similar problem, got it to work after adding '' (single quotes) around the json_encode string. Following from my js file:
var myJsVar = <?php echo json_encode($var); ?> ; -------> NOT WORKING
var myJsVar = '<?php echo json_encode($var); ?>' ; -------> WORKING
just thought of posting it in case someone stumbles upon this post like me :)

Categories