php json request: json_decode unicode string [duplicate] - php

This question already has answers here:
PHP json_decode() returns NULL with seemingly valid JSON?
(29 answers)
Closed 1 year ago.
I try to get the contents of this json URL:
http://www.der-postillion.de/ticker/newsticker2.php
Problem seems to be that the contents of "text" have Unicode within.
Everytime I try to get the json_decode, it fails with NULL...never had that issue before. always pulling json that way:
$news_url_postillion = 'http://www.der-postillion.de/ticker/newsticker2.php';
$file = file_get_contents($news_url_postillion, false, $context);
$data = json_decode($file, TRUE);
//debug
print_r(array($data));
$news_text = $data['tickers'];
//test
echo $news_text->text[0]; //echo first text element for test
foreach($news_text as $news){
$news_text_output = $news->{'text'};
echo 'Text:' . echo $news_text_output; . '<br>';
}
Anybody any idea what is wrong here? tries to get encoding working for hours with things like:
header("Content-Type: text/json; charset=utf-8");
or
$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"Content: type=application/json\r\n" .
"Content-Type: text/html; charset=utf-8"
)
);
$context = stream_context_create($opts);
but no luck :(
Thanks for your help!
Solution:
the json source has some unwanted elements in it, like the BOM character at json start. I could not influence the source json, so the solution walkingRed provided put me on the right track. Only the utf8_decode was needed due to his code is only for english language without special characters.
My working code solution for parsing and output the json is:
<?php
// Postillion Newsticker Parser
$news_url_postillion = 'http://www.der-postillion.de/ticker/newsticker2.php';
$json_newsDataPostillion = file_get_contents($news_url_postillion);
// Fix the strange json source BOM stuff
$obj_newsDataPostillion = json_decode(preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $json_newsDataPostillion), true);
//DEBUG
//print_r($result);
foreach($obj_newsDataPostillion['tickers'] as $newsDataPostillion){
$newsDataPostillion_text = utf8_decode($newsDataPostillion['text']);
echo 'Text:' . $newsDataPostillion_text . '<br>';
};
?>

I made some search and get this:
$result = json_decode(preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $file), true);
Original post

BOM! There is a BOM character at the beginning of the document which you linked and you need to remove it before you try to decode its content.
You can see it e.g. if you would download that json with wget and display it with less.

Related

PHP & JSON : Trying to write JSON to Mysql

I am having an issue where my PHP script opens a file with JSON code and needs to insert it into a MySQL database.
For some reason it only displays some of the output from the JSON.
Here is my code
$json = json_decode(file_get_contents('data.json'), true);
$data = $json;
// VAR's
$system = $data['System'];
$cid_from = $data["From"];
$cid_to = $data['To'];
//DEBUG USAGES
$array = print_r($data, true);
////// THIS ONE WORKS FINE
echo $data["System"];
////// THIS ONE DOESN'T WORK
echo $data["To"];
file_put_contents('output/json-local.txt',$array . "\r\n", FILE_APPEND);
////// BUT HERE IT ACTUALLY WORKS
file_put_contents('output/cli-from.txt',$data['From']. "\r\n", FILE_APPEND);
file_put_contents('output/cli-to.txt',$data['To']. "\r\n", FILE_APPEND);
// file_put_contents('json-sysid-local.txt',$systemid . "\r\n", FILE_APPEND);
Here is the contents of data.json
{"action":"call-data-record",
"System":"48130b83e2232f0ecd366a92d4d1261d",
"PrimaryCallID":"n1bWEfCdHcf#MSS.MTN.CO.ZA-b2b_1",
"CallID":"0440b807#pbx",
"From":"<sip:+27722080036#xxx.co.za>",
"To":"<sip:27102850816#xxx.co.za>",
"Direction":"O",
"RemoteParty":"",
"LocalParty":"",
"TrunkName":"",
"TrunkID":"",
"Cost":"",
"CMC":"",
"Domain":"xxx.co.za",
"TimeStart":"2018-08-14 16:03:21",
"TimeConnected":"",
"TimeEnd":"2018-08-14 16:03:23",
"LocalTime":"2018-08-14 18:03:21",
"DurationHHMMSS":"0:00:00",
"Duration":"0",
"RecordLocation":"",
"RecordUsers":"",
"Type":"hunt",
"Extension":"100",
"ExtensionName":"100",
"IdleDuration":"",
"RingDuration":"2",
"HoldDuration":"0",
"IvrDuration":"0",
"AccountNumber":"400",
"IPAdr":"",
"Quality":"VQSessionReport: CallTerm\r\nLocalMetrics:\r\nCallID:0440b807#pbx\r\nFromID:<sip:27102850816#xxx.co.za>\r\nToID:<sip:+27722080036#xxxx.co.za>;tag=1460166964\r\nx-UserAgent:Vodia-PBX/57.0\r\nx-SIPmetrics:SVA=RG SRD=91\r\nx-SIPterm:SDC=OK SDR=OR\r\n"}
Your "To" data is encapsulated in <>. This causes your browser to interpret it as an HTML tag and not display any content.
You can (should!) escape the special HTML control characters:
echo htmlspecialchars($data["To"]);
See http://php.net/htmlspecialchars
Edit: It doesn't hurt to precautionary add this to your other outputs aswell. If the string doesn't contain such characters, it will simply be returned onchanged. You eliminate possible XSS attack vectors this way.
The browser source clearly shows "To":"" is being written by PHP to the browser output correctly but the browser is interpreting as an HTML opening tag hence ignoring the rest of the content.
Wrap your output in the PHP htmlspecialchars() function to see the output as in the file.
Add - echo "TO : ".htmlspecialchars($data["To"]);

PHP: How to json encode of hindi language in response

I am working with a translation API but here is an issue. I am using JSON in response and when I do json_encode of Hindi then the out is like "\u092f\u0939 \u0915\u093e\u0930 \u0939\u0948"
My code is given below
$data = array();
$data['hindi'] = 'यह कार है';
$data['english'] = 'This is car';
echo json_encode($data); die;
and the response is
{"hindi":"\u092f\u0939 \u0915\u093e\u0930 \u0939\u0948","english":"This is car"}
If you are running PHP 5.4 or greater, pass the JSON_UNESCAPED_UNICODEparameter when calling json_encode
Example:
$data = array();
$data['hindi'] = 'यह कार है';
$data['english'] = 'This is car';
echo json_encode($data, JSON_UNESCAPED_UNICODE);
die;
This is correct json and when you display it in the browser and / or parse it, it will result in an object with the correct keys and values:
var json_string = '{"hindi":"\u092f\u0939 \u0915\u093e\u0930 \u0939\u0948","english":"This is car"}',
json = JSON.parse(json_string);
// or directly:
var json2 = {"hindi":"\u092f\u0939 \u0915\u093e\u0930 \u0939\u0948","english":"This is car"};
console.log(json_string);
console.log(json);
console.log(json2);
document.write(json_string);
document.write('<br>');
document.write(json.hindi);
document.write('<br>');
document.write(json2.hindi);
The reason for this is likely that these characters are not in UTF-8 (I could not find them, at least). From the PHP documentation on json_encode:
All string data must be UTF-8 encoded.
This means that it will have to convert it to a 'description' of the characters. Do not worry, if you decode it again it will very likely be correct.
I have a solution to this problem. It works fine for me.
$data = array();
$data['hindi'] = base64_encode('यह कार है');
$data['english'] = 'This is car';
$encoded_text=json_encode($data);
To retrieve the hindi part of data:
$hindi=base64_decode(json_decode($encoded_text)->hindi);

Part of string missing after urldecode() in php

I have an encoded string (it is too long to be posted here). When I use different utilities for decoding the string (http://www.the-art-of-web.com/javascript/escape/) the string looks perfect after urldecode(). However, when I actually pass the string through urldecode() in my php file on my testing environment the first 100 or so characters are missing. I cannot figure out why. I have tried both urldecode and rawurldecode. If you want to see the string I am trying to process you can make a GET request against this url http://pacific-wave-7885.herokuapp.com/api/opencart the string I am working with is the "contents" value of the JSON object
What i am trying to accomplish:
I want to make a php file that calls the above api address, gets the contents from the JSON object, decodes the string and parses the code.
here is what I have tried:
function utf8_urldecode($str) {
$str = preg_replace("/%u([0-9a-f]{3,4})/i","&#x\\1;",urldecode($str));
return html_entity_decode($str,null,'UTF-8');;
}
$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"Accept-language: en\r\n" .
"Cookie: foo=bar\r\n"
)
);
$context = stream_context_create($opts);
$file = file_get_contents('http://pacific-wave-7885.herokuapp.com/api/opencart', false, $context);
$contents_decode = utf8_urldecode($file);
echo $contents_decode;
You can see with this code that the "contents" starts with "language->load('shipping/usps')" and is missing the first 100 or so characters of that part of the string.
I have also tried this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://pacific-wave-7885.herokuapp.com/api/opencart");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
$data = json_decode($output, true);
$contents = $data[0]['contents'];
$contents_decode = urldecode($contents);
echo $contents_decode;
curl_close($ch);
This also produces the same result - part of the beginning of the string is missing.
to reiterate: if you grab the encoded string straight from the JSON object and use an online decoding tool the string looks great, but once it is passed through urldecode() in my php file the first part of the string is missing characters.
If anyone can see what I am missing I would be so grateful.
Just fyi: My php environment is the latest version of XAMPP with php5 and the JSON object is coming from a NODE server with express.js.
If anyone has a better idea of how I can pass php code as a string from a Node server to a PHP server and then parse it I would be open to that as well.
I'm going to make two assumptions:
It's showing up starting at language->load('shipping/usps');
You are viewing the returned string in your browser.
Good news is that you aren't missing any characters! The browser is simply misinterpreting the tags -
<?php
class ModelShippingUsps extends Model {
public function getQuote($address) {
$this->
The browser is trying to make sense of it - it doesn't know this is PHP, it thinks it is HTML. It sees < and thinks "oh cool, beginning of an HTML tag." And then it sees ? and it just tries to figure out what kind of malformed tag this is, and then it see's -> and it decides it must be an HTML comment, so it parses it as:
<!--?php
class ModelShippingUsps extends Model {
public function getQuote($address) {
$this--->
Change echo $contents_decode; to echo "<textarea>" . $contents_decode . "</textarea>" and you'll see the full string.
Why not just return the data (instead of a PHP block in the contents) and have your PHP class on the receiving end decode the JSON and complete the logic.
If you are insisting on getting the PHP block back...it might be easier to just return a JSON response from node/express and use json_decode to pull it into an array. In playing with this the only way I could make your PHP block JSON friendly was to escape it then encodeURIComponent.
<?php
$jsonString = '{
"name":"usps.php",
"version":1,
"platform":"opencart",
"contents":"%253C%253Fphp%250Aclass%2520ModelShippingUsps%2520extends...57D%250A%2509%2509%257D%250A%250A%2509%2509return%2520%2524method_data%253B%250A%2509%257D%250A%257D%250A%253F%253E",
"_id":"53c40942bddf820200000007",
"__v":0
}
';
$jsonArr = json_decode( $jsonString, true);
var_dump( urldecode( urldecode( $jsonArr["contents"] ) ) );
?>

json_decode in php

I want to use json_decode to this json file but something is going wrong. It was working fine a few days ago but now returns NULL.
<?php
$url = 'http://opendata.diavgeia.gov.gr/api/decisions?org=eot&output=json';
function works2($url)
{
$opts = array(
'http'=>array(
'method'=>"GET",
'header'=>"Connection:Keep-Alive\r\nAccept:*/*\r\n"
)
);
$context = stream_context_create($opts);
$retstr = file_get_contents($url,false,$context);
return $retstr;
}
var_dump(json_decode(works2($url)));
?>
Could anyone help me?
I used a JSON validator and the url seems to provide an INVALID JSON
try using jsonList to validate the URL here http://jsonlint.com/
I get
Parse error on line 822:
... "subject": "Ματαίωση του πρόχει
-----------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
If the URL provides a proper JSON string your code should work.
I don't think json_decode stopped working for you. It's rather server response. Did you print $retstr to make sure it's not empty or broken?
works2 result is ok.
Maybe the json encoded content is too big or (probably) contains unwanted characters or a malformed sequence.
Take a look to default opendata output. It is xml format, but far long to download.

PHP json_decode() returns NULL with seemingly valid JSON?

I have this JSON object stored on a plain text file:
{
"MySQL": {
"Server": "(server)",
"Username": "(user)",
"Password": "(pwd)",
"DatabaseName": "(dbname)"
},
"Ftp": {
"Server": "(server)",
"Username": "(user)",
"Password": "(pwd)",
"RootFolder": "(rf)"
},
"BasePath": "../../bin/",
"NotesAppPath": "notas",
"SearchAppPath": "buscar",
"BaseUrl": "http:\/\/montemaiztusitio.com.ar",
"InitialExtensions": [
"nem.mysqlhandler",
"nem.string",
"nem.colour",
"nem.filesystem",
"nem.rss",
"nem.date",
"nem.template",
"nem.media",
"nem.measuring",
"nem.weather",
"nem.currency"
],
"MediaPath": "media",
"MediaGalleriesTable": "journal_media_galleries",
"MediaTable": "journal_media",
"Journal": {
"AllowedAdFileFormats": [
"flv:1",
"jpg:2",
"gif:3",
"png:4",
"swf:5"
],
"AdColumnId": "3",
"RSSLinkFormat": "%DOMAIN%\/notas\/%YEAR%-%MONTH%-%DAY%\/%TITLE%/",
"FrontendLayout": "Flat",
"AdPath": "ad",
"SiteTitle": "Monte Maíz: Tu Sitio",
"GlobalSiteDescription": "Periódico local de Monte Maíz.",
"MoreInfoAt": "Más información aquí, en el Periódico local de Monte Maíz.",
"TemplatePath": "templates",
"WeatherSource": "accuweather:SAM|AR|AR005|MONTE MAIZ",
"WeatherMeasureType": "1",
"CurrencySource": "cotizacion-monedas:Dolar|Euro|Real",
"TimesSingular": "vez",
"TimesPlural": "veces"
}
}
When I try to decode it with json_decode(), it returns NULL. Why?
The file is readable (I tried echoing file_get_contents() and it worked ok).
I've tested JSON against http://jsonlint.com/ and it's perfectly valid.
What's wrong here?
This worked for me
json_decode( preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $json_string), true );
It could be the encoding of the special characters. You could ask json_last_error() to get definite information.
You could try with it.
json_decode(stripslashes($_POST['data']))
If you check the the request in chrome you will see that the JSON is text, so there has been blank code added to the JSON.
You can clear it by using
$k=preg_replace('/\s+/', '',$k);
Then you can use:
json_decode($k)
print_r will then show the array.
Maybe some hidden characters are messing with your json, try this:
$json = utf8_encode($yourString);
$data = json_decode($json);
I had the same problem and I solved it simply by replacing the quote character before decode.
$json = str_replace('"', '"', $json);
$object = json_decode($json);
My JSON value was generated by JSON.stringify function.
For me the php function stripslashes() works when receiving json from javascript. When receiving json from python, the second optional parameter to the json_decode call does the trick since the array is associative. Workes for me like a charm.
$json = stripslashes($json); //add this line if json from javascript
$edit = json_decode($json, true); //adding parameter true if json from python
this help you to understand what is the type of error
<?php
// A valid json string
$json[] = '{"Organization": "PHP Documentation Team"}';
// An invalid json string which will cause an syntax
// error, in this case we used ' instead of " for quotation
$json[] = "{'Organization': 'PHP Documentation Team'}";
foreach ($json as $string) {
echo 'Decoding: ' . $string;
json_decode($string);
switch (json_last_error()) {
case JSON_ERROR_NONE:
echo ' - No errors';
break;
case JSON_ERROR_DEPTH:
echo ' - Maximum stack depth exceeded';
break;
case JSON_ERROR_STATE_MISMATCH:
echo ' - Underflow or the modes mismatch';
break;
case JSON_ERROR_CTRL_CHAR:
echo ' - Unexpected control character found';
break;
case JSON_ERROR_SYNTAX:
echo ' - Syntax error, malformed JSON';
break;
case JSON_ERROR_UTF8:
echo ' - Malformed UTF-8 characters, possibly incorrectly encoded';
break;
default:
echo ' - Unknown error';
break;
}
echo PHP_EOL;
}
?>
This error means that your JSON string is not valid JSON!
Enable throwing exceptions when an error happens and PHP will throw an exception with the reason for why it failed.
Use this:
$json = json_decode($string, null, 512, JSON_THROW_ON_ERROR);
Just thought I'd add this, as I ran into this issue today. If there is any string padding surrounding your JSON string, json_decode will return NULL.
If you're pulling the JSON from a source other than a PHP variable, it would be wise to "trim" it first:
$jsonData = trim($jsonData);
The most important thing to remember, when you get a NULL result from JSON data that is valid is to use the following command:
json_last_error_msg();
Ie.
var_dump(json_last_error_msg());
string(53) "Control character error, possibly incorrectly encoded"
You then fix that with:
$new_json = preg_replace('/[[:cntrl:]]/', '', $json);
Here is how I solved mine https://stackoverflow.com/questions/17219916/64923728 .. The JSON file has to be in UTF-8 Encoding, mine was in UTF-8 with BOM which was adding a weird &65279; to the json string output causing json_decode() to return null
In my case , i was facing the same issue , but it was caused by slashes inside the json string so using
json_decode(stripslashes($YourJsonString))
OR
json_decode( preg_replace('/[\x00-\x1F\x80-\xFF]/', '', $YourJsonString), true );
If the above doesnt work, first replace the quotes from html quote , this might be happening if you are sending data from javascript to php
$YourJsonString = stripslashes($YourJsonString);
$YourJsonString = str_replace('"', '"', $YourJsonString);
$YourJsonString = str_replace('["', '[', $YourJsonString);
$YourJsonString = str_replace('"]', ']', $YourJsonString);
$YourJsonString = str_replace('"{', '{', $YourJsonString);
$YourJsonString = str_replace('}"', '}', $YourJsonString);
$YourJsonObject = json_decode($YourJsonString);
Will solve it,
It's probably BOM, as others have mentioned. You can try this:
// BOM (Byte Order Mark) issue, needs removing to decode
$bom = pack('H*','EFBBBF');
$response = preg_replace("/^$bom/", '', $response);
unset($tmp_bom);
$response = json_decode($response);
This is a known bug with some SDKs, such as Authorize.NET
If you are getting json from database, put
mysqli_set_charset($con, "utf8");
after defining connection link $con
Just save some one time. I spent 3 hours to find out that it was just html encoding problem. Try this
if(get_magic_quotes_gpc()){
$param = stripslashes($row['your column name']);
}else{
$param = $row['your column name'];
}
$param = json_decode(html_entity_decode($param),true);
$json_errors = array(
JSON_ERROR_NONE => 'No error has occurred',
JSON_ERROR_DEPTH => 'The maximum stack depth has been exceeded',
JSON_ERROR_CTRL_CHAR => 'Control character error, possibly incorrectly encoded',
JSON_ERROR_SYNTAX => 'Syntax error',
);
echo 'Last error : ', $json_errors[json_last_error()], PHP_EOL, PHP_EOL;
print_r($param);
So, html_entity_decode() worked for me. Please try this.
$input = file_get_contents("php://input");
$input = html_entity_decode($input);
$event_json = json_decode($input,true);
It took me like an hour to figure it out, but trailing commas (which work in JavaScript) fail in PHP.
This is what fixed it for me:
str_replace([PHP_EOL, ",}"], ["", "}"], $JSON);
I recommend creating a .json file (ex: config.json).
Then paste all of your json object and format it. And thus you will be able to remove all of that things that is breaking your json-object, and get clean copy-paste json-object.
I also face the same issue...
I fix the following steps... 1) I print that variable in browser 2) Validate that variable data by freeformatter 3) copy/refer that data in further processing
after that, I didn't get any issue.
This happen because you use (') insted {") in your value or key.
Here is wrong format.
{'name':'ichsan'}
Thats will be return NULL if you decode them.
You should pass the json request like this.
{"name":"ichsan"}
I've solved this issue by printing the JSON, and then checking the page source (CTRL/CMD + U):
print_r(file_get_contents($url));
Turned out there was a trailing <pre> tag.
you should ensure these points
1. your json string dont have any unknowns characters
2. json string can view from online json viewer (you can search on google as online viewer or parser for json) it should view without any error
3. your string dont have html entities it should be plain text/string
for explanation of point 3
$html_product_sizes_json=htmlentities($html);
$ProductSizesArr = json_decode($html_product_sizes_json,true);
to (remove htmlentities() function )
$html_product_sizes_json=$html;
$ProductSizesArr = json_decode($html_product_sizes_json,true);
For my case, it's because of the single quote in JSON string.
JSON format only accepts double-quotes for keys and string values.
Example:
$jsonString = '{\'hello\': \'PHP\'}'; // valid value should be '{"hello": "PHP"}'
$json = json_decode($jsonString);
print $json; // null
I got this confused because of Javascript syntax. In Javascript, of course, we can do like this:
let json = {
hello: 'PHP' // no quote for key, single quote for string value
}
// OR:
json = {
'hello': 'PHP' // single quote for key and value
}
but later when convert those objects to JSON string:
JSON.stringify(json); // "{"hello":"PHP"}"
Before applying PHP related solutions, validate your JSON format. Maybe that is the problem. Try this online JSON format validator.
For me, I had to turn off the error_reporting, to get json_decode() working correctly. It sounds weird, but true in my case. Because there is some notice printed between the JSON string that I am trying to decode.
I had exactly the same problem
But it was fixed with this code
$zip = file_get_contents($file);
$zip = json_decode(stripslashes($zip), true);
I had the same issue and none of the answers helped me.
One of the variables in my JSON object had the value Andaman & Nicobar. I removed this & and my code worked perfectly.
<?php
$json_url = "http://api.testmagazine.com/test.php?type=menu";
$json = file_get_contents($json_url);
$json=str_replace('},
]',"}
]",$json);
$data = json_decode($json);
echo "<pre>";
print_r($data);
echo "</pre>";
?>

Categories