Trouble decoding Json from Google - php

Well, I'm trying to retrieve some info, from the Google Suggest tool.
The thing is, the json returned after the request, doesn't seem decode-able (using json_decode) + JSONLint sees it as "invalid".
What's wrong?
{
e: "GooDUs7lFIeXO63LgBA",
c: 0,
u: "https://www.google.com/s?gs_rn\x3d24\x26gs_ri\x3dpsy-ab\x26tok\x3dt8ORbtI13MEFLoCQjPSv6w\x26cp\x3d2\x26gs_id\x3d3i\x26xhr\x3dt\x26q\x3dtemplate\x26es_nrs\x3dtrue\x26pf\x3dp\x26safe\x3doff\x26sclient\x3dpsy-ab\x26oq\x3d\x26gs_l\x3d\x26pbx\x3d1\x26bav\x3don.2,or.r_cp.r_qf.\x26bvm\x3dbv.50500085,d.bGE\x26fp\x3dc513cf9c63a02102\x26biw\x3d1304\x26bih\x3d437\x26tch\x3d1\x26ech\x3d20\x26psi\x3dFYkDUs-xCsrT4QTD9YGwDw.1375963413783.1",
p: true,
d: "[\x22template\x22,[[\x22template\\u003cb\\u003es\\u003c\\/b\\u003e\x22,0],[\x22template\\u003cb\\u003e monster\\u003c\\/b\\u003e\x22,0],[\x22template\\u003cb\\u003e c++\\u003c\\/b\\u003e\x22,0],[\x22template\\u003cb\\u003es for pages\\u003c\\/b\\u003e\x22,0]],{\x22t\x22:{\x22bpc\x22:false,\x22tlw\x22:false},\x22q\x22:\x22YjrI_EdhVrEkZrkqZwaGIJ_Ih4c\x22,\x22j\x22:\x223i\x22}]"
}
That's what JSONLint gives as an error :
Parse error on line 1:
{ e: "GooDUs7lFIeXO63L
-----^
Expecting 'STRING', '}'
P.S. Even after editing it like "e": and so on, it still gives out error regarding the value of u and claiming that it was expecting a STRING or NUMBER etc... :S

The code given in the question is not valid JSON.
In order to be valid JSON, it would be required to have the field named in quotes. There are no quotes around the e variable name, or any of the others.
This is what the JSON decoder is complaining about: It is expecting to see "e", not e.
In addition, JSON does not accept the \x escaping format (character reference in hex); it can only use the \u format (unicode character reference in decimal). The code you've provided includes escaped characters in both formats.
The question is, are you using an official Google API? Because they're usually pretty good at providing valid JSON. This isn't valid JSON, so it may be that you're not using the correct API. Another clue is that the variable names aren't very meaningful; offical APIs would normally give more meaningful variable names. If it is the correct API, you should try raising a ticket with Google to fix it; broken JSON is not good, but it should be pretty trivial for them to fix.
Assuming you can't get them to fix it and we can't find an alternative API location that does give valid data, how do we deal with what we've got?
While this code may not be valid JSON, it is valid as a Javascript object (the JSON rules are stricter that those of plain Javascript). It could therefore be run in a Javascript interpreter using eval(), if you trusted it enough for that.
The only other alternativate is to fix the string prior to parsing it so that the variable names are quoted. That's a bit of a pain, but would be do-able if the output was consistent. You'll have problems though if it ever changes (and again, if it's an unofficial API, that could happen at any time without warning).

The problem is with the backslashes in the strings (used for the escape characters)
In PHP 5.4, you can use JSON_UNESCAPED_SLASHES:
echo json_encode(JSON_STRING, JSON_UNESCAPED_SLASHES);
Otherwise, you can do the replacement-
str_replace('\\/', '/', json_encode(JSON_STRING));
Since \/ is a valid way to represent /

OK, so this is what I ended up doing (not elegant at all but it works) :
$content = preg_replace_callback(
"(\\\\x([0-9a-f]{2}))i",
function($a) {return chr(hexdec($a[1]));},
$content
);
$content = str_replace("e:","\"e\":",$content);
$content = str_replace("c:","\"c\":",$content);
$content = str_replace("u:","\"u\":",$content);
$content = str_replace("p:","\"p\":",$content);
$content = str_replace("d:","\"d\":",$content);
$content = str_replace("\"[","[",$content);
$content = str_replace("]\"","]",$content);
$content = json_decode($content);

Related

PHP string parameter best practice

I have a PHP function that internally builds an object graph using among other things one string parameter, then uses json_encode() to create a JSON string and then post the JSON string to a remote web service.
Like this:
function send($text)
{
$payload = array(
'text' => $text
// Set additional properties here ...
);
$payload_json = json_encode($payload);
// Post $payload_json to remote service with Curl ...
}
From the manual of json_encode (http://php.net/manual/en/function.json-encode.php)
All string data must be UTF-8 encoded.
I see a couple of options:
Attempt to validate that $text is in fact UTF-8 and throw an exception if it is not
Attempt to detect the encoding in $text and convert it to UTF-8 if necessary
Return false when $text is not UTF-8
Communicate with my API users in documentation that $text must be UTF-8
Check for error with json_last_error() and throw an exception if an error was encountered
What is the best practice?
You should always communicate to the users of the API what you're expecting.
If you expect UTF-8 encoded text, say so in your documentation.
Once it's in there, you should return a descriptive error such as "Invalid Encoding, for more information read the documentation: link" where link goes to the relevant page for the call that failed
This way, you're not responsible anymore, and the developers that use your API will know what is going wrong, and you don't have to worry about it in your API.
You're the developer, it's your API, and your API has it's own rules, if people want to use it, they need to follow the rules you set.

json_decode syntax error from valid json [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 11 months ago.
Improve this question
I have this simple JSON string :
{"type_voie":"ALL","indice_repetition":"B","num_voie":"1","nom_voie":"Azrty","infos_voie":"Zzz","distribution_speciale":"","cp":"34230","ville":"PLAISSAN","bureau_distributeur":""}
That I get from my utf8 database.
When I pass it to json_decode($json, true), the result is NULL and json_last_error() give me 4 (JSON_ERROR_SYNTAX)
If I do mb_detect_encoding, I can see that the string is in ASCII, but I only use UTF-8 everywhere.
The string was originally inserted in database from an AJAX query. I got the date from a form serialized with the jQuery function serialize().
I did this a lot in my project and never had problem like this.
I tried to force UTF8 using iconv, mb_convert_encoding or ut8_encode, but nothing change.
I also tried with html_entity_decode() because sometimes PDO is replacing the quotes, but same result.
EDIT :
OH MY GOD.
SORRY.
For some reasons, a was concatenated between the moment I get the json and the moment I send it to the parser.
There was just PEBKAC....
I'm so ashamed.
Will delete this post soon
As we found in the comments that there are invisible characters in the string, consider running this bit of code. It should show you each character in the string, it'd ascii value and it's printed version.
There should be a few characters in there which do not print anything; those are the ones preventing you from json_encoding the string. I cannot see why they are in the string, that has something to do with your database and the data you post into it, but at least you'll know where the problem is.
Odds are they are contained inside the inserted data somewhere, which means you probably have a piece of code somewhere that add something unprintable to their input.
You can strip them while exporting, but it would be better to find out which code adds them and remove them from the database. They will continue to bother you in the future otherwise. (As you've seen at this point even something simple like strlen() will fail to work properly due to these characters)
for( $i = 0 ; $i < mb_strlen($string) ; $i++ ) {
echo 'pos: ' . $i . ' | ord: ' . ord( $string[$i] ) . ' | char: ' . $string[$i] . '<br />';
}
As guys pointed in previous answers, there can be unprintable characters in json, which means they should be removed before calling json_decode.
One way to do that is this:
$json = preg_replace('/[[:^print:]]/', '', $json);
I am not saying this is the best solution, but so far the only one I found that worked for me.
You have to encode your source with UTF-8 before using json_decode()
try with this code :
$json = mb_convert_encoding($json, "UTF-8");
var_dump(json_decode($json, true));
PHP CURL to API with JSON response
Syntax error, malformed JSON
json_last_error() === 4
JSON_ERROR_SYNTAX
In my situation, I was using PHP CURL to get JSON data from a local API.
I accidentally included HTML in the API's JSON response which caused the error.
After removing the inserted HTML from the API response page, json_decode() executed properly.
Why did this error occur?
CURL requests return the entire page content's as a string. This becomes problematic for json_decode() if content other than JSON is included in the string.
When print/echo'ing a CURL's JSON string on a web page, the string's content can be hidden within the HTML document.
Tip: use var_dump() to compare string lengths to determine if extra content has been included in the JSON string or inspect the page source code to determine if additional code has been included.
your valid json may be like this add [] in your array then generate json
[{"type_voie":"ALL","indice_repetition":"B","num_voie":"1","nom_voie":"Azrty","infos_voie":"Zzz","distribution_speciale":"","cp":"34230","ville":"PLAISSAN","bureau_distributeur":""}]

PHP json_decode returns null

I'm writing PHP code that uses a database. To do so, I use an array as a hash-map.
Every time content is added or removed from my DB, I save it to file.
I'm forced by my DB structure to use this method and can't use mysql or any other standard DB (School project, so structure stays as is).
I built two functions:
function saveDB($db){
$json_db = json_encode($db);
file_put_contents("wordsDB.json", $json_db);
} // saveDB
function loadDB(){
$json_db = file_get_contents("wordsDB.json");
return json_decode($json_db, true);
} // loadDB
When echo-ing the string I get after the encoding or after loading from file, I get a valid json (Tested it on a json viewer) Whenever I try to decode the string using json_decode(), I get null (Tested it with var_dump()).
The json string itself is very long (~200,000 characters, and that's just for testing).
I tried the following:
Replacing single/double-quotes with double/single-quotes (Without any backslashes, with one backslash and three backslashes. And any combination I could think of with a different number of backslashes in the original and replaced string), both manually and using str_replace().
Adding quotes before and after the json string.
Changing the page's encoding.
Decoding without saving to file (Right after encoding).
Checked for slashes and backslashes. None to be found.
Tried addslashes().
Tried using various "Escape String" variants.
json_last_error() doesn't work. I get no error number (Get null, not 0).
It's not my server, so I'm not sure what PHP version is used, and I can't upgrade/downgrade/install anything.
I believe the size has something to do with it, because small strings seem to work fine.
Thanks Everybody :)
In your JSON file change null to "null" and it will solve the problem.
Check if your file is UTF8 encoded. json_decode works with UTF8 encoded data only.
EDIT:
After I saw uploaded JSON data, I did some digging and found that there are 'null' key. Search for:
"exceeding":{"S01E01.html":{"2217":1}},null:{"S01E01.html":
Change that null to be valid property name and json_decode will do the job.
I had a similar problem last week. my json was valid according to jsonlint.com.
My json string contained a # and a & and those two made json_decode fail and return null.
by using var_dump(json_decode($myvar)) which stops right where it fails I managed to figure out where the problem was coming from.
I suggest var_dumping and using find dunction to look for these king of characters.
Just on the off chance.. and more for anyone hitting this thread rather than the OP's issue...I missed the following, someone had htmlentities($json) way above me in the call stack. Just ensure you haven't been bitten by the same and check the html source.
Kickself #124

Not able to parse this json

I am trying to parse the json output from
http://www.nyc.gov/portal/apps/311_contentapi/services/all.json
And my php json_decode returns a NULL
I am not sure where the issue is, I tried running a small subset of the data through JSONLint and it validated the json.
Any Ideas?
The error is in this section:
{
"id":"2002-12-05-22-24-56_000010083df0188b4001eb56",
"service_name":"Outdoor Electric System Complaint",
"expiration":"2099-12-31T00:00:00Z",
"brief_description":"Report faulty Con Edison equipment, including dangling or corroded power lines or "hot spots.""
}
See where it says "hot spots." in an already quoted string. Those "'s should've been escaped. Since you don't have access to edit the JSON perhaps you could do a search for "hot spots."" and replace it with \"hot spots.\"" like str_replace('"hot spots.""', '\\"hot spots.\\""\, $str); for as long as that's in there. Of course that only helps if this is a one time thing. If the site continues to make errors in their JSON output you'll have to come up with something more complex.
What I did to identify the errors in the JSON ...
Since faulty quoting is the first thing to look for, I downloaded the JSON to a text file, opened in a text editor (I used vim but any full featured editor would do), ran a search and replace that removed all characters except double-quote and looked at the result. It was clear that correct lines should have 4 double-quotes so I simply searched for 5 double-quotes together and found the first bad line. I noted the line number and then undid the search and replace to get the original file back and looked at that line. This gives you what you need to get the developers of the API to fix the JSON.
Writing code to automatically fix the bad JSON before giving it to json_decode() would be quite a bit harder but doable using techniques like those in another answer.
According to the PHP manual:
In the event of a failure to decode, json_last_error() can be used to determine the exact nature of the error.
Try calling it to see where the error is.

Handling malformed JSON in PHP

I'm trying to write a php script that handles data from a webservice that delivers "json" as a string. The problem is the string isn't really json; it's javascript. Specifically, the keys are not quoted, although the variables are. Example (the actual data is much longer and more complicated):
{desc:'User defined payload'}
As described by the php manual, json_decode() correctly fails to interpret this string.
My question is, how can I successfully interpret a string like this in php?
The only solution I can think of is to write some regular expressions that fix the syntax, but then I'd have two problems.
EDIT
Hadvig's suggestion of using the Services_JSON pear module worked, and looks like a general solution. Once I had the module installed, my code looked like this:
require_once 'PEAR.php';
require_once 'Services/JSON.php';
$Services_JSON = new Services_JSON();
$data = $Services_JSON->decode($malformed_json);
Unfortunately, this is SLOW. To interpret the whole string (~400,000 chars) took > 36 seconds! Using a regular expression to fix the quotes and then using json_decode took ~0.04 seconds. Here's what I used:
// fix single quotes
$s = str_replace("'", '"', $malformed_json);
// fix unquoted keys
$valid_json = preg_replace('/([{\[,])\s*([a-zA-Z0-9_]+?):/', '$1"$2":', $s);
$data = json_decode($valid_json);
Of course, this will break if the data contains any quotes, brackets, or commas.
Ok. try to use this. http://pear.php.net/pepr/pepr-proposal-show.php?id=198 I just check your string
Depends on how complicated your data is :
$output = "{desc:'User defined payload',asc:'whatever'}";
function json_js_php($string){
$string = str_replace("{",'{"',$string);
$string = str_replace(":'",'":"',$string);
$string = str_replace("',",'","',$string);
$string = str_replace("'}",'"}',$string);
return $string;
}
echo json_decode(json_js_php($output))->{'desc'};
returns : User defined payload
Using regexp is a no-go. JSON grammar cannot be correctly parsed using regexp. You will open yourself to a ton of future bugs.
I recommend using some kind of a YAML parser. YAML is backwards-compatible with JSON and allows unquoted literals at the same time.
Symfony YAML component worked great for me.
And remember that there will be a performance penalty in comparison to json_decode because it's implemented natively.
If the problem is just the unquoted identifiers and the data can be assumed not to contain any curly brackets, this should do it:
$goodJson = preg_replace("/{\s*([a-zA-Z0-9_]+)/", '{ "$1"', $badJson);
(not tested!)
Try this:
$jsonString = "{result:true,username:'usr000242',password:'123456',message:'Cannot send username and password to email#test.com'}";
function manualFixInvalidJSON($jsonString=''){
$jsonString = preg_replace("/([{,])([a-zA-Z][^: ]+):/", "\$1\"$2\":", $jsonString);
$jsonString = preg_replace("/:([a-zA-Z\'][^:]+)([,}])/", ":\"$1\"$2", $jsonString);
$jsonString = json_decode($jsonString,true);
function trimer($val){
return trim(trim($val,"'"),"\"");
}
$jsonString = array_map('trimer', $jsonString);
return json_encode($jsonString);
}
echo jsonString($jsonString);

Categories