PHP-generarted JSON File Errors - php

I am currently using a PHP script to create a cache of JSON files. We have a PHP file on our server that queries a very large database, and returns result sets in JSON format (so we can use it on the site in jQuery and other frameworks).
The script for presenting the raw JSON from our database works great, and I made a simple cache building script that also works to a degree. However, we have noticed some odd things happening with the resulting cache file.
PHP seems to be adding slashes to the quote marks, as well as adding superfluous " to the beginning and end of the JSON.
Here is a sample of the JSON we're passing in (note that it's not complete):
[
{
"id":1580,
"name":"Fydell House",
"address1":"South Square",
"address2":null,
"towncity":"Boston",
"county":"Lincolnshire",
"postcode":"PE21 6HU",
"addressVerbose":"South Square
Boston
Lincolnshire
PE21 6HU
",
"addressVerboseLinked":"",
"longitude":-0.022778,
"latitude":52.975806,
"londonBorough":null,
"telno":"01205 351 520",
"faxno":null,
"email":null,
"webaddress":null,
"mainimg":null,
"venueType":"Historic Building",
"description":null,
"excerpt":" ",
"images":null,
"creationDate":943920000,
"londonfeatured":false,
"unusual":false,
"featured":false,
"active":true,
"trial":false,
"modifiedDate":1234709308,
"hits":"1579",
"allowReviews":false,
"amenities":null,
"imagealt":"Lincolnshire wedding reception venue in Boston, Fydell House",
"imagetitle":"Lincolnshire wedding venues in Boston",
"car_directions":null,
"pub_directions":null,
"additional_info":null,
"listedBy":0,
"listedByName":null,
"region":null
}
]
And the PHP code that outputs the JSON file and stores it on our server:
// Function to output the contents from the live data, and create a cache file:
function create_cache_file($url, $filename)
{
$url = str_replace(' ', '%20', $url);
$json_string = file_get_contents($url);
$file = fopen($filename, 'w');
// If there is a problem creating the file:
if(!$file)
{
die('error creating the file!');
}
else
{
fwrite($file, json_encode($json_string));
fclose($file);
echo $json_string;
}
}
And this is what the file looks like after it's been processed with PHP and stored on our server:
"[{\"id\":437,\"name\":\"Lanteglos Country House Hotel\",\"address1\":\"Lanteglos-by-Camelford\",\"address2\":null,\"towncity\":\"Camelford\",\"county\":\"Cornwall\",\"postcode\":\"PL32 9RF\",\"addressVerbose\":\"Lanteglos-by-Camelford<br \\\/>Camelford<br \\\/>Cornwall<br \\\/>PL32 9RF<br \\\/>\",\"addressVerboseLinked\":\"\",\"longitude\":-4.695491,\"latitude\":50.612462,\"londonBorough\":null,\"telno\":\"01840 213 551\",\"faxno\":null,\"email\":null,\"webaddress\":null,\"mainimg\":null,\"venueType\":\"Hotel\",\"description\":null,\"excerpt\":\" \",\"images\":null,\"creationDate\":943920000,\"londonfeatured\":false,\"unusual\":false,\"featured\":false,\"active\":true,\"trial\":false,\"modifiedDate\":1234662248,\"hits\":\"1145\",\"allowReviews\":false,\"amenities\":null,\"imagealt\":\"Cornwall wedding reception venue in Camelford, Lanteglos Country House Hotel\",\"imagetitle\":\"Cornwall wedding venues in Camelford\",\"car_directions\":null,\"pub_directions\":null,\"additional_info\":null,\"listedBy\":0,\"listedByName\":null,\"region\":null},{\"id\":438,\"name\":\"Rosehill Public House\",\"address1\":\"Little Petherick\",\"address2\":null,\"towncity\":\"Wadebridge\",\"county\":\"Cornwall\",\"postcode\":\"PL27 7QT\",\"addressVerbose\":\"Little Petherick<br \\\/>Wadebridge<br \\\/>Cornwall<br \\\/>PL27 7QT<br \\\/>\",\"addressVerboseLinked\":\"\",\"longitude\":-4.94093,\"latitude\":50.51259,\"londonBorough\":null,\"telno\":\"01841 540 777\",\"faxno\":null,\"email\":null,\"webaddress\":null,\"mainimg\":null,\"venueType\":\"Inn \\\/ Pub\",\"description\":null,\"excerpt\":\" \",\"images\":null,\"creationDate\":943920000,\"londonfeatured\":false,\"unusual\":false,\"featured\":false,\"active\":true,\"trial\":false,\"modifiedDate\":1234752874,\"hits\":\"818\",\"allowReviews\":false,\"amenities\":null,\"imagealt\":\"Cornwall wedding reception venue in Wadebridge, Rosehill Public House\",\"imagetitle\":\"Cornwall wedding venues in Wadebridge\",\"car_directions\":null,\"pub_directions\":null,\"additional_info\":null,\"listedBy\":0,\"listedByName\":null,\"region\":null}]"
This is causing some serious problems elsewhere on the site when we try to decode the JSON. While we can use stripslashes to remove the slashes, some of the other areas are also causing parse errors (that weren't present using the raw data served up by the first PHP file).
Can anyone suggest why PHP is adding slashes and erroneous quote marks around the string? Ideally, we would like a work-around at the point of creating the JSON file on the server, not reading it from it...

I take it that file_get_contents($url) returns you a JSON encoded file? Then your problem is that you're JSON encoding it again. "[{\"id\":437,\"nam... is the proper JSON encoded representation of the string [{"id":1580,"nam....
If you json_decoded it when reading from the cache file, you'd get the original JSON string back. Or just don't json_encode it.

Related

Trying to use a JSON API that isn't technically an API

I just started playing D&D, and I'm working on a project for my group to help us play the game a littler easier. We all created Characters on D&D Beyond. They don't have an official API, but apparently, if you type in "your-character-sheet/json" - you can get some JSON output about your character.
I attempted to grab this data in php as I normally would
<?php
$request = "https://www.dndbeyond.com/profile/PixiiBomb/characters/9150025/json";
$url = file_get_contents($request);
$json = json_decode($url, true);
echo $json["character"]["id"];
?>
This should echo out: "9150025" - but I actually see nothing. It's totally blank. So I tried to echo $url, and this is what I see.
Which of course, it not what I should be seeing (because that is formatting from their website, minus the stylesheets)
I went back to the JSON view and manually saved the data as pixii.json This time using the following:
$request = "uploads/dndbeyond_sheets/pixii.json"; // Saved to my harddrive
$url = file_get_contents($request);
$json = json_decode($url, true);
echo $json["character"]["id"];
THIS WORKS if I manually save the file. But that would mean that every time we level up, I would have to manually save all of our JSON data, and run some code to read it and update it.
Is there something else I can try to allow me to use this URL instead of having to manually save the page?
Ideally, I would like to have all of my code written , and then when we level up, I press a button and grab the new data from the site (which I don't need help with, that's not part of the question). Instead of manually visiting each website, saving the data, and then parsing it.

Reading or Writing multiple JSON objects to a single file

I have built a script to scrape some records and store them as a json object for later use. This is the last step in the scraping process (Area->Location->ReportMeta->ReportDetails) that are all working fine with this method of storing the data.
The issue is there are a lot of them, on the order of several hundred thousand. I tried accumulating them all into an array then encoding and writing that to a file, but it's maxing out memory before it gets close to being finished. I could up the memory but I am looking for a more stable/replicatable/"out of the box" way of doing this. Best practice if you will.
My first thought was just write them to the file as each one is scraped. That is working but I am left with a single file with many individual json objects which is nigh unreadable unless I do some special formatting to bring it back in.
I am looking for a better way of doing this, or some advice.
$reports_obj = new Report();
foreach($reports_array as $report){
$report_details = $reports_obj->getReport($report['report_id'], $report['report_type']);
$fp = fopen('report_details.json', 'a');
fwrite($fp, json_encode($report_details));
fclose($fp);
}
This leaves me with a whole bunch of this:
{
"report_id": "12345",
"report_type": "Type A",
"facility_name": "Name here",
"facility_type": "building",
"report_date": "26-February-2018"
}
{
"report_id": "12345",
"report_type": "Type A",
"facility_name": "Name here",
"facility_type": "building",
"report_date": "26-February-2018"
}
{
"report_id": "12345",
"report_type": "Type A",
"facility_name": "Name here",
"facility_type": "building",
"report_date": "26-February-2018"
}
Would it be best to try and find/replace the large file after the fact with proper json structure, or is there a better way of storing this? I can't open the file, re-read the data and then array push, for instance, as this would ultimately have the same limitation as just accumulating them all into an array to begin with.
As for "why" json? It's just a soft preference. I would like to stay with it if possible.
Maybe , you can try this :
$reports_obj = new Report();
foreach($reports_array as $report){
$report_details[] = $reports_obj->getReport($report['report_id'],$report['report_type']);
}
$jsonjson=json_encode($report_details);
$report="{\"report\":".$jsonjson."}";
$fp = fopen('report_details.json', 'a');
fwrite($fp,$report);
fclose($fp);
if you have , sample, maybe i can check ?
You should look for a NoSQL database.
If you dont want/cant for what ever reason it is better to Loop through all reports, generate the JSON and write after instead of opening and closing a file each time
$result="";
foreach($reports_array as $report){
$report_details = $reports_obj->getReport($report['report_id'], $report['report_type']);
$result .= json_encode($report_details)."\n\r";
}
$fp = fopen('report_details.json', 'a');
fwrite($fp,$result);
fclose($fp)

CakePHP 2 requestAction breaks my JSON encoding to Swift... invisibly

I've got CakePHP 2 outputting JSON, and I've got Swift 2 code in the Xcode iPhone emulator loading that JSON data.
In my Controller, here's the code that generates the JSON:
public function iosDemoIndex() {
$studentDataIn = $this->Student->getStudentsByUserId(298, 'rASC');
$studentDataOut = [];
foreach ($studentDataIn as $student) {
//$currentIep = $this->Iep->requestAction('/ieps/getCurrentIep/' . $student['Student']['id']);
$goals = $this->Goal->find('all', array('conditions' => array('Goal.student_id =' => $student['Student']['id'])));
$goalHolder = array();
foreach ($goals as $goal) {
array_push($goalHolder, $goal['Goal']['goal_title']);
}
array_push($studentDataOut, array('studentName' => $student['Student']['l_name']. ", " . $student['Student']['f_name'], 'studentGoals' => $goalHolder));
}
$this->set(array(
'students' => $studentDataOut,
'_serialize' => array('students')
));
}
Which works fine. My Swift code succesfully loads the students array via the JSON into an array of structs. But if I simply un-comment line 5 (which grabs data that I ultimately hope to use to restrict the find in line 6), Xcode gives me:
Error with Json: Error Domain=NSCocoaErrorDomain Code=3840 "Invalid value around character 2." UserInfo={NSDebugDescription=Invalid value around character 2.}
Line 5 shouldn't alter the JSON document in any way. And when I look at the JSON itself, it looks the same regardless of whether the line's commented out.
So, any ideas what an unrelated, ought-to-be invisible requestAction could be doing that's breaking my JSON... in a way that I can't see when I look at it? Thanks.
For posterity (future-me):
It was a login issue. The reason the JSON looked identical in my browser was that I had an active user session in the browser at the time. Without that session, (for instance when the iOS emulator tried to load the JSON page) the CakePHP was returning a redirect to login. Viable solutions include replicating the requestAction code in the JSON-generating controller (though that's not very DRY), or establishing proper login handling on the Swift side of things.

Parse iTunes API (which uses JSON) w/ PHP

This is the first time I have came in contact with JSON, and I literally have no idea how to parse it with PHP. I know that functions to decode JSON exist in PHP, but I am unsure how to retrieve specific values defined in the JSON. Here's the JSON for my app:
http://itunes.apple.com/search?term=enoda&entity=software
I require a few values to be retrieved, including the App Icon (artworkUrl100), Price (price) and Version (version).
The things I am having issues with is putting the URL of the App Icon into an actual HTML image tag, and simply retrieving the values defined in the JSON for the Price and Version.
Any help/solutions to this would be fantastic.
Thanks,
Jack
Yeah, i have something similar, for my App review website, here is a bit code:
$context = stream_context_create(array('http' => array('header'=>'Connection: close')));
$content = file_get_contents("http://ax.phobos.apple.com.edgesuite.net/WebObjects/MZStoreServices.woa/wa/wsLookup?id=$appid&country=de");
$content = json_decode($content);
$array = $content->results["0"];
$version = $array->version;
$artistname = $array->artistName;
$artistid = $array->artistId;
Thats what I used to get Information from the AppStore, maybe you can change the link and some names and it would work for you.

Extract/scrape specific data from JSON file

This been bugging me for quite a few hours. I've been searching a lot and I have found a lot of information. The problem is, I'm not that good, I'm actually a beginner to the max. I'd like to achieve this with Python (if it's possible!). Maybe with JavaScript and PHP also? Let me explain.
I just found this website http://listeningroom.net and it's great. You can create/join rooms and upload tracks and listen to them together with friends.
I'd like to extract/scrape/get some specific data from a .json file.
This file contains artist, album title, track title and more. I'd like to extract just the artist, album and track title.
http://listeningroom.net/room/chillasfuck/spins.json The .json file Contains the tracks played in the past 24 hours.
I managed to scrape the whole .json file with Python after looking around, (local .json file) with the following probably not so valid code.
json_data=open('...\spins.json')
data = json.load(json_data)
pprint(data)
json_data.close()
This prints out the following:
[{u'endTime': u'1317752614105',
u'id': u'cf37894e8eaf886a0d000000',
u'length': 492330,
u'metadata': {u'album': u'Mezzanine',
u'artist': u'Massive Attack',
u'bitrate': 128000,
u'label': u'Virgin',
u'length': 17494.479054779807,
u'title': u'Group Four'},
Just a part of the print
1. I'd like to scrape it from an url (the one provided at the top)
2. Just get 'album', 'artist' and 'title'
3. Make sure it prints it as simple as possible like this:
Artist
Track title
Album
Artist
Track title
Album
4. If it's not too much, save it to a .txt file
I hope I could get some help, I really want to create this for myself, so I can check out more music!
Marvin
Python (after you loaded the json)
for elem in data:
print('{artist}\n{title}\n{album}\n'.format(**elem['metadata']))
To save in a file:
with open('the_file_name.txt','w') as f:
for elem in data:
f.write('{artist}\n{title}\n{album}\n\n'.format(**elem['metadata']))
You're already really close.
data = json.load(json_data)
is taking the JSON string and converting it to a Python object - in this case, a list of dictionaries (plus 'metadata', which is a dictionary of dictionaries).
To get this into the format that you want, you just need to loop through the items.
for song in data:
artist = song['metadata']['artist'] # This tells it where to look in the dictionary. It's looking for the dictionary item called 'metadata'. Then, looking inside that dictionary for 'artist'.
album = song['metadata'['album']
songTitle = song['metadata']['title']
print '%s\n%s\n%s\n' % (artist, album, songTitle)
Or, to print it to a file:
with open('the_file_name.txt','w') as f:
for song in data:
artist = song['metadata']['artist']
album = song['metadata'['album']
songTitle = song['metadata']['title']
f.write('%s\n%s\n%s\n' % (artist, album, songTitle))
Okay this is a bit short but the thing about json is that it translate an array into a string
eg.
array['first'] = 'hello';
array['second'] = 'there';
will become
[{u'first': u'hello', u'second': 'there'}];
after a jsonencode
run that sting throu jsondecode and you get your array back
so simply run you json file thou a decoder and then you should be able to reach your data through:
array['metadata'].album
array['metadata'].artist
...
have never used python but it should be the same.
have a look at http://www.php.net/manual/en/function.json-decode.php it might clear upp a thing or two.
For PHP you need json.decode
<?php
$json = file_get_contents($url);
$val = json_decode($json);
$room = $val[0]->metadata;
echo "Album : ".$room->album."\n";
echo "Artist : ".$room->artist."\n";
echo "Title : ".$room->title."\n";
?>
Outputs
Album : Future Sandwich
Artist : Them, Roaringtwenties
Title : Fast Acting Nite-Nite Spray With Realistic Uncle Beard
Note its a truck load of JSON data there so you'll have to iterate adequately

Categories