How would I scrape this JSON info using PHP and MySQL? - php

Here's the info I'm trying to break up into a database. I'm going to be using this only for my own use to analyse statistics and all that. I have been manually doing it with Excel but I'd like to save myself some work in future.
URL IS: http://fantasy.premierleague.com/web/api/elements/537/
Any idea how to scrape that info or easily convert it to excel format? I know a bit of php and mysql, but nothing about JSON and very little about scraping (I tried messing with SIMPLE_HTML_DOM).

You need to JSON_decode the data in PHP.
$obj = JSON_decode($mydata));
print_r($obj);
Extra information for you:
http://php.net/manual/en/function.json-decode.php

You can convert it into an array as
$array = json_decode(file_get_contents('http://fantasy.premierleague.com/web/api/elements/537/'));
json_decode()
You could use PEAR excel writer to convert it into excel

PHP has a json parser function json_decode().
So:
Use the file_get_contents() function to read the json content from the URL into a string.
Use json_decode() to create a PHP structure representation.
Use PEAR Spreadsheet_Excel_Writer module to create your excel spreadsheet.
Yeah. Easy as 1, 2, 3.

<?php
$x=json_decode(file_get_contents('http://fantasy.premierleague.com/web/api/elements/537/'));
print_r($x);//$x will contain all the values in an array format.
?>

<?php
// Create a stream
$opts = array(
'http'=>array(
'method'=>"GET"
)
);
$context = stream_context_create($opts);
// Open the file using the HTTP headers set above
$json= file_get_contents('http://fantasy.premierleague.com/web/api/elements/537/', false, $context);
$arr= json_decode($json);
$xml = new SimpleXMLElement('<root/>');
array_walk_recursive($arr, array ($xml, 'addChild'));
print $xml->asXML();

PHP makes it very easy:
$str = file_get_contents('http://fantasy.premierleague.com/web/api/elements/537/');
$jsonarray = json_decode($str, true);
var_dump($jsonarray);
Of course you will have to analyze the structure of the array and figure out how to decompose it to what you are actually looking for.

Try using $obj = json_decode($jsonStr) after you receive your response string by running curl on the URL you mentioned in PHP. Then you can grab params from the json object like
$obj['paramName'];
Then you can do anything you want with the information including putting it into a database.
For simple MySQL interaction in php, check out MySQLConnector class.
http://jakesankey.com/blog/2011/12/php-mysql-helper-class/

use just json_decode and get the converted data like this edit
$arr = json_decode('your JSON data',true);
echo $arr['transfers_out']; // output 490374 //for array
echo $arr->transfers_out; // output 490374 //for stdClass

Related

Decode json With PHP not working

I am trying to decode json with PHP but dont know where am i wrong. Here is my code
$rr ='var modelsGlobal = [{"value":"FAFW3801LW","productdetailurl":"/Washers-Dryers/Washers/Front-Load/FAFW3801LW/"}{"value":"FAFW3801LW","productdetailurl":"/Washers-Dryers/Washers/Front-Load/FAFW3801LW/"}]';
$json = json_decode($rr, true);
foreach($json['modelsGlobal'] as $json){
$prod_id = $json["value"];
}
Please help
You are trying to decode (broken) JavaScript, not JSON.
JSON wouldn't include var modelsGlobal = and array members need a , between them.
Run your data through a linter.
After you fix the errors which are preventing the parsing, the JSON doesn't start with an object with a modelsGlobal, so loop over the array in $json directly.
Your JSON is incorrect. It is not JSON but JavaScript and it lacks a comma to separate the two objects of the array.
If you use PHP 5.3+ use json_last_error to check errors with json_encode/json_decode.

JSON Parsing Syntax - Simple PHP

I have the following json output - I've tried multiple things however not having any luck - I just want to get the value for kind (software-package). Any ideas:
{"items":
[{"assets":
[{"kind":"software-package","url":"__URL__"}],
"metadata":{"bundle-identifier":"SimpleCalculator",
"bundle-version":"000","kind":"software",
"title":"com.work.demo","subtitle":"1.0"}
Thanks,
Try to decode it with json_encode like
$result_arr = json_decode($my_arr,true);
print_r($result_arr['items']['assets']['kind']);
That is not valid JSON.
Lets attempts to format that a little more sensibly.
{"items": [
{"assets": [
{"kind":"software-package","url":"__URL__"}
],
"metadata":{"bundle-identifier":"SimpleCalculator",
"bundle-version":"000","kind":"software",
"title":"com.work.demo","subtitle":"1.0"
}
You're missing a lot of closing brackets. :(
How are you acquiring this JSON. Is it hand formatted or generated by some software? They're may be bugs in the source, and not your code.
If you are using a version of PHP above 5.3 then there are built-in functions to decode and encode JSON strings already.
$some_json_string = {......};
$json_string_as_object = json_decode($some_json_string);
$json_string_as_array = json_decode($some_json_string, true);
You can also do the reverse:
$some_array = array(...);
$json_string = json_encode($some_array);
Json Decode : http://php.net/manual/en/function.json-decode.php
Json Encode: http://www.php.net/manual/en/function.json-encode.php

Simple JSON request in PHP

I have the following json
country_code({"latitude":"45.9390","longitude":"24.9811","zoom":6,"address":{"city":"-","country":"Romania","country_code":"RO","region":"-"}})
and i want just the country_code, how do i parse it?
I have this code
<?php
$json = "http://api.wipmania.com/jsonp?callback=jsonpCallback";
$jsonfile = file_get_contents($json);
var_dump(json_decode($jsonfile));
?>
and it returns NULL, why?
Thanks.
<?php
$jsonurl = "http://api.wipmania.com/json";
$json = file_get_contents($jsonurl);
var_dump(json_decode($json));
?>
You just need json not jsonp.
You can also try using json_decode($json, true) if you want to return the array.
you're requesting jsonp with http://api.wipmania.com/jsonp?callback=jsonpCallback, which returns a function containing JSON like:
jsonpCallback({"latitude":"44.9718","longitude":"-113.3405","zoom":3,"address":{"city":"-","country":"United States","country_code":"US","region":"-"}})
and not JSON itself. change your URL to http://api.wipmania.com/json to return pure JSON like:
{"latitude":"44.9718","longitude":"-113.3405","zoom":3,"address":{"city":"-","country":"United States","country_code":"US","region":"-"}}
notice the second chunk of code doesn't wrap the json in the jsonpCallback() function.
The website doesn't return pure JSON, but wrapped JSON. This is meant to be included as a script and will call a callback function. If you want to use it, you first need to remove the function call (the part until the first paranthesis and the paranthesis at the end).
If your server implements JSONP, it will assume the callback parameter to be a JSONP signal and the result will be similar to a JavaScript function, like
jsonpCallback("{yada: 'yada yada'}")
And then, json_decode won't be able to parse jsonpCallback("{yada: 'yada yada'}") as a valid JSON string
If country_code( along with closing parenthesis are include in your json, remove them.
This is not a valid json syntax: json
You are being returned JSONP, not JSON. JSONP is for cross-domain-requests in JavaScript. You don't need to use it when using PHP because you aren't affected by cross-domain-policies.
Since you are getting a string from the file_get_contents() function you can do a replacement of the country_code( text (this is the JSONP specific part of the response):
<?php
$json = "http://api.wipmania.com/jsonp?callback=jsonpCallback";
$jsonfile = substr(file_get_contents($json)), 13, -1);
var_dump(json_decode($jsonfile));
?>
Note
This works but JKirchartz's solution looks better, just request the correct data rather than messing around with the incorrect data.
Obviously in this situation, using the correct URL to access the API will return pure jSON.
"http://api.wipmania.com/json"
A lot of people are providing an alternative to the API in use, rather than answering the OP's question, so here is a solution for those looking for a way of handling jSONp in PHP.
First, the API allows you to specify a callback method, so you can either use Jasper's method of getting the jSON sub string, or you can give a callback method of json_decode, and modify the result to use with a call to eval. This is my alternative to Jasper's code example since I don't like to be a copy cat:
$json = "http://api.wipmania.com/jsonp?callback=json_decode";
$jsonfile eval(str_replace("(", "('", str_replace(")", "')", file_get_contents($json)))));
var_dump($jsonfile);
Admittedly this seems a little longer, more insecure, and not as clear to read as Jasper's code:
$json = "http://api.wipmania.com/jsonp?callback=jsonpCallback";
$jsonfile = substr(file_get_contents($json)), 13, -1);
var_dump(json_decode($jsonfile));
Then the jSON "address":{"city":"-","country":"Romania","country_code":"RO","region":"-"} tells us to access the country_code like so:
$jsonfile->{'address'}->{'country_code'};

PHP json_encode not returning valid json

I am running a Debian box with PHP v5.2.17. I am trying to get around the cross-domain issue with an XML file and am using this got to fetch any xml and return json:
<?php
header('content-type: application/json; charset=utf-8');
if( strlen($_GET["feed"]) >= 13 ) {
$xml = file_get_contents(urldecode($_GET["feed"]));
if($xml) {
$data = #simplexml_load_string($xml, "SimpleXMLElement", LIBXML_NOCDATA);
$json = json_encode($data);
echo isset($_GET["callback"]) ? "{$_GET[’callback’]}($json)" : $json;
}
}
?>
The problem is, its not returning valid json to jquery.. The start character is "(" and the end is ")" where jquery wants "[" as the start and "]" as the end. I've taken the output and used several online validation tools to check it..
Is there a way I can change these characters prior to sending back or pass json_encode options?
You could change json_encode($data) to json_encode(array($data)) if it expects an array (like you're saying):
$json = json_encode(array($data));
EDIT: Also, I believe the SimpleXml call will result in a bunch of SimpleXmlElements, perhaps json_encode then thinks it should be objects, instead of arrays? Perhaps casting to an array will yield the correct results.
You cannot json_encode() SimpleXMLElements (that's the type that is returned by simplexml_load_string(). You have to convert the data from the XML file into some native PHP type (most likely an array).
SORRY that's wrong. json_encode() can in fact encode SimpleXMLElements (at least on my PHP version 5.3.4). So if your client-side code expects an array you must wrap your $data in an array:
$json = json_encode(array($data));
We can use json_encode() function most probably on array. so you first take XML content into PHP array and then apply json_encode().I think this will solve your problem..
It seems that you are sending an empty callback parameter or something, but the callback parameter in jQuery must look exactly like this: callback=?

Get a JSON result into a PHP array. How?

This is what I tried:
$doc = new DOMDocument();
$jsonurl = "http://v1.syndication.nhschoices.nhs.uk/.json?apikey=xxxxxx";
$doc->load($jsonurl);
var_dump(json_decode($doc));
var_dump(json_decode($doc, true));
The output is NULL NULL for the 2 var_dumps.
The JSON returned from the url looks like this (after view source):
[{"Text":"Live Well","Uri":"http:\/\/v1.syndication.nhschoices.nhs.uk\/livewell?apikey=xxxxx"},{"Text":"Conditions","Uri":"http:\/\/v1.syndication.nhschoices.nhs.uk\/conditions?apikey=xxxxx"},{"Text":"Organisations","Uri":"http:\/\/v1.syndication.nhschoices.nhs.uk\/organisations?apikey=xxxxx"},{"Text":"Carers Direct","Uri":"http:\/\/v1.syndication.nhschoices.nhs.uk\/carersdirect?apikey=xxxxx"}]
Is this valid?
If that URL returns a JSON string, you should not use the DOMDocument class to load it : that class is to manipulate XML data.
Instead, you probably should do something like this, using file_get_contents() to do the HTTP request, and get its raw result :
$url = 'http://v1.syndication.nhschoices.nhs.uk/.json?apikey=xxxxxx';
$json_string = file_get_contents($url);
$data = json_decode($json_string);
var_dump($data);
As a sidenote : using file_get_contents() to make HTTP requests will only work if allow_url_fopen is enabled.
If allow_url_fopen is not enabled, you'll have to be ready to fallback to a solution based on curl.
You're trying to somehow decode a DOMDocument. json_decode works on strings. What is the relationship between a DOM document and a JSON string? Not very much at all. Pick one!
#Pascal MARTIN
With a slight modification it works:
$jsonurl = 'http://v1.syndication.nhschoices.nhs.uk/.json?apikey=XXXXXX';
$jsonstr = json_encode(file_get_contents($jsonurl));
$data = json_decode($jsonstr);
var_dump(json_decode($data));
I added json_encode around the file_get_contents call.
Thanks all for the help :)
http://v1.syndication.nhschoices.nhs.uk/.json
I think your url is not valid? /.json maybe should be /something.json or maybe /json

Categories