I have a rather big json file with coordinates in the following format
"[[3.2,1],[4.8,2]]"
which represents (3.2,1) and (4.8,2)
I'm using these coördinates to generate a D3 geographic map, but when php is modelling this information into a geoJSONobject I encounter the following error:
I need to transform the coordinates into a array for which I use json_decode. However:
json_decode("[[3.2,1],[4.8,2]]")
returns
Array
(
[0] => Array
(
[0] => 3
[1] => 1
)
[1] => Array
(
[0] => 4
[1] => 2
)
)
Where I lose the decimals. How can I prevent this?
Edit:
{"type": "FeatureCollection",
"features": [{
"type": "Feature",
"geometry": {
"type": "Polygon",
"coordinates": "[[[8.7, 11], [8.89, 12.13],[9.27, 12.13], [9.9, 12], [9.7, 10.8], [8.7, 11]]]"
},
"properties": {
"name": "04",
"count": "25"
}
}]
}
This is an example of the data I'm getting as output. (It is supposed to represent a map of rooms which are get a density color by its usage)
I am able to parse this using jQuery.parseJSON(data), but running the following D3 code generates the weirdest errors:
val(svgname).append("g")
.selectAll("path")
.data(geoJSONobject.features)
.enter().append("path")
.attr("d", path)
...
I think it's because of the quotes around the array of coordinates.
Edit (2) - actual solution
The solution I accepted was a workaround, but the true issue was localized php-settings. using:
echo json_encode($dataset, JSON_NUMERIC_CHECK);
in the php-file, all the issues were resolved. Though I'd update the question since it is still being looked at (if anyone would encouter the issue)
I had the same problem. I solved it using the followin regex
SOLUTION 1
$yourJsonVariable = preg_replace('/:\s*(\-?\d+(\.\d+)?([e|E][\-|\+]\d+)?)/', ': "$1"', $yourJsonVariable);
Convert it into array
$array = json_decode($yourJsonVariable, true);
Credits goes to this SO LINK
SOLUTION 2
You can set ini_set('precision',1);
SOLUTION 3
$decoded = json_decode($encoded, true, null, JSON_BIGINT_AS_STRING);
NOTE: The Last solution will work only for PHP > 5.4
You might want to take a look at this Blog
Just wrap the values in quotes: json_decode('[["3.2","1"],["4.8","2"]]');
Related
I have this data in MongoDB:
{
"_id": ObjectId("542bxxxxxxxxxxxx"),
"TEMP_C": 13,
"time": ISODate("2014-08-21T05:30:00Z")
}
I want to group it by day and feed it to Highcharts, to display Temp average per day.
Like this: http://jsfiddle.net/tw7n6wxb/3/
Using MongoDB Aggregation Pipeline, I was able to do the grouping, based on some other examples and this great post: http://www.kamsky.org/stupid-tricks-with-mongodb/stupid-date-tricks-with-aggregation-framework
Side question: Why is it so complicated to group by date in MongoDB??? The most annoying part is having to re-compose the date object after splitting it into '$dayOfMonth', '$month', and '$year'.
Is there any simpler way of doing this?
In any case, I got this part working (I think). This is the result:
{
"_id" : {"sec":1409346800,"usec":0},
"avg" : 12
},
{
"_id" : {"sec":1409356800,"usec":0},
"avg" : 15
},
But, Highcharts series take arrays of value pairs as input:
Example: data: [[5, 2], [6, 3], [8, 2]].
The first value on each pair is the X value, and this value has to be a number (when X axis is configured as datetime, X values are in milliseconds).
The PROBLEM I'm having is that MongoDB returns the date as a MongoDate Object with two values inside, 'sec' and 'usec', while Highcharts is expecting one number.
Is there anyway to convert a MongoDate Object to integer in the pipeline? using a $project for example?
I'm using PHP, but I would like to avoid post-pocessing in the application (like PHP date formatting).
Or, any other ideas on how to solve this?
Thanks,
You seem to just want the timestamp values returned from the result. There is indeed a simple way to do this in the aggregation framework without using the date aggregation operators. You can use basic "date math" instead, and with a trick that can be used to extract the "timestamp" value from the date object and manipulate it:
db.collection.aggregate([
{ "$group": {
"_id": {
"$subtract": [
{ "$subtract": [ "$time", new Date("1970-01-01") ] },
{ "$mod": [
{ "$subtract": [ "$time", new Date("1970-01-01") ] },
1000 * 60 * 60 * 24
]}
]
},
"avg": { "$avg": "$TEMP_C" }
}}
])
So the basic "trick" there is that when subtract one date object from another ( or similar operation ) the result returned is a number for "milliseconds" of the time difference between the two. So by using the "epoch" date of "1970-01-01", you get the "epoch timestamp" value for the date object as a number.
Then the basic date math is applied by subtracting from this value the modulo ( or remainder ) from the milliseconds in a day. This "rounds" the value to represent the "day" on which the entry is recorded.
I like posting the JSON because it parses everywhere, but in a more PHP way, then like this:
$collection->aggregate(array(
array( '$group' => array(
'_id' => array(
'$subtract' => array(
array( '$subtract' => array(
'$time', new MongoDate(strtotime("1970-01-01 00:00:00"))
) ),
array( '$mod' => array(
array( '$subtract' => array(
'$time', new MongoDate(strtotime("1970-01-01 00:00:00"))
) ),
1000 * 60 * 60 * 24
))
)
),
"avg" => array( '$avg' => '$TEMP_C' )
))
))
So that is a little cleaner than using the date aggregation operators to get to your intended result. Of course this is still not "all the way" to how you want the data to be presented where you can use it in the client.
The real thing to do here is manipulate the result so that you get the output format you want. This is probably better suited to your server code doing the manipulation before you return the response, but if you have MongoDB 2.6 or greater then it is "possible" to do this within the aggregation pipeline itself:
db.collection.aggregate([
{ "$group": {
"_id": {
"$subtract": [
{ "$subtract": [ "$time", new Date("1970-01-01") ] },
{ "$mod": [
{ "$subtract": [ "$time", new Date("1970-01-01") ] },
1000 * 60 * 60 * 24
]}
]
},
"avg": { "$avg": "$TEMP_C" }
}},
{ "$group": {
"_id": null,
"data": {
"$push": {
"$map": {
"input": { "$literal": [ 1,2 ] },
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el", 1 ] },
"$$_id",
"$avg"
]
}
}
}
}
}}
])
So this is pretty sneaky really. After the initial "grouping" is done to determine the averages for each day you get two fields in your result documents per day for _id and avg. What the $map operator does here is takes and array as input ( in this case, just a numbered template with a pair of values to identify position ) and processes each element to return an array equal to the elements present in the original.
The $cond operator here allows you to look at the value of the current element of that array and "swap it" with another value present in the current document. So for each document, the results contain something that is a paired array like:
[ 1409346800000, 12 ]
Then all that happens is all results are pushed into a single document with a "data" array that appears as follows:
{ "_id": null, "data": [ [..,..], [..,..], (...) ] }
Now your data element in that one result is an array of array pairs representing the points you want.
Of course though, operators like $map are only available from MongoDB 2.6 and onwards, so if you have that available then you can use them but otherwise just process the results in code with a similar "map" operation:
function my_combine($v) {
return array($v["_id"],$v["avg"])
}
$newresult = array_map( "my_combine", $result )
So this really comes down to array manipulation from whichever way you approach it, but the date manipulation trick should also save you some work in obtaining the results as the expected timestamp values as well.
I have a multidimensional array that looks like this:
{
"groups": [
{
"__v": 0,
"_create_date": "2014-08-20T23:00:12.901Z",
"_id": "53f5287ca78473a969001827",
"_last_message_date": "2014-08-20T23:04:36.347Z",
"activity": 0,
"blocked_users": [],
"created_by": {
"_id": "53e84b0eba84943c6d0003f8",
"_last_modified": "2014-08-20T00:11:05.399Z",
"first_name": "Jegg",
"last_name": "V"
},
"curated": false,
"diversity": 0,
"featured": false,
"flagged": false,
"last_message": {
"text": "let's talk beo",
"created_by": {
"_id": "53e84b0eba84943c6d0003f8",
"first_name": "Jegg",
"last_name": "V"
},
"_id": "53f52984a78473a969001833",
"_create_date": "2014-08-20T23:04:36.347Z"
},
"member_count": 1,
"messages_count": 1,
"name": "Test",
"public": true,
"recency": 52182276.347,
"score": 52182276.347,
"tags": []
},
This structure repeats over 3000 times creating a very large multidimensional array. I think I can use array_chunk($array, 300) to break the array into smaller chunks. But I can't figure out how to access them exactly.
What I want to do is independently loop through the newly chunked arrays. So I'd like to end up with something like:
$array1 = {...}
$array2 = {...}
$array3 = {...}
$array4 = {...}
... and so on
THen I could loop through each of the newly created arrays, which are essentially smaller groups of the original array, but of 3000 arrays in one multidimensional array as I have in the first place, I end up with these smaller ones of 300 arrays each.
I hope this makes sense, I'm kinda out of my league. Help is always appreciated.
I think your array is in json format.
First decode it and then pass to array_chunk method.
array_chunk($input_array, 300));
then access them as $input_array[0][0], $input_array[0][1]....... $input_array[0][299], $input_array[1][0], $input_array[1][1].....
EDIT: oh, somehow I entirely misread the question. array_chunk is something worth looking into.
You could try using extract to fetch array values to the "global" variable namespace.
extract takes three arguments: the array you wish to extract, flags, and prefix if needed.
I'm not sure how non-associative arrays are extracted, but you could try
$full_array = array(
array( ... ),
array( ... ),
array( ... ),
array( ... ),
...
);
// EXTR_PREFIX_ALL prefixes all extracted keys with wanted prefix (the third param).
$extract_amount = extract( $full_array, EXTR_PREFIX_ALL, 'prefix' );
Now you should have the array extracted and available for use with variable names $prefix0, $prefix1, $prefix2 and so on.
I'm not sure how smart it is to extract an array with hundreds of available values.
MongoDB can not get the total.
MYSQL :
select sum(play) as toplam from videos
MongoDB :
$group = $naytu->db->videos->group(array(), array("toplam" => 0), "function(obj, prev) {prev.toplam = prev.toplam + obj.play - 0;}");
results (print_r)
Array
(
[retval] => Array
(
[0] => Array
(
[toplam] => NAN
)
)
[count] => 78656
[keys] => 1
[ok] => 1
)
where is the problem ?
In PHP #Neils answer is:
$mongo->videos->aggregate([
['$group' => ['_id' => null, 'toplam' => ['$sum' => '$play']]]
]);
The group() function is kind of an old form of achieving results like this, it will also not work with sharded collections. Better to familiarize yourself with aggregate:
db.collection.aggregate([
{"$group": { "_id": null, "toplam": {"$sum": "$play"} }}
]);
Output for you driver will just be a cursor just like using find.
for PHP Syntax
For the record, your usage seems to be way off. Much as above you just translate the relative JSON to Dicts, and if that is still a problem then use a JSON parser to show you how. All the arguments are valid JSON, so I'm reminding myself to do this for all examples now, so people can easily dump into their own language.
db.collection.group({
"key": {},
"reduce": "function(curr, result) { result.total += curr.play; }",
"initial": { "total": 0 }
})
But still, don't use this anyway. Stick to aggregate, it's a useful tool to learn.
Of course in all cases, make sure that all your data being applied to a sum is actually a number, or things will blow up on you.
Maybe this can help
"function(obj, prev) {prev.toplam = new NumberInt(prev.toplam) + new NumberInt(obj.play);}"
I'm trying to extract a specific value from json content . Here it is link with the json code http://www.ebayclassifieds.com/m/AreaSearch?jsoncallback=json&lat=41.1131514&lng=-74.0437521 As you may see the the code displayed is
json({items:[{url:"http://fairfield.ebayclassifieds.com/",name:"Fairfield"},{url:"http://newyork.ebayclassifieds.com/",name:"New York City"}],error:null});
I need to extract the first url which in this case is "http://fairfield.ebayclassifieds.com/" and its name value which is "Fairfield" , I could do it with regex but I would prefer to use json_decode. Unfortunately when I try to decode it doesn't work
$json = getContent("http://www.ebayclassifieds.com/m/AreaSearch?jsoncallback=json&lat=41.1131514&lng=-74.0437521");
$test = json_decode($json, true);
As danp already said, the returned JSON is enclosed in a function call (specified by jsoncallback=json). You cannot get rid of this totally but, just using AreaSearch?jsoncallback=&lat=41.1131514&lng=-74.0437521 removes at least the json at the beginning of the string and you can get rid of the brackets by:
$json = trim(trim($json), "();");
with gives:
{items:[{url:"http://fairfield.ebayclassifieds.com/",name:"Fairfield"},{url:"http://newyork.ebayclassifieds.com/",name:"New York City"}],error:null}
Unfortunately, the JSON string is not valid. The keys (items, url, ...) have to be enclosed in quotes ". You can easily check that you get a syntax error with json_last_error() (error code 4, JSON_ERROR_SYNTAX).
Update:
According to this question: Invalid JSON parsing using PHP , you can make the JSON string valid with:
$json = preg_replace('/(\w+):/i', '"\1":', $json);
This encloses the keys in quotes.
If the string would be valid, then you could generate an array via:
$a = json_decode($json, true);
which would give you:
Array
(
[items] => Array
(
[0] => Array
(
[url] => http://fairfield.ebayclassifieds.com/
[name] => Fairfield
)
[1] => Array
(
[url] => http://newyork.ebayclassifieds.com/
[name] => New York City
)
)
[error] =>
)
So you could get the first URL and name via $a['items'][0]['url'] and $a['items'][0]['name'] resp.
But I repeat, the JSON you get as response is not valid and you cannot parse it with json_decode() in its original form.
Its not valid JSON. The keys should be wrapped inside quotes.
You can validate your json using the excellent JSON Lint site.
This is a valid version of the data returned:
{
"items": [
{
"url": "http://fairfield.ebayclassifieds.com/",
"name": "Fairfield"
},
{
"url": "http://newyork.ebayclassifieds.com/",
"name": "New York City"
}
],
"error": "null"
}
I'm pretty sure it's an obvious error somewhere on this - but let me explain what I'm donig :
I'm calling a PHP file via jQuery as follows :
$.getJSON("/phpincs/getbucket.php?awskey="+awskey+"&awssecret="+awssecret+"&bucket="+bucket,
function(json){
$.each(json,function(i,item){
$(new Option(item.name,item.name)).appendTo('#bucket_contents_0');
});
}
and the JSON file it returns is as follows :
Array
(
[bt_shop.png] => Array
(
[name] => bt_shop.png
[time] => 1260393948
[size] => 156985
[hash] => 8a4eba621d5e08b84c962a0ad21ec2ae
)
[me_profile.jpg] => Array
(
[name] => me_profile.jpg
[time] => 1260393952
[size] => 2714
[hash] => 4f5d185b0c671e6165902762681f098b
)
[thumbnail.aspx.jpeg] => Array
(
[name] => thumbnail.aspx.jpeg
[time] => 1260393951
[size] => 5268
[hash] => 6939efa58ff7ffcac17218ffdd0d5d8c
)
)
true
for some reason it doesn't seem to fire the function(json){} - I've stuck an alert(''); in and it doesn't do anything.
Can someone quickly explain to me what seems to be going wrong there?
Cheers,
Carl
It is more than likely not calling the callback function because it doesn't look like what you are returning is json. If you have an $variable defined that contains your array...call
echo json_encode($jsondata); exit;
At the end of your script.
I've changed the names of the inner arrays as your previous labels are going to cause problems with the dot. You will get an error like:
myArray.bt_shop is undefined
when you try to call
alert(myArray.bt_shop.png.name);
the only way it could be called is with
alert(myArray["bt_shop.png"].name);
So having changed your code a bit, this is the JSON version of your arrays...
{
"one":
{
"name": "bt_shop.png",
"time": "1260393948",
"size": "156985",
"hash": "8a4eba621d5e08b84c962a0ad21ec2ae"
},
"two":
{
"name": "me_profile.jpg",
"time": "1260393952",
"size": "2714",
"hash": "4f5d185b0c671e6165902762681f098b"
},
"three":
{
"name": "thumbnail.aspx.jpeg",
"time": "1260393951",
"size": "5268",
"hash": "6939efa58ff7ffcac17218ffdd0d5d8c"
}
}
Then you reference your fields like this when you have the object:
myArray["two"]["name"]
myArray["two"].name
myArray.two.name
Your returned file isn't JSON. Unless you're using PHP syntax to describe your JSON object for us, you need to encode it into JSON format using json_encode.
What you call the JSON file isn't JSON. Or maybe do you use some PHP library that converts that strange format into JSON ?