I created a REST API using Laravel (Lumen). Basically I'm taking some values from database and need to do some functions. Here's the problem.
There's a field called web_data in my result set. It's Perl code. This is how it looks when I query the database using normal SQL editor like phpmyadmin and workbench.
{"caption" => "Genes (Comprehensive set from GENCODE 26)","colour_key" => "[biotype]","default" => {"MultiBottom" => "collapsed_label","MultiTop" => "gene_label","alignsliceviewbottom" => "as_collapsed_label","contigviewbottom" => "transcript_label","contigviewtop" => "gene_label","cytoview" => "gene_label"},"key" => "ensembl","label_key" => "[biotype]","multi_name" => "GENCODE 26 Comprehensive gene set","name" => "Comprehensive Gene Annotations from GENCODE 26"}
It's very clear and nothing wrong. But when I get the same result via my REST API, I'm getting the following output. It has many "\" symbols. Here's that output.
{\"caption\" => \"Genes (Comprehensive set from GENCODE 26)\",\"colour_key\" => \"[biotype]\",\"default\" => {\"MultiBottom\" => \"collapsed_label\",\"MultiTop\" => \"gene_label\",\"alignsliceviewbottom\" => \"as_collapsed_label\",\"contigviewbottom\" => \"transcript_label\",\"contigviewtop\" => \"gene_label\",\"cytoview\" => \"gene_label\"},\"key\" => \"ensembl\",\"label_key\" => \"[biotype]\",\"multi_name\" => \"GENCODE 26 Comprehensive gene set\",\"name\" => \"Comprehensive Gene Annotations from GENCODE 26\"}
Can you please tell me what's the reason for this? Is this an issue with PHP or something like that ?
And also I have another issue. This is Perl code containing "=>" symbols to seperate values. I need to convert this into a JSON object using PHP. Is that possible ?
Need to convert like following object.
{ "caption" : "Genes (Comprehensive set from GENCODE 26)", "colour_key" : "[biotype]", "default" : { "MultiBottom" : "collapsed_label" ...
This is the php code i'm using to create the REST output using SQL data.
public function testDualDatabaseConnection()
{
$testGene = DB::connection('mysql2')->select('select a.analysis_id, logic_name, ad.description, ad.display_label, ad.web_data FROM ( select distinct(analysis_id) as analysis_id from gene ) as a join analysis on (a.analysis_id = analysis.analysis_id) join analysis_description as ad on (analysis.analysis_id = ad.analysis_id) where ad.displayable = :wow1', ['wow1' => "1"]);
return response()->json($testGene);
}
And also this is the full JSON output i'm getting via REST API.
https://gist.github.com/chanakaDe/0d1e5916d8bc788bb101afa9b92dc9cd
In this response, you can see "web_data", that's also the part I need to convert to JSON again.
If on the off chance you really do have
{\"caption\" => \"Genes ...\", ...}
Then you have a JSON fragment, and the first you thing you need to do is turn it into a complete JSON document so it can be decoded.
$json_fragment_from_api = '{\"caption\" => \"Genes ...\", ...}';
$json_from_api = '{"result":"' . $json_fragment_from_api . '"}';
$response = json_decode($json_from_api, true);
$perl_code = $response["result"];
That said, I think it's far more likely that you only showed a portion of your actual input. You're far more likely to have received something that looks more like
{ "result": "{\"caption\" => \"Genes ...\", ...}" }
Since you already have a complete JSON document, simply start by decoding it.
$json_from_api = '{ "result": "{\"caption\" => \"Genes ...\", ...}" }';
$response = json_decode($json_from_api, true);
$perl_code = $response["result"];
At this point, you have the value from the database.
{"caption" => "Genes ...", ...}
That value is Perl code. More specifically, it's surely Perl code that was produced by Data::Dumper. This is going to be hard to convert into JSON.
You could use the following, but it won't handle everything:
$desired_json = str_replace(' =>', ':', $perl_code);
Here are some things it won't handle:
Undefined values
Strings containing =>
Strings containing $
Strings containing #
Strings containing non-ASCII or some non-printable characters.
As shown here:
$ perl -e'
use feature qw( say );
use Data::Dumper qw( Dumper );
local $Data::Dumper::Useqq = 1;
local $Data::Dumper::Terse = 1;
local $Data::Dumper::Indent = 0;
say(Dumper({ x => undef }));
say(Dumper({ x => " => " }));
say(Dumper({ x => q{$} }));
say(Dumper({ x => q{#} }));
say(Dumper({ x => "\x7F" }));
'
{"x" => undef} # Gives {"x": undef} instead of {"x": null}
{"x" => " => "} # Gives {"x": ": "} instead of {"x": " => "}
{"x" => "\$"} # Gives {"x": "\$"} instead of {"x": "$"}
{"x" => "\#"} # Gives {"x": "\#"} instead of {"x": "#"}
{"x" => "\177"} # Gives {"x": "\177"} instead of {"x": "\u007f"}
Not sure where that escaping happens (if you can, give some more details)
But, try to replace => with : and \" with " and you'll have yourself a valid json.
$string = '{\"caption\" => \"Genes (Comprehensive set from GENCODE 26)\",\"colour_key\" => \"[biotype]\",\"default\" => {\"MultiBottom\" => \"collapsed_label\",\"MultiTop\" => \"gene_label\",\"alignsliceviewbottom\" => \"as_collapsed_label\",\"contigviewbottom\" => \"transcript_label\",\"contigviewtop\" => \"gene_label\",\"cytoview\" => \"gene_label\"},\"key\" => \"ensembl\",\"label_key\" => \"[biotype]\",\"multi_name\" => \"GENCODE 26 Comprehensive gene set\",\"name\" => \"Comprehensive Gene Annotations from GENCODE 26\"}';
$string = str_replace('\"','"', $string);
$string = str_replace('=>',':', $string);
$json = json_decode($string, true);
print_r($json);
You can check it out here: https://3v4l.org/UBmGA
Apparently this answer got more hate than i'd expect. Nevertheless, it did help OP and i'm sticking by it. In the end, this is what SO stands for.
I would like to know though... why so many downvotes? I do take constructive criticism well and would like to know how to improve future answers.
Related
I have a below array:
Array
(
[0] => CLICK
[1] => CSC
)
After json_encode I got below one.
"["CLICK","CSC"]"
How to convert this into "[\"CLICK\",\"CSC\"]" this.
Any one help.
Some info was missing which I got after discussion.
They are manually replacing a lot of characters before returning json. Out of them they also include [ => "[ and ] => ]" due to their backend implications.
A simple json_encode was solution for this along with skipping those character replacement for specific this key.
Source:
<?php
$arr = [
"CLICK", "CSC"
];
echo json_encode(json_encode($arr)) . "\n";
Result:
"[\"CLICK\",\"CSC\"]"
I have a PHP file that runs a Perl script, using popen and the perl script outputs the following back to the php
{ 'City' => [ 'LA', 'Chicago', 'NY' ], 'Name' => 'Kevin Bridges', 'Id' => '7075', 'Last-Status-Change' => { 'Time' => 14172911, 'User' => 'kbridge', 'To' => 'LAX', 'From' => 'ORD' }}
I cannot modify the perl script, and I really don't know the contents of it. But it looks like it is outputting JSON. I have tried using json_encode to grab the contents of the output but no success. Can anyone tell me if it is possible to parse this or do I have to manually write a parser?
Convert => to : and ' to ". After that use json_decode to create associative array from the string.
$array = json_decode(str_replace(["=>", "'"], [":", '"'], $a), true);
That's not JSON but is "almost" the PHP [] array syntax, except it uses some {}. You could try:
eval('$array = ' . str_replace(['{','}'], ['[',']'], $output) . ';');
print_r($array);
It does not looks like to be a valid JSON, it seems to be a PERL hash, so i think you gonna need to parse that manually... a simple way to you accomplish that in PHP, it would be to replace the {-} to [] and you can eval that string to be consider an array in PHP
Im looking to use a PHP array obtained from a RESTful service.
They array is just a simple array outputted as follows,
array (
0 =>
stdClass::__set_state(array(
'id' => '375',
'primary_name' => 'Beaufort 3',
'price' => '',
'sqft' => '2435',
'bdrm' => '3',
'bthm' => '2.5',
'display_title' => 'Traditional A1',
'full_img_path' => '',
'thumb_path' => '',
'available_in' => 'a:2:{i:0;s:1:"2";i:1;s:1:"5";}',
'display_first' => '',
)),
)
I'm obtaining the data using file_get_contents but of course its no longer an array at this point. What is the best way to convert this back to a usable array?
I have never seen a service that provides this, probably because it is something people would avoid using, but it does look like it could be something they intend for you to be able to use with eval(). I suppose this could be intended as a convenience. I think they have made an error, though. If you use var_export, (which I assume is how this was generated) on an array like this:
$example = array (
array(
'id' => '375',
'primary_name' => 'Beaufort 3',
'price' => '',
'sqft' => '2435',
'bdrm' => '3',
'bthm' => '2.5',
'display_title' => 'Traditional A1',
'full_img_path' => '',
'thumb_path' => '',
'available_in' => 'a:2:{i:0;s:1:"2";i:1;s:1:"5";}',
'display_first' => '',
)
);
then you would get something you could eval into a variable and use in your code. However, if you var_export an array of anonymous objects instead of an array of associative arrays, you will get the type of response you are getting, which I don't know of any way to use. I would guess they are var_exporting the results of a query and they are using FETCH_OBJ instead of FETCH_ASSOC for the fetch style.
EDIT:
I was looking through the comments on var_export after writing this and came across this way of doing it that should work.
$str = str_replace("stdClass::__set_state", "(object)", $str);
eval('$array=' . $str . ';');
But just because something is possible doesn't mean we should do it.
You can convert it back into an array with eval(). http://php.net/manual/en/function.eval.php
eval('$array='.$response.';');
However, this can be dangerous, because eval will take any PHP code given it - if the service is compromised, your code would execute anything passed to it. JSON, if the service supports it, is much safer and natively supported in PHP since 5.2 via json_decode(). http://php.net/manual/en/function.json-decode.php
I have a problem.
For a mistake I have a lot of not valid JSON strings like this:
{
"d": {
"results": [
{
"__metadata": {
"uri": "https://api.datamarket.azure.com/Data.ashx/Bing/Search/Web?Query=u0027non supporting iframesu0027&Market=u0027it-ITu0027&Adult=u0027Offu0027&Options=u0027DisableLocationDetectionu0027&WebSearchOptions=u0027DisableQueryAlterationsu0027&$skip=0&$top=1",
"type": "WebResult"
},
"ID": "7858fc9f-6bd5-4102-a835-0fa89e9f992a",
"Title": "something good",
"Description": "something "WRONG" here!",
"DisplayUrl": "www.devx.com/Java/Article/27685/1954",
"Url": "http://www.devx.com/Java/Article/27685/1954"
}
],
"__next": "https://api.datamarket.azure.com/Data.ashx/Bing/Search/Web?Query=u0027non%20supporting%20iframesu0027&Market=u0027it-ITu0027&Adult=u0027Offu0027&Options=u0027DisableLocationDetectionu0027&WebSearchOptions=u0027DisableQueryAlterationsu0027&$skip=50"
}
}
As you can see the field Description contains a bad string (" into "), so I'm not able to parse the json using php's json_decode, infact it returns NULL.
I've 1 million of wrong json, much more big than this (10 times).
How can I do in php?
In your case you could exploit the fact that strings in json could not be over a line. That is a snappy point to grab with s multi-line aware search and replace with a regular expression function like preg_match_callback in PHP.
/^\s+"[a-z_"]+": "([^"]*".*)",?$/mi
Whitespace at the beginning of the line; member-name in form of a valid name (only characters and underscore here) as a string; the : and then the broken string until the end of the line optionally followed by a comma ,?.
This regex already matches only invalid lines. However if your json also contains a valid string with \" inside, this regex does not really work.
So it's also good to place some checks that the replacement would do what it is intended.
$like = '... json-like but broken json string as in question ...';
// Fixing #1: member strings containing double-quotes on the same line.
$fix1Pattern = '/^(\s+"[a-z_]+": ")([^"]*".*)(",?)$/mi';
$fix1Callback = function ($matches) {
list($full, $prefix, $string, $postfix) = $matches;
$fixed = strtr($string, ['"' => '\"']);
if (!is_string(json_decode("\"$fixed\""))) {
throw new Exception('Fix #1 did not work as intended');
}
return "$prefix$fixed$postfix";
};
// apply fix1 onto the string
$buffer = preg_replace_callback($fix1Pattern, $fix1Callback, $like);
// test if it finally works
print_r(json_decode($buffer));
Keep in mind that this is limited. You might need to learn about regular expressions first which is a world of it's own. But the principle is often very similar: You search the string for the patterns that are the broken parts and then you do some string manipulation to fix these.
If the json string is much more broken, then this needs even more love, probably not to be easily solved with a regular expression alone.
Exemplary output for the code-example and the data provided:
stdClass Object
(
[d] => stdClass Object
(
[results] => Array
(
[0] => stdClass Object
(
[__metadata] => stdClass Object
(
[uri] => https://api.datamarket.azure.com/Data.ashx/Bing/Search/Web?Query=u0027non supporting iframesu0027&Market=u0027it-ITu0027&Adult=u0027Offu0027&Options=u0027DisableLocationDetectionu0027&WebSearchOptions=u0027DisableQueryAlterationsu0027&$skip=0&$top=1
[type] => WebResult
)
[ID] => 7858fc9f-6bd5-4102-a835-0fa89e9f992a
[Title] => something good
[Description] => something "WRONG" here!
[DisplayUrl] => www.devx.com/Java/Article/27685/1954
[Url] => http://www.devx.com/Java/Article/27685/1954
)
)
[__next] => https://api.datamarket.azure.com/Data.ashx/Bing/Search/Web?Query=u0027non%20supporting%20iframesu0027&Market=u0027it-ITu0027&Adult=u0027Offu0027&Options=u0027DisableLocationDetectionu0027&WebSearchOptions=u0027DisableQueryAlterationsu0027&$skip=50
)
)
I have a result of export variable in Perl like this string:
$VAR1 = {
'guard' => undef,
'work_hand' => undef,
'images' =>
{'1' =>
{
'mini_height' => 150,
'width' => 150,
'extension' => 'jpg',
'filename' => 'object_1.1330907414.96873.jpg',
'mini_width' => 150,
'class' => 'Ontico::Image',
'height' => 150,
'mini_filename' => 'object_1.1330907414.96873.mini.jpg',
'size' => 26053,
'symname' => 'big_logo'
},
'2' =>
{
'width' => 48,
'extension' => 'jpg',
'alt' => 'Даниэле Галлоппа',
'height' => 48,
'mini_filename' => 'object_91.1235312905.mini.jpg',
'size' => 12809,
'symname' => 'logo',
'mini_height' => 150,
'filename' => 'object_91.1235312905.jpg',
'mini_width' => 150,
'class' => 'Ontico::Image'
}
},
'show_league_banner' => 0,
'back_hand' => undef,
'weight_category' => undef,
'stick_position' => undef
};
How can I deserialize this data in PHP?
P.S. I already have data in this format in DB, I cannot change it to json or another.
You've got a number of suggestions for trying to parse it one way or another, but the real question is why?
Why not just have a small Perl program that loads it, and spits out an equivalent JSON string.
You could then either call that Perl program from within your PHP to do the conversion; this would mean you are using Perl to read the Perl format, which would guarantee correct conversion.
Or (better yet) run it against your entire database in a batch, to get rid of the Perl-specific data format from the DB; then you can just use PHP's standard JSON functions.
That would then make life so much simpler in your PHP code (or in any other language you need to read the data with at a later date).
The obvious and only robust solution is to use Perl to deserialize and reserialize the input to a standard format. The Perl program that can accomplish this task does not need to be very large, either.
// receive input in Perl's Data::Dumper format and produce PHP object output
function perl_dd_to_php( $dd_output ) {
$process = proc_open( "perl -000 -MJSON -e 'print encode_json eval <>'",
array( array("pipe","r"), array("pipe","w") ),
$pipes );
fwrite($pipes[0], $dd_output );
fclose($pipes[0]);
$json_string = stream_get_contents($pipes[1]);
fclose($pipes[1]);
return json_decode($json_string);
}
Since it's not JSON but it looks like JSON, you could try to modify a JSON library to work with that format. I took this JSON library, replaced : with => and added undef as you can see here (lines 496, 671 and 681). It's pretty straightforward, really, and I assume you can work around other differences in a similar manner.
Result is:
stdClass Object
(
[guard] =>
[work_hand] =>
[images] => stdClass Object
(
[1] => stdClass Object
(
[mini_height] => 150
[width] => 150
[extension] => jpg
[filename] => object_1.1330907414.96873.jpg
[mini_width] => 150
[class] => Ontico::Image
[height] => 150
[mini_filename] => object_1.1330907414.96873.mini.jpg
[size] => 26053
[symname] => big_logo
)
[2] => stdClass Object
(
[width] => 48
[extension] => jpg
[alt] => Даниэле Галлоппа
[height] => 48
[mini_filename] => object_91.1235312905.mini.jpg
[size] => 12809
[symname] => logo
[mini_height] => 150
[filename] => object_91.1235312905.jpg
[mini_width] => 150
[class] => Ontico::Image
)
)
[show_league_banner] => 0
[back_hand] =>
[weight_category] =>
[stick_position] =>
)
Is that what you're looking for?
use JSON;
(or any other data interchange format like XML)
JSON documentation and examples are available at CPAN
If you can change the Perl code, then do as amon suggests and use some standard serialization format like JSON or XML or YAML that you can deserialize in PHP.
You could even make Perl output PHP's native serialization format if you really wanted to, although I wouldn't generally recommend that. (What about when you next want to deserialize the same data in, say, Python?)
If you can't change the Perl code, you'll just have to bite the bullet and try to parse the Data::Dumper output in PHP. I couldn't find any existing code to do that, so it looks like you may have to write your own. This could be job for a parser generator, although the format is (usually) simple enough that you might be able to just hand-code it.
Edit: Since you say that you have this serialized data in a database, why not just write a Perl program to read the data and convert it to a more standard serialization format like JSON?
Since you stated you cannot change format:
I don't like using eval, but because your syntax is so close to the expected PHP array syntax, I think we can let it slide.
Set $string equal to the contents from your database that fits the format below. See below for a working example using the data you provided. At the end of the day, PHP will set the variable at the beginning of your perl var to the new parsed array.
Since it is going to be a textblock/large string, do:
<?php
$string = "\$VAR1 = {
'guard' => undef,
'work_hand' => undef,
'images' =>
{'1' =>
{
'mini_height' => 150,
... // truncated for readability
};";
$res = str_replace(array("{", "}", 'undef'), array("array(", ")", "''"), $string);
eval($res);
print_r($VAR1);
Your result is:
Array
(
[guard] =>
[work_hand] =>
[images] => Array
(
[1] => Array
(
[mini_height] => 150 ...
Note: I would suggest you take the time now to retrofit and upgrade your database content to a more standard format simply for the fact that it will be easier to maintain in the future.
You can loop through your database, grab all the contents row by row, at which point you run the data into the function above, and wrap it in json_encode() and update your database row with the new JSON string. This will save you a headache in the future and allow you to update all your data for the new standard.