Any JSON to CSV Converter - php

I am trying to build common function for JSON to CSV Converter. currently for different json file i have to make some changes in existing function.
current code:
function JsonToCSV($jfilename, $cfilename) {
if (($json = file_get_contents($jfilename)) == false)
die('Error reading json file...');
$data = json_decode($json, true);
$fp = fopen($cfilename, 'w');
$header = false;
foreach ($data as $row) {
if (empty($header)) {
$header = array_keys($row);
fputcsv($fp, $header);
$header = array_flip($header);
}
fputcsv($fp, array_merge($header, $row));
}
fclose($fp);
return;
}
Above code is working for below json
[
{
"Id": "1",
"Name": "Juned Ansari",
"Position": "Full Stack Developer",
"Salary": "$99999"
},
{
"Id": "2",
"Name": "Mayur Marolia",
"Position": "Data Analyst",
"Salary": "$6789000"
},
{
"Id": "3",
"Name": "Mitesh Panchal",
"Position": "Team Leader",
"Salary": "$2324540"
}
]
but the problem is if my json structure changed then i have to rewrite above function like it is not working for below json
[
{
"BILLING_SOFTWARE_API_KEY": "ABCD1234",
"DISTRIBUTOR_API_KEY": "11380",
"SALESMANS": [
{
"sm_code": 1,
"sm_name": "DEEPAK MEHTA 7044524144"
},
{
"sm_code": 2,
"sm_name": "Juned Ansari"
}
]
}
]

The problem is that JSON is unstructured, while CSV is structured.
To clear this hurdle you must first of all gather all the JSON fields in all the structure, and since the header must be written first, you need to cycle through the JSON twice.
$columns = [ ];
// This could be a foreach
// foreach($data as $row) { $columns = array_merge($columns, array_keys($row)); }
array_map(function($row) use (&$columns) {
$columns = array_unique(array_merge($columns, array_keys($row)));
}, $data);
// Now columns contain all columns in all rows of the JSON.
$fp = fopen($cfilename, 'w');
fputcsv($fp, $columns);
// Set all empty fields to some default
$template = array_fill_keys($columns, '');
foreach ($data as $row) {
fputcsv($fp, array_values(array_merge($template, $row)));
}
fclose($fp);
The above will not function out of the box for complex data (if a column has sub-information, like in your example). There you need a more complex step:
foreach ($data as $row) {
$collapsed = array_map(function($value) {
if (is_array($value)) {
return implode(', ', $value);
}
return $value;
}, $row);
fputcsv($fp, array_merge($template, $collapsed));
}
Still more complex information in the JSON is a clear indication that you're doing this wrong. Your best bet is then to re-encode the complex value as JSON and store it as is in the CSV field (use json_encode() instead of implode, above).
The Great Column Name Massacre
For those cases where you need to throw bad money after worse, you can implement what I called the Great Column Name Massacre. In its easiest form, you code
{
"address": {
"street": "Piazza Vieusseux",
"number": 2,
"locality" : {
"type": "city",
"name": "Florence"
}
}
}
as
[
"address_street" => "Piazza Vieusseux",
"address_number" => 2,
"address_locality_type" => "city",
"address_locality_name" => "Florence"
]
I'm feeling of two minds about this. Please do not take this wrong, but I'm feeling sort of like you asked me why your Smith&Wesson battery-operated hair dryer is not working, even if you put all six batteries in the drum, pointed it to your head and pulled the trigger.
And I feel like I'm telling you "Oh, there's a safety switch on the side. You need to move it from SAFE to FIRE, or it will not work".
So bearing in mind that this looks like a very bad idea, the folding function I mentioned in the comments is this (you can adapt it to your needs, see later):
function fold($arry, $prefix = '') {
$retval = [ ];
foreach ($arry as $key => $value) {
$newkey = $prefix.$key;
if (is_array($value)) {
$folded = fold($value, $newkey . '_');
foreach ($folded as $subkey => $subval) {
$retval[$subkey] = $subval;
}
} else {
$retval[$newkey] = $value;
}
}
return $retval;
}
Once each element of the array has been folded, it can be analyzed to find out the column names (you can do this while folding) and then everything proceeds like above.
Testing
The folding function works properly with the provided JSON sample, and yields
Array
(
[BILLING_SOFTWARE_API_KEY] => ABCD1234
[DISTRIBUTOR_API_KEY] => 11380
[SALESMANS_0_sm_code] => 1
[SALESMANS_0_sm_name] => DEEPAK MEHTA 7044524144
[SALESMANS_1_sm_code] => 2
[SALESMANS_1_sm_name] => Juned Ansari
)
This of course immediately raises the first problem; "DISTRIBUTOR_API_KEY" is what we would expect from {"DISTRIBUTOR": {"API": {"KEY": 11380}}}. It works, but the decoding is ambiguous.
To overcome this limitation the quickest way is to change the delimiter from '_' to something else, or encode it differently in the keys.
Be warned: there will be no end of problems with this approach. If I had the time, I flatter myself fantasizing that I might end up with a post to rival the Famous Answer.
Having not the time, I hereby decline all responsibility for the frightful damage, loss of property, loss of productivity, loss of time, dereliction of duty, alienation of spouse and family ties and dire health consequences (included, but not limited to, a painful and prolonged death) that I believe are sure to follow.

I use simple client SIDE to convert JSON/HTML/XML to CSV,EXCEL...
because it is easy to download by attaching the file to download attribute of anchor tag...
here is an example you may like ...
The JS FIDDLE
$(document).ready(function(){
$('button').click(function(){
var data = $('#txt').val();
if(data == '')
return;
JSONToCSVConvertor(data, "Vehicle Report", true);
});
});
function JSONToCSVConvertor(JSONData, ReportTitle, ShowLabel) {
//If JSONData is not an object then JSON.parse will parse the JSON string in an Object
var arrData = typeof JSONData != 'object' ? JSON.parse(JSONData) : JSONData;
var CSV = '';
//Set Report title in first row or line
CSV += ReportTitle + '\r\n\n';
//This condition will generate the Label/Header
if (ShowLabel) {
var row = "";
//This loop will extract the label from 1st index of on array
for (var index in arrData[0]) {
//Now convert each value to string and comma-seprated
row += index + ',';
}
row = row.slice(0, -1);
//append Label row with line break
CSV += row + '\r\n';
}
//1st loop is to extract each row
for (var i = 0; i < arrData.length; i++) {
var row = "";
//2nd loop will extract each column and convert it in string comma-seprated
for (var index in arrData[i]) {
row += '"' + arrData[i][index] + '",';
}
row.slice(0, row.length - 1);
//add a line break after each row
CSV += row + '\r\n';
}
if (CSV == '') {
alert("Invalid data");
return;
}
//Generate a file name
var fileName = "MyReport_";
//this will remove the blank-spaces from the title and replace it with an underscore
fileName += ReportTitle.replace(/ /g,"_");
//Initialize file format you want csv or xls
var uri = 'data:text/csv;charset=utf-8,' + escape(CSV);
// Now the little tricky part.
// you can use either>> window.open(uri);
// but this will not work in some browsers
// or you will not get the correct file extension
//this trick will generate a temp <a /> tag
var link = document.createElement("a");
link.href = uri;
//set the visibility hidden so it will not effect on your web-layout
link.style = "visibility:hidden";
link.download = fileName + ".csv";
//this part will append the anchor tag and remove it after automatic click
document.body.appendChild(link);
link.click();
document.body.removeChild(link);
}

Related

Cross compare two sets of data

I have a report table. A report can have many documents (which are csv file paths). A document has many mappings.
So I initially loop my documents for a report and load the csv data using ParseCsv
foreach ($this->report->uploadedDocuments->documents as $document) {
$csv = new ParseCsv\Csv();
$csv->limit = 1;
$csv->parse(storage_path("app/" . $document->fullName));
$this->csvData = $csv->data;
}
If I output this, I see something like the following for each document (only showing one CSV row here from one document)
[
{
"id":"1",
"field1":"Some data",
"something":"Some data",
"something":"Some data",
"something":"Some data",
"something":"Some data",
}
]
Now each document has mappings so I can also do something like this
foreach ($this->report->uploadedDocuments->documents as $document) {
//Load CSV data
foreach ($document->mappings as $column) {
Log::debug(json_encode($column));
}
}
The output for the above is something like this
{
"document_id":"6434fc74-18f7-43ff-be33-3a8b0cf1cadc",
"field1":1,
"field2":null,
"field3":null,
"created_at":"2020-10-26T10:19:36.000000Z",
"updated_at":"2020-10-26T10:19:36.000000Z"
}
So from the above, I can see that field1 is located in column (index) 1 in the csv data. I can also see that this CSV does not have a field2 or field3. So my idea is to turn the original CSV data into this
[
{
"field1":"Some data",
}
]
So this is my thinking. I first need to loop the documents for the reports, load the CSV files, and read the data
foreach ($this->analysis->uploadedDocuments->documents as $document) {
$csv = new ParseCsv\Csv();
$csv->limit = 5;
$csv->parse(storage_path("app/" . $document->fullName));
if (empty($csv->data)) {
throw new Exception('Not data found in file.');
}
foreach ($csv->data as $dataKey => &$item) {
foreach ($item as $key => $value) {
//Now I have access to the keys and values from csv
}
}
}
But then how can I cross check this against my document mappings? I obviously have to loop the mappings but where should I do this? I have something which I think could work but it is very messy, and relies on my knowing the column names e.g.
foreach ($this->report->uploadedDocuments->documents as $document) {
$csv = new ParseCsv\Csv();
$csv->limit = 5;
$csv->parse(storage_path("app/" . $document->fullName));
if (empty($csv->data)) {
throw new Exception('Not data found in file.');
}
foreach ($document->mappings as $column) {
foreach ($csv->data as $dataKey => &$item) {
foreach ($item as $key => $value) {
//match the mapping column to the csv column
if ($column->fieldOne !== NULL && $key === $column->fieldOne) {
//now I have access to the values for that column
}
}
}
}
}
Would there be a better way of achieving something like this?
Thanks
I suggest the following as I understand from your question
<?PHP
$csvData = $csv->data;
$csvData = json_encode($csvData);
$arr = json_decode($csvData, true);
$field1 = array_column($arr, "field1");
print_r($field1);
?>
This line convert object into json string
$csvData = json_encode($csvData);
This line convert json string to pure array
$arr = json_decode($csvData, true);
This line extract column value from multidimensional array
$field1 = array_column($arr, "field1");
I hope this will work.

How to loop different csv and merge them?

I have a series of csv i need to grab from a url, they have different namings but the same extention .csv and the same exact format.
Then convert it to json.
The following is what i use for one but how to loop for any .csv?
Also once downloaded them how to merge them? So let's say we have different dates, we should expect a merge like:
{
"date": 2/4/20,
"state": "AK",...
},
{
"date": 3/4/20,
"state": "AK"...
This is the php I am using for a single csv
header('Content-Type: application/json');
if (($handle = fopen("example.com/NAME.csv", "r")) !== FALSE) {
$csvs = [];
while(! feof($handle)) {
$csvs[] = fgetcsv($handle);
}
$datas = [];
$column_names = [];
foreach ($csvs[0] as $single_csv) {
$column_names[] = $single_csv;
}
foreach ($csvs as $key => $csv) {
if ($key === 0) {
continue;
}
foreach ($column_names as $column_key => $column_name) {
$datas[$key-1][$column_name] = $csv[$column_key];
}
}
$json = json_encode($datas, JSON_PRETTY_PRINT);
fclose($handle);
print_r($json);
}
What if we have
example.com/NAME.csv
example.com/NAME2.csv
example.com/NAME3.csv
UPDATE
This is the series of csv and its data
Since there is not a way (as I know) to just scrape a site and find all files with the same extension (as per comments said and as I could not find how to using php), I ended up creating an array of names (in my case are dates) and then loop it, then the output will be a merged json:
$arr = array("04-12-2020",
"04-13-2020",
"04-14-2020",
"04-15-2020",
"04-16-2020",
"04-17-2020",
"04-18-2020",
"04-19-2020",
"04-20-2020",
"04-21-2020",
"04-22-2020",
"04-23-2020",
"04-24-2020",
"04-25-2020",
"04-26-2020",
"04-27-2020",
"04-28-2020",
"04-29-2020",
"04-30-2020",
"05-01-2020",
"05-02-2020",
"05-03-2020",
"05-04-2020",
"05-05-2020"
);
foreach($arr as $date) {
$url = $url.$date.".csv";..
// here the rest of the code as per the question
Not sure if this is the best way but works.
The result is a mega json with all dates and merged data as a single obj

how do json data become dynamic html tables using php?

how the json key is used as the table name, and displays data in the table, without the table name being created
{
"status": "sukses",
"title": "OK",
"img": "ok.jpg",
"data": {
"network": {
"technology": "No cellular connectivity",
"2g_bands": " N\/A",
"gprs": "No",
"edge": "No"
}
}
}
I want to make it like this
statuss title img technology 2g_bands gprs edge
sukses ok ok.jpg No cellular connectivity N\/A No No
with php Below, I am confused how to retrieve the json key to retrieve the name of the table I need to be dynamic
<?php
$Libs = new Library();
$result = $Libs->pilihmodel($idbrand,$idmodel);
if ($result->rowCount()>0) {
while ($datas = $result->fetch(PDO::FETCH_OBJ)) {
$datass = $datas->jsondetail;
$hasiljson = json_decode($datass, TRUE);
$binatang = array($hasiljson);
echo "<table>";
foreach ($binatang as $jenis) {
foreach ($jenis as $nama) {
echo "<td>$nama</td>";
}
}
echo "</table>";
}
You're about halfway there. Getting the keys is easy in a PHP foreach loop using $key => $value) syntax. Then you can, in parallel, build the header row and data row using the key and value from each field. Because of the structure of the JSON you also need to make a special case for the "data" object.
(If the structure is unpredictable you could make a recursive function which would traverse the structure and either print a field if it's a plain string, or find its children if it's another array, but that's probably overkill for what you're doing here. If you really need that though, please mention it.)
This will work:
$hasiljson = json_decode($datass, TRUE);
$header = "";
$body = "";
foreach ($hasiljson as $key => $value)
{
if ($key != "data")
{
$header .= "<td>$key</td>";
$body .= "<td>$value</td>";
}
else
{
foreach ($hasiljson["data"]["network"] as $key1 => $value1)
{
$header .= "<td>$key1</td>";
$body .= "<td>$value1</td>";
}
}
}
echo "<table><tr>$header</tr><tr>$body</tr></table>";
Demo: http://sandbox.onlinephpfunctions.com/code/8b61f2933b0ebf8a3ea1091c1e4474882e8c90a1

Create JSON response from Parsed XML PHP

I am parsing thorugh a eBay API response. I want to deliver this back to a website cleaner and easier to parse with JavaScript. I successfuly Parsed through the XML... but now turning that into JSON to resend back to the client is giving me some headaches.
NOTE: $resp is the response from eBay. It's their full length XML that is successfully parsed with the code below.
For example... $valueName could be Grade. And then I go into the next foreach loop and get the values for this. These values may be 10, 9.5, 9 etc.
Here is my PHP code.
$arrayName = array();
$arrayValue = array();
foreach($resp->aspectHistogramContainer->aspect as $name) {
$nameAspect = $name['name'];
//$arrayName["aspectName"] = $nameAspect;
foreach($name->valueHistogram as $value) {
$valueAspect = $value['valueName'];
//$arrayValue["aspectValue"] = $valueAspect;
}
//array_push($arrayName, $arrayValue);
}
echo json_encode($arrayName);
So, without me trying to create my own JSON, I am getting that I need. I echos results and it was similar to this...
NAME
----- Value
----- Value
----- Value
NAME
----- Value
NAME
etc etc
For a JSON response... Im looking for something like...
[
{
"name": "NAME",
"value": ["value", "value"]
}, {
"name": "name",
"value": ["value", "value"]
}
]
Any help and guidance would be greatly appreciated.
eBay's response is like this (there are A LOT more <aspect> and <valueHistogram>)
<getHistogramsResponse xmlns="http://www.ebay.com/marketplace/search/v1/services">
<ack>Success</ack>
<version>1.13.0</version>
<timestamp>2018-11-07T15:32:20.380Z</timestamp>
<aspectHistogramContainer>
<domainDisplayName>Baseball Cards</domainDisplayName>
<aspect name="Card Manufacturer">
<valueHistogram valueName="Ace Authentic">
<count>19</count>
</valueHistogram>
<valueHistogram valueName="American Caramel">
<count>2024</count>
</valueHistogram>
<valueHistogram valueName="APBA">
<count>10554</count>
</valueHistogram>
<valueHistogram valueName="Bazooka">
<count>8826</count>
</valueHistogram>
<valueHistogram valueName="Be A Player">
<count>17</count>
</valueHistogram>
<valueHistogram valueName="Bell Brand Dodgers">
<count>334</count>
To encode it (and assuming SimpleXML), then it's just a case of building each internal $aspect data array and then adding the values to it. I use (string) to ensure the data is not stored as a SimpleXMLElement, which can cause side effects...
$arrayName = array();
foreach($resp->aspectHistogramContainer->aspect as $name) {
$aspect = [ "name" => (string)$name['name']];
foreach($name->valueHistogram as $value) {
$aspect["value"][] = (string)$value['valueName'];
}
$arrayName[] = $aspect;
}
echo json_encode($arrayName);
with the sample XML, this gives...
[{"name":"Card Manufacturer","value":["Ace Authentic","American Caramel","APBA","Bazooka","Be A Player","Bell Brand Dodgers"]}]
Create one single array $resultArray and store values in it. By maintaining your current code structure with minimal changes, here is the updated code snippet,
$resultArray = array();
$i = 0; // Maintain Array Index value
foreach($resp->aspectHistogramContainer->aspect as $name) {
$resultArray[$i]["aspectName"] = (string)$name['name'];;
foreach($name->valueHistogram as $value) {
$resultArray[$i]["aspectValue"][] = (string)$value['valueName'];
}
$i++; // Increment array index to store next value
}
echo json_encode($resultArray);
$results = array();
// Parse the XML into a keyed array
foreach($resp->aspectHistogramContainer->aspect as $name) {
$nameAspect = (string) $name['name'];
$values = array();
foreach($name->valueHistogram as $value) {
$values[] = (string) $value['valueName'];
}
$results[$nameAspect] = $values;
}
// This keeps things simple - rewrite to the required JSON format
$outputForJSON = array();
foreach ($results as $name => $values) {
$outputForJSON[] = array(
"name" => $name,
"values" => $values
);
}
echo json_encode($outputForJSON);

How can I get my foreach loop to output the result as one combined array instead of separate ones?

I'm receiving a JSON input, and I need to run it through a script and get the values returned in a new JSON array. However when I do so, it yields eg 3 separate arrays, not one big one as it received. Example:
Input:
[{
"auth": "aa03e76d0a8bab11e08250000c29b481",
"topic": "edafdff398fb22847a2f98a15ca3186e/1",
"value": "1000"
},
{
"auth": "aa03e76d0a8bab11e08250000c29b481",
"topic": "edafdff398fb22847a2f98a15ca3186e/1",
"value": "2000"
},
{
"auth": "aa03e76d0a8bab11e08250000c29b481",
"topic": "edafdff398fb22847a2f98a15ca3186e/2",
"value": "3000"
}]
I run it through the following script:
function clean($string) {
$string = str_replace(' ', '-', $string); // Replaces all spaces with hyphens.
return preg_replace('/[^A-Za-z0-9\-]/', '', $string); // Removes special chars.
}
//Get inputs from request
$data = array();
$data = json_decode(file_get_contents('php://input'), true);
$reply_array = array();
//Start looping through data
foreach($data as $mid => $input) {
//Dig out the installation id and register
$identify = explode('/',trim($input['topic'],'/'));
$installation = preg_replace('/[^a-z0-9_]+/i','',array_shift($identify));
$RegisterID = array_shift($identify)+0;
// Extract the cleaning supplies needed to clean and validate the data
$sql_get_detergent="SELECT ProfileID, Description, Lvl4, Register, BitNo, DataFormat, Units, LowAct, HighAct, Multiplier, FaultCondition, FaultText FROM DeviceProfiles WHERE RegisterID = '".$RegisterID."' ORDER BY RegisterID ASC";
$result_get_detergent = mysqli_query($con,$sql_get_detergent);
while($row_clean = mysqli_fetch_array($result_get_detergent)) {
//Write out the reply
$semantic = strtolower($row_clean['Lvl4']);
$reply_array['topic'] = $installation."/".clean($semantic);
}
print_r(json_encode($reply_array));
}
Output is not what I need - as seen here it yields 3 separate arrays, not one big one.
Current output:
{"topic":"edafdff398fb22847a2f98a15ca3186e\/"}
{"topic":"edafdff398fb22847a2f98a15ca3186e\/"}
{"topic":"edafdff398fb22847a2f98a15ca3186e\/"}
Desired output:
[{
"topic":"edafdff398fb22847a2f98a15ca3186e\/"
},
{
"topic":"edafdff398fb22847a2f98a15ca3186e\/"
},
{
"topic":"edafdff398fb22847a2f98a15ca3186e\/"
}]
What am I doing wrong? Appreciate your help!
You are printing each result in the foreach loop, and in the next iteration of the loop you overwrite the previous result.
Of course you get three single results that way.
Change your code like this:
$all_replies = array();
foreach($data as $mid => $input) {
// [...]
while($row_clean = mysqli_fetch_array($result_get_detergent)) {
$reply_array = array();
// Do you logic here - you can fill $reply_array as you want.
$all_replies[] = $reply_array;
}
}
print_r(json_encode($all_replies));
You could also use this approach, which is a smaller change:
$reply_array = array();
foreach($data as $mid => $input) {
// [...]
while($row_clean = mysqli_fetch_array($result_get_detergent)) {
// [...]
$reply_array[]['topic'] = $installation . "/" . clean($semantic);
}
}
print_r(json_encode($reply_array));
That will work for your example case, but you'll not be able to set another value apart from topic in a new assignment. You'd have to do it this way:
$reply_array = array();
foreach($data as $mid => $input) {
// [...]
while($row_clean = mysqli_fetch_array($result_get_detergent)) {
// [...]
$reply_array[] = array(
'topic' => $installation . "/" . clean($semantic),
'auth' => $your_auth_value,
);
}
}
print_r(json_encode($reply_array));
I prefer the first solution with an extra variable, but the other one works, too.
There is a small mistake in your code.
Replace below code :
$reply_array['topic'] = $installation."/".clean($semantic);
with :
$reply_array[]['topic'] = $installation."/".clean($semantic);

Categories