Sum data in multi-dimensional arrays with variable number of grouping keys - php

Question

I have data coming back from MySQL and the goal is to pull data from multiple servers.
The data can be grouped by one or more fields, and I want to pull data from X number of MySQL servers and sum the data that should be summed, using the other data as a key or index.
Example:
Result from Mysql[$x]:
[{"date":"2017-10-10","account":"1","trees":"1","people":"0","pets":"4"}
,{"date":"2017-10-10","account":"2","trees":"2","people":"5","pets":"1"}
,{"date":"2017-10-11","account":"1","trees":"3","people":"3","pets":"4"}
,{"date":"2017-10-11","account":"2","trees":"4","people":"1","pets":"4"}]
Result from Mysql[$x+1]:
[{"date":"2017-10-10","account":"1","trees":"5","people":"1","pets":"1"}
,{"date":"2017-10-10","account":"2","trees":"5","people":"2","pets":"1"}
,{"date":"2017-10-11","account":"1","trees":"5","people":"0","pets":"2"}
,{"date":"2017-10-11","account":"2","trees":"5","people":"1","pets":"2"}]
Desired end result:
[{"date":"2017-10-10","account":"1","trees":"6","people":"1","pets":"5"}
,{"date":"2017-10-10","account":"2","trees":"7","people":"7","pets":"2"}
,{"date":"2017-10-11","account":"1","trees":"8","people":"3","pets":"6"}
,{"date":"2017-10-11","account":"2","trees":"9","people":"2","pets":"6"}]
in this case, there are 2 keys, date and account, but in reality there can be between 1 and 4.
The grouping keys are known, so the code knows that in this case, date and account are the keys.
I'm trying to find a better way than iterating through each row of mysql[$x] and then iterating through each row of mysql[$x+1] ... mysql[$x+n] and checking each value of the known keys to see if they match the current row's key values and then summing the data.
Also note that in reality, there may not be a row for each set of keys, hence the number and order of rows from each server can be variable.
I need an efficient way to do this because there are potentially tens of thousands of rows with a dozen+ columns.
I'm experimenting with code like this:
$res = array();
$keys = array('date','account');
array_walk($t,'rehash',$res);
function rehash($data,$key,&$result) {
global $keys;
global $res;
$keydata = array_slice($data,0,count($keys));
$vals = array_slice($data,count($keys));
//this is how the data would ideally be structured, my model
//$res[$keydata[$keys[0]]][$keydata[$keys[1]]] = $vals;
//this code below successfully creates rows in the right format
for($x = count($keys) - 1; $x>= 0; $x--) {
if($x == count($keys)-1) {
$tmp = array($keydata[$keys[$x]] => $vals);
//} else if($x ==0) {
// maybe reconcile here?
} else {
$tmp = array($keydata[$keys[$x]] => $tmp);
}
}
}
But how do i reconcile this? I can't simply do $res[key1] = $tmp[key1] because with multiple keys it would remove all the other rows.
Also with nested arrays, how would I iterate through each nested array if the number of levels is variable?
Lastly if I do this, I would need to convert it back into the original format for the frontend, so... joy. :D (this part would be easy, just is worth taking into consideration from an efficiency standpoint)

Answer 1

You don't need to use array_walk, since your data isn't hierarchical. Just use an ordinary foreach loop.
$res = array();
foreach ($t as $row) {
$cur = &$res;
// Drill down into result array using the key fields, creating nested arrays
// as necessary
foreach ($keys as $keycol) {
$key = $row[$keycol];
if (!isset($cur[$key])) {
$cur[$key] = array();
}
$cur = &$cur[$key];
}
// Now go through the data columns in the row, adding them to the totals
foreach ($row as $key => $val) {
if (in_array($key, $keys)) {
// Put the key fields in the result row
$cur[$key] = $val;
} else {
// Accumulate other values in the result row
if (!isset($cur[$key])) {
$cur[$key] = 0;
}
$cur[$key] += $row[$key];
}
}
unset($cur); // Break the reference link
}

Answer 2

Here is a possible solution based on the structure of the sample data given in the question.
$jsn1 = '[{"date":"2017-10-10","account":"1","trees":"1","people":"0","pets":"4"}
,{"date":"2017-10-10","account":"2","trees":"2","people":"5","pets":"1"}
,{"date":"2017-10-11","account":"1","trees":"3","people":"3","pets":"4"}
,{"date":"2017-10-11","account":"2","trees":"4","people":"1","pets":"4"}]';
$jsn2 = '[{"date":"2017-10-10","account":"1","trees":"5","people":"1","pets":"1"}
,{"date":"2017-10-10","account":"2","trees":"5","people":"2","pets":"1"}
,{"date":"2017-10-11","account":"1","trees":"5","people":"0","pets":"2"}
,{"date":"2017-10-11","account":"2","trees":"5","people":"1","pets":"2"}]';
/* Decode the json strings into associative arrays: */
$arr1 = json_decode($jsn1, true);
$arr2 = json_decode($jsn2, true);
/* Put in an array the keys whose values are to be added: */
/* Do not include date and account keys if they are not to be added */
$arr_keys = ['trees', 'people', 'pets'];
/* Loop through both $arr1 and $arr2 while storing new arrays in arr: */
/* Be adding up the values for the keys that are in $arr_keys */
$arr = [];
foreach($arr1 as $k => $v){
$arr[$k]['date'] = $arr1[$k]['date']; $arr[$k]['account'] = $arr1[$k]['account'];
foreach($arr_keys as $y){
if($arr1[$k][$y] !== null && $arr2[$k][$y] !== null){
$arr[$k][$y] = $arr1[$k][$y] + $arr2[$k][$y];
}}}
/* encode the array to the required json string: */
$json = json_encode($arr);
echo $json;
/* Output:
[{"date":"2017-10-10","account":"1","trees":"6","people":"1","pets":"5"}
,{"date":"2017-10-10","account":"2","trees":"7","people":"7","pets":"2"}
,{"date":"2017-10-11","account":"1","trees":"8","people":"3","pets":"6"}
,{"date":"2017-10-11","account":"2","trees":"9","people":"2","pets":"6"}]
*/