Related
First of all, I have following data returned from database. I will have two different data as below respectively
sum1
count1
sm__state_name__
om__order_date__year
om__order_date__quarter
om__order_date__month
5645000
4
Luanda
2017
3
8
213985939.8600001
1606
Luanda
2017
3
9
7729331.52
119
Benguela
2017
3
9
1012936
17
Zaire
2017
3
9
1054883
19
Bie
2017
3
9
2347944
26
Cuando Cubango
2017
3
9
428769.6000000001
60
Bengo
2017
3
9
6444569
86
Huila
2017
3
9
4914030
25
Cunane
2017
3
9
1167200
26
Cuanza North
2017
3
9
750080
10
Cuanza Sul
2017
3
9
2178100
6
Huambo
2017
3
9
1099934
25
Lunda North
2017
3
9
410135
12
Malange
2017
3
9
In array format
array (
0 =>
array (
'sum1' => '5645000',
'count1' => '4',
'sm__state_name__' => 'Luanda',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '8',
),
1 =>
array (
'sum1' => '213985939.8600001',
'count1' => '1606',
'sm__state_name__' => 'Luanda',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
2 =>
array (
'sum1' => '352839.60000000003',
'count1' => '9',
'sm__state_name__' => NULL,
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
3 =>
array (
'sum1' => '7729331.52',
'count1' => '119',
'sm__state_name__' => 'Benguela',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
4 =>
array (
'sum1' => '1012936',
'count1' => '17',
'sm__state_name__' => 'Zaire',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
5 =>
array (
'sum1' => '1054883',
'count1' => '19',
'sm__state_name__' => 'Bie',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
6 =>
array (
'sum1' => '2347944',
'count1' => '26',
'sm__state_name__' => 'Cuando Cubango',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
7 =>
array (
'sum1' => '428769.6000000001',
'count1' => '60',
'sm__state_name__' => 'Bengo',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
8 =>
array (
'sum1' => '6444569',
'count1' => '86',
'sm__state_name__' => 'Huila',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
9 =>
array (
'sum1' => '4914030',
'count1' => '25',
'sm__state_name__' => 'Cunane',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
10 =>
array (
'sum1' => '1167200',
'count1' => '26',
'sm__state_name__' => 'Cuanza North',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
11 =>
array (
'sum1' => '750080',
'count1' => '10',
'sm__state_name__' => 'Cuanza Sul',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
12 =>
array (
'sum1' => '2178100',
'count1' => '6',
'sm__state_name__' => 'Huambo',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
13 =>
array (
'sum1' => '1099934',
'count1' => '25',
'sm__state_name__' => 'Lunda North',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
14 =>
array (
'sum1' => '410135',
'count1' => '12',
'sm__state_name__' => 'Malange',
'om__order_date__year' => '2017',
'om__order_date__quarter' => '3',
'om__order_date__month' => '9',
),
)
Array
(
"sm__state_name__",
"om__order_date__year",
"om__order_date__quarter",
"om__order_date__month",
)
ABOBE ARRAY INCLUDES CAN BE ANY NUMBER OF FIELDS
Below is sample output of What I want in return
{
"data": [
{
"key": "Luanda",
"items": [
{
"key": 2017,
"items": [
{
"key": 3,
"items": [
{
"key": 8,
"items": null,
"count": 4,
"summary": [
438380.9935
]
},
{
"key": 9,
"items": null,
"count": 1606,
"summary": [
438380.9935
]
},
],
"summary": [
1285085.9636
]
}
],
"summary": [
1285085.9636
]
}
],
"summary": [
1285085.9636
]
},
{
"key": "Benguela",
"items": [
{
"key": 2017,
"items": [
{
"key": 3,
"items": [
{
"key": 9,
"items": null,
"count": 679,
"summary": [
4781987.8575
]
},
],
"summary": [
15017212.0305
]
}
],
"summary": [
15017212.0305
]
}
],
"summary": [
15017212.0305
]
},
{...},
{...},
{...},
],
"totalCount": 22854
}
Don't mind the summary value. I just put dummy values there.
Is this kind of process even possible? Because I think of many different things, recursion, multiple loops, triple loop but couldn't think of way this could work.
I know it's not an issue or bug. Sorry for that. But it would be great if someone could point me to right direction.
With a variable array of columns you need to group in hierarchical order, you'll certainly want a recursive solution to this problem. For each step in your recursive calls, check to see if a particular grouping level exists yet, and if not, then initialize it. Group using associative arrays for your items, then convert to flat arrays after. It's very simple conceptually, although perhaps a little confusing to look at:
function aggregateData($data, $db_row, $columns, $first_column = true) {
// Base case: with no more columns left, we just take the sum and return.
if(empty($columns)) {
$data['summary'] += $db_row['sum1'];
return $data;
}
$column = array_shift($columns);
$value = $db_row[$column];
if($first_column) {
// First column is a special case. We don't add anything here because every level's summary is the sum of its nested items.
if(!array_key_exists($value, $data)) {
$data[$value] = [
'key'=>$value,
'items'=>empty($columns) ? null : [],
'summary'=>0
];
}
$data[$value] = aggregateData($data[$value], $db_row, $columns, false);
} else {
// For all other columns, we add the sum to each nested level.
if(!array_key_exists($value, $data['items'])) {
$data['items'][$value] = [
'key'=>$value,
'items'=>empty($columns) ? null : [],
'summary'=>0
];
}
$data['summary'] += $db_row['sum1'];
$data['items'][$value] = aggregateData($data['items'][$value], $db_row, $columns, false);
}
return $data;
}
function flattenData($data) {
foreach($data as $key=>$value) {
if(is_null($value['items'])) {
break;
}
$data[$key]['items'] = flattenData($value['items']);
}
return array_values($data);
}
$db_rows = /* your DB retrieval code here */;
$columns = /* columns to group by in hierarchical order */;
$data = [];
foreach($db_rows as $db_row) {
$data = aggregateData($data, $db_row, $columns);
}
$data = flattenData($data);
To help understand what's going on, consider the top-most level, grouping by state. After the aggregateData() calls, before flattening the arrays, it will produce a structure that looks like the following:
{
"Luanda": {
"key": "Luanda",
"items": {...},
"summary": ...,
},
"Benguela": {
"key": "Benguela",
"items": {...},
"summary": ...,
}
}
Notice that because each entry is associated with its key in an object, instead of an index in an array, this allows for easy lookups so we can aggregate information at each level. After flattening, we instead get this:
[
{
"key": "Luanda",
"items": [...],
"summary": ...,
},
{
"key": "Benguela",
"items": [...],
"summary": ...,
}
]
Each entry is no longer associated with its key, instead being the desired flat array. We lose the ability to do simple lookups, but we no longer need that capability at the end of our calculations.
The above doesn't solve the entirety of your problem as there are points of data not being included in this result, but as stackoverflow is not a free coding service and you have not provided any of your own code, I will be leaving the necessary modifications as an exercise. This should, however, remove the bulk of the work required and serve as a strong starting point for your solution.
Ok, so basically if I understand this correctly, you have a database with a bunch of records. Then you want to create some massive JSON based off the database that will have 4 dimensions based on the columns for the state, year, quarter, and month. Then the outer-most array will contain the records of the database grouped by the state, then the items for any specific state will further constrict into a set of items based on the key for the year, etc.
I'm thinking the most efficient way to do this is a single loop where you'd pull all the records in the database then loop through it in PHP and construct some new arrays based on the current record iteration which can be referred to later for look-up purposes.
Begin by setting:
$dataItems = [];
This would be used for storing and structuring data for easily look up and calculations.
Iterate through the list. The first item would be:
sum1
count1
state_name
order_year
order_quarter
order_month
5645000
4
Luanda
2017
3
8
Then you'd run code for the iteration that would look something like this to help populate the array or arrays you'd be constructing:
if (empty($dataItems['Luanda'])) {
$dataItems['Luanda'] = [];
}
if (empty($dataItems['Luanda'][2017])) {
$dataItems['Luanda'][2017] = [];
}
if (empty($dataItems['Luanda'][2017][3])) {
$dataItems['Luanda'][2017][3] = [];
}
if (empty($dataItems['Luanda'][2017][3][8])) {
$dataItems['Luanda'][2017][3][8] = [];
}
$dataItems['Luanda'][2017][3][8][] = ['sum1' => 5645000, 'count1' => 4];
The second item would be:
sum1
count1
state_name
order_year
order_quarter
order_month
213985939.8600001
1606
Luanda
2017
3
9
The PHP code for this iteration would look like this:
if (empty($dataItems['Luanda'])) {
$dataItems['Luanda'] = [];
}
if (empty($dataItems['Luanda'][2017])) {
$dataItems['Luanda'][2017] = [];
}
if (empty($dataItems['Luanda'][2017][3])) {
$dataItems['Luanda'][2017][3] = [];
}
if (empty($dataItems['Luanda'][2017][3][9])) {
$dataItems['Luanda'][2017][3][9] = [];
}
$dataItems['Luanda'][2017][3][9][] = ['sum1' => 213985939.8600001, 'count1' => 1606];
Etc.
Then you'd loop through the $dataItems structure once it's built and do stylizing logic like having a key called "items", etc and create your desired output structure, then finally output in JSON format by using json_encode.
If you need something like the count for the entire year, in your initial loop, you can write to a separate array to help keep track of it, adding it together along the way, then refer to it later when creating an array with the desired output structure. Things like summary you can easily add into the loop and keep track of that along the way.
Sounds like some fun, but yeah you'd only need one loop to create a lookUp or multiple lookUp sort of arrays then a second loop to refer to the loopUp array(s) to get your data in the desired output format.
Let's assume we have a "big" date and time range, like
$big = [
'start' => '2018-09-01 00:00:00',
'stop' => '2018-09-01 23:59:59'
]
I have to create an array with all ranges not included in another array, like the following:
$exclude = [
[
'start' => '2018-09-01 12:00:00',
'stop' => '2018-09-01 14:59:59'
],
[
'start' => '2018-09-01 18:00:00',
'stop' => '2018-09-01 19:59:59'
]
]
so that the resulting structure would be something like this:
$results = [
[
'start' => '2018-09-01 00:00:00',
'stop' => '2018-09-01 11:59:59'
],
[
'start' => '2018-09-01 15:00:00',
'stop' => '2018-09-01 17:59:59'
],
[
'start' => '2018-09-01 20:00:00',
'stop' => '2018-09-01 23:59:59'
]
]
tl;dr
I have a big date time range (like the ones wrote above) and I have to create a resulting one by removing one or more ranges coming from another structure. Something like extracting the work breaks from the day, thus keeping only the real working hours.
I'm using Carbon and Laravel, any native solution to this or should I parse the whole structure and make the holes on my own? I don't like to reinvent the wheel.
I don't know a lot on carbon but I know that Carbon extends the native DateTime class so the following code can be adapted to achieve your purpose:
$result=[['start'=>$big['start'],'stop'=>''],['start'=>'','stop'=>''],['start'=>'','stop'=>$big['stop']]];//you can build this with a loop if you want...
foreach($result as $k =>$value){
switch($k){
case 0:
$result[$k]['stop']=(($date=date_create($exclude[0]['start']))&&$date->sub(new dateInterval('PT1S')))?$date->format('Y-m-d H:i:s'):'';
unset($date);
break;
case 1:
$result[$k]['start']=(($date=date_create($exclude[0]['stop']))&&$date->add(new dateInterval('PT1S')))?$date->format('Y-m-d H:i:s'):'';
$result[$k]['stop']=(($date=date_create($exclude[1]['start']))&&$date->sub(new dateInterval('PT1S')))?$date->format('Y-m-d H:i:s'):'';
unset($date);
break;
case 2:
$result[$k]['start']=(($date=date_create($exclude[1]['stop']))&&$date->add(new dateInterval('PT1S')))?$date->format('Y-m-d H:i:s'):'';
unset($date);
break;
default:
break;
}
}
I am using PHP array and outputing to JSON string. I want to convert "0" in the json format to lets say calling them "items". Since while importing this to oracle db it says expecting names instead of zero. When I change the index ( 0, 1, 2 ) to be called just items, it works normally.
Here is the PHP array, and I am outputting it as json.
$data = array(
'INDIVIDUAL_SALUTATION' => $salution,
'INDIVIDUAL_FIRST_NAME' => $firstname,
'INDIVIDUAL_LAST_NAME' => $lastname,
'GENDER' => $gender,
'DATE_OF_BIRTH' => $result['Birthday'],
'EMAIL_ID' => $result['Email'],
'MOBILE_NUMBER' =>$result['Phone'],
'MOBILE_COUNTRY_CODE' => substr($result['Phone'], 1, 3),
'OCCUPATION' => null,
'OCCUPATION_STATUS' => null,
'ADDRESS_LINE1' => $result['Address_street'],
'TOWN' => $result['Address_city'],
'POSTAL_CODE' => $result['Address_zip'],
'COUNTRY' => $result['country_name'],
'CUSTOMER_NUMBER' => $result['Owner'],
'POLICY_START_DATE' => $result['paid_thru_date'],
'POLICY_END_DATE' => $result['duedate'],
'LOAN_AGREEMENT_NUMBER' => $result['ORIG_ID'],
'REPAYABLE_AMOUNT' => $result['repayable_amount'],
'FINANCE_TERM_MONTHS' => $result['finance_term_months'],
'MONTHLY_INSTALLMENT' => $result['monthly_installment'],
'AMOUNT_INSURED' => $result['amount_insured'],
'CURRENCY_ID' => $result['Abbreviation']
);
$jsonArray[] = $data;
}
$mainInfo = array(
'SRC_NAME' => 'AEX',
"RUN_NUMBER" => 1,
"RUN_DATE" => date("Ymd"),
"RUN_NO_OF_RECORDS" => $arrayCount,
"YTD_NO_OF_RECORDS" => $arrayCount
);
$finalArray = array_merge($jsonArray , $mainInfo);
$this->output->set_content_type('application/json');
$this->output->set_output(json_encode($finalArray));
$this->output->set_content_type('application/json');
$this->output->set_output(json_encode($mainInfo));
Here is the output
{
"0":{
"INDIVIDUAL_SALUTATION":"MR",
"INDIVIDUAL_FIRST_NAME":"borrower",
"INDIVIDUAL_LAST_NAME":"three",
"GENDER":"M",
"DATE_OF_BIRTH":"1993-09-17",
"EMAIL_ID":"borrowerthree#aurorax.co",
"MOBILE_NUMBER":"+3581466144569",
"MOBILE_COUNTRY_CODE":"358",
"OCCUPATION":null,
"OCCUPATION_STATUS":null,
"ADDRESS_LINE1":"Vaskivuorentie 22B",
"TOWN":"Vantaa",
"POSTAL_CODE":"01600",
"COUNTRY":"Finland",
"CUSTOMER_NUMBER":"772",
"POLICY_START_DATE":"2017-01-02",
"POLICY_END_DATE":"2017-07-01",
"LOAN_AGREEMENT_NUMBER":"7",
"REPAYABLE_AMOUNT":"50.42",
"FINANCE_TERM_MONTHS":"6",
"MONTHLY_INSTALLMENT":"8.40",
"AMOUNT_INSURED":"50.42",
"CURRENCY_ID":"EUR"
},
"1":{
"INDIVIDUAL_SALUTATION":"MR",
"INDIVIDUAL_FIRST_NAME":"borrower",
"INDIVIDUAL_LAST_NAME":"three",
"GENDER":"M",
"DATE_OF_BIRTH":"1993-09-17",
"EMAIL_ID":"borrowerthree#aurorax.co",
"MOBILE_NUMBER":"+3581466144569",
"MOBILE_COUNTRY_CODE":"358",
"OCCUPATION":null,
"OCCUPATION_STATUS":null,
"ADDRESS_LINE1":"Vaskivuorentie 22B",
"TOWN":"Vantaa",
"POSTAL_CODE":"01600",
"COUNTRY":"Finland",
"CUSTOMER_NUMBER":"772",
"POLICY_START_DATE":"2017-01-02",
"POLICY_END_DATE":"2017-07-01",
"LOAN_AGREEMENT_NUMBER":"9",
"REPAYABLE_AMOUNT":"40.35",
"FINANCE_TERM_MONTHS":"6",
"MONTHLY_INSTALLMENT":"6.73",
"AMOUNT_INSURED":"40.35",
"CURRENCY_ID":"EUR"
},
"2":{
"INDIVIDUAL_SALUTATION":"MR",
"INDIVIDUAL_FIRST_NAME":"borrower",
"INDIVIDUAL_LAST_NAME":"two",
"GENDER":"M",
"DATE_OF_BIRTH":"1993-09-17",
"EMAIL_ID":"borrowertwo#aurorax.co",
"MOBILE_NUMBER":"+358466144569123",
"MOBILE_COUNTRY_CODE":"358",
"OCCUPATION":null,
"OCCUPATION_STATUS":null,
"ADDRESS_LINE1":"Vaskivuorentie 22B",
"TOWN":"Vantaa",
"POSTAL_CODE":"01600",
"COUNTRY":"Finland",
"CUSTOMER_NUMBER":"770",
"POLICY_START_DATE":"2017-01-02",
"POLICY_END_DATE":"2017-07-01",
"LOAN_AGREEMENT_NUMBER":"11",
"REPAYABLE_AMOUNT":"99.84",
"FINANCE_TERM_MONTHS":"6",
"MONTHLY_INSTALLMENT":"16.64",
"AMOUNT_INSURED":"99.84",
"CURRENCY_ID":"EUR"
},
"RUN_NUMBER":1,
"RUN_DATE":"20170109"
}
What I am trying to do is make it look like instead of 0 have 'items' : {} and then move on, so I can import it to oracle.
How can I achieve this with php?
Don't use array_merge, put the array in an element of $mainInfo:
$mainInfo['items'] = $jsonArray;
$this->output->set_output(json_encode($mainInfo));
Create index variable and make it to zero
$index = 0
And change this $jsonArray[] = $data; to this $jsonArray['item' .$index++] = $data;
Hope that helps
This is my invoices mongo table
{
'_id': ObjectId("565d78336fe444611a8b4593"),
'TransactionID': 'X020',
'Type': 'SALESINVOICE',
'InvoiceNumber': 'ABC020',
'Date': '2015-11-01 00:00:00',
'DueDate': '2015-12-01 00:00:00',
'CurrencyCode': 'GBP',
'CurrencyRate': NumberLong(1.2),
'Total': NumberLong(200),
'PlannedPaymentDate': '',
'HasAttachments': 0,
'UpdatedDate': '2015-12-01 10:36:35',
'AmountDue': NumberLong(200),
'AmountPaid': 0,
'Status': 'OPEN',
'AmountCredited': 0,
'AmountDebited': 0,
'Source': 1,
'InvoiceID': 'csv_1_X020',
'ExpectedPaymentDate': '2015-12-01 00:00:00',
'ContactID': 1,
'ValidateStatus': 'IMPORT',
'StatusChangeDate': ''
}
What i want the multiplication of two fields (Total * CurrencyRate)
I tried this query ::
DB::connection($this->MongoSchemaName)->collection($this->InvoicesTable)->where('Type', 'PAYMENT')->where('ContactID', (int)$customer->ContactID)->select(DB::raw('sum(Total*CurrencyRate)'))->first();
and tried more ::
DB::connection($this->MongoSchemaName)->collection($this->InvoicesTable)->where('Type', 'PAYMENT')->where('ContactID', (int)$customer->ContactID)->sum(DB::raw('Total * CurrencyRate'));
but not getting the exact out put all time i get 0
I believe aggregation operators like sum expect exact column name as a parameter. You can try to project the multiplication first, then sum the result:
DB::connection($this->MongoSchemaName)
->collection($this->InvoicesTable)
->where('ContactID', (int)$customer->ContactID)
->project([
'ContactID' => 1,
'TotalInBaseCurrency' => ['$multiply' => ['$Total', '$CurrencyRate']]
])
->sum('TotalInBaseCurrency')
or use aggregation directly:
DB::connection($this->MongoSchemaName)
->collection($this->InvoicesTable)
->raw(function($collection) use ($customer){
return $collection->aggregate([
['$match' => [
'ContactID' => (int)$customer->ContactID,
'Type' => 'PAYMENT'
]
],
['$group' => [
'_id' => '$ContactID',
'TotalInBaseCurrency' => [
'$sum' => ['$multiply' => ['$Total', '$CurrencyRate']]
]
]
]
]);
})
I'm using an API which gives an example of how they want the data of the POST request I'm about to make to be formatted. This is their example:
un=chris&
key=xxxx&
origin=plot&
platform=lisp&
args=[[0, 1, 2], [3, 4, 5], [1, 2, 3], [6, 6, 5]]&
kwargs={"filename": "plot from api",
"fileopt": "overwrite",
"style": {
"type": "bar"
},
"traces": [1],
"layout": {
"title": "experimental data"
},
"world_readable": true
}
I'm confused about how I should put together this data from existing arrays in PHP. From what I understand the example show an encoded "string" that is just partly encoded? As of now I am putting the string together all by myself through extracting the keys and values from the arrays.
I'm looking for a more neat way of doing this with existing methods?
I believe using http_build_query will solve this for you.
Given the example, here's sample of how to use it:
$args = array(array(0,1,2), array(3,4,5), array(1,2,3), array(6,6,5));
$kwargs = array(
'filename' => 'plot from api',
'fileopt' => 'overwrite'
'style' => array('type' => 'bar'),
'traces' => array(1),
'layout' => array('title' => 'experimental data'),
'word_readable' => true
);
$request = array(
'un' => 'chris',
'key' => 'xxx',
'origin' => 'plot',
'platform' => 'lisp',
'args' => $args,
'kwargs' => $kwargs
);
$queryString = http_build_query($request);
echo $queryString;
More info: https://php.net/http_build_query