mySQL with PHP to JSON for clientside chart - php

I am hoping to get a better way to do what I am doing. I have several charts to do and want to go about it the best way.
I am using HighCharts which requires a JSON object for data. I am using Laravel as my back end. I am sending 2 variables (1 for labels & 1 for values) to my view. I see i can do this in the controller but haven't started yet
My Issue:
I have this table
| date | printer | quant |
| 2017-01-23 | JP1 | 20 |
| 2017-01-23 | JP1 | 12 |
| 2017-01-23 | JP2 | 18 |
| 2017-01-24 | JP5 | 16 |
| 2017-01-24 | JP1 | 8 |
| 2017-01-26 | JP2 | 12 |
I think that hits all the cases, variable printers, missed days, need to sum quantity per day and make 0 if missing. Need an array for each printer which I dont have a premade list because they change a lot. my querys will be with a date range. The list of printers will be distinct for that date range.
What I have done is make a pivot mySQL query which gets me this
[
{
"date": "2017-01-23",
"sum": "1416",
"JP4_3": "375",
"JP4_1": "533",
"JP4_2": "508",
"B3": null,
"A2": null
},
{
"date": "2017-01-24",
"sum": "2151",
"JP4_3": "847",
"JP4_1": "499",
"JP4_2": "805",
"B3": null,
"A2": null
},
{
"date": "2017-01-25",
"sum": "2097",
"JP4_3": "284",
"JP4_1": "917",
"JP4_2": "896",
"B3": null,
"A2": null
}
]
I can loop through and grab each property to make an array.
I am wondering is there a better way. I need to get a label and data to work with high charts. Here is an example.
xAxis: {
categories: ['Apples', 'Bananas', 'Oranges']
},
series: [{
name: 'Jane',
data: [1, 0, 4]
}, {
name: 'John',
data: [5, 7, 3]
}]
I know this is a complicated and long winded question. Thank you for any help and direction.

return just array of data from laravel and in view balde use
var data = JSON.parse( '{!! json_encode($chartData) !!}' );
I used this in my code

Wanted to give a code example for how I did this. Kind of a pain but from Samsquanch i guess this is normal. This does several database calls but still plenty fast for me. This is using laravel framework.
// Main chart info
$PrinterChartArray ["chart"] = array (
'type' => 'line'
);
$PrinterChartArray ["title"] = array (
"text" => "Units per Printer"
);
$PrinterChartArray ["credits"] = array (
"enabled" => false
);
$PrinterChartArray ["xAxis"] = array (
"categories" => array ()
);
$PrinterChartArray ["tooltip"] = array (
"valueSuffix" => " units"
);
$PrinterChartArray ["yAxis"] = array (
"title" => array (
"text" => "Units"
)
);
// Get series Names
$printerNames = PrintJob::distinct()->select('printer')
->whereDate('job_date', '>=', date($from))
->whereDate('job_date', '<=', date($to))
->get();
// loop through series names and set series name and data for high charts
$chartSeries = [];
foreach($printerNames as $printer)
{
$printName = $printer->printer;
// Used where inside join so wouldnt miss dates
// used a calendar table with all dates to fill in missing dates
$data = PrintJob::select(DB::raw('COALESCE(SUM(print_jobs.quantity), 0) as sum'), DB::raw('calendar.datefield'))
->rightJoin('calendar', function($join) use ($printName)
{
$join->on('print_jobs.job_date', '=', 'calendar.datefield')
->where('printer', '=', $printName);
})
->whereDate('calendar.datefield', '>=', date($from))
->whereDate('calendar.datefield', '<=', date($to))
->groupBy('calendar.datefield')
->orderBy('calendar.datefield', 'ASC')
//->toSql();
->get();
//->pluck('sum');
// Needed to convert the returned value from a string value to a int value (apparently a driver issue with php and mysql)
foreach ($data as $key => $var) {
$data[$key] = (int)$var->sum;
}
// Add series to high chart
$chartSeries[] = array(
"name" => $printName,
"data" => $data
);
}
// Unessesary to use PrintJob for this but keeping similar to series to not mess up label
$chartLabel = PrintJob::select(DB::raw('calendar.datefield'))
->rightJoin('calendar', function($join) use ($printName)
{
$join->on('print_jobs.job_date', '=', 'calendar.datefield');
})
->whereDate('calendar.datefield', '>=', date($from))
->whereDate('calendar.datefield', '<=', date($to))
->groupBy('calendar.datefield')
->orderBy('calendar.datefield', 'ASC')
//->toSql(); --- This will show the actual query
//->get(); --- Will give array of objects
->pluck('calendar.datefield'); // --- Will give a array of just the values wanted
$PrinterChartArray ["series"] = $chartSeries;
$PrinterChartArray ["xAxis"] = array (
"categories" => $chartLabel
);
//return $PrinterChartArray; -- used to see output without view
return view('print.charts', compact('jobsByDayLabels', 'jobsByDayValues', 'unitsByDayLabels', 'unitsByDayValues', 'PrinterChartArray'));
And this is the view part
<!-- Chart for Units by Printer by Day -->
<div class="row">
<div class="col-md-12 col-sm-12 col-xs-12">
<div class="x_panel">
<div id="printerUnitsByDay" style="min-width: 310px; height: 500px; margin: 0 auto"></div>
</div>
</div>
</div>
and this below is the script on the view
$(function() {
$('#printerUnitsByDay').highcharts({!! json_encode($PrinterChartArray) !!})
});

Related

Design patterns to override attributes of object and validate them

GOAL:
I want to build a "market validation control engine" that validates players' actions and limits on the market.
SUMMARY:
Consider that we have a Character object that can buy only a given number of Item objects, e.g Arrow. The number depends on Race, Tire and Area where a lower layout of hierarchy overrides the upper; if the limit isn't specified lower layout inherits it from the upper:
For example:
Pro ELF in Forest can buy up to 2000 arrows
Basic Ork in Forest and Basic Ork in Desert can buy 50 arrows.
Additionally, the Character object can Buy and Sell a specific amount of arrows on the market. The amount and action depend on the Area. Otherway to say is every node might have its validation rules.
For example:
Basic Elf in Desert can sell 50% arrows
Basic Orc in Desert can sell 100% of the arrows.
Lastly, based on the Tier players can Exchange arrows to another item. For example :
Pro ELF can exchange 100% of its arrows
STRUGGLES:
Building hierarchy
Validation rules. Every layout of hierarchy (Tier or Area) have rules
How to make 1 and 3 work together
Scalability. Adding a new layout (e.g. Clan) or attaching a new validation rule to Area
Maintenance
I am almost sure that I need multiple different patterns here. Currently, I'm thinking of the Decorator Pattern for creating hierarchy object with all the limits and the Chain of Responsibility for the rule validation. However, I am not sure how well these patterns will work together in the long ran.
UPD: In the example, I used Arrow - one of the hundreds of items on the market and Desert with Forest - two of the 9 areas.
There's quite a few different ways to implement this so I'm assuming you're not really sure where to start.
I've written up some example code using classes. In this, there's 2 ways that we could store the item (arrow) data.
Nested Array - This allows for a more "structured" approach. It also allows you to easily add more attributes to each "child" array if necessary.
Serialized Map - This has a much smaller code footprint and is generally quicker.
NOTE: This code is not setup to be well-performing (in reference to speed) but as a way to see how you could setup this system.
There are 3 files here index.php, Character.class.php, Market.class.php.
There are 2 main methods on the Market class: getNestedLimit() and getSerializedLimit(). Each one shows how you would get the limit based on those example structures.
<?php
// index.php
require_once __DIR__ . "/Character.class.php";
require_once __DIR__ . "/Market.class.php";
$market = new Market;
$orc_a = new Character("orc", "basic", "desert");
$orc_b = new Character("orc", "pro", "forest");
$market->getNestedLimit($orc_a, "arrow"); // 50
$market->getNestedLimit($orc_b, "arrow"); // 150
$elf_a = new Character("elf", "pro", "forest");
$elf_b = new Character("elf", "pro", "desert");
$market->getSerializedLimit($elf_a, "arrow"); // 2000
$market->getSerializedLimit($elf_b, "arrow"); // 50
<?php
// Character.class.php
class Character {
public string $race;
public ?string $tier = null;
public ?string $area = null;
public function __construct(
string $race,
?string $tier = null,
?string $area = null,
) {
$this->race = $race;
$this->tier = $tier;
$this->area = $area;
}
}
// Market.class.php
class Market {
public $nested_items = [
"arrow" => [
"limit" => [
"_base" => 0,
"race" => [
"orc" => [
"_base" => 50,
"tier" => [
"basic" => [
"area" => [],
],
"pro" => [
"area" => [
"desert" => 50,
"forest" => 150,
],
],
],
],
"elf" => [
"_base" => 1000,
"tier" => [
"basic" => [
"area" => [
"desert" => 150,
"forest" => 1000,
],
],
"pro" => [
"area" => [
"desert" => 50,
"forest" => 2000,
],
],
],
],
],
],
],
];
public function getNestedLimit(Character $character, string $item_id) {
$item = $this->nested_items[$item_id];
$item_limit = $item["limit"];
$actual_limit = $item_limit["_base"];
// check race
if (isset($item_limit["race"][$character->race])) {
$race_limit = $item_limit["race"][$character->race];
if (isset($race_limit["_base"])) {
// value exists - use this
$actual_limit = $race_limit["_base"];
}
// check tier
if (isset($race_limit["tier"][$character->tier])) {
$tier_limit = $race_limit["tier"][$character->tier];
if (isset($tier_limit["_base"])) {
// value exists - use this
$actual_limit = $tier_limit["_base"];
}
// check area
if (isset($tier_limit["area"][$character->area])) {
$actual_limit = $tier_limit["area"][$character->area];
}
}
}
return $actual_limit;
}
/** Serialize syntax:
* {race}[-{tier}[-{area}]]
* e.g.
* "Pro tier orc in forest" -> "o-p-f"
* "Basic tier elf" -> "e-b"
* "Orc" -> "o"
*
* When determining the limit, pick the most specific first.
*/
public $serialized_items = [
"arrow" => [
"limit" => [
"_base" => 0,
// orc
"o" => 50,
"o-p-d" => 50,
"o-p-f" => 150,
// elf
"e" => 1000,
"e-b-d" => 150,
"e-b-f" => 1000,
"e-p-d" => 50,
"e-p-f" => 2000,
]
]
];
/** This "map" is purely for example - you may end up using ids
* or something more consistent.
* I would recommend using a different system, such as enums or constants.
*/
public $serial_keys = [
"orc" => "o",
"elf" => "e",
"basic" => "b",
"pro" => "p",
"desert" => "d",
"forest" => "f",
];
public function getSerializedLimit(Character $character, string $item_id) {
$item = $this->serialized_items[$item_id];
$item_limit = $item["limit"];
$actual_limit = $item_limit["_base"];
// serialize character
// this could be done in various ways - this is purely for example
// start with most specific
$serial = $this->serial_keys[$character->race] . "-" . $this->serial_keys[$character->tier] . "-" . $this->serial_keys[$character->area];
while ($serial !== "") {
if (isset($item_limit[$serial])) {
$actual_limit = $item_limit[$serial];
break;
}
// remove most specific section
// here we are expected each section to be a single character
// be removing 2 characters, we remove the hyphen as well
// NOTE: this hyphen is only here to help visualize
// you could very easily use 1 character without
// the hyphen and change this to `-1`
$serial = substr($serial, -2);
}
return $actual_limit;
}
}
EDIT: Adding example compiling nested array
As mentioned in the comments, you won't have to manually adjust the array - instead you could have it compile on load/initialise of the market. The code below shows an example datasource setup and code to compile the nested array.
This can also easily be adjusted to re-created the serialized array.
<?php
Datasource: `item`
| item_id | name | limit |
|---------|-------|-------|
| arrow | Arrow | 0 |
Datasource: race
| race_id | `name` |
|---------|--------|
| elf | Elf |
| orc | Orc |
Datasource: `tier`
| tier_id | name |
|---------|-------|
| basic | Basic |
| pro | Pro |
Datasource: `area`
| area_id | name |
|---------|--------|
| forest | Forest |
| desert | Desert |
Datasource: `item_limit`
| item_id | race_id | tier_id | area_id | limit |
|---------|---------|---------|---------|-------|
| arrow | orc | NULL | NULL | 50 |
| arrow | orc | pro | forest | 150 |
| arrow | elf | NULL | NULL | 1000 |
| arrow | elf | basic | desert | 150 |
| arrow | elf | pro | desert | 50 |
| arrow | elf | pro | forest | 2000 |
// Using these - compile to nested array (i.e. you only need to maintain the datasources, not the array)
$items = ... // results from `item` datasource;
$races = ... // results from `race` datasource;
$tiers = ... // results from `tier` datasource;
$areas = ... // results from `area` datasource;
$nested_array = [];
foreach($items as $item){
$limit_data = ["_base" => $item["limit"]];
// populate races
$limit_data["race"] = [];
foreach($races as $race){
$race_data = ["tier" => []];
// populate tiers
foreach($tiers as $tier){
$tier_data = ["area" => []];
foreach($areas as $area){
$tier_data["area"][$area["area_id"]] = [];
}
$race_data["tier"][$tier["tier_id"]] = $tier_data;
}
$limit_data["race"][$race["race_id"]] = $race_data;
}
$item_limits = ... // results from `item_limit` datasource filtered by `item_id`
foreach($item_limits as $item_limit){
// expects "less specific" columns to always be set
// i.e. if `area` is set, `tier` and `race` should be too
// assigning variable by reference to keep this example short
$parent =& $limit_data["race"][$item_limit["race_id"]];
if(!isset($item_limit["tier_id"])){
$parent["_base"] = $item_limit["limit"];
continue;
}
$parent =& $parent["tier"][$item_limit["tier_id"]];
if(!isset($item_limit["area_id"])){
$parent["_base"] = $item_limit["limit"];
continue;
}
$parent["area"][$item_limit["area_id"]] = $item_limit["limit"];
}
$nested_array[$item["item_id"]] = ["limit" => $limit_data];
}
Memory Considerations
The above code shows how you could implement a system like this. However, in a live environment with many items, it would be better to only initialise/compile the nested/serialized array for a specific item as needed. Caching is your friend here.
Alternatively you can take this same approach and implement it in such a way that it only needs to query the datasource to reduce the memory footprint.
There are two questions actually.
How to build the character hierarchy with design pattern
How to rule the market with validation
This is my modeling suggestion
To the first problems. There are different properties to define a Character. Race and Tier. I am not sure if Area is the location of the character position. Or it is the biome of the character. If latter, then the character properties become three. I suggest to model the hierarchy with composition over inheritance. Otherwise, using traditional inheritance would result in many combination of classes (default Orc, Basic Orc, Pro Orc etc). Not to mention if more Race or Tier to come in the future.
To the second problem, I would not recommend chain of responsibility. Because with CoR, you need to define a long chain. Each processing object handle a variation of Character. You will need many processing object similar to the first problem.
I suggest to have a data structure registered what kind of Character could trade how many items. In my suggestion this is the responsibility of MarketAuthority. Imagine it has a map-like structure inside, and each entry it has a record call MarketRegulation. MarketRegulation define what kind of Character can buy or sell how many items. When trade happens, pass the character to MarketAuthority and find which MarketRegulation match the character. From the MarketRegulation result you know how many the character can trade.
Below is the example of MarketAuthority implementation.
class MarketAuthority
{
private $regulations;
function __construct()
{
$this->regulations = array();
// setting up regulations. Ideally they should be come from data source
// orc
$orc = new Character(null, null, new Orc());
array_push($this->regulations, new MarketRegulation($orc, new ActualLimit(50), TradeType::BUY));
.
.
.
// basic orc
$basicOrc = new Character(new Basic(), null, new Orc());
array_push($this->regulations, new MarketRegulation($basicOrc, new ActualLimit(50), TradeType::BUY));
$proForestOrc = new Character(new Pro(), new Forest(), new Orc());
array_push($this->regulations, new MarketRegulation($proForestOrc, new ActualLimit(150), TradeType::BUY));
// more to be set ...
}
public function buyCheck(Character $character, int $quantity): bool
{
$regulation = $this->getRegulation($character, TradeType::BUY);
return $regulation->getLimit()->allow($character, $quantity);
}
public function sellCheck(Character $character, int $quantity): bool
{
$regulation = $this->getRegulation($character, TradeType::SELL);
return $regulation->getLimit()->allow($character, $quantity);
}
public function buy(Character $character, int $quantity)
{
if ($this->buyCheck($character, $quantity)) {
$arrow = $character->getArrow() + $quantity;
$character->setArrow($arrow);
}
}
public function sell(Character $character, int $quantity)
{
if ($this->sellCheck($character, $quantity)) {
$arrow = $character->getArrow();
$character->setArrow($arrow - $quantity);
}
}
public function getRegulation(Character $character, TradeType $tradeType): MarketRegulation
{
$regulation = array_filter($this->regulations, function (MarketRegulation $r) use ($character, $tradeType) {
return $r->getCharacterConfig()->equals($character) && $r->getTradeType() == $tradeType;
});
return array_pop($regulation);
}
}
For a runnable prototype, please visit https://github.com/stackoverflow-helpdesk/free-market. Below is the prototype console output, simulating how the MarketAuthority regulate trades.
Basic Desert Elf [0] can buy up to 150 arrow
Basic Desert Elf [0] buy arrow*150
Basic Desert Elf [150]
Basic Desert Elf [150] only sell up to 50%. i.e.75
Basic Desert Elf [150] sell arrow*80
Basic Desert Elf [150]
Basic Desert Elf [150] cannot sell arrow*80
Orc [0] can buy up to 50 arrow
Orc [0] buy arrow*80
Orc [0]
Orc [0] cannot buy arrow*80

How to create json from array in php like APIs

I have a sample 2018 student pass record of CSV file like this...
1 | Jonathan | 4CS-10 | 1,2,3,4 | Pass with credit
2 | Peter Jo | 4CS-11 | 1,2,3,4 | Pass with credit
and I want to convert it into Json. So firstly I thought I need to change it into php array and then i have to use json_encode function.
I have fetched CSV data into array with fgetcsv().
The final json result I want to achieve is the following:
{"2018" : {
"4CS-10" : {
"name" : "Jonathan",
"destinations": "1,2,3,4"
},
"4CS-11" : {
"name" : "Peter Joe",
"destinations": "1,2,3,4"
}
}
}
To produce that structure you need to iterate your CSV array and transform the rows to associative arrays, then add each row to your output array using the second column as a key.
foreach ($your_array as $row) {
// ↓ use 4CS-10, etc. as keys
$data[$row[2]] = ['name' => $row[1], 'destinations' => $row[3]];
// convert numeric keys ↑ to string keys ↑
}
echo json_encode(['2018' => $data]);
You may have trouble with this if the column you're using as key (values like 4CS-10, etc.) is not unique. If any values there are repeated, you'll only get the last row from the CSV because you'll overwrite the original value of that key. If that's the case, you'll need to change your structure a little if you want to get all the data, and make the value under 4CS-10 an array of objects instead of a single object.
$data[$row[2]][] = ['name' => $row[1], 'destinations' => $row[3]];

PHP - Google Charts

I'm trying to use Google Charts and i'm having some trouble to create array for API.
https://developers.google.com/chart/interactive/docs/gallery/linechart#creating-material-line-charts
My source of data has 3 columns, which is quantity, date and user name.
(you can see in attachment).
How to convert his form of array to use with google charts api (line charts)?
I think i need to convert each different value from column "responsavel" to a new column and set these column the value of column "qt_pedido"
In this example, the final array should be something like this:
Date | user1 | user2
-------------------------------
03/09/2018 | 58 | 19
05/09/2019 | 23 | 5
Result from sql server
it would be easier to use google data table methods to transform the rows to columns,
see following working snippet...
google.charts.load('current', {
packages: ['line']
}).then(function () {
// create data table
var data = google.visualization.arrayToDataTable([
['Date', 'responsavel', 'qt_pedido'],
['20/10/2018', 'user1', 10],
['20/10/2018', 'user2', 20],
['20/10/2018', 'user3', 30],
['20/11/2018', 'user1', 40],
['20/11/2018', 'user2', 50],
['20/11/2018', 'user3', 60],
['20/12/2018', 'user1', 70],
['20/12/2018', 'user2', 80],
['20/12/2018', 'user3', 90],
]);
// create data view
var view = new google.visualization.DataView(data);
// column arrays
var aggColumns = [];
var viewColumns = [0];
// build view & agg columns for each responsibility
data.getDistinctValues(1).forEach(function (responsibility, index) {
viewColumns.push({
calc: function (dt, row) {
if (dt.getValue(row, 1) === responsibility) {
return dt.getValue(row, 2);
}
return null;
},
label: responsibility,
type: 'number'
});
aggColumns.push({
aggregation: google.visualization.data.sum,
column: index + 1,
label: responsibility,
type: 'number'
});
});
// set view columns
view.setColumns(viewColumns);
// sum view by date
var aggData = google.visualization.data.group(
view,
[0],
aggColumns
);
// draw chart
var chart = new google.charts.Line(document.getElementById('chart_div'));
chart.draw(aggData);
});
<script src="https://www.gstatic.com/charts/loader.js"></script>
<div id="chart_div"></div>

Right way of validating xlsx files before database inserting

while playing with PHPExcel I came across some questions how properly handle validation/and inserting values into a database. I do not need any codes, just the general concept how to do it.
Firstly I iterate through first row to check if the columns are matching the given one ( if it fits the schema ).
On the next step, I get the rows and meanwhile its beeing validated row/column wise. If the type doesn't match I will get an error.
While validating the row, I need to get the Worker name and convert it to id get_worker_id().
Question number #1.
Is such solution a good practice? It will produce upto 100 queries. Foreach row - 1.
Question number #2
I also need to validate the rows once again, I would take the worker_id, the F and G column to check if such record isn't present in the database. I would simply introduce a function similar to get_worker_id() but it would return true/false if entry exists.
But again is this the proper way of doing it? By raw calculations my method would produce 100 selects ( get_worker_id ), 100 selects ( validate if exists ), 100 insert ( if all is ok ).
Im not sure if I am doing it properly. Could you hit me up with some advices?
Thanks in forwards.
Model for handling the xlsx file.
class Gratyfikant_model extends CI_Model {
private $_limit = 100;
const columns = array(
'A' => "Z",
'B' => "KS",
'C' => "G",
'D' => "S",
'E' => "Numer",
'F' => "Miesiąc", // required
'G' => "Data wypłaty", // required
'H' => "Pracownik", // required
'I' => "Brutto duże", // required
'J' => "ZUS pracownik", // required
'K' => "ZUS pracodawca", // required
'L' => "Do wypłaty", // required
'M' => "Obciążenie", // required
'N' => "FW");
const validators = array(
'F' => 'date',
'G' => 'date',
'H' => 'string',
'I' => 'float',
'J' => 'float',
'K' => 'float',
'L' => 'float',
'M' => 'float',
);
const validators_errors = array(
'float' => "Wartość nie jest liczbą",
'string' => "Wartość nie jest poprawna",
'date' => "Wartość nie jest datą"
);
protected $_required = array(
'H', 'I', 'J', 'K', 'L', 'M'
);
private $_sheet = array();
private $_sheet_pracownicy = array();
private $_agregacja = array();
protected $_invalid_rows = array();
public function __construct() {
parent::__construct();
}
public function read_data(array $dane) {
if (count($dane) > $this->_limit) {
throw new Exception('Limit wierszy to ' . $this->_limit);
}
$this->_sheet = $dane;
return $this;
}
public function column_validation() {
foreach ($this->_required as $r) {
if (!isset($this->_sheet[1][$r]) || $this->_sheet[1][$r] != self::columns[$r] || !array_key_exists($r, $this->_sheet[1])
) {
throw new Exception('Kolumna - ' . $r . ' - Wartość nagłówka nie pasuje do szablonu, powinno być ' . self::columns[$r]);
}
}
return $this;
}
function validateDate($date) {
$d = DateTime::createFromFormat('Y-m-d', $date);
return $d && $d->format('Y-m-d') === $date;
}
private function row_validation($k, $a, $v, $f) {
switch ($v) {
case "date":
$cellval = $this->validateDate(PHPExcel_Style_NumberFormat::toFormattedString($f, PHPExcel_Style_NumberFormat::FORMAT_DATE_YMD));
break;
case "float":
$cellval = is_float($f);
break;
case "string":
$cellval = is_string($f);
break;
default:
break;
}
if (!$cellval) {
$this->_invalid_rows[$a][$k] = $v;
}
}
public function get_sheet_data() {
$dane = $this->_sheet;
unset($dane[1]); // remove first col
$zus_pracownik = 0;
$zus_pracodawca = 0;
$zus_lacznie = 0;
$do_wyplaty = 0;
$obciazenie = 0;
$brutto = 0;
foreach ($dane as $a => $d) {
foreach (self::validators as $k => $v) {
echo $this->row_validation($k, $a, $v, $d[$k]);
}
if (!is_null($d["H"]) && !empty($d["H"])) {
// $this->_sheet_pracownicy[$d["H"]]["numer"] = PHPExcel_Style_NumberFormat::toFormattedString($d["E"], PHPExcel_Style_NumberFormat::FORMAT_DATE_DDMMYYYY);
$this->_sheet_pracownicy[] = array(
"pracownik" => $d["H"],
"miesiac" => PHPExcel_Style_NumberFormat::toFormattedString($d["F"], PHPExcel_Style_NumberFormat::FORMAT_DATE_YMD),
"data_wyplaty" => PHPExcel_Style_NumberFormat::toFormattedString($d["G"], PHPExcel_Style_NumberFormat::FORMAT_DATE_YMD),
"zus_pracownik" => $d["J"],
"zus_pracodawca" => $d["K"],
"zus_lacznie" => bcadd($d["K"], $d["J"]),
"do_wyplaty" => $d["L"],
"obciazenie" => $d["M"],
"brutto" => $d["I"],
"id_prac" => $this->get_worker_id($d["H"]));
$zus_pracownik = bcadd($zus_pracownik, $d["J"]);
$zus_pracodawca = bcadd($zus_pracodawca, $d["K"]);
$zus_lacznie = bcadd($zus_lacznie, bcadd($d["K"], $d["J"]));
$do_wyplaty = bcadd($do_wyplaty, $d["L"]);
$obciazenie = bcadd($obciazenie, $d["M"]);
$brutto = bcadd($brutto, $d["I"]);
}
}
$this->_agregacja = array(
"zus_pracownik" => $zus_pracownik,
"zus_pracodawca" => $zus_pracodawca,
"zus_lacznie" => $zus_lacznie,
"do_wyplaty" => $do_wyplaty,
"obciazenie" => $obciazenie,
"brutto" => $brutto
);
return $this;
}
public function display_result() {
if (empty($this->_invalid_rows)) {
return array(
"wartosci" => $this->_sheet_pracownicy,
"agregacja" => $this->_agregacja
);
}
}
public function display_errors() {
foreach ($this->_invalid_rows as $k => $a) {
foreach ($a as $key => $value) {
throw new Exception('Pole ' . $key . '' . $k . ' ' . self::validators_errors[$value]);
}
}
return $this;
}
public function get_worker_id($getAd) {
$this->db->select('id_pracownika as id')
->from('pracownicy')
->like('CONCAT( imie, \' \', nazwisko )', $getAd)
->or_like('CONCAT( nazwisko, \' \', imie )', $getAd);
$query = $this->db->get();
$result = $query->result_array();
if (isset($result[0]["id"])) {
return $result[0]["id"];
} else {
throw new Exception('Nie odnaleziono ' . $getAd . ' w bazie danych, proszę dodać pracownika a następnie ponownie wczytać plik');
}
}
}
Display
try {
$data['s'] = $this->gm
->read_data($sheetData)
->column_validation()
->get_sheet_data()
->display_errors()
->display_result();
} catch (Exception $e) {
$data['ex'] = $e->getMessage();
}
XLSX file example
+---+---------------+---+---+------------+---------+--------------+-----------+-------------+---------------+----------------+------------+------------+--------+
| Z | KS | G | S | Numer | Miesiąc | Data wypłaty | Pracownik | Brutto duże | ZUS pracownik | ZUS pracodawca | Do wypłaty | Obciążenie | FW |
+---+---------------+---+---+------------+---------+--------------+-----------+-------------+---------------+----------------+------------+------------+--------+
| | nieprzekazany | G | | 03.08.2017 | sie.17 | 08.09.2017 | Worker1 | 2000 | 274,2 | 392,2 | 1459,48 | 2392,2 | (brak) |
| | nieprzekazany | G | | 03.08.2017 | sie.17 | 08.09.2017 | Worker2 | 1000 | 137,1 | 171,6 | 768,24 | 1171,6 | (brak) |
| | nieprzekazany | G | | 03.08.2017 | sie.17 | 08.09.2017 | Worker3 | 2000 | 274,2 | 392,2 | 1413,88 | 2392,2 | (brak) |
| | nieprzekazany | G | | 03.08.2017 | sie.17 | 08.09.2017 | Worker4 | 2000 | 274,2 | 392,2 | 1418,88 | 2392,2 | (brak) |
+---+---------------+---+---+------------+---------+--------------+-----------+-------------+---------------+----------------+------------+------------+--------+
This really depends on the scale of your application and how frequently this Excel file will be imported. For example, if your application receives little to no traffic then running several queries per line is not the end of the world. If you already have the server and database setup and running then you might as well make use of them. Conversely, if your application is under constant heavy load then trying to minimize the amount of queries you run may be a good idea.
Option 1
If your application is small and/or doesn't get much traffic then don't worry about the ~300 queries you need to make. MySQL is not fragile and if you have indexed your data well your queries will be very fast.
Option 2
Move to querying the data you need first and storing it in memory so you can perform your logic checks in PHP.
This means for Question 1 you should get all of your workers in one query and then build a lookup array in PHP.
Here is a very rough example:
// Get all workers
SELECT worker_name, worker_id FROM workers;
// Build a lookup array from the query results
$worker_array = array(
'Worker1' => 1,
'Worker2' => 2,
...
);
// Then as you loop each row check if the work is in your lookup array
if ( ! isset($worker_array[$excel_row['worker_name']])) {
// do something
}
Likewise for Question 2 you could get your unique data samples in one query (you don't need the entire record, just the unique fields). However, this may present a problem if you have a lot of unique data samples.
Option 3
Create a temporary table in MySQL and import your Excel data without performing any logic checks. Then you can perform your logic checks entirely in SQL.
Here is a very rough example without knowing anything about your data structure:
-- Get all records in the Excel data that match unique data samples
SELECT
*
FROM
temporary_table tt
JOIN
workers w
ON w.worker_name=tt.worker_name
JOIN
data d
ON d.worker_id=w.worker_id
AND d.col_f=tt.col_f
AND d.col_g=tt.col_g
If there are no issues with the data then you can perform an INSERT from your temporary table into your data table. This limits your queries to the initial insert (you can batch this for better performance as well), the data check and the insert from temp to real data.
Recap
It all comes down to your application. If you can get away with doing Option 1 and you've already got it implemented then that's fine for now. You don't need to over optimize things if you don't see this application growing like crazy.
However, if you are worried about scale and growth then I'd personally look at implementing Option 3.
There are multiple concerns here:
Split import into stages.
Validate headers. (Break importing if errors found) 2) Iterate
over each row.
Validate row.
Import if valid.
Log error if any.
Stop processing file if all rows where processed or go back to 2.
As to weather you need some chunking it depend on how much time and memory your script is consuming. If you need it, it's as simple as reading X rows to memory and then processing it. At extreme you can load each record separately.
If you do not need it just load it all to array.
chunking - consuming up to X rows in a single iteration, then clearing memory then consuming next chunk...
100 queries just doesn't sound right to me for a single php instance.
https://secure.phabricator.com/book/phabcontrib/article/n_plus_one/
Or just search for n plus 1 query problem

db result to php array

I am trying to use morris area chart dynamicly
from my sql query I get the following result
period | item | amount
------ | ---- | ------
20170801 | iphone | 327
20170801 | ipad | 278
20170801 | ipod | 125
20170802 | iphone | 528
20170802 | ipad | 325
20170802 | ipod | 250
etc
now I need to store this in an array like this
data: [{
period: '20170801',
iphone: 327,
ipad: 278,
ipod: 125
}, {
period: '20170802',
iphone: 528,
ipad: 325,
itouch: 250
}, ...]
I am using a while loop through the table for each date it needs to create a new object. For a date in an object it needs to add an element to that object.
I have absolutely no clue how to achieve this
<?php
class test {
private $period = '';
private $item = '';
private $amount = '';
public function __construct($period, $item, $amount) {
$this->period = $period;
$this->item = $item;
$this->amount = $amount;
}
}
$obj = [
new test('20170801', 'iphone', '327'),
new test('20180801', 'iphone2', '328')
];
var_dump($obj);
But most usefull use ORM and after you get data - serialize it to string via php function "json_encode"
http://php.net/manual/en/function.json-encode.php
http://php.net/manual/en/function.json-decode.php
Ok since you have no idea how to start I coded a little script to get you started:
$db = new mysqli('localhost', 'root', '', 'db');
$result = $db->query('SELECT * FROM table');
$data = $result->fetch_all(MYSQLI_ASSOC);
$json = json_encode($data, JSON_PRETTY_PRINT);
print_r($json);
You have to change the database server data and you table name in SQL query, other then that it should just output the whole table data in JSON format. If you want to join all data from teh same date you have to add soem more logic, but that might cause duplicate properties.

Categories