i am iterating an array with more than 3000 items in it
array looks something like that:
[
[
'id' => 1,
'type' => 'income'
'amount' => 10
],
[
'id' => 2,
'type' => 'expense',
'amount' => 20
],
.......
]
while iterating i call functions that manipulate another array in the same class
something like that:
$this->data->revenue->each(function($row) use($user)
{
if ($row->isIncome())
{
$this->addRevenueRowToColumn($row, 'income');
$this->addRevenueRowToColumn($row, 'total');
}
if ($row->isExpense())
{
$this->addRevenueRowToColumn($row, 'expense');
$this->subtractRevenueRowToColumn($row, 'total');
}
}
this is what the functions do:
protected function addRevenueRowToColumn(&$row, $columnName)
{
$this->report['byMonth'][$row->getMonthTableKey()]['byDepartment'][$row->department_id][$columnName] += $row->amount;
$this->report['byMonth'][$row->getMonthTableKey()]['total'][$columnName] += $row->amount;
$this->report['totals']['byDepartment'][$row->department_id][$columnName] += $row->amount;
$this->report['totals']['total'][$columnName] += $row->amount;
}
protected function subtractRevenueRowToColumn(&$row, $columnName)
{
$this->report['byMonth'][$row->getMonthTableKey()]['byDepartment'][$row->department_id][$columnName] -= $row->amount;
$this->report['byMonth'][$row->getMonthTableKey()]['total'][$columnName] -= $row->amount;
$this->report['totals']['byDepartment'][$row->department_id][$columnName] -= $row->amount;
$this->report['totals']['total'][$columnName] -= $row->amount;
}
it takes about 11 seconds to process the data and display it
what should i do?
thanks in advance!
speed up php only solution
Apart from the suggested use of a database to organize your stuff, if you still want it the hard way ;) you can avoid iterating (let php do it internaly) over the whole array by using one of the php functions below:
array_map
— Applies the callback to the elements of the given arrays
and
array_walk
— Apply a user function to every member of an array
I see in your code, you have the 'use' clause, so I presume you are PHP > 5.3. In that case, you can do the following:
$yourdata = array_map(
function($row) use ($user)
{
/*$user->doStuff();*/
return $row;
}, $yourdata
);
Furthermore, a lot of overhead is the display rendering part. If you have a lot of things to display, for example using a simple echo, it's faster to do:
$result = "";
$result .= $something;
$result .= $somethingelse;
echo $result;
than
echo $something;
echo $somethingelse;
enhance solution to use database
But besides that, it will surely be beneficial if you use the database. The most obvious thing is to store your data in some db tables and use some sql-ish solution to query it. It will speed your script up for sure.
speed up the db+php solution
Next, you can get a major performance boost by doing most of your calculations (business logic) inside the database engine in form of stored procedures.
For example if you'd go with mysql, you could create a cursor and iterate over your table rows in a loop. On every row you do some stuff, while ofcourse having direct access to all the tables/columns of your scheme (and possibly other stored procedures/functions). If you are doing a lot of math-ish calculations it's a great solution, but ofcourse IMHO it's usually less convenient (more sophisticated) to write your stuff in SQL rather than PHP ;)
Related
Good day, basically I want to insert some related data all at once using eloquent. My current code is:
$allStudies = Study::chunk(50, function ($studies) use ($request, $questionData, $answerData) {
foreach ($studies as $study) {
$evaluationInsert = Evaluation::create([
'study_id' => $study->id,
'questionnaire' => $request->questionnaire,
'description' => $request->description
]);
$evaluationQuestions = $evaluationInsert
->question()
->createMany($questionData);
foreach ($evaluationQuestions as $question) {
$question->answer()->createMany($answerData);
}
}
});
The result of $allStudies is a collection of Study model that currently have around 150-ish data. $questionData is just a static array of arrays that consist of 38 elements and $answerData is an array of arrays that have 4 elements which consist the answer options of each questions. However, the code does work but it takes too long time to execute because of big loops and increasing the timeout in php seems not an ideal way to solve this. What is the elegant way to solve this kind of case?
Laravel 5.5 + Redis.
Got the following code in controller:
$products = Cache::remember('category_'.$category->alias.'_page_'.$page, 1440, function() use ($childrenCategoriesIndexes){
return Product::whereIn('category_id', $childrenCategoriesIndexes)
->userFilter()
->paginate(15);
});
It caches each page. But what if there are too many custom filters? This is scopeUserFilter() from Product model:
public function scopeUserFilter($query) {
if (request('price_from')) {
$query->where('price', '>', request('price_from'));
}
if (request('price_to')) {
$query->where('price', '<', request('price_to'));
}
return $query;
}
And there're only 2 variables. But what if there will be 10 and more variables, how to cache this data? I think keys like this are not good:
'category_'.$category->alias.'_page_'.$page.'_'.request('price_from').'_'.request('price_to')
Hashed the params, then you can include as many as you can:
$params = [
'page' => 1,
'price_from' => '',
'price_to' => '',
'param0' => '',
...
];
foreach (array_keys($params) as $param) {
if (request()->has($param))
$params[$param] = request()->input($param);
}
$prefix = 'category_';
$hashed = md5(json_encode($params));
$cache_key = $prefix . $hashed;
Instead of:
defining an array of all possible parameters
looping through that array to check whether the request contains any of those parameters and if so, assigning that value to the relevant key in the array
creating a hash of the array to use as the key for your cache
...an alternative, and possibly more flexible, approach (which I picked up from this article) would be to:
create an array of all request parameters and sort them alphabetically
rebuild the url using this sorted array
create a hash of this url to use as the key for your cache
This means your code would look something like this:
$url = request()->url();
$queryParams = request()->query();
ksort($queryParams);
$queryString = http_build_query($queryParams);
$fullUrl = "{$url}?{$queryString}";
$rememberKey = sha1($fullUrl);
return Cache::remember($rememberKey, $minutes, function () use ($data) {
return $data;
});
While #Ben's answer does address how we can cache multiple params, it's not really a good practice to cache all requests.
Caching is typically used for the most popular requests (highly frequent reads, infrequent writes). For example, caching the top 10 param combinations. If you start caching the long tail, you're defeating the purpose of caching as you drift towards more frequent writes and less frequent reads. Eventually you'll run out of memory if you're using in-memory caching engines
I would suggest rethink your caching strategy
Please have a look to below code
function GetAreaName($AreaCode)
{
switch ($AreaCode)
{
case 201: return 'New Jersey';
case 202: return 'Washington';
// this goes on till
case 999: return '';
}
}
Let's say if the AreaCode is 998 then it would have to go through so many cases!
How could we optimize this function? (No using databases.)
I'm thinking to build an array and do a binary search over it? But this means every time the function is called the array will be rebuild? How do we build the array once, cache it and re-use every time this function is called?
Why not just use a hash table?
class Area {
private $areaCodes = array(
201 => 'New Jersey',
202 => 'Washington',
// this goes on till
999 => '';
);
function getStateByAreaCode ($areaCode) {
if (array_key_exists($areaCode, $this->areaCodes)) {
return $this->areaCodes[$areaCode];
} else {
return false;
}
}
}
Call it like this:
$area = new Area();
$city = $area->getStateByAreaCode(303);
Just save your class in a file and include it when you need it.
You Asked How to Prevent the Array From Being Created Every Request:
By putting this in a class you at least keep it clean. It technically still gets created each request, but unless your array is enormous (WAY bigger than the area codes in the U.S.) it shouldn't pose a performance issue. If you are worried about building the array every time you have a request, then take a look at a code optimizer like APC or the Zend Optimizer. This essentially takes the byte code that PHP generates at run time and caches it.
Sounds like you should just store it in your database.
But if you can't do that, either abstract it into a config file of some kind and store it in some kind of persisted object, or just use a static variable:
function foo($key) {
static $cache = array(1 => 'abc', 2 => 'def', 3 => 'ghi');
if (array_key_exists($key, $cache)) {
return $cache[$key];
} else {
//Somehow signal an error (throw an exception, return boolean false, or something)
}
}
In the above, $cache would only exist once. (If you knew that the values would never be null, you could use isset instead of array_key_exists.)
This isn't very flexible though since changing the data requires you to edit your code. You typically want your data and your code to be decoupled.
That could mean storing it in some kind of file (json, xml, php, whatever), and loading it into some kind of structure that you only create once. You would then pass that object or array around wherever it was needed. (Or, if you wanted to be hacky, you could use a static class. I suggest against this though.)
Switch condition is evaluated once only:
In a switch statement, the condition is evaluated only once and the result is compared to each case statement. In an elseif statement, the condition is evaluated again. If your condition is more complicated than a simple compare and/or is in a tight loop, a switch may be faster. ➫➫➫
There is no optimization required. However, read:
In PHP what's faster, big Switch statement, or Array key lookup
If you want to build a config file, you can consider something like:
$areas = array
(
1 => 'abc',
2 => 'def',
..
);
Then simply compare:
if (!isset($areas[$some_code]))
{
// do something
}
else
{
// ok
}
Try below pseudo code
$areas = array('201' => 'New Jersey',
'202' => 'Washington',
......
........
'999' => '');
function GetAreaName($AreaCode)
{
if(isset($areas[$AreaCode])) {
return $areas[$AreaCode];
} else {
// do something
}
}
I am seeking for someone's knowledge out here.
Right now, I would need to merge several arrays into a bigger one, but all of those arrays depend on a function.
This function returns a numeric array containing different quantities of numbers between 1 and 7 :
function Possible($i, $j, $grid)
$possible = Possible($i, $j, $grid)
I'm working with a grid, and the function returns a different array for every case of the grid. What I would like to do is to merge those 7 arrays into another one. Some numbers may be present more than once in this big array, but I want it this way.
I tried using for loops, while loops and some other techniques, but nothing worked. In this case, it is not possible to manually define every array, since they change depending of what is contained in the grid and there are too many. It has to be done automatically, and this is where I get stuck.
for ($jj=0; $j<7; $j++){
$possRow = array_merge( ###what do I add here or what do I change in this code to make everything work###
Thank you if someone can help me out!
Etpi
hope this help:
$biggerOneArray = array();
for($k=0;$k<7;$k++) {
$biggerOneArray[] = Possible($i,$j,$grid);
}
Then you can check your bigger array, may contains all arrays of the iterations of the loop (7 arrays merged).
var_dump($biggerOneArray);
The output should be this:
array(
(int) 0 => array(
'key' => 'value',
'key2' => 'value2'
),
(int) 1 => array(
'key3' => 'value3',
'key4' => 'value4'
)
)
etc...
I'm sorry but your description isn't very clear. But just to get you started you might look at this solution.
function Possible($i, $j, $grid) {
// some code ... e.g. $array[] = "some data";
return $array;
}
By creating a small array for each grid and returning it using return $array you get a few little arrays which you can inturn place into a for loop to merge it into one larger array. However i believe the var $jj must have some meaning in the function it self as well.
for($jj=0;$jj<7;$jj++) {
$merged_array[$jj] = Possible($i,$j,$grid);
}
Maybe if you descripe your problem a little more and post an exmple of the array's your working with i can give you a better answer.
I have an array of arrays, each array containing details of a scan by a medical device. I'm getting this data from text logs that are dumped nightly. The format of which is this:
$this->scans = array(
array(
'patientid' => (int),
'patientname' => 'John Skeet',
'reviewed' => 0 or 1
//plus more irrelevant
),
array(
//same as above
), //etc
)
The important array key here is reviewed, as each scan may be reviewed if it is of high enough quality. However, the text logs dump out EVERY scan that is acquired, then goes back through and re-lists the ones that are reviewed.
Now in order to prevent duplicates , I figured I could just use an array_filter to filter out scans that have been both acquired and reviewed (keeping the reviewed version). However, the filter function is filtering out the entire array (except in some rare cases). If someone could take a look and let me know why they think it's happening that would be much appreciated.
$this->scans = array_filter($this->scans, array($this, "scan_cleanup"));
.
private function scan_cleanup($scan) {
//only if the scan was not reviewed
if ($scan['reviewed'] == 0) {
//change reviewed status to see if there is a duplicate
$scan['reviewed'] == 1;
//return false to remove this copy (and keep reviewed)
if (in_array($scan, $this->scans)) {
return false;
}
}
return true;
}
$scan['reviewed'] == 1;
vs
$scan['reviewed'] = 1;
One is a conditional, that does nothing in this context, the other is not there.
You are also not running the return false very often. I'd change the logic a little to make it a little clearer, and simpler by a little refactoring (pulling out a condition-check).
if ($scan['reviewed'] and hasDupe($scan)) {
return false; // filter out
}
return true; // it is passed back, and is output
hasDupe() does the best checks you know for a duplicate record and returns true/false.
Simple case of "==" vs. "=" as far as I can see.
$scan['reviewed'] = 1;
That oughta do the trick. Sometimes the simplest problems are the hardest to spot ;-)