I know this is more of PHP problem because of it's loose Typing of arrays but I see this problem all over the place in a project I took over and not sure of the best way to refactor it. Suppose you have two sets of data, both multi dimensional arrays, $results_by_entity and $target_limits and we want to check what the target is foreach result_by_entity so we can set some state
foreach ($results_by_entity AS $result_by_entity) {
foreach ($target_limits AS $target_limit) {
if ($target_limit['activity_id'] == $result_by_entity['activity_id']) {
$result_by_entity->target = $target_limit->quantity;
$result_by_entity->progress = $target_limit->score;
}
}
}
There are a couple of main problems here
1-The data is really strongly tied together, so it is really hard to refactor $results_by_entity into it's own class and $target_limits into it's own class
2-The time taken to process this grows exponentially as the data size grows
I read the Refactoring book by Martin Fowler and it was really helpful but this style of problem doesn't really show up I think mostly because his examples are in JAVA which is strongly typed. The class is super run on so really hard to debug and extend but all the data is so tied together primarily because of these types of loops so not to sure how to solve. Any recommendations would be really appreciated
What you want is to index your data pre-emptively if possible:
$results_index = array();
foreach ($results_by_entity AS $result_by_entity) {
//Index this value
$results_index[$result_by_entity['activity_id']] = $result_by_entity; //Add a & in front if it's a scalar value, but it looks like it's an object in your example
}
foreach ($target_limits AS $target_limit) {
//Find the corresponding activity id in results
if (isset($results_index[$target_limit['activity_id']])) {
$result_by_entity = $results_index[$target_limit['activity_id']];
$result_by_entity->target = $target_limit->quantity;
$result_by_entity->progress = $target_limit->score;
}
}
Related
I am trying to get all stores on a Magento shop. By all stores, I mean all stores from all websites. I wrote this code and it works, but I'm a little concerned about the complexity of the nested foreach loop. Please take a look at it and advise me if you think I can do something different.
public function getAllStoresCustom(){
$all_stores = array();
foreach (Mage::app()->getWebsites() as $website) {
foreach ($website->getGroups() as $group) {
$all_stores [] = $group->getStores();
}
}
return $all_stores;
}
I've only found these functions in Magento, so I think I had to use those and this seemed the only combination that worked.
Thanks a lot
Try this:
$allStores = Mage::getModel('core/store')->getCollection();
Then loop through $allStores when needed
foreach ($allStores as $store) {
//do something with $store
}
Notice: You will get a store with id 0. That is the admin store view. If you want all stores without the admin store view use this:
$allStores = Mage::getModel('core/store')->getCollection()->setWithoutDefaultFilter()
foreach() is an incredibly efficient PHP function so you can bet that it is not going to be your slowdown. If you are looking to optimize something then look into the code for the getStores() and getGroups() functions because those are being called within the iterations whereas getWebsites() gets called only once.
If you want more guidance then please feel free to update your question with the contents of those functions.
You may also want to try https://codereview.stackexchange.com/ for more experienced opinions especially since you don't have any specific programming issue/error =)
If anyone has an idea, I couldn't think of a better way to phase the question.
I'll try to not make this to complicated an explination.
I'm writing a "quotes" class that is the main class. This class has "overall" functions that preform calculations based on "items" stored in its array. Suffice it to say, the end-developer will call it as $q = new apiQuote/quote().
Then, before it's of any use, the first item must be added and it's properties set so it can do it's "own" calculations. Something like $q->createItem(). Once the item is created with this call, an "item" of the class "item" is added to an array in "quotes" named "items".
The currently editable item, $q->item is always the last one added to the array via the createItem method. Which looks like:
public function createNewItem() {
$this->items[] = new item();
$this->item = $this->items[count($this->items)-1];
}
I added setItem method, whereby the parameter would be an integer representing item index in the array and would set $q->item to the item index chosen. This works, but still seems "not as productive" as I'd like".
What I'm curious about, is if anyone has any suggestions on a better way to go about this. I tried looking for a "cards/deck" php example, but all I could find was array shuffles, which is kinda useless here. I know how to do such associations in .NET and thought this would be just as easy, but I don't have the same property abilities in PHP that I have in a .NET language, thus negating what I'm used to in created this kind of "class/subclass[items]" type structure.
Really I would just like to know if anyone has done anything similar and if I'm doing things to the "best of ability" or if there might be a better way, via PHP, to allow an "end-developer" to call on one main class and create a list "items" based on a subclass that can later be used for methods of the main class?
I really hope this sums it all up well and I havn't gone outside the guidelines of "question asking" here, but I can't think of a better place, other than maybe Code Review to pose such a question and get great developer feed back. If y'all feel I need move it to Code Review, let me know. My main reason for choosing this over CR is this site tends to get faster responses.
Perhaps a view of what I have and what I "might" like to see:
Way it works now
$q = new apiQuote\quote(TRUE);
$q->createNewItem();
$q->item->totalHeight = 100;
$q->item->totalWidth = 250;
...
$q->createNewItem();
$q->item->totalHeight = 300;
$q->item->set_coverage('25%');
...
$q->setItem(1);
$q->item->totalHeight = 250;
...
$q->getTotalsCurv(); // an array to create a graph curve of totals from each item
What I "think" I might like:
$q = new apiQuote\quote(TRUE);
$q->items[] = new apiQuote\item();
$q->items[0]->totalHeight = 100;
$q->items[0]->totalWidth = 250;
...
$q->items[] = new apiQuote\item();
$q->items[1]->totalHeight = 300;
$q->items[1]->set_coverage('25%');
...
$q->items[0]->totalHeight = 250;
...
$q->getTotalsCurv();
However, something like the second idea mean leaving the "items" array public, which could lead to a vast amount of "other" problems as I'm trying to set this whole thing up to be "near dummy proof". Thus the usage of protected variables with specific get/set methods as if they where C# properties.
I think your problem is how to identify an "item" outside of the quote instance. And by using its array index you feel you are going to run into the problems. And you will, when you will try to delete an item. It would successfully invalidate any index already known/stored outside. The simplest patch to it is to give every item a unique ID and store them in the map instead of storing it as a vector.
Also, in your solution item by itself cannot provide you with any helpful information how to access this item in your collection of items (a quote).
public function createNewItem() {
static $counter;
$id = $counter++;
return $this->item = $this->items[$id] = new item($id);
}
public function editItem($id) {
return $this->item = $this->items[$id];
}
public function removeItem($id) {
$this->item = null;
unset($this->item[$id]);
}
Alternatively I recommend you not to reinvent the wheel and take a look here:
http://php.net/manual/en/spl.datastructures.php
and here in specific
http://www.php.net/manual/en/class.splobjectstorage.php
Anything that implements Iterator interface can be iterated with foreach
http://www.php.net/manual/en/class.iterator.php
Ah well you could return the new item from the createItem function.
An equivalent of your .NET example would be to simply return the new item
public function createNewItem() {
$this->items[] = new item();
$this->item = $this->items[count($this->items)-1];
return $this->items[count($this->items)-1];
}
then you could do
$item = $q->createNewItem();
$item->totalHeight = 100;
...
And as you are already adding the new item to the array from within createNewItem so no need for something like $q->add($item);
And to get at any other item you could do a
function getItem($index){
return $this->items[count($this->items)-1];
}
$otheritem = $q->getItem(3);
$otheritem->totalHeight = 100;
I have a Propel 1.6 generated class Group that has Inits related to it, and Inits have Resps related to them. Pretty straightforward.
I don't understand the difference between these two pieces of Propel code. Here in the first one, I re-create the $notDeleted criteria on every loop. This code does what I want -- it gets all the Resps into the $data array.
foreach ($group->getInits() as $init) {
$notDeleted = RespQuery::create()->filterByIsDeleted(false);
foreach ($init->getResps($notDeleted) as $resp) {
$data[] = $resp;
}
}
Here in the second code, I had the $notDeleted criteria pulled out of the loop, for (what I thought were) obvious efficiency reasons. This code does not work the way I want -- it only gets the Resps from one of the Inits.
$notDeleted = RespQuery::create()->filterByIsDeleted(false);
foreach ($group->getInits() as $init) {
foreach ($init->getResps($notDeleted) as $resp) {
$data[] = $resp;
}
}
I thought it must be something to do with how the getResps() method caches the results, but that's not how the docs or the code reads in that method. The docs and the code say that if the criteria passed in to getResps() is not null, it will always get the results from the database. Maybe some other Propel cache?
(First off, I'm guessing you meant to use $init versus $initiative in your loops. That or there's some other code we're not seeing here.)
Here's my guess: In your second example you pull out the $notDeleted Criteria object, but each time through the inner foreach the call to getResps($notDeleted) is going to make Propel do a filterByInit() on the Criteria instance with the current Init instance. This will add a new WHERE condition to the SQL, but obviously a Resp can only have one Init.Id value, hence the lone result.
I don't think there is a good reason to pull that out though, under the covers Propel is just creating a new Criteria object, cloning the one you pass in - thus no real memory saved.
I was previously mostly scripting in PHP and now considering getting "more serious" about it :)
I am working on a hiking website, and I needed to put some values into an object that I then try to pass back to the calling code.
I tried doing this:
$trailhead = new Object ();
But the system sort of barfed at me.
Then I didn't declare the object at all, and started using it like this:
$trailhead->trailhead_name = $row['trailhead_name'];
$trailhead->park_id = $row['park_id'];
That seemed to work reasonably ok. But there are at least 3 problems with this:
Since that code gets executed when getting things from the database, what do I do in case there is more than one row?
When I passed the $trailhead back to the calling code, the variables were empty
I actually am maybe better off making a real object for Trailhead like this:
class Trailhead
{
private $trailhead_name;
private $park_id;
private $trailhead_id;
public function __construct()
{
$this->trailhead_name = NULL;
$this->park_id = NULL;
$this->trailhead_id = NULL;
}
}
What do people generally do in these situations and where am I going wrong in my approach? I know its more than one place :)
$trailheads[] = $trailhead;
I'd do a print_r() of $trailhead to check that it's what you expect it to be. The default object type in PHP is going to be stdClass.
Yes, that's going to be better, as it'll allow your Trailhead objects to have functions. The way you're currently doing it is basically taking advantage of none of PHP's object functionality - it's essentially an array with slightly different syntax.
I think you should get in "contact" with some of the basics first. Objects in PHP have sort of a history. They are a relative to the array and there are two sorts of objects:
data-objects
class objects
data objects are called stdClass and that's actually what you were initially looking for:
$trailhead = new Object();
in PHP is written as:
$trailhead = new stdClass;
(with or without brackets at the end, both works)
You then can dynamically add members to it, like you did in the second part without declaring the variable (that works, too in PHP):
$trailhead->trailhead_name = $row['trailhead_name'];
$trailhead->park_id = $row['park_id'];
If you want to more quickly turn $row into an object, just cast it:
$trailhead = (object) $row;
That works the other way, too:
$array = (array) $trailhead;
As you can see, those basic data based objects in PHP do not hide their relationship to the array data type. In fact you can even iterator over them:
foreach($trailhead as $key=>$value) {
echo $key, ' is ', $value, "<br>\n";
}
You can find lots of information about this in the PHP manual, it's a bit spread around, but worth to know the little basics before repeating a lot of names only to pass along the data that belongs together.
Next to these more or less stupid data objects, you can code complete classes that - next to what every stdClass can do - can have code, visibility, inheritance and all the things you can build nice stuff from - but this can be pretty complex as the topic is larger.
So it always depends on what you need. However, both type of objects are "real objects" and both should work.
class myClass {
function importArray(array $array) {
foreach($array as $key=>$value) {
if(!is_numeric($key)) $this->$key=$value;
}
}
function listMembers() {
foreach($this as $key=>$value) {
echo $key, ' is ', $value, "<br>\n";
}
}
}
$trailhead = new myClass();
$trailhead->importArray($row);
echo $trailhead->park_id;
Keep in mind that instead of creating a set of objects that merely does the same in each object (store data), you should just take one flexible class that is handling the job flexible (e.g. stdClass) because that will keep your code more clean.
Instead of coding a getter/setter orgy you can then spend the time thinking about how you can make the database layer more modular etc. .
Just pass back an array:
$trailhead = array(
'trailhead_name' => $row['trailhead_name'],
'park_id' => $row['park_id'],
'trailhead_id' => $row['trailhead_id'],
)
Then either access it like:
$trailhead['park_id'];
or use the nifty list() to read it into variables:
list($trailhead_name, $park_id, $trailhead_id) = $trailhead;
I realize the knee-jerk response to this question is that "you dont.", but hear me out.
Basically I am running on an active-record system on a SQL, and in order to prevent duplicate objects for the same database row I keep an 'array' in the factory with each currently loaded object (using an autoincrement 'id' as the key).
The problem is that when I try to process 90,000+ rows through this system on the odd occasion, PHP hits memory issues. This would very easily be solved by running a garbage collect every few hundred rows, but unfortunately since the factory stores a copy of each object - PHP's garbage collection won't free any of these nodes.
The only solution I can think of, is to check if the reference count of the objects stored in the factory is equal to one (i.e. nothing is referencing that class), and if so free them. This would solve my issue, however PHP doesn't have a reference count method? (besides debug_zval_dump, but thats barely usable).
Sean's debug_zval_dump function looks like it will do the job of telling you the refcount, but really, the refcount doesn't help you in the long run.
You should consider using a bounded array to act as a cache; something like this:
<?php
class object_cache {
var $objs = array();
var $max_objs = 1024; // adjust to fit your use case
function add($obj) {
$key = $obj->getKey();
// remove it from its old position
unset($this->objs[$key]);
// If the cache is full, retire the eldest from the front
if (count($this->objs) > $this->max_objs) {
$dead = array_shift($this->objs);
// commit any pending changes to db/disk
$dead->flushToStorage();
}
// (re-)add this item to the end
$this->objs[$key] = $obj;
}
function get($key) {
if (isset($this->objs[$key])) {
$obj = $this->objs[$key];
// promote to most-recently-used
unset($this->objs[$key]);
$this->objs[$key] = $obj;
return $obj;
}
// Not cached; go and get it
$obj = $this->loadFromStorage($key);
if ($obj) {
$this->objs[$key] = $obj;
}
return $obj;
}
}
Here, getKey() returns some unique id for the object that you want to store.
This relies on the fact that PHP remembers the order of insertion into its hash tables; each time you add a new element, it is logically appended to the array.
The get() function makes sure that the objects you access are kept at the end of the array, so the front of the array is going to be least recently used element, and this is the one that we want to dispose of when we decide that space is low; array_shift() does this for us.
This approach is also known as a most-recently-used, or MRU cache, because it caches the most recently used items. The idea is that you are more likely to access the items that you have accessed most recently, so you keep them around.
What you get here is the ability to control the maximum number of objects that you keep around, and you don't have to poke around at the php implementation details that are deliberately difficult to access.
It seems like the best answer was still getting the reference count, although debug_zval_dump and ob_start was too ugly a hack to include in my application.
Instead I coded up a simple PHP module with a refcount() function, available at: http://github.com/qix/php_refcount
Yes, you can definitely get the refcount from PHP. Unfortunately, the refcount isn't easily gotten for it doesn't have an accessor built into PHP. That's ok, because we have PREG!
<?php
function refcount($var)
{
ob_start();
debug_zval_dump($var);
$dump = ob_get_clean();
$matches = array();
preg_match('/refcount\(([0-9]+)/', $dump, $matches);
$count = $matches[1];
//3 references are added, including when calling debug_zval_dump()
return $count - 3;
}
?>
Source: PHP.net
I know this is a very old issue, but it still came up as a top result in a search so I thought I'd give you the "correct" answer to your problem.
Unfortunately getting the reference count as you've found is a minefield, but in reality you don't need it for 99% of problems that might want it.
What you really want to use is the WeakRef class, quite simply it holds a weak reference to an object, which will expire if there are no other references to the object, allowing it to be cleaned up by the garbage collector. It needs to be installed via PECL, but it really is something you want in every PHP installation.
You would use it like so (please forgive any typos):
class Cache {
private $max_size;
private $cache = [];
private $expired = 0;
public function __construct(int $max_size = 1024) { $this->max_size = $max_size; }
public function add(int $id, object $value) {
unset($this->cache[$id]);
$this->cache[$id] = new WeakRef($value);
if ($this->max_size > 0) && ((count($this->cache) > $this->max_size)) {
$this->prune();
if (count($this->cache) > $this->max_size) {
array_shift($this->cache);
}
}
}
public function get(int $id) { // ?object
if (isset($this->cache[$id])) {
$result = $this->cache[$id]->get();
if ($result === null) {
// Prune if the cache gets too empty
if (++$this->expired > count($this->cache) / 4) {
$this->prune();
}
} else {
// Move to the end so it is culled last if non-empty
unset($this->cache[$id]);
$this->cache[$id] = $result;
}
return $result;
}
return null;
}
protected function prune() {
$this->cache = array_filter($this->cache, function($value) {
return $value->valid();
});
}
}
This is the overkill version that uses both weak references and a max size (set it to -1 to disable that). Basically if it gets too full or too many results were expired, then it will prune the cache of any empty references to make space, and only drop non-empty references if it has to for sanity.
PHP 7.4 now has WeakReference
To know if $obj is referenced by something else or not, you could use:
// 1: create a weak reference to the object
$wr = WeakReference::create($obj);
// 2: unset our reference
unset($obj);
// 3: test if the weak reference is still valid
$res = $wr->get();
if (!is_null($res)) {
// a handle to the object is still held somewhere else in addition to $obj
$obj = $res;
unset($res);
}
I had a similar problem with the Incredibly Flexible Data Storage (IFDS) file format with trying to keep track of references to objects in an in-memory data cache. How I solved it was to create a ref-counting class that wrapped a reference to the underlying array. I generally prefer arrays over objects as PHP has traditionally tended to handle arrays better than objects with regards to unfortunate things like memory leaks.
class IFDS_RefCountObj
{
public $data;
public function __construct(&$data)
{
$this->data = &$data;
$this->data["refs"]++;
}
public function __destruct()
{
$this->data["refs"]--;
}
}
Since 'refs' is tracked as a regular value in the data, it is possible to know when the last reference to the data has gone away. Regardless of whether multiple variables reference the refcounting object or it is cloned, the refcount will always be non-zero until all references are gone. I don't need to care how many actual references there are internally in PHP as long as the value is correctly zero vs. non-zero. The IFDS implementation also tracks an estimated amount of RAM being used by each object (again, being exact isn't super important as long as it is in the ballpark), allowing it to prioritize writing and releasing unused objects that are occupying system resources first and then writing and releasing portions of still-referenced objects that are caching large quantities of DATA chunk information.
To get back to the topic/question, with this ref-counting class-based approach, it is, for example, mostly straightforward to prune to ~5,000 records in a cache upon hitting 10,000 records in the cache. General strategy is to not get rid of records still being referenced plus keep the most recently requested/used records that aren't being referenced because they are likely to be referenced again. Upon every new reference, unset() and then setting the item again will move the item to the end of the array so that the oldest probably unreferenced items appear first and the newest probably still referenced items appear last.
Weak references, as several people have suggested, won't solve every caching issue. They don't work in caching scenarios where you don't want to remove an item from the cache until the application is done working with it (i.e. deleting an item that the application later attempts to use) but also want to keep it around as long as RAM overhead permits even if the application stops referencing it temporarily but might need it again in a moment. Weak references are also incapable of working in scenarios where the item in the cache is holding onto unwritten data that may or may not be fine with staying unwritten even if there are no references to it in the application. In short, when there is a balancing act to maintain, weak references cannot be used.