PHP Garbage Collection for Singleton Properties - php

(Specifically, I'm working inside Magento, but this is a general PHP question.)
I have a singleton class with an array which stores large objects. Please consider the following:
protected $_categoryObjectCache = array();
/**
* #param $key
* #param $value
*/
public function setCategoryObjectCache($key, $value) {
$this->_categoryObjectCache[$key] = $value;
if ($this->nearingMemoryLimit()) {
unset($this->_categoryObjectCache);
if (function_exists('gc_collect_cycles')) {
gc_collect_cycles();
}
$this->_categoryObjectCache = array();
}
}
/**
* #return bool
*/
protected function nearingMemoryLimit() {
$memory_limit = ini_get('memory_limit');
if (preg_match('/^(\d+)(.)$/', $memory_limit, $matches)) {
if ($matches[2] == 'M') {
$memory_limit = $matches[1] * 1024 * 1024; // nnnM -> nnn MB
} else if ($matches[2] == 'K') {
$memory_limit = $matches[1] * 1024; // nnnK -> nnn KB
}
}
return ((memory_get_usage() / $memory_limit) > 0.75);
}
(The method setCategoryObjectCache() doesn't get called very often. Usually I'm just reading from that array with a method called getCategoryObjectCache())
I built this singleton cache as part of an importer which goes through thousands of products and assigns them to various various categories. However, this requires loading each category model and using it's built in methods to look up parent/child relationships, status, etc. Looking up each category that a product wants to relate to, over and over for every product incurred thousands of duplicate model instantiations for the same category. This cache solves that problem.
The issue though, is that I'm worried about the cache getting too big. Magento is notorious for having performance issues because of these huge model objects. So I wrote the above code to look at the current scripts memory usage and clear the cache if the script gets too close to maxing out memory.
What I don't know, is whether this is really going to work. PHP garbage collection is a big ball of mystery to me, and I'm not sure if I could just call $this->_categoryObjectCache = array(); and expect PHP to clean up, or if I really needed to call unset($this->_categoryObjectCache); and gc_collect_cycles();
How can I make sure to clear out the RAM when my cache starts getting too big?

Related

PHP Multi Threading - Synchronizing a cache file between threads

I created a script, that, for a game situation tries to find the best possible solution. It does this, by simulating each and every possible move, and quantifying them, thus deciding which is the best move to take (which will result in the fastest victory). To make it faster, I've implemented PHP's pthread, in the following way: each time the main thread needs to find a possible move (let's call this JOB), it calculates all the possible moves in the current depth, then starts a Pool, and adds to it, each possible move (let's call this TASK), so the threads develop the game tree for each move separately, for all the additional depths.
This would look something like this:
(1) Got a new job with 10 possible moves
(1) Created a new pool
(1) Added all jobs as tasks to the pool
(1) The tasks work concurently, and return an integer as a result, stored in a Volatile object
(1) The main thread selects a single move, and performs it
.... the same gets repeated at (1) until the fight is complete
Right now, the TASKS use their own caches, meaning while they work, they save caches and reuse them, but they do not share caches between themselves, and they do not take caches over from a JOB to another JOB. I tried to resolve this, and in a way managed, but I don't think this is the intended way, because it makes everything WAY slower.
What I tried to do is as follows: create a class, that will store all the cache hashes in arrays, then before creating the pool, add it to a Volatile object. Before a task is being run, it retrieves this cache, uses it for read/write operation, and when the task finished, it merges it with the instance which is in the Volatile object. This works, as in, the caches made in JOB 1, can be seen in JOB 2, but it makes the whole process way much slower, then it was, when each thread only used their own cache, which was built while building the tree, and then destroyed, when the thread finished. Am I doing this wrong, or the thing I want is simply not achieavable? Here's my code:
class BattlefieldWork extends Threaded {
public $taskId;
public $innerIterator;
public $thinkAhead;
public $originalBattlefield;
public $iteratedBattlefield;
public $hashes;
public function __construct($taskId, $thinkAhead, $innerIterator, Battlefield $originalBattlefield, Battlefield $iteratedBattlefield) {
$this->taskId = $taskId;
$this->innerIterator = $innerIterator;
$this->thinkAhead = $thinkAhead;
$this->originalBattlefield = $originalBattlefield;
$this->iteratedBattlefield = $iteratedBattlefield;
}
public function run() {
$result = 0;
$dataSet = $this->worker->getDataSet();
$HashClassShared = null;
$dataSet->synchronized(function ($dataSet) use(&$HashClassShared) {
$HashClassShared = $dataSet['hashes'];
}, $dataSet);
$myHashClass = clone $HashClassShared;
$thinkAhead = $this->thinkAhead;
$innerIterator = $this->innerIterator;
$originalBattlefield = $this->originalBattlefield;
$iteratedBattlefield = $this->iteratedBattlefield;
// the actual recursive function that will build the tree, and calculate a quantify for the move, this will use the hash I've created
$result = $this->performThinkAheadMoves($thinkAhead, $innerIterator, $originalBattlefield, $iteratedBattlefield, $myHashClass);
// I am trying to retrieve the common cache here, and upload the result of this thread
$HashClassShared = null;
$dataSet->synchronized(function($dataSet) use ($result, &$HashClassShared) {
// I am storing the result of this thread
$dataSet['results'][$this->taskId] = $result;
// I am merging the data I've collected in this thread with the data that is stored in the `Volatile` object
$HashClassShared = $dataSet['hashes'];
$HashClassShared = $HashClassShared->merge($myHashClass);
}, $dataSet);
}
}
This is how I create my tasks, my Volatile, and my Pool:
class Battlefield {
/* ... */
public function step() {
/* ... */
/* get the possible moves for the current depth, that is 0, and store them in an array, named $moves */
// $nextInnerIterator, is an int, which shows which hero must take an action after the current move
// $StartingBattlefield, is the zero point Battlefield, which will be used in quantification
foreach($moves as $moveid => $move) {
$moves[$moveid]['quantify'] = new BattlefieldWork($moveid, self::$thinkAhead, $nextInnerIterator, $StartingBattlefield, $this);
}
$Volatile = new Volatile();
$Volatile['results'] = array();
$Volatile['hashes'] = $this->HashClass;
$pool = new Pool(6, 'BattlefieldWorker', [$Volatile]);
foreach ($moves as $moveid => $move) {
if (is_a($moves[$moveid]['quantify'], 'BattlefieldWork')) {
$pool->submit($moves[$moveid]['quantify']);
}
}
while ($pool->collect());
$pool->shutdown();
$HashClass = $Volatile['hashes'];
$this->HashClass = $Volatile['hashes'];
foreach ($Volatile['results'] as $moveid => $partialResult) {
$moves[$moveid]['quantify'] = $partialResult;
}
/* The moves are ordered based on quantify, one is selected, and then if the battle is not yet finished, step is called again */
}
}
And here is how I am merging two hash classes:
class HashClass {
public $id = null;
public $cacheDir;
public $battlefieldHashes = array();
public $battlefieldCleanupHashes = array();
public $battlefieldMoveHashes = array();
public function merge(HashClass $HashClass) {
$this->battlefieldCleanupHashes = array_merge($this->battlefieldCleanupHashes, $HashClass->battlefieldCleanupHashes);
$this->battlefieldMoveHashes = array_merge($this->battlefieldMoveHashes, $HashClass->battlefieldMoveHashes);
return $this;
}
}
I've benchmarked each part of the code, to see where am I losing time, but everything seems to be fast enough to not warrant the time increase I am experiencing. What I am thinking is, that the problem lies in the Threads, sometimes, it seems that no job is being done at all, like they are waiting for some thread. Any insights on what could be the problem, would be greatly appreciated.

updating several records with one request in laravel

Some quick context:
I have a sql table and a eloquent model for JobCards and each JobCard has several Operations belonging to it. I have a table and model for Operations. The users of my application browse and edit JobCards, but when I say editing a Jobcard this can include editing Operations associated with a JobCard. I have a page where a user can edit the Operations for a certain JobCard, I submit the the data as an array of Operations.
I want a clean way to update the data for the Operations of a JobCard. There are 3 different actions I may or may not need to do:
Update an existing Operation with new data
Create a new Operation
Delete an Operatation
I tried dealing with the first 2 and things are getting messy already. I still need a way of deleting an Operation if it is not present in the array sent in the request.
Heres my code:
public function SaveOps(Request $a)
{
$JobCardNum = $a -> get('JobCardNum');
$Ops = $a -> get('Ops');
foreach ($Ops as $Op) {
$ExistingOp = JobCardOp::GetOp($JobCardNum, $Op['OpNum'])->first();
if(count($ExistingOp)==0) {
$NewOp = new JobCardOp;
$NewOp -> JobCardNum = $JobCardNum;
$NewOp -> fill($Op);
$NewOp -> save();
$this->UpdateNextOpStatus($JobCardNum, $NewOp);
}
else {
$ExistingOp -> fill($Op);
$ExistingOp -> save();
}
}
Can anyone help with the deletion part and/or help make my code tidier.
This is how your method should look like. Please note that, I added a new method getCache($JobCardNum) this method will get an array of operations per job card (assuming that your model is designed to be related this way) this method will go to the DB only once, to get all the Operations that you need for this method call instead of getting them one-by-one (in the foreach loop), this way you make sure that the expensive call to the DB is done only once, on the other hand you got your JobCard's operations in the form of an array ready to compare with the new ones (coming in the request), the return of this method will be in the form of (key=>value with the key being the operation number and the value being the operation object it self).
/**
* This function will get you an array of current operations in the given job card
* #param $JobCardNum
* #return array
*/
public function getCache($JobCardNum)
{
/**
* asuming that the relation in your model is built that way. if not you should then
* use JobCardOp::all(); (Not recommended because it will get a lot of unnecessary
* data )
*/
$ExistingOps = JobCardOp::where('job_card_id', '=', $JobCardNum);
$opCache = array();
foreach ($ExistingOps as $Op) {
$opCache[(string)$Op->OpNum] = $Op;
}
return $opCache;
}
public function SaveOps(Request $a)
{
$strOpNum = (string)$Op['OpNum'];
$JobCardNum = $a->get('JobCardNum');
$Ops = $a->get('Ops');
$opCache = $this->getCache($JobCardNum);
foreach ($Ops as $Op) {
if (!isset($opCache[$strOpNum])) {
$NewOp = new JobCardOp;
$NewOp->JobCardNum = $JobCardNum;
$NewOp->fill($Op);
$NewOp->save();
$this->UpdateNextOpStatus($JobCardNum, $NewOp);
} else {
$ExistingOp = $opCache[$strOpNum];
$ExistingOp->fill($Op);
$ExistingOp->save();
}
unset($opCache[$strOpNum]);
}
/*
* at this point any item in the $opCache array must be deleted because it was not
* matched in the previous for loop that looped through the requested operations :)
*/
foreach ($opCache as $op) {
$op->delete();
}
}

Ordering Doctrine Collection based on associated Entity when it is not possible to use the #orderBy annotation

I would like to understand the best way to order a Doctrine Collection based on associated Entity. In this case, it is not possible to use the #orderBy annotation.
I have found 5 solutions on the Internet.
1) Adding a method to the AbstractEntity (according to Ian Belter https://stackoverflow.com/a/22183527/1148260)
/**
* This method will change the order of elements within a Collection based on the given method.
* It preserves array keys to avoid any direct access issues but will order the elements
* within the array so that iteration will be done in the requested order.
*
* #param string $property
* #param array $calledMethods
*
* #return $this
* #throws \InvalidArgumentException
*/
public function orderCollection($property, $calledMethods = array())
{
/** #var Collection $collection */
$collection = $this->$property;
// If we have a PersistentCollection, make sure it is initialized, then unwrap it so we
// can edit the underlying ArrayCollection without firing the changed method on the
// PersistentCollection. We're only going in and changing the order of the underlying ArrayCollection.
if ($collection instanceOf PersistentCollection) {
/** #var PersistentCollection $collection */
if (false === $collection->isInitialized()) {
$collection->initialize();
}
$collection = $collection->unwrap();
}
if (!$collection instanceOf ArrayCollection) {
throw new InvalidArgumentException('First argument of orderCollection must reference a PersistentCollection|ArrayCollection within $this.');
}
$uaSortFunction = function($first, $second) use ($calledMethods) {
// Loop through $calledMethods until we find a orderable difference
foreach ($calledMethods as $callMethod => $order) {
// If no order was set, swap k => v values and set ASC as default.
if (false == in_array($order, array('ASC', 'DESC')) ) {
$callMethod = $order;
$order = 'ASC';
}
if (true == is_string($first->$callMethod())) {
// String Compare
$result = strcasecmp($first->$callMethod(), $second->$callMethod());
} else {
// Numeric Compare
$difference = ($first->$callMethod() - $second->$callMethod());
// This will convert non-zero $results to 1 or -1 or zero values to 0
// i.e. -22/22 = -1; 0.4/0.4 = 1;
$result = (0 != $difference) ? $difference / abs($difference): 0;
}
// 'Reverse' result if DESC given
if ('DESC' == $order) {
$result *= -1;
}
// If we have a result, return it, else continue looping
if (0 !== (int) $result) {
return (int) $result;
}
}
// No result, return 0
return 0;
};
// Get the values for the ArrayCollection and sort it using the function
$values = $collection->getValues();
uasort($values, $uaSortFunction);
// Clear the current collection values and reintroduce in new order.
$collection->clear();
foreach ($values as $key => $item) {
$collection->set($key, $item);
}
return $this;
}
2) Creating a Twig extension, if you need the sorting just in a template (according to Kris https://stackoverflow.com/a/12505347/1148260)
use Doctrine\Common\Collections\Collection;
public function sort(Collection $objects, $name, $property = null)
{
$values = $objects->getValues();
usort($values, function ($a, $b) use ($name, $property) {
$name = 'get' . $name;
if ($property) {
$property = 'get' . $property;
return strcasecmp($a->$name()->$property(), $b->$name()->$property());
} else {
return strcasecmp($a->$name(), $b->$name());
}
});
return $values;
}
3) Transforming the collection into an array and then sorting it (according to Benjamin Eberlei https://groups.google.com/d/msg/doctrine-user/zCKG98dPiDY/oOSZBMabebwJ)
public function getSortedByFoo()
{
$arr = $this->arrayCollection->toArray();
usort($arr, function($a, $b) {
if ($a->getFoo() > $b->getFoo()) {
return -1;
}
//...
});
return $arr;
}
4) Using ArrayIterator to sort the collection (according to nifr https://stackoverflow.com/a/16707694/1148260)
$iterator = $collection->getIterator();
$iterator->uasort(function ($a, $b) {
return ($a->getPropery() < $b->getProperty()) ? -1 : 1;
});
$collection = new ArrayCollection(iterator_to_array($iterator));
5) Creating a service to gather the ordered collection and then replace the unordered one (I have not an example but I think it is pretty clear). I think this is the ugliest solution.
Which is the best solution according to you experience? Do you have other suggestions to order a collection in a more effective/elegant way?
Thank you very much.
Premise
You proposed 5 valid/decent solutions, but I think that all could be reduced down to two cases, with some minor variants.
We know that sorting is always O(NlogN), so all solution have theoretically the same performance. But since this is Doctrine, the number of SQL queries and the Hydration methods (i.e. converting data from array to object instance) are the bottlenecks.
So you need to choose the "best method", depending on when you need the entities to be loaded and what you'll do with them.
These are my "best solutions", and in a general case I prefer my solution A)
A) DQL in a loader/repository service
Similar to
None of your case (somehow with 5, see the final notes note). Alberto Fernández pointed you in the right direction in a comment.
Best when
DQL is (potentially) the fastest method, since delegate sorting to DBMS which is highly optimized for this. DQL also gives total controls on which entities to fetch in a single query and the hydrations mode.
Drawbacks
It is not possible (AFAIK) to modify query generated by Doctrine Proxy classes by configuration, so your application need to use a Repository and call the proper method every time you load your entities (or override the default one).
Example
class MainEntityRepository extends EntityRepository
{
public function findSorted(array $conditions)
{
$qb = $this->createQueryBuilder('e')
->innerJoin('e.association', 'a')
->orderBy('a.value')
;
// if you always/frequently read 'a' entities uncomment this to load EAGER-ly
// $qb->select('e', 'a');
// If you just need data for display (e.g. in Twig only)
// return $qb->getQuery()->getResult(Query::HYDRATE_ARRAY);
return $qb->getQuery()->getResult();
}
}
B) Eager loading, and sorting in PHP
Similar to case
Case 2), 3) and 4) are just the same thing done in different place. My version is a general case which apply whenever the entities are fetched. If you have to choose one of these, then I think that solution 3) is the most convenient, since don't mess with the entity and is always available, but use EAGER loading (read on).
Best when
If the the associated entities are always read, but it is not possible (or convenient) to add a service, then all entities should loaded EAGER-ly. Sorting then can be done by PHP, whenever it makes sense for the application: in an event listener, in a controller, in a twig template... If the entities should be always loaded, then an event listener is the best option.
Drawbacks
Less flexible than DQL, and sorting in PHP may be a slow operation when the collection is big. Also, the entities need to be hydrated as Object which is slow, and is overkill if the collection is not used for other purpose. Beware of lazy-loading, since this will trigger one query for every entity.
Example
MainEntity.orm.xml:
<?xml version="1.0" encoding="utf-8"?>
<doctrine-mapping>
<entity name="MainEntity">
<id name="id" type="integer" />
<one-to-many field="collection" target-entity="LinkedEntity" fetch="EAGER" />
<entity-listeners>
<entity-listener class="MainEntityListener"/>
</entity-listeners>
</entity>
</doctrine-mapping>
MainEntity.php:
class MainEntityListener
{
private $id;
private $collection;
public function __construct()
{
$this->collection = new ArrayCollection();
}
// this works only with Doctrine 2.5+, in previous version association where not loaded on event
public function postLoad(array $conditions)
{
/*
* From your example 1)
* Remember that $this->collection is an ArryCollection when constructor is called,
* but a PersistentCollection when are loaded from DB. Don't recreate the instance!
*/
// Get the values for the ArrayCollection and sort it using the function
$values = $this->collection->getValues();
// sort as you like
asort($values);
// Clear the current collection values and reintroduce in new order.
$collection->clear();
foreach ($values as $key => $item) {
$collection->set($key, $item);
}
}
}
Final Notes
I won't use case 1) as is, since is very complicated and introduce inheritance which reduce encapsulation. Also, I think that it has the same complexity and performance of my example.
Case 5) is not necessarily bad. If "the service" is the application repository, and it use DQL to sort, then is my first best case. If is a custom service only to sort a collection, then I think is definitely not a good solution.
All the codes I wrote here is not ready for "copy-paste", since my objective was to show my point of view. Hope it would be a good starting point.
Disclaimer
These are "my" best solutions, as I do it in my works. Hope will help you and others.

Magento - Blank white screen search results. Many things broken

Whenever a user searches, I get this error:
2012-06-26 11:05:21.671 [NOTICE] [208.69.120.120:48175-0#hostname] [STDERR] PHP Fatal error: Call to undefined method Mage_Catalog_Model_Resource_Product_Flat::getEntityTablePrefix() in /chroot/home/SITENAME/DOMAIN.COM/html/app/code/core/Mage/Eav/Model/Entity/Attribute/Abstract.php on line 505
And instead of the user's results appearing, they get a blank white page- no error on their end, no UI, just white. This was the first issue I noticed, but on the same day the following issues started coming up:
White Search Results
Sub-category product count for all sub-categories in layered nav is showing 0.
Some customers can not view orders from their front-end UI when logged in.
Our order export script is returning blank fields (1.7mb instead of 4.3).
Our "Made-in-the-usa" and "best sellers" pages are returning more products than they should.
Now, I know these are all incorrect because if I reindex the entire site, for some period while it is processing the index, all of the above works. However, when the index is complete, it all breaks again. The same day this happened we had an error page appear that stated one of the tables had broken and should be repaired. We ran PHPMyAdmin's repair and optimize functions on all tables and it fixed that error- but all of these are still broken.
Any ideas at all? Any ideas of what could be tried to fix this? I cant find this error anywhere- and the guys over at Nexcess haven't been able to find anything for this, either.
Thank you for your time.
As per the comments above, Magento's telling you that it's trying to call the method getEntityTablePrefix on an object whose classes don't have that method defined. Specifically in this method
#File: app/code/core/Mage/Eav/Model/Entity/Attribute/Abstract.php
public function getBackendTable()
{
if ($this->_dataTable === null) {
if ($this->isStatic()) {
$this->_dataTable = $this->getEntityType()->getValueTablePrefix();
} else {
$backendTable = trim($this->_getData('backend_table'));
if (empty($backendTable)) {
$entityTable = array($this->getEntity()->getEntityTablePrefix(), $this->getBackendType());
$backendTable = $this->getResource()->getTable($entityTable);
}
$this->_dataTable = $backendTable;
}
}
return $this->_dataTable;
}
Given this happens from the following class
Mage_Catalog_Model_Resource_Product_Flat
It says to me that you have an extension and/or customization done that assumes you're not using the flat catalog data tables and wasn't coded to work with the flat table.
Dropping in a debugging call like this
if(!is_callable(array($this->getEntity()),'getEntityTablePrefix'))
{
mageDebugBacktrace();
//debug_print_backtrace();
exit;
}
right before the offending call (in a local code pool override, of course), will print out a call-stack that should point the offending code.
The seems that problem is in Mage_CatalogSearch_Model_Resource_Search_Collection::_getSearchEntityIdsSql that is not compatible with using the product flat index.
You can rewrite class Mage_CatalogSearch_Model_Resource_Search_Collection and do two little modifications.
1) Add new function _getSearchEntityIdsSqlUsingFlatIndex to rewrited class. This new function (I hope) does exactly the same thing as original _getSearchEntityIdsSql, but with using the product flat index.
2) Modify function _getSearchEntityIdsSql so that it calls new _getSearchEntityIdsSqlUsingFlatIndex if the catalog product flat index is enabled and built.
See source code:
class VENDOR_MODULE_Model_PATHTOREWRITECLASS extends Mage_CatalogSearch_Model_Resource_Search_Collection {
/**
* Retrieve SQL for search entities using product flat index.
*
* #param $query
* #return Varien_Db_Select
*/
protected function _getSearchEntityIdsSqlUsingFlatIndex($query)
{
/* #var $coreHelper Mage_Core_Model_Resource_Helper_Abstract */
$coreHelper = Mage::getResourceHelper('core');
$likeOptions = array('position' => 'any');
$flatTableName = $this->getTable('catalog/product_flat').'_'.$this->getStoreId();
/** #var Varien_Db_Select $select */
$select = $this->getConnection()
->select()
->from($flatTableName, array('entity_id'));
foreach ($this->_getAttributesCollection() as $attribute) {
/** #var Mage_Catalog_Model_Entity_Attribute $attribute */
if ($this->_isAttributeTextAndSearchable($attribute)) {
$attributeCode = $attribute->getAttributeCode();
$dbFieldName = in_array($attribute->getFrontendInput(), array('select', 'multiselect'))
? $attributeCode.'_value'
: $attributeCode;
if ($this->getConnection()->tableColumnExists($flatTableName, $dbFieldName)) {
$select->where($coreHelper->getCILike($dbFieldName, $this->_searchQuery, $likeOptions));
} else {
Mage::log(__METHOD__.": Attribute '$attributeCode' is missing in flat index.", Zend_Log::NOTICE);
}
}
}
return $select;
}
/**
* Retrieve SQL for search entities
*
* #param unknown_type $query
* #return string
*/
protected function _getSearchEntityIdsSql($query)
{
// HACK - make compatibility with flat index
/** #var Mage_Catalog_Helper_Product_Flat $flatHelper */
$flatHelper = Mage::helper('catalog/product_flat');
if ($this->getStoreId() > 0
&& $flatHelper->isEnabled($this->getStoreId())
&& $flatHelper->isBuilt($this->getStoreId())
) {
return $this->_getSearchEntityIdsSqlUsingFlatIndex($query);
}
// END HACK
return parent::_getSearchEntityIdsSql($query);
}
}

How do one use ACL to filter a list of domain-objects according to a certain user's permissions (e.g. EDIT)?

When using the ACL implementation in Symfony2 in a web application, we have come across a use case where the suggested way of using the ACLs (checking a users permissions on a single domain object) becomes unfeasible. Thus, we wonder if there exists some part of the ACL API we can use to solve our problem.
The use case is in a controller that prepares a list of domain objects to be presented in a template, so that the user can choose which of her objects she wants to edit. The user does not have permission to edit all of the objects in the database, so the list must be filtered accordingly.
This could (among other solutions) be done according to two strategies:
1) A query filter that appends a given query with the valid object ids from the present user's ACL for the object(or objects). I.e:
WHERE <other conditions> AND u.id IN(<list of legal object ids here>)
2) A post-query filter that removes the objects the user does not have the correct permissions for after the complete list has been retrieved from the database. I.e:
$objs = <query for objects>
$objIds = <getting all the permitted obj ids from the ACL>
for ($obj in $objs) {
if (in_array($obj.id, $objIds) { $result[] = $obj; }
}
return $result;
The first strategy is preferable as the database is doing all the filtering work, and both require two database queries. One for the ACLs and one for the actual query, but that is probably unavoidable.
Is there any implementation of one of these strategies (or something achieving the desired results) in Symfony2?
Assuming that you have a collection of domain objects that you want to check, you can use the security.acl.provider service's findAcls() method to batch load in advance of the isGranted() calls.
Conditions:
Database was populated with test entities, with object permissions of MaskBuilder::MASK_OWNER for a random user from my database, and class permissions of MASK_VIEW for role IS_AUTHENTICATED_ANONYMOUSLY; MASK_CREATE for ROLE_USER; and MASK_EDIT and MASK_DELETE for ROLE_ADMIN.
Test Code:
$repo = $this->getDoctrine()->getRepository('Foo\Bundle\Entity\Bar');
$securityContext = $this->get('security.context');
$aclProvider = $this->get('security.acl.provider');
$barCollection = $repo->findAll();
$oids = array();
foreach ($barCollection as $bar) {
$oid = ObjectIdentity::fromDomainObject($bar);
$oids[] = $oid;
}
$aclProvider->findAcls($oids); // preload Acls from database
foreach ($barCollection as $bar) {
if ($securityContext->isGranted('EDIT', $bar)) {
// permitted
} else {
// denied
}
}
RESULTS:
With the call to $aclProvider->findAcls($oids);, the profiler shows that my request contained 3 database queries (as anonymous user).
Without the call to findAcls(), the same request contained 51 queries.
Note that the findAcls() method loads in batches of 30 (with 2 queries per batch), so your number of queries will go up with larger datasets. This test was done in about 15 minutes at the end of the work day; when I have a chance, I'll go through and review the relevant methods more thoroughly to see if there are any other helpful uses of the ACL system and report back here.
Itinerating over the entities is not feasible if you have a couple of thousandth entities - it will keep getting slower and consuming more memory, forcing you to use doctrine batching capabilities, thus making your code more complex (and innefective because after all you need only the ids to make a query - not the whole acl/entities in memory)
What we did to solve this problem is to replace acl.provider service with our own and in that service add a method to make a direct query to the database:
private function _getEntitiesIdsMatchingRoleMaskSql($className, array $roles, $requiredMask)
{
$rolesSql = array();
foreach($roles as $role) {
$rolesSql[] = 's.identifier = ' . $this->connection->quote($role);
}
$rolesSql = '(' . implode(' OR ', $rolesSql) . ')';
$sql = <<<SELECTCLAUSE
SELECT
oid.object_identifier
FROM
{$this->options['entry_table_name']} e
JOIN
{$this->options['oid_table_name']} oid ON (
oid.class_id = e.class_id
)
JOIN {$this->options['sid_table_name']} s ON (
s.id = e.security_identity_id
)
JOIN {$this->options['class_table_nambe']} class ON (
class.id = e.class_id
)
WHERE
{$this->connection->getDatabasePlatform()->getIsNotNullExpression('e.object_identity_id')} AND
(e.mask & %d) AND
$rolesSql AND
class.class_type = %s
GROUP BY
oid.object_identifier
SELECTCLAUSE;
return sprintf(
$sql,
$requiredMask,
$this->connection->quote($role),
$this->connection->quote($className)
);
}
Then calling this method from the actual public method that gets the entities ids:
/**
* Get the entities Ids for the className that match the given role & mask
*
* #param string $className
* #param string $roles
* #param integer $mask
* #param bool $asString - Return a comma-delimited string with the ids instead of an array
*
* #return bool|array|string - True if its allowed to all entities, false if its not
* allowed, array or string depending on $asString parameter.
*/
public function getAllowedEntitiesIds($className, array $roles, $mask, $asString = true)
{
// Check for class-level global permission (its a very similar query to the one
// posted above
// If there is a class-level grant permission, then do not query object-level
if ($this->_maskMatchesRoleForClass($className, $roles, $requiredMask)) {
return true;
}
// Query the database for ACE's matching the mask for the given roles
$sql = $this->_getEntitiesIdsMatchingRoleMaskSql($className, $roles, $mask);
$ids = $this->connection->executeQuery($sql)->fetchAll(\PDO::FETCH_COLUMN);
// No ACEs found
if (!count($ids)) {
return false;
}
if ($asString) {
return implode(',', $ids);
}
return $ids;
}
This way now we can use the code to add filters to DQL queries:
// Some action in a controller or form handler...
// This service is our own aclProvider version with the methods mentioned above
$aclProvider = $this->get('security.acl.provider');
$ids = $aclProvider->getAllowedEntitiesIds('SomeEntityClass', array('role1'), MaskBuilder::VIEW, true);
if (is_string($ids)) {
$queryBuilder->andWhere("entity.id IN ($ids)");
}
// No ACL found: deny all
elseif ($ids===false) {
$queryBuilder->andWhere("entity.id = 0")
}
elseif ($ids===true) {
// Global-class permission: allow all
}
// Run query...etc
Drawbacks: This methods have to be improved to take into account the complexities of ACL inheritance and strategies, but for simple use cases it works fine. Also a cache has to be implemented to avoid the repetitive double query (one with class-level, another with objetc-level)
Coupling Symfony ACL back to application and using it as sorting, is not good approach. You are mixing and coupling 2 or 3 layers of application together.
ACL functionality is to answer "YES/NO" to question "Am I allowed to do this?" If you need some sort of owned/editable articles, you can use some column like CreatedBy or group CreatedBy by criteria from another table. Some usergroups or accounts.
Use joins, and in case you're using Doctrine, get it to generate joins for you, as they are almost always faster. Therefore you should design your ACL schema that doing these fast filters are feasible.

Categories