Using Symfony and Doctrine, I’m writing a component that populates the database with entries of arbitrary entities during the deployment of an application.
I want to make this component generic, so I’m trying to resolve dependencies between entities automatically and insert them in the correct order.
I am currently get the dependencies of each single entity through the entity metadata, recursively:
public function getEntityDeps($eName)
{
$deps = [];
foreach ($this->entityManager->getClassMetadata($eName)->getAssociationMappings() as $mapping)
{
$deps[] = $mapping['targetEntity'];
$deps = array_merge($deps, $this->getEntityDeps($mapping['targetEntity']));
}
return $deps;
}
The result is obviously a list of the following kind:
// NOTE: The real list of course contains class names instead of entity aliases.
[
"FooBundle:EntityA" => [],
"FooBundle:EntityB" => ["FooBundle:EntityA", "FooBundle:EntityC"],
"FooBundle:EntityC" => ["FooBundle:EntityA"],
"BarBundle:EntityA" => ["BarBundle:EntityB"],
"BarBundle:EntityB" => []
]
The next step would be apply some type of topological sorting to the list, I guess.
However, I’m not sure if I can use a generic algorithm here or if I’m forgetting something. Especially as entities are not necessarily related (so that, in fact, we may have more than one dependency graph).
Also, I’d hope that there’s some internal Doctrine functionality which can do the sorting for me.
So what would be the most reliable way to sort an arbitrary set of Doctrine entities? Is there some reusable Doctrine feature?
So, after digging through the Doctrine source code for a while, I found that they have indeed an implementation of topological sorting (DFS for that matter). It's implemented in Doctrine\ORM\Internal\CommitOrderCalculator.
And where is this CommitOrderCalculator used? In UnitOfWork, of course.
So, instead of manually calculating the correct commit order, I just needed … to move the call to $em->flush() outside the foreach loop that went through the entities and let UOW do the work itself.
Duh.
Related
I have a strange problem with \Doctrine\ORM\UnitOfWork::getScheduledEntityDeletions used inside onFlush event
foreach ($unitOfWork->getScheduledEntityDeletions() as $entity) {
if ($entity instanceof PollVote) {
$arr = $entity->getAnswer()->getVotes()->toArray();
dump($arr);
dump($entity);
dump(in_array($entity, $arr, true));
dump(in_array($entity, $arr));
}
}
And here is the result:
So we see that the object is pointing to a different instance than the original, therefore in_array no longer yields expected results when used with stick comparison (AKA ===). Furthermore, the \DateTime object is pointing to a different instance.
The only possible explanation I found is the following (source):
Whenever you fetch an object from the database Doctrine will keep a copy of all the properties and associations inside the UnitOfWork. Because variables in the PHP language are subject to “copy-on-write” the memory usage of a PHP request that only reads objects from the database is the same as if Doctrine did not keep this variable copy. Only if you start changing variables PHP will create new variables internally that consume new memory.
However, I did not change anything (even the created field is kept as it is). The only operations that were preformed on entity are:
\Doctrine\ORM\EntityRepository::findBy (fetching from DB)
\Doctrine\Common\Persistence\ObjectManager::remove (scheduling for removal)
$em->flush(); (triggering synchronization with DB)
Which leads me to think (I might be wrong) that the Doctrine's change tracking method has nothing to do with the issue that I'm experiencing. Which leads me to following questions:
What causes this?
How to reliably check if an entity scheduled for deletion is inside a collection (\Doctrine\Common\Collections\Collection::contains uses in_array with strict comparison) or which items in a collection are scheduled for deletion?
The problem is that when you tell doctrine to remove entity, it is removed from identity map (here):
<?php
public function scheduleForDelete($entity)
{
$oid = spl_object_hash($entity);
// ....
$this->removeFromIdentityMap($entity);
// ...
if ( ! isset($this->entityDeletions[$oid])) {
$this->entityDeletions[$oid] = $entity;
$this->entityStates[$oid] = self::STATE_REMOVED;
}
}
And when you do $entity->getAnswer()->getVotes(), it does the following:
Load all votes from database
For every vote, checks if it is in identity map, use old one
If it is not in identity map, create new object
Try to call $entity->getAnswer()->getVotes() before you delete entity. If the problem disappears, then I am right. Of cause, I would not suggest this hack as a solution, just to make sure we understand what is going on under the hood.
UPD instead of $entity->getAnswer()->getVotes() you should probably do foreach for all votes, because of lazy loading. If you just call $entity->getAnswer()->getVotes(), Doctrine probably wouldn't do anytning, and will load them only when you start to iterate through them.
From the doc:
If you call the EntityManager and ask for an entity with a specific ID twice, it will return the same instance
So calling twice findOneBy(['id' => 12]) should result in two exact same instances.
So it all depends on how both instances are retrieved by Doctrine.
In my opinion, the one you get in $arr is from a One-to-Many association on $votes in the Answer entity, which results in a separate query (maybe a id IN (12)) by the ORM.
Something you could try is to declare this association as EAGER (fetch="EAGER"), it may force the ORM to make a specific query and keep it in cache so that the second time you want to get it, the same instance is returned ?
Could you have a look at the logs and post them here ? It may indicates something interesting or at least relevant to investigate further.
According to FOSElasticaBundle documentation it is possible to configure application to use custom query builder method like this:
user:
persistence:
elastica_to_model_transformer:
query_builder_method: createSearchQueryBuilder
But is it possible to choose QB method live, e.g. in controller action?
I'd like to be able to control what's being fetched from DB while transforming Elastica results to Doctrine entities. E.g. sometimes I'll want to do eager fetch on some relations, but can't do that by default.
Since FOSElasticaBundle documentation is not very precise, I went through its code and found it impossible to control what query builder is used on controller level.
It is possible to change whole elastica_to_model_transformer to a custom service, but still it's statically defined in configuration. Maybe with some dirty solution it would be possible going this way, but I don't think it's worth it.
I decided to just not using this feature of FOSElasticaBundle. The main problem I had was that when you use fos_elastica.index instead of fos_elastica.finder or elastica repository (in order to get plain not transformed results Elastica\Resultset), there's no findPaginated method with returns Pagerfanta paginator object, which is very helpful in my case.
Fortunately although it's not mentioned in documentation it's possible to create the Pagerfanta this way too, but a little bit more manually.
Here's a code snippet:
//generate ElaticaQuery somehow.
$browseQuery = $browseData->getBrowseQuery();
$search = $this->container->get('fos_elastica.index.indexName.typName');
//create pagerfanta's adapter manually
$adapter = new \Pagerfanta\Adapter\ElasticaAdapterElasticaAdapter($search, $browseQuery);
// now you can create the paginator too.
$pager = new Pagerfanta($adapter);
//do some paging work on it...
$pager->setMaxPerPage($browseData->getPerPage());
try {
$pager->setCurrentPage($browseData->getPage());
} catch(OutOfRangeCurrentPageException $e) {
$pager->setCurrentPage(1);
}
//and get current page results.
/** #var Result[] $elasticaResults */
$elasticaResults = $pager->getCurrentPageResults();
// we have to grab ids manyally, but it's done the same way inside FOSElasticaBundle with previous approach
$ids = array();
foreach($elasticaResults as $elasticaResult) {
$ids[] = $elasticaResult->getId();
}
//use regular Doctrine's repository to fetch Entities any way you want.
$entities = $this->getDoctrine()->getRepository(MyEntity::class)->findByIdentifiers($ids);
This actually has a few advantages. In general it gives you back control over your data and doesn't tie ElasticSearch with Doctrine. Therefore you can resign on fetching data from Doctrine if you have all needed data in ElasticSearch (if they are read only data of course). This lets you optimize your application performance but reducing amount of SQL queries.
The code above may be wrapped with some kind of service in order to prevent making mess in controllers.
I have an Extbase-based extension for Typo3, which has a hierarchical data model. I had to insert an additional layer to this model, ie the original structure was Project contains multiple items. Now, I have Project contains multiple sub-projects and Sub-project contains multiple items. Everything is modelled using MM-relation tables and works in the backend. I can add, remove, sort the sub-projects and items.
However, the fluid template does not show anything and if I pass eg the sub-Project to t3lib_utilities_debug::Debug, I get
You should never see this warning. If you do, you probably used PHP
array functions like current() on the
Tx_Extbase_Persistence_ObjectStorage. To retrieve the first result,
you can use the rewind() and current() methods.
when printing the ObjectStorage for the items. I assume that the MM-relation I added is somehow broken, but I cannot figure out how. Furthermore, it seems that the __construct method of the domain model is not called (I have added a debug output, which is not printed).
The enumeration works if I pass the result of a call to findAll of the repository, but it does not work for my filtered calls (which worked before I added the additional layer). The filtering method looks like eg for the item
public function findBySubProject(SubProject $p) {
$query = $this->createQuery();
$query->getQuerySettings()->setRespectStoragePage(false);
$query->matching($query->equals('subproject', $p));
return $query->execute();
}
As I said, the query yields results, but they are somehow broken wrt. their relations.
Any ideas how to fix that?
I don't know on which version of Extbase you're developing on.
But on TYPO3 4.6+ you should be aware of the object and reflection caching. During development you can disable this caching by:
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_reflection']['backend'] = 't3lib_cache_backend_NullBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_object']['backend'] = 't3lib_cache_backend_NullBackend';
Since your problem has something to do with modifications in your model you should try to truncate the tables cf_extbase_object, cf_extbase_object_tags, cf_extbase_reflection and cf_extbase_reflection_tags after any change.
If this doesn't help you to resolve your problem then you should give us more insight about your configuration (especially the TCA configuration because Extbase rely on it).
How to test a Extbase QueryResult
$items = $this->itemRepository->findAll();
echo count($items);
if ($items) {
echo '<pre>';
foreach ($items as $item) {
print_r($item);
}
echo '</pre>';
}
-- edit --
Did you define the field subproject in your TCA? It should be atleast available as type passtrough:
'subproject' => array(
'config' => array(
'type' => 'passthrough',
),
),
I case someone else encounters the same issue: I accidentally used an object without items as test object. If you try to enumerate/debug/display an empty ObjectStorage, the warning is printed.
I'm a little confused as to what is going on here, it looks to me like a method is calling itself? I'm trying to learn about Magento's models. I was working my way back from a helper (catalog/category) and I got to a call on this method "GetCategories". I don't know whats going on here. If anyone could shed light on this code snippet I greatly appreciate it.
getCategories ( $parent,
$recursionLevel = 0,
$sorted = false,
$asCollection = false,
$toLoad = true
){
$categories = $this->getResource()
->getCategories($parent, $recursionLevel, $sorted, $asCollection, $toLoad);
return $categories;
}
Not much to add to #hakra's answer. Just a portion of Magento-specific logic.
So to work with Magento models you should know, that Magento has 2 types of Models: normal models, and resource models (we can call assign Blocks to the models too, as a view models - but that is more connected to the V part of MVC).
The resource models were created as a DB adapters that contain only DB-related logic, and often are connected to some DB table, hence contain the logic for CRUD operations with that table. So you'll see smth like this regularly - for the simplicity someMethod is a part of normal model, but since it contains DB-related logic, all the implementation of the method was moved to the resource model, so the body of someMethod in the regular model will be something like that:
public function someMethod($args)
{
return $this->getResource()->someMethod($args);
}
It is hard to say for the code you've posted. Even both methods share the same name (getCategories) it must not mean that they are of the same class or even object.
If you want to find out you would need to compare:
var_dump($this === $this->getResource());
Apart from that, it is also common in programming recursion that a method calls itself, hence recursion. However for that chunk of code, it would run against the wall.
So technically speaking I would do the assumption that in your example this is not the exact same object method.
Please take note that this answer is independent to Magento, it's just how PHP works generally.
I am currently a beginner in CakePHP, and have played around with CakePHP 1.3, but recently CakePHP 2.0 has been released.
So far I like it but the only thing is being a pain is the fact that it doesn't return Objects, rather it just returns arrays. I mean, it hardly makes sense to have to do $post['Post']['id']. It is (in my opinion) much more practical to just do $post->id.
Now after Google I stumbled upon this link, however, this kept generating errors about indexes not being defined when using the Form class (guessing this is because it was getting the objectified version rather than the array version).
I am following the Blog tutorial (already have followed it under 1.3 but going over it again for 2.0)
So, anyone know how to achieve this without it interfering with the Form class?
Hosh
Little known fact: Cake DOES return them as objects, or well properties of an object, anyway. The arrays are the syntactical sugar:
// In your View:
debug($this->viewVars);
Shwoing $this is a View object and the viewVars property corresponds with the $this->set('key', $variable) or $this->set(compact('data', 'for', 'view')) from the controller action.
The problem with squashing them into $Post->id for the sake of keystrokes is Cake is why. Cake is designed to be a heavy lifter, so its built-in ORM is ridiculously powerful, unavoidable, and intended for addressing infinity rows of infinity associated tables - auto callbacks, automatic data passing, query generation, etc. Base depth of multidimensional arrays depends on your find method, as soon as you're working with more than one $Post with multiple associated models (for example), you've introduced arrays into the mix and there's just no avoiding that.
Different find methods return arrays of different depths. From the default generated controller code, you can see that index uses $this->set('posts', $this->paginate()); - view uses $this->set('post', $this->Post->read(null, $id)); and edit doesn't use $this->set with a Post find at all - it assigns $this->data = $this->Post->read(null, $id);.
FWIW, Set::map probably throws those undefined index errors because (guessing) you happen to be trying to map an edit action, amirite? By default, edit actions only use $this->set to set associated model finds to the View. The result of $this->read is sent to $this->data instead. That's probably why Set::map is failing. Either way, you're still going to end up aiming at $Post[0]->id or $Post->id (depending on what you find method you used), which isn't much of an improvement.
Here's some generic examples of Set::map() property depth for these actions:
// In posts/index.ctp
$Post = Set::map($posts);
debug($Post);
debug($Post[0]->id);
// In posts/edit/1
debug($this-viewVars);
debug($this->data);
// In posts/view/1
debug($this-viewVars);
$Post = Set::map($post);
debug($Post->id);
http://api13.cakephp.org/class/controller#method-Controllerset
http://api13.cakephp.org/class/model#method-Modelread
http://api13.cakephp.org/class/model#method-ModelsaveAll
HTH.
You could create additional object vars. This way you wouldn't interfere with Cake's automagic but could access data using a format like $modelNameObj->id; format.
Firstly, create an AppController.php in /app/Controller if you don't already have one. Then create a beforeRender() function. This will look for data in Cake's standard naming conventions, and from it create additional object vars.
<?php
App::uses('Controller', 'Controller');
class AppController extends Controller {
public function beforeRender() {
parent::beforeRender();
// camelcase plural of current model
$plural = lcfirst(Inflector::pluralize($this->modelClass));
// create a new object
if (!empty($this->viewVars[$plural])) {
$objects = Set::map($this->viewVars[$plural]);
$this->set($plural . 'Obj', $objects);
}
// camelcase singular of current model
$singular = lcfirst(Inflector::singularize($this->modelClass));
// create new object
if (!empty($this->viewVars[$singular])) {
$object = Set::map($this->viewVars[$singular]);
$this->set($singular . 'Obj', $object);
}
}
}
Then in your views you can access the objects like so:
index.ctp
$productsObj;
view.ctp
$productObj->id;
All we're doing is adding 'Obj' to the variable names that Cake would already provide. Some example mappings:
Products -> $productsObj
ProductType -> $productTypesObj
I know this is not perfect but it would essentially achieve what you wanted and would be available across all of your models.
While I like the idea Moz proposes there are a number of existing solutions to this problem.
The quickest one I found is https://github.com/kanshin/CakeEntity - but it looks like you might need to refactor it for 2.x - there might even already be a 2.x branch or fork but I didn't look.
I also ran this question couple of time in my head. Now a few Cake based apps later, I see the benefit to be able to branch and merge (am, in_array etc.) result sets more conveniently with arrays than using objects.
The $Post->id form would be a sweet syntactic sugar, but not a real benefit over arrays.
You could write a function that iterates over your public propertys (see ReflectionClass::getProperties) and save it in an array (and return the array).
If you have access to the class, you can implement the ArrayAccess Interface and easily access your object as an array.
P.S.: Sorry, i've never used CakePHP but i think object-to-array conversion doesn't have to be a framework specific problem