Doctrine load relations from an Arraycollection - php

I'm looking for an elegant way to do the following:
Lets say I have an Entity with a OneToMany Relation, e.g.
class Parent
{
/**
* #ORM\OneToMany(targetEntity="Child")
*/
private $children;
public function __construct()
{
$this->children = new ArrayCollection();
}
}
class Child
{
/**
* #ORM\ManyToOne(targetEntity="Parent")
*/
private $parent;
}
Now in my logic I have to filter those parents which can't be done by queries only. So I end up with an ArrayCollection of parents, e.g:
$parents = new ArrayCollection([
$parent1,
$parent2,
$parent3
]);
Now from all those parents, I would like to join the children with one query.
How could I do this with doctrine.
I know I could just loop over the collection and call ->getChildren() on each parent.
But I possibly have hundreds of parents (which would mean hundreds of queries). Which I would like to avoid.
Before anyone says 'do a LEFT JOIN on the parents when you get them', I cannot filter the parents I need with only queries. So this is why I cannot just LEFT JOIN.
Cookies for thoughs !

In order to avoid the N + 1 selects problem, I would suggest the following solution which doesn’t need JOINs and uses two separate queries. This is the most efficient solution.
First, retrieve all parents:
$parents = $em->createQueryBuilder()
->select("p")
->from("YourFoobarBundle:Parent", "p")
->where(/*...*/)
->setParameter(/*...*/)
->indexBy("p.id")
->getQuery()->getResult();
Now load all children of those parents:
$children = $em->createQueryBuilder()
->select("c")
->from("YourFoobarBundle:Child", "c")
->where("IDENTITY(child.parent) IN (?1)")
->setParameter(1, array_keys($parents))
->getQuery()->getResult();
The great thing about Doctrine is that now all needed entities are stored in memory. So when you do a $parent->getChildren(), you don’t trigger a new DB query (unless the children have additional relations themselves).
NOTE: If you always need all children of the selected parents, you should mark the $children for eager loading:
/**
* #ORM\OneToMany(targetEntity="Child", fetch="EAGER")
*/
private $children;
In this case, Doctrine will always (!) fetch the needed children automatically.

You can make a sub query and use it inside a clause (a where in expression) of your other query. It is very much like #lxg describes, but you can do all this in one single query for increased performance (You don't have to execute the queries separately).
$qb = $entityManager->createQueryBuilder();
$sub = $qb->select('p')
->from('Application\Entity\Parent', 'p')
->where(/*...*/)
->setParameter(/*...*/)
$children = $qb->select('c')
->from('Application\Entity\Child', 'c')
->where($qb->expr()->in('c.parent', $sub->getDQL()))
->getQuery()
->getResult();

Related

Doctrine fetch associated array for multiple objects

I've got two classes, Post and Comment, as components I'm using to build a news feed. Post has a one-to-many association with Comment.
When building my news feed, I query the posts table and typically return a chunk of 20 results. With these 20 results, I'd also like to load the comments associated with it. Logic dictates that I can simply use fetch="EAGER" on the #OneToMany annotation, but in this use case it doesn't scale well as searching through a comments table with 1,000's of entries repetitively for each post is not performant.
Ideally, I'd like to separately preload them in a single query - which is what I've done.
class Post {
// ...
/**
* #var Comment[]
*
* #ORM\OneToMany(targetEntity="App\Entity\PostComment", mappedBy="post", cascade={"persist", "remove"}, orphanRemoval=true)
*/
protected $comments;
public function getComments() {
return $this->comments;
}
public function setComments(array $comments) {
$this->comments = $comments;
}
}
class NewsFeedService {
private function preloadComments(array $posts) {
$postIds = array_column($posts, 'id'); // Get the post ID's
$comments = $this->manager->getRepository(Comment::class)->findCommentByPostIds($postIds);
// Group the comments by post
$groupedComments = [];
foreach($comments as $comment) {
$groupedComments[$comment->getPost()->getId()][] = $comment;
}
// Populate each $post object with it's comments
foreach($groupedComments as $postId => $postComments) {
$post = $posts[$postId];
$post->setComments($postComments);
}
}
}
class CommentRepository extends EntityRepository {
public function findCommentsByPostIds(array $postIds) {
return $this->createQueryBuilder('c')
->where('c.post IN(:ids)')
->setParameter('ids', $postIds)
->getQuery()
->getResult();
}
}
Essentially, when calling preloadComments() I run a single query on the DB and then force the results into each Post. Compared to EAGER fetching this saves me on average 35-40% with my current dataset (60,000 comments). I mean it works, but..
My question is if there is a better, perhaps even doctrine native way of doing this. There are also some things that I haven't tested such as if calling $post->setComments() and adding externally fetched data will cause issues if I just so happen to update the Post object and flush the changes. I feel like going about the way I'm doing isn't optimal and may cause some small headaches over the performance I'm saving.

Symfony 4 Sorting Filtered Array Collection

I am having trouble sorting a collection resulting from a one-to-many relationship that has been filtered. I have a quiz that has questions:
class Quiz
{
/**
* One quiz has many questions. This is the inverse side.
* #ORM\OneToMany(targetEntity="Question", mappedBy="assessment")
* #ORM\OrderBy({"num" = "ASC"})
*/
private $questions;
public function __construct() {
$this->questions = new ArrayCollection();
}
This works as expected. However, when I modify the getter to exclude inactive (soft-deleted) questions, the sort order is lost.
public function getQuestions()
{
// filter to never return soft deleted questions
$criteria = Criteria::create()->where(Criteria::expr()->eq("active", true));
return $this->questions->matching($criteria);
}
In fact, with this getter in place, if I modify the order by clause to a nonexistent column, I do not get an unrecognized field exception as I would expect:
#ORM\OrderBy({"nonexistantcolumn" = "ASC"})
This leads me to believe that somehow the criteria filtering is overriding the annotation. Any ideas on how to resolve this would be much appreciated.
Besides filtering, Criteria can also sort a collection:
public function getQuestions()
{
// filter to never return soft deleted questions
$criteria = Criteria::create()
->where(Criteria::expr()->eq("active", true))
->orderBy(["num" => Criteria::ASC]);
return $this->questions->matching($criteria);
}
However, consider adding another unfiltered getter since this will prevent you from actually deleting inactive elements, or moving this logic to a repository method.

avoid duplication in many to many relationship - Doctrine DQL

noob alert;
I have Post and Tag Entities like this:
Post.php
/**
* Many posts can have many tags. This is the owning side.
* #ORM\ManytoMany(targetEntity="Tag", inversedBy="posts")
* #ORM\JoinTable(name="post_tag")
*/
protected $tags;
Tag.php
/**
* Many tags can belong to many different posts. This is the inverse side
* #ORM\ManytoMany(targetEntity="Post", mappedBy="tags")
*/
protected $posts;
Now, I want to query all posts with their tags.
For that, I'm using queryBuilder in my Repository and successfully able to retrieve results.
$query = $this->createQueryBuilder('P')
->select('P.title, T.name')
->leftJoin('P.tags', 'T')
->where('P.id = 1')
->getQuery()
->execute();
But as you can possibly imagine, this query fetches duplicates. So, if I had two tags for a post, there would be two similar posts inside the array.
Output
array(
array(title=>'Post One', name=>'php'),
array(title=>'Post One', name=>'python'),
)
Is there a doctrine way to turn these tags into an array collection and stuff this back into the final post array.

Elegant way to walk backward through OneToOne table entities with Doctrine

I have a very simply structured entity that contains a simple association
Database_Entity_Tenant
id (primary key)
parentId (id of the parent entry)
code (a simple identifier for the tenant, unique)
I defined parentId in my entity accordingly:
/**
* #Column(type="integer")
* #OneToOne(targetEntity="Tenant")
* #JoinColumn(name="parentTenantId", referencedColumnName="id")
* **/
protected $parentId;
This works fine - the generated database schema resembles my choices and its good.
Now i am writing my first method which basically has to return an array of all the tenants that are chained together, in reverse order (i use this for walking backward through a chain of tenants).
In order to do that i came up with the idea to use a while() loop.
$currentTenant = {DATABASE_ENTITY_TENANT}; // In my real code i fetch the entity object of the current tenant
$chain[] = $currentTenant;
$repository = Database::entityManager()->getRepository('Database_Entity_Tenant');
while(!$currentTenant->getParentId()){
$currentTenant = $repository->findOneBy(array(
'id' => $currentTenant->getParentId()
));
$chain[] = $currentTenant;
}
Any tenant that has no parent (such as the base tenant) will have no parent id (or null), so that would end the while loop.
Now all this may work, but it seems really rough to me. I am fairly new to Doctrine so i don't know much about it but i am sure there is some way to do this more elegantly.
QUESTION
Does Doctrine 2 provide me with any set of functions i could use to solve the above problem in a better way?
If not, then is there any other way to do this more elegantly?
If I'm not getting your problem wrong, you just need to find all the entries in your association table ordered by the parentId. In Doctrine2 you can do the following:
$currentTenant = {DATABASE_ENTITY_TENANT}; // assuming a valid entity
$repository = Database::entityManager()
->getRepository('Database_Entity_Tenant')
->createQueryBuilder('t')
->where('t.parentId IS NOT NULL')
->andWhere('t.parentId < :current') /* < or > */
->setParameter('current', $currentTenant->getParentId()->getId())
->orderBy('t.parentId', 'ASC') /* ASC or DESC, no array_reverse */
->getQuery()
->getResult();
/* At this point $repository contains all what you need because of Doctrine,
* but if you want a chain variable: */
$chain = array();
foreach ($repository as $tenant) {
$chain[] = $tenant->getCode(); // your tenant entity if your entity is mapped correctly
}
Hope this helps!

Doctrine - self-referencing entity - disable fetching of children

I have a very simple entity(WpmMenu) that holds menu items connected to one another in a self-referencing relationship (adjecent list it's called)?
so in my entity I have:
protected $id
protected $parent_id
protected $level
protected $name
with all the getters/setters the relationships are:
/**
* #ORM\OneToMany(targetEntity="WpmMenu", mappedBy="parent")
*/
protected $children;
/**
* #ORM\ManyToOne(targetEntity="WpmMenu", inversedBy="children", fetch="LAZY")
* #ORM\JoinColumn(name="parent_id", referencedColumnName="id", onUpdate="CASCADE", onDelete="CASCADE")
*/
protected $parent;
public function __construct() {
$this->children = new ArrayCollection();
}
And everything works fine. When I render the menu tree, I get the root element from the repository, get its children, and then loop through each child, get its children and do this recursively until I have rendered each item.
What happens (and for what I am seeking a solution)is this:
At the moment I have 5 level=1 items and each of these items have 3 level=2 items attached (and in the future I will be using level=3 items as well). To get all elements of my menu tree Doctrine executes:
1 query for the root element +
1 query to get the 5 children(level=1) of the root element +
5 queries to get the 3 children(level=2) of each of the level 1 items +
15 queries (5x3) to get the children(level=3) of each level 2 items
TOTAL: 22 queries
So, I need to find a solution for this and ideally I would like to have 1 query only.
So this is what I am trying to do:
In my entities repository(WpmMenuRepository) I use queryBuilder and get a flat array of all menu items ordered by level. Get the root element(WpmMenu) and add "manually" its children from the loaded array of elements. Then do this recursively on children. Doing this way I could have the same tree but with a single query.
So this is what I have:
WpmMenuRepository:
public function setupTree() {
$qb = $this->createQueryBuilder("res");
/** #var Array */
$res = $qb->select("res")->orderBy('res.level', 'DESC')->addOrderBy('res.name','DESC')->getQuery()->getResult();
/** #var WpmMenu */
$treeRoot = array_pop($res);
$treeRoot->setupTreeFromFlatCollection($res);
return($treeRoot);
}
and in my WpmMenu entity I have:
function setupTreeFromFlatCollection(Array $flattenedDoctrineCollection){
//ADDING IMMEDIATE CHILDREN
for ($i=count($flattenedDoctrineCollection)-1 ; $i>=0; $i--) {
/** #var WpmMenu */
$docRec = $flattenedDoctrineCollection[$i];
if (($docRec->getLevel()-1) == $this->getLevel()) {
if ($docRec->getParentId() == $this->getId()) {
$docRec->setParent($this);
$this->addChild($docRec);
array_splice($flattenedDoctrineCollection, $i, 1);
}
}
}
//CALLING CHILDREN RECURSIVELY TO ADD REST
foreach ($this->children as &$child) {
if ($child->getLevel() > 0) {
if (count($flattenedDoctrineCollection) > 0) {
$flattenedDoctrineCollection = $child->setupTreeFromFlatCollection($flattenedDoctrineCollection);
} else {
break;
}
}
}
return($flattenedDoctrineCollection);
}
And this is what happens:
Everything works out fine, BUT I end up with each menu items present twice. ;) Instead of 22 queries now I have 23. So I actually worsened the case.
What really happens, I think, is that even if I add the children added "manually", the WpmMenu entity is NOT considered in-sync with the database and as soon as I do the foreach loop on its children the loading is triggered in ORM loading and adding the same children that were added already "manually".
Q: Is there a way to block/disable this behaviour and tell these entities they they ARE in sync with the db so no additional querying is needed?
With immense relief (and a lots of learning about Doctrine Hydration and UnitOfWork) I found the answer to this question. And as with lots of things once you find the answer you realize that you can achieve this with a few lines of code. I am still testing this for unknown side-effects but it seems to be working correctly.
I had quite a lot of difficulties to identify what the problem was - once I did it was much easier to search for an answer.
So the problem is this: Since this is a self-referencing entity where the entire tree is loaded as a flat array of elements and then they are "fed manually" to the $children array of each element by the setupTreeFromFlatCollection method - when the getChildren() method is called on any of the entities in the tree (including the root element), Doctrine (NOT knowing about this 'manual' approach) sees the element as "NOT INITIALIZED" and so executes an SQL to fetch all its related children from the database.
So I dissected the ObjectHydrator class (\Doctrine\ORM\Internal\Hydration\ObjectHydrator) and I followed (sort of) the dehydration process and I got to a $reflFieldValue->setInitialized(true); #line:369 which is a method on the \Doctrine\ORM\PersistentCollection class setting the $initialized property on the class true/false. So I tried and IT WORKS!!!
Doing a ->setInitialized(true) on each of the entities returned by the getResult() method of the queryBuilder (using the HYDRATE_OBJECT === ObjectHydrator) and then calling ->getChildren() on the entities now do NOT trigger any further SQLs!!!
Integrating it in the code of WpmMenuRepository, it becomes:
public function setupTree() {
$qb = $this->createQueryBuilder("res");
/** #var $res Array */
$res = $qb->select("res")->orderBy('res.level', 'DESC')->addOrderBy('res.name','DESC')->getQuery()->getResult();
/** #var $prop ReflectionProperty */
$prop = $this->getClassMetadata()->reflFields["children"];
foreach($res as &$entity) {
$prop->getValue($entity)->setInitialized(true);//getValue will return a \Doctrine\ORM\PersistentCollection
}
/** #var $treeRoot WpmMenu */
$treeRoot = array_pop($res);
$treeRoot->setupTreeFromFlatCollection($res);
return($treeRoot);
}
And that's all!
Add the annotation to your association to enable eager loading. This should allow you to load the entire tree with only 1 query, and avoid having to reconstruct it from a flat array.
Example:
/**
* #ManyToMany(targetEntity="User", mappedBy="groups", fetch="EAGER")
*/
The annotation is this one but with the value changed
https://doctrine-orm.readthedocs.org/en/latest/tutorials/extra-lazy-associations.html?highlight=fetch
You can't solve this problem if using adjacent list. Been there, done that. The only way is to use nested-set and then you would be able to fetch everything you need in one single query.
I did that when I was using Doctrine1. In nested-set you have root, level, left and right columns which you can use to limit/expand fetched objects. It does require somewhat complex subqueries but it is doable.
D1 documentation for nested-set is pretty good, I suggest to check it and you will understand the idea better.
This is more like a completion and more cleaner solution, but is based on the accepted answer...
The only thing needed is a custom repository that is going to query the flat tree structure, and then, by iterating this array it will, first mark the children collection as initialized and then will hydratate it with the addChild setter present in the parent entity..
<?php
namespace Domain\Repositories;
use Doctrine\ORM\EntityRepository;
class PageRepository extends EntityRepository
{
public function getPageHierachyBySiteId($siteId)
{
$roots = [];
$flatStructure = $this->_em->createQuery('SELECT p FROM Domain\Page p WHERE p.site = :id ORDER BY p.order')->setParameter('id', $siteId)->getResult();
$prop = $this->getClassMetadata()->reflFields['children'];
foreach($flatStructure as &$entity) {
$prop->getValue($entity)->setInitialized(true); //getValue will return a \Doctrine\ORM\PersistentCollection
if ($entity->getParent() != null) {
$entity->getParent()->addChild($entity);
} else {
$roots[] = $entity;
}
}
return $roots;
}
}
edit: the getParent() method will not trigger additional queries as long as the relationship is made to the primary key, in my case, the $parent attribute is a direct relationship to the PK, so the UnitOfWork will return the cached entity and not query the database.. If your property doesn't relates by the PK, it WILL generate additional queries.

Categories