I am working on a Symfony 3.4 + Doctrine based project. The production database has grown quite large and I would like to be able to copy some of the data / entities to a second database which can be used as sandbox for running tests, evaluations, etc. on the data.
Adding a second database connection to the project was no problem. Second step was query the database schema / table structure form Doctrine to re-create the exact same tables for some entities in this second DB.
Now I would like to query entities from the production DB using the entity manager as usual and persist them to the second DB. Since the second DB holds only some of the data/entities/tables I cannot use a second entity manager here but have to insert the data manually.
I am looking for something like:
// Load entity from production DB via entity manager
$repo = $this->em->getRepository(SomeEntity::class);
$entity = $repo->findOneById('xy12');
// Pseudocode(!): Get SQL code to save the entity
$saveQuery = $repo->getSaveQueryForEntity(entity); <<< HOW TO DO THIS?
$saveSql = saveQuery->getSql();
// Run SQL on sandbox connection
$sandboxConnection = $doctrine->getConnection('sandbox');
$sandboxConnection->executeQuery($saveSql);
Of course I could create the INSERT query completely manually. However, this would be quite cumbersome and error prone. On the other hand creating the SQL code already build into Doctrine and all I need is a way to access/get this code to run it on a different connection?
Or is this approach completely wrong and there is a better way to get an entity from one DB to the other?
EDIT:
Dumping the complete database and importing it into a sandbox DB is not an option. The database holds data of many registered users and each user can decide if and when he wants to transfer some data to the sandbox. Copying a several GB large database with all user data to a sandbox because User 123 wants to run some tests on entities A and B is not very effective, is it?
I do not want to describe the complete internal logic of the project here, since this does not really help the question. So the question is how to copy / move a single entity to another database by getting the SQL from doctrine :-)
You said:
Since the second DB holds only some of the data/entities/tables I cannot use a second entity manager here but have to insert the data manually.
but you can still declare two different entity managers, that both map the same entity, but with different mapping options (maybe you don't want to map all fields with the other entity manager, for example).
You will need to have two distinct mappings that are bound to the same entity, so better go with separate YML files (instead of annotations). You can have something like:
config/orm/default/User.orm.yml // User mapping for the default EM
config/orm/other/User.orm.yml // User mapping for the other EM
Also, loading the entity with the default entity manager and persisting with the other will not work as expected. You will have to use merge() instead of persist(), since the entity will be managed by the default entity manager:
$user = $defaultEntityManager->getRepository(User::class)->find(1);
$otherEntityManager->merge($user);
$otherEntityManager->flush();
Given it's absolutely unsuitable to do it on a database level using mysqldump, I would probably enable two entity managers for the project and maintain exactly the same data schema for both of them. It gives an opportunity to persist similar objects when needed. When your user would select entities to copy on a web page, we can pass those ids to a handler to fetch entity from the main entity manager and sync it into a sandbox one. It should be pretty straightforward and less hacky than getting insert SQL queries from Doctrine. Here is a simple example to give you a starting point.
<?php
declare(strict_types=1);
namespace App\Sync;
use Doctrine\ORM\EntityManagerInterface;
class SandboxDbSyncHandler
{
/** #var EntityManagerInterface */
private EntityManagerInterface $em;
/** #var EntityManagerInterface */
private EntityManagerInterface $sandboxEm;
public function __construct(EntityManagerInterface $em, EntityManagerInterface $sandboxEm)
{
$this->em = $em;
$this->sandboxEm = $sandboxEm;
}
public function sync(string $class, array $ids): void
{
$repo = $this->em->getRepository($class);
$sandBoxRepo = $this->sandboxEm->getRepository($class);
$entities = $repo->findBy(['id' => $ids]);
foreach ($entities as $entity) {
if (!$entity instanceof UpdatableFromEntity) {
continue;
}
$sandBoxEntity = $sandBoxRepo->find($entity->getId());
if (!$sandBoxEntity) {
$sandBoxEntity = $class()::createFromEntity();
}
$sandBoxEntity->updateFromEntity($entity);
$this->sandboxEm->persist($entity);
}
$this->sandboxEm->flush();
}
}
<?php
declare(strict_types=1);
namespace App\Entity;
interface UpdatableFromEntity
{
public static function createFromEntity($entity);
public function updateFromEntity($entity): void;
}
<?php
declare(strict_types=1);
namespace App\Entity;
class SomeEntity implements UpdatableFromEntity
{
private string $id;
private ?string $name;
public function __construct(string $id)
{
$this->id = $id;
}
public function getId(): string
{
return $this->id;
}
public function getName(): ?string
{
return $this->name;
}
public function updateFromEntity($entity): void
{
if (!$entity instanceof SomeEntity::class) {
throw new \Exception('Cannot update an entity from the entity of a different type');
}
$this->name = $entity->getName();
}
public static function createFromEntity($entity)
{
if (!$entity instanceof SomeEntity::class) {
throw new \Exception('Cannot create an entity from the entity of a different type');
}
return new static($entity->getId());
}
}
As we can see in official documentation Doctrine implements entity listeners which are executed only when something happens on a specific entity.
However there is a different injection between lifecycle event listeners/subscribers and entity listeners. In fact for listeners/subscribers Doctrine injects only a LifecycleEventArgs object into defined callbacks
// Event listener/subscriber
public function postUpdate(LifecycleEventArgs $args)
{
$entity = $args->getEntity();
}
which gives you anyway access to the entity, but then in an entity listener the entity is also injected explicitly as first argument
// Entity listener
public function postUpdate(object $entity, LifecycleEventArgs $args)
{
$entity2 = $args->getEntity();
}
and it's still available in $args. This is also reported in documentation:
An entity listener method receives two arguments, the entity instance
and the lifecycle event.
But then what's exactly the difference between $entity and $args->getEntity() in an entity listener?
It's the same entity.
As LifecycleEventArgs is injected even in "generic" doctrine listener, you need to retrieve entity object. On the others side the LifecycleEventArgs has a lot of things you can retrieve in both situation.
For this rease, they've used the same object for both operations: it's pretty common and acceptable from my POV.
You can verify it yourself by doing something like
// Entity listener
public function postUpdate(object $entity, LifecycleEventArgs $args)
{
$entity2 = $args->getEntity();
dump(spl_obj_hash($entity) == spl_obj_hash($entity)); // you can echo this, or log, or VarDump, or whatever
}
Is there a way to force a doctrine event ( like preUpdate ) on a parent associated entity ?
So for example: I have a order entity with one-to-many orderItem entities.
Now, I want to do a bunch of checkup's and possible changes to the order entity or even one of it's orderItem entities ( where I need to access many other services) whenever any of the orderItems change. But the doctrine events do not fire on the order entity when one of its orderItem entities changes.
Note: this post entirely focuses on the particular case of the preUpdate event. It is possible to dispatch an event manually by using the event manager. The problem lies in the fact that simply triggering the preUpdate event of an entity is not enough to have its new state persisted to the database if the preUpdate method modified something.
There are multiple ways to do this but none of them are really straightforward. Considering only the case of the preUpdate event, I had quite a lot of trouble to find how to do this in a clean way as association updates are simply not built in a way to handle such cases as discussed in the Doctrine documentation.
Either way, if you want to do this, among the solutions I found, there were many that suggested to directly mess up with the UnitOfWork of Doctrine. This can be quite powerful but then you have to be careful about what you use and when you use it as Doctrine might not be able to actually dispatch the event you want in some cases discussed below.
Anyway, I ended up implementing something that makes use of a change of tracking policy for the parent entity. By doing so, the parent entity preUpdate event can be triggered if one of its properties is modified or if one of its "children" was modified.
Main concerns with the UnitOfWork
If you wish to use the UnitOfWork (that you can retrieve by using $args->getEntityManager()->getUnitOfWork() with any type of arguments of lifecycle events), you can use the public method scheduleForUpdate(object $entity). However, if you wish to use this method, you will need to call it before the commit order is computed inside of the unit of work. Moreover, if you have a preUpdate event associated to the entity you scheduled for update, it will raise an error if your entity has an empty change set (which is exactly the case we are dealing with when the main entity is not modified but one of its related entities is).
Thus calling $unitOfWork->scheduleForUpdate($myParentEntity), in a preUpdate of a child entity is not an option as explained in the documentation (performing calls to the UnitOfWork API is strongly discouraged as it does not work as it would outside of the flush operation).
It should be noted that $unitOfWork->scheduleExtraUpdate($parentEntity, array $changeset) can be used in that specific context but this method is marked as "INTERNAL". The following solutions avoid using it but it might be a good approach if you know what you are getting into.
Possible solutions
Note: I did not test the implementation of the wanted behaviour with the onFlush event but it was often presented as the most powerful approach. For the other two possibilities listed here, I tried them successfully with a OneToMany association.
In the following section, when I'm talking about a parent entity, I refer to the entity that has the OneToMany association while children entities are refering to the entities that have the ManyToOne association (thus, the children entities are the owning side of the association).
1. Using onFlush event
You can try to work your way out of this by using the onFlush event however, in that case you have to deal with the UnitOfWork internals as suggested in the documentation. In that case, you can't do it within an Entity listener (introduced in 2.4) as the onFlush event is not among the possible callbacks. Some examples based on what's given by the official doc can be found on the web. Here is a possible implementation: Update associated entities in doctrine.
The main drawback here is that you don't really trigger the preUpdate event of your entity, you just handle the behaviour you wanted somewhere else. It seemed a bit too heavy handed for me, so I searched for other solutions.
2. Using the UnitOfWork in preFlush event of the child entities
One way to actually trigger the preUpdate event of the parent entity, is to add another entity listener to the child entity and to use the UnitOfWork. As explained before, you can't simply do this in the preUpdate event of the child entity.
In order for the commit order to be properly computed, we need to call scheduleForUpdate and propertyChanged in the preFlush event of the child entity listener as shown below:
class ChildListener
{
public function preFlush(Child $child, PreFlushEventArgs $args)
{
$uow = $args->getEntityManager()->getUnitOfWork();
// Add an entry to the change set of the parent so that the PreUpdateEventArgs can be constructed without errors
$uow->propertyChanged($child->getParent(), 'children', 0, 1);
// Schedule for update the parent entity so that the preUpdate event can be triggered
$uow->scheduleForUpdate($child->getParent());
}
}
As you can see, we need to notify the UnitOfWork that a property has changed so that everything works properly. It looks a bit sloppy but it gets the work done.
The important part is that we mark the children property (the OneToMany association of the parent) as changed so that the change set of the parent entity is not empty. A few important notes about the internals at stake with this propertyChanged call:
The method expects a persistent field name (non-persistent ones will be ignored), any mapped field will do, even associations, that is why using children works here.
The change set that is modified consecutively to this call does not have any side effects here as it will be recomputed after the preUpdate event.
The main problem of this approach is that the parent entity is scheduled for update even if it is not needed. As there is no direct way to tell if the child entity has changed in its preFlush event (you could use the UnitOfWork but it would become a bit redundant with its internals), you will trigger the preUpdate event of the parent at every flush where a child entity is being managed.
Moreover, with this solution, Doctrine will begin a transaction and commit even if there are no queries performed (e.g. if nothing was modified at all, you will still find in the Symfony Profiler, two consecutives entries "START TRANSACTION" and "COMMIT" in the Doctrine logs).
3. Change the tracking policy of the parent and handle the behaviour explicitly
Since I've been messing with the internals of the UnitOfWork quite a bit, I stumbled upon the propertyChanged method (that was used in the previous solution) and noticed that it was part of the interface PropertyChangedListener. It happens that this is linked to a documented topic: the tracking policy. By default, you can just let Doctrine detect the changes but you can also change this policy and manage everything manually as explained here, in the documentation.
After reading about this, I eventually came up with the following solution that cleanly handles the wanted behaviour, the cost being that you have to do some extra work in your entities.
Thus, to have exactly what I desired, my parent entity follows the NOTIFY tracking policy and children notify the parent when one of their properties is modified. As described in the official documentation, you have to implement the NotifyPropertyChanged interface and then notify the listeners of properties changes (the UnitOfWork automatically adds itself to the listeners if it detects that one of the managed entities implements the interface). After that, if the annotation #ChangeTrackingPolicy is added, at commit times, Doctrine will rely on the change set that was built via propertyChanged calls and not on an automatic detection.
Here is how you would do it for a basic Parent entity:
namespace AppBundle\Entity;
use Doctrine\Common\NotifyPropertyChanged;
use Doctrine\Common\PropertyChangedListener;
/**
* ... other annotations ...
* #ORM\EntityListeners({"AppBundle\Listener\ParentListener"})
* #ORM\ChangeTrackingPolicy("NOTIFY")
*/
class Parent implements NotifyPropertyChanged
{
// Add the implementation satisfying the NotifyPropertyChanged interface
use \AppBundle\Doctrine\Traits\NotifyPropertyChangedTrait;
/* ... other properties ... */
/**
* #ORM\Column(name="basic_property", type="string")
*/
private $basicProperty;
/**
* #ORM\OneToMany(targetEntity="AppBundle\Entity\Child", mappedBy="parent", cascade={"persist", "remove"})
*/
private $children;
/**
* #ORM\Column(name="other_field", type="string")
*/
private $otherField;
public function __construct()
{
$this->children = new \Doctrine\Common\Collections\ArrayCollection();
}
public function notifyChildChanged()
{
$this->onPropertyChanged('children', 0, 1);
}
public function setBasicProperty($value)
{
if($this->basicProperty != $value)
{
$this->onPropertyChanged('basicProperty', $this->basicProperty, $value);
$this->basicProperty = $value;
}
}
public function addChild(Child $child)
{
$this->notifyChildChanged();
$this->children[] = $child;
$child->setParent($this);
return $this;
}
public function removeChild(Child $child)
{
$this->notifyChildChanged();
$this->children->removeElement($child);
}
/* ... other methods ... */
}
with the trait taken from the code given in the documentation:
namespace AppBundle\Doctrine\Traits;
use Doctrine\Common\PropertyChangedListener;
trait NotifyPropertyChangedTrait
{
private $listeners = [];
public function addPropertyChangedListener(PropertyChangedListener $listener)
{
$this->listeners[] = $listener;
}
/** Notifies listeners of a change. */
private function onPropertyChanged($propName, $oldValue, $newValue)
{
if ($this->listeners)
{
foreach ($this->listeners as $listener)
{
$listener->propertyChanged($this, $propName, $oldValue, $newValue);
}
}
}
}
and the following Child entity with the owning side of the association:
namespace AppBundle\Entity;
class Child
{
/* .. other properties .. */
/**
* #ORM\ManyToOne(targetEntity="AppBundle\Entity\Parent", inversedBy="children")
*/
private $parentEntity;
/**
* #ORM\Column(name="attribute", type="string")
*/
private $attribute;
public function setAttribute($attribute)
{
// Check if the parentEntity is not null to handle the case where the child entity is created before being attached to its parent
if($this->attribute != $attribute && $this->parentEntity)
{
$this->parentEntity->notifyChildChanged();
$this->attribute = $attribute;
}
}
/* ... other methods ... */
}
And there it is, you have everything fully working. If, your child entity is modified, you explicitly call notifyChildChanged that will then notify the UnitOfWork that children field has changed for the parent entity thus cleanly triggering the update process and the preUpdate event if one is specified.
Unlike the solution #2, the event will be triggered only if something has changed and you can control with precision why it should be marked as changed. For example, you could mark the children as changed if only a certain set of attributes is changed and ignore other changes as you have full control other what is eventually notified to the UnitOfWork.
Note:
With the NOTIFY tracking policy, apparently, preFlush events won't be triggered in the Parent entity listener (preFlush event being triggered in computeChangeSet which is simply not called for entities using this policy).
It is necessary to track every "normal" property to trigger updates if normal properties are changed. One solution to do this without having to modify all your setters is to use magic calls as shown below.
It is safe to set a children entry in the change set as it will be simply ignored when the update query is created since the parent entity is NOT the owning side of the association. (i.e. it does not have any foreign keys)
Use of magic calls to handle notifications easily
In my application, I added the following trait
namespace AppBundle\Utils\Traits;
trait MagicSettersTrait
{
/** Returns an array with the names of properties for which magic setters can be used */
abstract protected function getMagicSetters();
/** Override if needed in the class using this trait to perform actions before set operations */
private function _preSetCallback($property, $newValue) {}
/** Override if needed in the class using this trait to perform actions after set operations */
private function _postSetCallback($property, $newValue) {}
/** Returns true if the method name starts by "set" */
private function isSetterMethodCall($name)
{
return substr($name, 0, 3) == 'set';
}
/** Can be overriden by the class using this trait to allow other magic calls */
public function __call($name, array $args)
{
$this->handleSetterMethodCall($name, $args);
}
/**
* #param string $name Name of the method being called
* #param array $args Arguments passed to the method
* #throws BadMethodCallException if the setter is not handled or if the number of arguments is not 1
*/
private function handleSetterMethodCall($name, array $args)
{
$property = lcfirst(substr($name, 3));
if(!$this->isSetterMethodCall($name) || !in_array($property, $this->getMagicSetters()))
{
throw new \BadMethodCallException('Undefined method ' . $name . ' for class ' . get_class($this));
}
if(count($args) != 1)
{
throw new \BadMethodCallException('Method ' . $name . ' expects 1 argument (' . count($args) . ' given)');;
}
$this->_preSetCallback($property, $args[0]);
$this->$property = $args[0];
$this->_postSetCallback($property, $args[0]);
}
}
which I could then use in my entities. Here is an example of my Tag entity whose preUpdate event needed to be called when one of its aliases was modified:
/**
* #ORM\Table(name="tag")
* #ORM\EntityListeners({"AppBundle\Listener\Tag\TagListener"})
* #ORM\ChangeTrackingPolicy("NOTIFY")
*/
class Tag implements NotifyPropertyChanged
{
use \AppBundle\Doctrine\Traits\NotifyPropertyChangedTrait;
use \AppBundle\Utils\Traits\MagicSettersTrait;
/* ... attributes ... */
protected function getMagicSetters() { return ['slug', 'reviewed', 'translations']; }
/** Called before the actuel set operation in the magic setters */
public function _preSetCallback($property, $newValue)
{
if($this->$property != $newValue)
{
$this->onPropertyChanged($property, $this->$property, $newValue);
}
}
public function notifyAliasChanged()
{
$this->onPropertyChanged('aliases', 0, 1);
}
/* ... methods ... */
public function addAlias(\AppBundle\Entity\Tag\TagAlias $alias)
{
$this->notifyAliasChanged();
$this->aliases[] = $alias;
$alias->setTag($this);
return $this;
}
public function removeAlias(\AppBundle\Entity\Tag\TagAlias $alias)
{
$this->notifyAliasChanged();
$this->aliases->removeElement($alias);
}
}
I can then reuse the same trait in my "child" entity named TagAlias:
class TagAlias
{
use \AppBundle\Utils\Traits\MagicSettersTrait;
/* ... attributes ... */
public function getMagicSetters() { return ['alias', 'main', 'locale']; }
/** Called before the actuel set operation in the magic setters */
protected function _preSetCallback($property, $newValue)
{
if($this->$property != $newValue && $this->tag)
{
$this->tag->notifyAliasChanged();
}
}
/* ... methods ... */
}
Note: If you chose to do this, you might encounter errors when Forms are trying to hydrate your entities as magic calls are disabled by default. Simply add the following to your services.yml to enable magic calls. (taken from this discussion)
property_accessor:
class: %property_accessor.class%
arguments: [true]
A more pragmatic approach is to version your parent entity. A simple example of this would be a timestamp (e.g. updated_at) that is updated when the collection of child entities is modified. This assumes you update all the child entities through its parent.
I'm trying to work with doctrine event in a symfony project,
following the symfony doc I have this code
public function postPersist(LifecycleEventArgs $args)
{
$entity = $args->getEntity();
if (!$entity instanceof Rubrique) {
return;
}
$entityManager = $args->getEntityManager();
// do some stuff
}
The problem is that $entity is not the expected object Rubrique but an instance of Gedmo\Loggable\Entity\LogEntry maybe because Rubrique is Loggable. How can I access to my entity for manipulate it as I want ?
Thanks
This event listener is a "generic" one and not an doctrine entity listener
This means that the event is raised for each entity persisted: if you have a relation between Rubrique and LogEntry, than is possible that you're checking only for the "first" of them being "postPersisted".
If you need a specific listener only for that kind of entity, think about using doctrine entity listener (linked above).
Moreover remember that "generic" listener will listen (or will be subscribed) for events of every entity (so, basically, it could be invoked a lot of times) wheres doctrine entity listener not.
New guy at Symfony/Doctrine. So kindly guide me.
Requirement: To create a custom EntityManager which would override some of the methods like remove (instead of remove, i want to perform an update and modify a parameter like isValid in my class, so that the records are never deleted) and a find ( find a record which has a non zero isValid ) etc and use it instead of Doctrine's EntityManager.
I started reading through this thread:
Is there a way to specify Doctrine2 Entitymanager implementation class in Symfony2? and found the answer by user2563451 to be not so straightforward. I got lost when he talks about not to follow certain approaches (again no location of the files to be modified).
I have looked at the EntityManager.php and it specifically tells not to use extend the EntityManager class. Rather it asks to extend the EntityManagerDecorator. On looking at the EntityManagerDecorator, there are no methods available inside it (like create, persist, etc which I found in EntityManager) Does it mean I need to create new methods for each and every single Entity Manager functionality ?
Since there is no clear defined way to get this done, I am confused to get this thing started. Also Doctrine cookbook is of little use to me as it does not have any information to achieve this.
So any help regarding the extending of EntityManagerDecorator or EntityManager is appreciated.
Best if you can provide me step by step directions to achieve the same.
Thanks !
Edit 1: my requirement is to use my custom EntityManager instead of Doctrine's EntityManager (EM) and modify those 'remove' and 'find' methods as per my requirements. I am not sure whether I need to reuse the functionality provided by Doctrine's EM or write from scratch.
I think you may be confusing a Manager with a Repository.
An EntityManager is really nothing more than a Service you use to manage that specific or a collection of entities.
A repository extends \Doctrine\ORM\EntityRepository and is what tells Doctrine how to store your entity in the database.
You can use the combination of these two to achieve what you want.
For example. Let's take our entity Foo
class Foo
{
//...other properties
protected $isValid;
//...Getters and setters
}
We then have a manager for Foo.
class FooManager
{
protected $class;
protected $orm;
protected $repo;
public function __construct(ObjectManager $orm , $class)
{
$this->orm = $orm;
$this->repo = $orm->getRepository($class);
$metaData = $orm->getClassMetadata($class);
$this->class = $metaData->getName();
}
public function create()
{
$class = $this->getClass();
$foo = new $class;
return $foo;
}
public function findBy(array $criteria)
{
return $this->repo->findOneBy($criteria);
}
public function refreshFoo(Foo $foo)
{
$this->orm->refresh($foo);
}
public function updateFoo(Foo $foo, $flush = true)
{
$this->orm->persist($foo);
if($flush)
{
$this->orm->flush();
}
}
public function getClass()
{
return $this->class;
}
}
We have some basic functions for Creating and Updating our object. And now if you wanted to "remove" it without actually deleting it, you can add the following function in the Manager.
public function remove(Foo $foo)
{
$foo->setIsValid(false);
return $this->update($foo);
}
This way, we update the isValid fields to false and persist it to the database. And you'd use this like any service inside your controller.
class MyController extends Controller
{
public function someAction()
{
$fooManager = $this->get('my_foo_manager');
$newFoo = $fooManager->create();
//...
$fooManager->remove($newFoo);
}
}
So now we've got the remove part.
Next, we only want to find entities that isValid set to TRUE.
Honestly, the way I'd handle this is not even modify the find and instead in your controller
if(!$foo->getIsValid())
{
//Throw some kind of error. Or redirect to an error page.
}
But if you want to do it the other way. You can just make a repo.
use Doctrine\ORM\EntityRepository;
class FooRepository extends EntityRepository
{
public function find($id, $lockMode = LockMode::NONE, $lockVersion = null)
{
//Some custom doctrine query.
}
}
We override EntityRepository's native find() function with our own.
Finally we get all of this registered in the right places. For the manager you've got to make a service.
services:
my_foo_manager:
class: AppBundle\Manager\FooManager
arguments: [ "#doctrine.orm.entity_manager" , 'AppBundle\Entity\Foo']
And for the repository, you must specify the repositoryClass in the ORM definition of your entity.
AppBundle\Entity\Foo:
type: entity
repositoryClass: AppBundle\Entity\FooRepository
table: foos
id:
id:
type: integer
generator: {strategy: AUTO}
options: {unsigned: true}
fields:
isValid:
type: boolean
Knowing all of this you can now do some pretty cool things with Entities. I hope this helped. Good luck!
Regarding your use cases, you should instead use Doctrine Lifecycle Callbacks for the remove case and simple overide the find method in your entity repository.