I am importing users from an excel file. I have created entity manager:
$em = $this->getDoctrine()->getManager();
This is my loop:
...
for ($i=2; $i<=count($usersdatas); $i++) { // $userdatas: an array of users imported from excel
$nuser = new User();
$nuser->setEmail($usersdatas[$i]['A']); // email is unique in db
$nuser->setFname($usersdatas[$i]['B']);
$nuser->setLname($usersdatas[$i]['C']);
$nuser->addChannel($channel); //ManytoMany relation
$em->persist($nuser);
$em->flush();
}
...
Email field is unique in db and I do not want to check duplicate before flush (db validation)
I want to flush one by one for log reasons (not batch insert).
All my other codes and files (entity, services, configurations, ...) are correct and has no errors.
The first problem is if one of the rows has an email that exist in db (duplicate), the loop breaks by an exception from db.
I have solved the problem with this simple change:
...
try {
$em->persist($nuser);
$em->flush();
} catch (\Exception $e) {
...
}
...
The next problem is the "The EntityManager is closed" exception. If one email is duplicate, EntityManager ($em) will close automatically.
I have solved this by creating the $em before try/catch:
...
if (!$em->isOpen()) {
$em = $em->create(
$em->getConnection(), $em->getConfiguration());
}
...
Everything is OK. But I have a big problem with adding the channel to user:
...
$nuser->addChannel($channel); //ManytoMany relation
...
$channel is not a new one. But after closing and creating the EntityManager, the system thinks it is new on line persist (The $channel will successfully add if there is no errors):
ORMInvalidArgumentException {#1400 ▼
#message: "A new entity was found through the relationship 'My\UserBundle\Entity\User#channels' that was not configured to cascade persist operations for entity: My\PostBundle\Entity\Channel#000000002768057d0000000046371762. To solve this issue: Either explicitly call EntityManager#persist() on this unknown entity or configure cascade persist this association in the mapping for example #ManyToOne(..,cascade={"persist"}). If you cannot find out which entity causes the problem implement 'My\PostBundle\Entity\Channel#__toString()' to get a clue."
#code: 0
#file: "...\vendor\doctrine\orm\lib\Doctrine\ORM\ORMInvalidArgumentException.php"
#line: 91
-trace: array:13 [▶]
}
All my problems are from recreating EntityManager.
I think its a bug. Validating data in database side (Unique, Null, Foreign Keys, ...) is common. Closing the EntityManager after db errors means we can not communicate with db after any error.
I have 2 questions:
Is there any way to prevent EntityManager close?!
Can I use DBAL Doctrine to import my users? (users may have duplicated emails)
Exceptions are for exceptional situations. They're designed to let you clean up and then return a friendly error. They're not designed for validation.
How you're using them here is to say "Is it possible to add this user?" and if not, restart the database. That is, stop the entity manager manager from throwing exceptions, instead of trying to carry on after it has thrown an exception.
So if you've identified that the entity manager is throwing an exception and closing when you try to add a duplicate user, then check whether the user is a duplicate before trying to add them.
However, you said above that you don't want to use this approach. In that case, you will need to write your own SQL queries and can use DBAL (or just raw PDOs) and handle the SQL responses yourself.
Related
Versions:
PHP 8.1
Worked in Symfony 5.3 finding behavior in Symfony 5.4.
"doctrine/doctrine-fixtures-bundle": "^3.4"
"doctrine/doctrine-bundle": "^2.2"
"doctrine/orm": "^2.8"
General Problem:
Multiple Fixture classes causes errors with references from other fixture classes
Expectations from old 5.3 Project:
On the old project I am able to make tons of separate Fixtures classes and run them all with DependentFixturesInterface and use the references already created (persisted) to then create the relations needed for the other fixtures.
Example:
I create Users first and then Teams, for each Team these is a ManyToOne $createdUser column that relates to a User that creates it. But I can make a UserFixtures class and safe the references (as seen in symfony casts etc.) then runs the TeamFixtures class to use the references in UserFixtures for TeamFixtures (all standard understanding)
Behavior in new 5.4 project:
In the new project I am no way able to create multiple fixture classes. In the same exact example above when I try to create the same relationship I get the follow error
A new entity was found through the relationship 'App\Entity\Team#createdUser' that was not configured to cascade persist operations for entity: App\Entity\User#whateveruser. To solve this issue: Either explicitly call EntityManager#persist() on this unknown entity or configure cascade persist this association in the mapping for example #ManyToOne(..,cascade={"persist"}). If you cannot find out which entity causes the problem implement 'App\Entity\User#__toString()' to get a clue.
So then I listen to the exception and add cascade={"persist"} to the entity relation, run the fixtures again and I get the following error:
Duplicate entry *** for key ****
To me this means somehow it is not correctly persisting the Users in the first place or I am completely off as to how Doctrine works.
This is the loader in my main fixture class
public function loadData(): void
{
$this->generateMockIntervals();
// Users, Contacts
$this->setGroup(self::TEST, 3)->createMany('generateFakeUser', [$this->testPasswordHash, self::TEST_EMAIL_ITERATED]);
$this->setGroup(self::DUMMY, 500)->createMany('generateFakeUser', [$this->dummyPasswordHashed]);
$this->setGroup(self::teamGroupName(), 100)->createMany('generateTeams');
$configArray = array(
MemberInterface::MEMBERS => array(
self::GEN => [$this, 'generateUserMembers'],
self::RANGE => 20,
self::R => self::REF_TYPES[0],
),
);
if (!empty($configArray)) {
$this->groupForAll($configArray);
} else {
$this->pr("Class %s not configured yet", static::class);
}
}
The createMany function loops through each item and creates the new User references and saves it in the reference Repo (straight from SymfonyCasts).
groupAll does the same thing but loops through the references that is configured for each new reference key. I super enhanced symfony casts createMany function. If you have not seen it this is the function that EVERY entity is passed to.
protected function manageReference($entity, $groupName)
{
if (null === $entity) {
throw new \LogicException('Did you forget to return the entity object from your callback to BaseFixture::createMany()?');
}
$this->manager->persist($entity);
// store for usage later as App\Entity\ClassName_#COUNT#
$storage = sprintf('%s_%d', $groupName, $this->i);
$this->addReference($storage, $entity);
}
ONLY DIFFERENCE between 5.3 project and 5.4 project:
The only major difference I can see that is causing this problem is that in my old (5.3) project I had ALL Bidirectional variables built into the entity. In 5.4 I removed ALL/MOST of the Bidirectional relationships for my entities in theory to generate less queries.
Basically if I were to take out this part here
$this->setGroup(self::teamGroupName(), 100)->createMany('generateTeams');
$configArray = array(
MemberInterface::MEMBERS => array(
self::GEN => [$this, 'generateUserMembers'],
self::RANGE => 20,
self::R => self::REF_TYPES[0],
),
);
if (!empty($configArray)) {
$this->groupForAll($configArray);
} else {
$this->pr("Class %s not configured yet", static::class);
}
And put it into a new fixture I start getting the Duplicate Entry and "must persist" errors. But as you can see I am Persisting every single entity and I am flushing every 25 entities are Peristed
When I want to save 1 record to the database, everything works correctly.
When I want to save 2 records right after each other - an error occurs:
[Doctrine\ORM\ORMInvalidArgumentException]
A new entity was found through the relationship 'App\Entity\User#directory' that was not configured to cascade persist
operations for entity:. To solve this issue: Either explicitly call EntityManager#persist() on this unknown entity
or configure cascade persist this association in the mapping for example #ManyToOne(..,cascade={"persist"}). If you
cannot find out which entity causes the problem implement 'Main\Entity\Directory#__toString()' to get a clue.
I just started learning symfony and can't figure out what the problem is.
In my case: I create a new user who takes certain information from a directory that is already in my database.
For example, my new user is a cat lover, then the entry "loves cats" should pull up from the directory.
The text of the error says that I have to set up a connection and save a new record about the love of cats. But my reference book is already full, it does not need to be supplemented, just take information from there.
When I create one user, everything is fine. When I create two users who are supposed to take information from the directory and save the data, an error occurs.
Help me please.
I tried in different places to use method: $this->em->clear(). But it did not help.
class User:
ManyToOne(targetEntity="App\Entity\Directory")
JoinColumn(nullable=true)
private $directory;
public function createData($data) {
$this->setData($data);
$this->em->flush();
}
public function setData($data) {
$user = new User();
$directory = $this->em->getRepository(Directory::class)->findOneBy(['id' => $data['id']]);
$user->setDirectory($directory);
return $this;
}
You need to configure cascade operations for Directory entity which relates to user. Or you can do $em->persist($directory) before flushing
imagine I have some doctrine Entity and I can have some records of this entity in the database which I dont want to be deleted, but I want them to be visible.
In general I can have entities, for which I have default records, which must stay there - must not be deleted, but must be visible.
Or for example, I want to have special User account only for CRON operations. I want this account to be visible in list of users, but it must not be deleted - obviously.
I was searching and best what I get was SoftDeletable https://github.com/Atlantic18/DoctrineExtensions/blob/v2.4.x/doc/softdeleteable.md It prevents fyzical/real deletion from DB, but also makes it unvisible on the Front of the app. It is good approach - make a column in the Entity's respective table column - 1/0 flag - which will mark what can not be deleted. I would also like it this way because it can be used as a Trait in multiple Entities. I think this would be good candidate for another extension in the above Atlantic18/DoctrineExtensions extension. If you think this is good idea (Doctrine filter) what is the best steps to do it?
The question is, is this the only way? Do you have a better solution? What is common way to solve this?
EDIT:
1. So, we know, that we need additional column in a database - it is easy to make a trait for it to make it reusable
But
2. To not have any additional code in each repository, how to accomplish the logic of "if column is tru, prevent delete" with help of Annotation? Like it is in SoftDeletable example above.
Thank you in advance.
You could do this down at the database level. Just create a table called for example protected_users with foreign key to users and set the key to ON DELETE RESTRICT. Create a record in this table for every user you don't want to delete. That way any attempt to delete the record will fail both in Doctrine as well as on db level (on any manual intervention in db). No edit to users entity itself is needed and it's protected even without Doctrine. Of course, you can make an entity for that protected_users table.
You can also create a method on User entity like isProtected() which will just check if related ProtectedUser entity exists.
You should have a look at the doctrine events with Symfony:
Step1: I create a ProtectedInterface interface with one method:
public function isDeletable(): boolean
Step2: I create a ProtectionTrait trait which create a new property. This isDeletable property is annotated with #ORM/Column. The trait implements the isDeletable(). It only is a getter.
If my entity could have some undeletable data, I update the class. My class will now implement my DeleteProtectedInterface and use my ProtectionTrait.
Step3: I create an exception which will be thrown each time someone try to delete an undeletable entity.
Step4: Here is the tips: I create a listener like the softdeletable. In this listener, I add a condition test when my entity implements the ProtectedInterface, I call the getter isDeleteable():
final class ProtectedDeletableSubscriber implements EventSubscriber
{
public function onFlush(OnFlushEventArgs $onFlushEventArgs): void
{
$entityManager = $onFlushEventArgs->getEntityManager();
$unitOfWork = $entityManager->getUnitOfWork();
foreach ($unitOfWork->getScheduledEntityDeletions() as $entity) {
if ($entity instanceof ProtectedInterface && !$entity->isDeletable()) {
throw new EntityNotDeletableException();
}
}
}
}
I think that this code could be optimized, because it is called each time I delete an entity. On my application, users don't delete a lot of data. If you use the SoftDeletable component, you should replace it by a mix between this one and the original one to avoid a lot of test. As example, you could do this:
final class ProtectedSoftDeletableSubscriber implements EventSubscriber
{
public function onFlush(OnFlushEventArgs $onFlushEventArgs): void
{
$entityManager = $onFlushEventArgs->getEntityManager();
$unitOfWork = $entityManager->getUnitOfWork();
foreach ($unitOfWork->getScheduledEntityDeletions() as $entity) {
if ($entity instanceof ProtectedInterface && !$entity->isDeletable()) {
throw new EntityNotDeletableException();
}
if (!$entity instance SoftDeletableInterface) {
return
}
//paste the code of the softdeletable subscriber
}
}
}
Well the best way to achieve this is to have one more column in the database for example boolean canBeDeleted and set it to true if the record must not be deleted. Then in the delete method in your repository you can check if the record that is passed to be deleted can be deleted and throw exception or handle the situation by other way. You can add this field to a trait and add it to any entity with just one line.
Soft delete is when you want to mark a record as deleted but you want it to stay in the database.
I use doctrine ODM to work with MongoDB. I have documents to save which can duplicate time to time. I need only 1 copy of each event, so I use hashed uniq key to ensure event is only 1.
So I do several ->persist($document);
And when I do ->flush();
I'm getting an exception:
localhost:27017: E11000 duplicate key error index: dbname.event.$eventKey_1 dup key: { : "keyValue" }
And all data never persisted to MongoDB. So question is: is any way to persist uniq data and ignore existing without doing:
try {
->persist();
->flush();
} catch (\Exception $e) {}
for each document?
Thank you.
Updated:
thanks for your answers and your time, but I found exact solution :)
Mongo function insert has an option "ordered: "
https://docs.mongodb.org/manual/reference/method/db.collection.insert/
which allow continue insertion even after errors.
Doctrine use Pecl extension for mongo.
doctrine flush() use this method:
http://www.php.net/manual/en/mongocollection.batchinsert.php
which has option "continueOnError"
So if you do this way:
$documentManager->flush(null, ['continueOnError' => true]);
It will save all documents without errors and skip all with errors. Though it will throw "\MongoDuplicateKeyException". So all you need - catch this exception and handle or simply ignore it (depending on your needs).
Something like this :)
The native Doctrine methods do not support filtering unique values - you need to do this on your own.
To insert those data without any errors you have to do a few things, depending on your entity structure:
Find all existing entities with the unique keys you have
Find unique keys that are duplicated between the entities you are trying to persist
Replace the already existing entities with the entities you found
Persist and flush
There is absolutely no chance to do this without at least one additionally query. If you had the primary key of the existing entities, you could use those to get a reference object. But unfortunately, there is no support for getting references by unique keys according to the doctrine documentation:
http://doctrine-orm.readthedocs.org/en/latest/reference/limitations-and-known-issues.html#join-columns-with-non-primary-keys
It is not possible to use join columns pointing to non-primary keys. Doctrine will think these are the primary keys and create lazy-loading proxies with the data, which can lead to unexpected results. Doctrine can for performance reasons not validate the correctness of this settings at runtime but only through the Validate Schema command.
I was unable to get the ['continueOnError' => true] to work, so solved it differently:
$collection = $documentManager->getDocumentCollection(Content::class);
try {
$collection->insertMany($arrayData, ['ordered' => false]);
} catch (BulkWriteException $exception) {
// ignore duplicate key errors
if (false === strpos($exception->getMessage(), 'E11000 duplicate key error')) {
throw $exception;
}
}
You can just do: Search your existing id edit it and save..
With Doctrine + symfony:
$Document = $EntityManager->getRepository("ExpBundle:Document")->find(123);
$Document->setTitile("Doc");
$EntityManager->flush();
With Doctrine
$Document = $EntityManager->find('Document', 1234);
$Document->setTitle("edited");
$EntityManager->flush();
i'm creating my Doctrine record this way:
$user = new User();
$user->fromArray($_POST);
$user->save();
This throws an exception:
Uncaught exception 'Doctrine_Validator_Exception' with message 'Validation failed in class User 1 field had validation error: * 1 validator failed on dni (type)
Of course, the "dni" field is of type: integer, and the HTTP POST has all the values as strings. This is what makes the validation fail. The only way of passing the validation is doing this:
$_POST['dni'] = (int) $_POST['dni'];
But it "feels" wrong.
As a note, I'm working with Doctrine integrated into CodeIgniter. This didn't ever happened to me when worked with Symfony.
Many thanks.
This is old but unanswered so here it is:
$manager->setAttribute(Doctrine_Core::ATTR_VALIDATE, VALIDATE_ALL & ~VALIDATE_TYPES);
You can instruct Doctrine to validate all or no combination of: Types (your issue), Length, Constraints, All and None.
You can do it on the global level.
Probably also on the connection level.
And definetly per table but I could not find it on the doctrine orm documentation.
and you don't need to worry about having a wrong data type inserted in the database, the db won't let you and that's outside of doctrine's hand.