ULID is falsely stored as UUID in database (vanilla symfony app) - php

Yesterday I tried to do what Symfony shouted out in some blog post (https://symfony.com/blog/new-in-symfony-5-2-doctrine-types-for-uuid-and-ulid) but failed hard. I want to store ULID (format "TTTTTTTTTTRRRRRRRRRRRRRRRR") in the database because they are not only sorteable but also contain a timestamp which is just perfect for my application. But when I tell a property to be "type=ulid" then it is stored as UUID (format: "xxxxxxxx-xxxx-Mxxx-Nxxx-xxxxxxxxxxxx") in the database.
I debugged half day long and got so annoyed by this that I started from scratch and still the problem exists.
Where did I go wrong?
(skip to the ULID heading if you like, the following may look long but 50% of it is just basic stuff)
Symfony
The thing I do over and over again taken from https://symfony.com/doc/5.4/setup.html :
stat shyt # does not exist
composer create-project symfony/skeleton:5.4.* shyt
cd shyt; composer require webapp
Do you want to include Docker configuration from recipes? YES (default)
bin/console about shows Symfony Version 5.4.10 and PHP 7.4
ORM
Taken from https://symfony.com/doc/5.4/doctrine.html :
composer require symfony/orm-pack
composer require --dev symfony/maker-bundle
docker-compose up -d
ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network
So I add some lines to the docker-compose.override.yml file:
networks:
default:
ipam:
config:
- subnet: 10.1.2.1/24
services:
database:
# [...] doctrine/doctrine-bundle stuff [...]
networks:
default:
ipv4_address: 10.1.2.3
In ".env" set DATABASE_URL for the host "10.1.2.3"
bin/console doctrine:database:create (silly but as documented)
Could not create database "app" for connection named default
An exception occurred while executing a query: SQLSTATE[42P04]: Duplicate database: 7 ERROR: database "app" already exists
Well, yes. Docker already did this.
The make:entity is postponed until we have ULID capabilities.
ULID
We already lean into https://symfony.com/doc/5.4/components/uid.html (especially section ULIDs) :
composer require symfony/uid
bin/console make:entity Product
"someProperty" as "ulid" not nullable
Inspect the Product entity
Looks almost like in the documentation except it has an additional field (the primary key, an integer) and some getter/setter.
bin/console make:migration
Test the ULID entity
In between we use tests to programatically create a DB entry:
composer require phpunit to programatically create a database entry
bin/console --env=test doctrine:migrations:migrate
bin/console --env=test doctrine:database:create
The file "tests/FooTest.php" contains:
<?php
namespace App\Tests;
use App\Entity\Product;
use App\Repository\ProductRepository;
use Doctrine\ORM\EntityManager;
use Symfony\Bundle\FrameworkBundle\Test\KernelTestCase;
use Symfony\Component\Uid\Ulid;
class FooTest extends KernelTestCase
{
public function testFoo(): void
{
$product = new Product();
$product->setSomeProperty(new Ulid());
static::assertNotNull($product->getSomeProperty());
self::getContainer()->get(ProductRepository::class)
->add($product);
self::getContainer()->get('doctrine.orm.entity_manager')
->flush();
}
}
bin/console --env=test doctrine:query:sql 'TRUNCATE product' just to be sure
bin/phpunit
bin/console --env=test doctrine:query:sql 'SELECT * FROM product'
A UUID is shown instead of a ULID.
Shows ULID instead of UUID in the database
Using ULID as Primary Key
First clean up a bit then do the example shown in https://symfony.com/doc/5.4/components/uid.html#ulids :
rm migrations/* to start again
bin/console --env=test doctrine:database:drop --force
bin/console --env=test doctrine:database:create
bin/console doctrine:database:drop --force
bin/console --env=test doctrine:database:create
Edit "src/Entity/Product.php" to only contain the second ULID example from the documentation:
<?php
namespace App\Entity;
use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Uid\Ulid;
use App\Repository\ProductRepository;
/**
* #ORM\Entity(repositoryClass=ProductRepository::class)
*/
class Product
{
/**
* #ORM\Id
* #ORM\Column(type="ulid", unique=true)
* #ORM\GeneratedValue(strategy="CUSTOM")
* #ORM\CustomIdGenerator(class="doctrine.ulid_generator")
*/
private $id;
public function getId(): ?Ulid
{
return $this->id;
}
// ...
}
(example from documentation was missing the repository line)
bin/console make:migration
bin/console --env=test doctrine:migrations:migrate
Test is now a bit simpler:
public function testFoo(): void
{
self::getContainer()->get(ProductRepository::class)
->add(new Product());
self::getContainer()->get('doctrine.orm.entity_manager')
->flush();
}
bin/phpunit (Risky is ok)
bin/console --env=test doctrine:query:sql 'SELECT * FROM product'
Again those UUID instead of ULID
Database shows UUID instead of ULID

It's bit late but for anyone whom facing this issue, I too have the database showing UUID instead of ULID and it seems to be the expected behavior from doctrine since you can use UUID/ULID interchangeably meaning even if you have UUID stored in the database but your entity is mapped on ULID you will have an ULID when retrieving the object from the database, also you can retrieve your same object by using ULID or UUID.
so for instance I have a User entity having ULID for its identifier so the stored object will have uuid as below :
018477e6-eebc-164c-12e3-22ca8f1a88f3 myemail#mail.com []
if I retrieve my user with that UUID I'll get :
App\Entity\User {#430 ▼
-id: "01GHVYDVNW2S615RS2SA7HN27K"
-email: "myemail#mail.com"
-roles: []
#createdAt: DateTime #1668458933 {#423 ▶}
#updatedAt: DateTime #1668458998 {#428 ▶}
-password: "$2y$13$/2i9Ovc2lCQBRfSVgsnmoul1FhF.Kyki3irF6GQvrMrjacQX6ees6"
-isVerified: true
}
now if you use that ULID to retrieve your user it will work too !
going further if you inspect that UUID you'll see that the returned object has that uuid converted to base 32 :
bash-5.1$ bin/console uuid:inspect 018477e6-eebc-164c-12e3-22ca8f1a88f3
----------------------- --------------------------------------
Label Value
----------------------- --------------------------------------
Version 1
toRfc4122 (canonical) 018477e6-eebc-164c-12e3-22ca8f1a88f3
toBase58 1BsMRvKcgozP4Kw2m4Fb1C
toBase32 01GHVYDVNW2S615RS2SA7HN27K
----------------------- --------------------------------------
and finally you can get that stored uuid by converting it to refc4122 as you can see below:
bin/console ulid:inspect 01GHVYDVNW2S615RS2SA7HN27K
---------------------- --------------------------------------
Label Value
---------------------- --------------------------------------
toBase32 (canonical) 01GHVYDVNW2S615RS2SA7HN27K
toBase58 1BsMRvKcgozP4Kw2m4Fb1C
toRfc4122 018477e6-eebc-164c-12e3-22ca8f1a88f3
---------------------- --------------------------------------
Time 2022-11-14 20:48:53.948 UTC
---------------------- --------------------------------------
I'm not sure why doctrine doesn't just store the ULID but its current behavior is not blocking you from using ULID in your project.
Hope this help !

I'm trying out the ULID as well and followed the same documentation (https://symfony.com/blog/new-in-symfony-5-2-doctrine-types-for-uuid-and-ulid). I noticed you did one thing different in your code compared to the documentation.
In the entity change:
* #ORM\CustomIdGenerator(class="doctrine.ulid_generator")
to
* #ORM\CustomIdGenerator(class=UlidGenerator::class)
And add the following
use Symfony\Bridge\Doctrine\IdGenerator\UlidGenerator;
This should the make the ULID work.
The dump of this would be something like this (sorry this is a dump in the console as I was testing the ulid, but I hope it gives you the information you need):
^ Symfony\Component\Uid\Ulid^ {#842
#uid: "01G7MCSCANWA2XXZ1NT1TZV6KQ"
toBase58: "1BoCbKTaPcpbAuQVieEW54"
toRfc4122: "0181e8cc-b155-e285-defc-35d075fd9a77"
time: "2022-07-10 15:48:57.813 UTC"
}
In the database it looks as followed:
Ulid in mariaDB
I hope this helps.

Related

Symfony 4.4 Upgrade - DoctrineUpdateCommand no longer called from console command

I am upgrading our Symfony application from version 3.4 to version 4.4, which includes an upgrade of Doctrine from 1.12 to 2.3. I had previously written a class that modified the results to the doctrine:schema:update command, which worked great, but appears not to be working now. The class is below.
To modify doctrine:schema:update, I created a class called DoctrineUpdateCommand, which extended \Doctrine\ORM\Tools\Console\Command\SchemaTool\UpdateCommand, and placed it in the Command folder of the bundle. This was all that was needed. I referenced this answer to figure out how to do it: How to set up entity (doctrine) for database view in Symfony 2.
We need to override the doctrine:schema:update command because one of our entities refers to a MySQL view instead of a MySQL table. Further, the entity is referenced as both a stand alone entity, and as a many-to-many join. The override class caused the entity and join to be ignored. It also added a sql statement to create the view.
After the upgrade, if I run php console doctrine:schema:update --dump-sql, I get these results:
19:08:16 CRITICAL [console] Error thrown while running command "doctrine:schema:update --dump-sql". Message: "The table with name 'nest_qa.assignedrole_privilegerole' already exists." ["exception" => Doctrine\DBAL\Schema\SchemaException^ { …},"command" => "doctrine:schema:update --dump-sql","message" => "The table with name 'nest_qa.assignedrole_privilegerole' already exists."]
In SchemaException.php line 112:
The table with name 'nest_qa.assignedrole_privilegerole' already exists.
I am fairly certain that the extension of the doctrine command is no longer being called, and it's using the default command instead, but I can't figure out how to change that. Any help would be appreciated.
Here is the original class:
<?php
namespace ApiBundle\Command;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;
use Symfony\Component\Console\Style\SymfonyStyle;
use Doctrine\ORM\Tools\SchemaTool;
class DoctrineUpdateCommand extends \Doctrine\ORM\Tools\Console\Command\SchemaTool\UpdateCommand{
protected $ignoredEntities = array(
'ApiBundle\Entity\AssignedrolePrivilegerole',
);
protected $ignoreAssociationMappings = array(
'ApiBundle\Entity\Privilegerole' => 'assignedroles',
'ApiBundle\Entity\Assignedrole' => 'privilegeroles',
);
protected function executeSchemaCommand(InputInterface $input, OutputInterface $output, SchemaTool $schemaTool, array $metadatas, SymfonyStyle $ui) {
/** #var $metadata \Doctrine\ORM\Mapping\ClassMetadata */
$newMetadatas = array();
foreach ($metadatas as $metadata) {
if (array_key_exists($metadata->getName(), $this->ignoreAssociationMappings)) {
if(array_key_exists($this->ignoreAssociationMappings[$metadata->getName()], $metadata->getAssociationMappings())){
unset($metadata->associationMappings[$this->ignoreAssociationMappings[$metadata->getName()]]);
}
}
//If metadata element is not in the ignore array, add it to the new metadata array
if (!in_array($metadata->getName(), $this->ignoredEntities)){
array_push($newMetadatas, $metadata);
}
}
parent::executeSchemaCommand($input, $output, $schemaTool, $newMetadatas, $ui);
$output->writeln("------Create view for assignedrole_privilegerole");
$output->writeln("CREATE VIEW `assignedrole_privilegerole` AS select `Assignedrole`.`id` AS `assignedrole_id`,`Privilegerole`.`id` AS `privilegerole_id` from (`Assignedrole` join `Privilegerole`) where ((`Assignedrole`.`role_id` = `Privilegerole`.`role_id`) and ((`Assignedrole`.`unit_id` = `Privilegerole`.`unit_id`) or `Privilegerole`.`unit_id` in (select `Unittree`.`ancestor_id` from `Unittree` where (`Unittree`.`descendant_id` = `Assignedrole`.`unit_id`)))) ;");
}
}
Console commands in symfony <4.x were registered by scanning Command folder inside a bundle. Since bundles are obsolete in symfony 4+, you have to define commands in your services definition by tagging the command class, or use DI autoconfiguration.
Option 1: explicitly add console.command tag to the service definition:
services:
ApiBundle\Command\DoctrineUpdateCommand:
tags:
- { name: twig.extension }
Option 2: use DI autoconfiguration:
services:
ApiBundle\Command\DoctrineUpdateCommand:
autoconfigure: true
After your class is registered as a console command, it must override the default one.
See symfony docs for more: Console Command

Doctrine Migrations with Symfony 3 addIndex not found. Why?

I've inherited a Symfony 3 project that had no migrations.
Following the documentation, I added Doctrine Migrations Bundle to Symfony.
$ composer require doctrine/doctrine-migrations-bundle "^1.0"
Added the configuration to config.yml
doctrine_migrations:
dir_name: "%kernel.root_dir%/DoctrineMigrations"
namespace: Application\Migrations
table_name: migration_versions
name: Application Migrations
Then added the loader to appKernel.php
new Doctrine\Bundle\MigrationsBundle\DoctrineMigrationsBundle(),
The console now shows doctrine:migrations:* as available features, so I then generated my migration file
php bin/console doctrine:migrations:generate
Generated new migration class to "/path/to/project/app/DoctrineMigrations/Version20100621140655.php"
Then continued following along with the Doctrine migrations documentation:
public function up(Schema $schema)
{
// alter my table
$options = ['fields' => ['type_id']];
$schema->addIndex('advertiser', 'idx_type_id', $options);
...
}
Now, when I run the migration:
Migrating up to 20170714165306 from 20170714155549
++ migrating 20170714165306
[Symfony\Component\Debug\Exception\UndefinedMethodException]
Attempted to call an undefined method named "addIndex" of class "ProxyManagerGeneratedProxy\__PM__\Doctrine\DBAL\Schema\Schema\Generatedaca9b50393335f8354881e485c485329".
Now when I got that error, I searched on that and found ocramius/ProxyManager. I installed that with
composer require ocramius/proxy-manager "^2.0"
Still got the same error.
Granted, in most of the Doctrine Migrations documentation, I see mainly addSql( 'ALTER TABLE ...') type statements, but I want to use the wrapper methods addIndex, removeIndex, etc.
Can someone explain how to get these features working?
I figured out my problem. The problem is solved by doing the following in the migration file:
public function up(Schema $schema)
{
// alter my table
$table = $schema->getTable('advertiser');
$table->addIndex(['type_id'], 'idx_type_id');
...
}
The basic idea is that you need to grab a reference to the table, and then you can addIndex to the specific column on the table. Contrary to the Doctrine docs, you can't addIndex on a $schema reference passing the table name within a Symfony2/3 migration.

Symfony2 doctrine:schema:update keeps updating

I've just added a new attribute in my model:
/**
* #ORM\Column(name="has_donation", type="boolean", options ={"default":false})
*/
private $hasDonation;
And then I tried running:
php app/console doctrine:schema:update --force
which gave me the following result:
Updating database schema...
Database schema updated successfully! "1" queries were executed
The problem is that each time I run the schema:update command, I get the same result - without making any changes to my entities. Any idea why? Thanks!
Update
Here's the out for php app/console doctrine:schema:update --dump-sql:
ALTER TABLE Clinic CHANGE has_donation has_donation TINYINT(1) DEFAULT '0' NOT NULL;
I can assume that this issue arises because of usage of illegal or irrelevant mapping information.
In case of the question author, the option "default":false was illegal.
I had same issue with code:
/**
* #ORM\Column(
* type="boolean",
* nullable=false,
* options={"unsigned":true, "comment":"Is file uploaded manually?"}
* )
*/
protected $manual = false;
In my case the option "unsigned":true was irrelevant. After removing this option, I got the code working.
First, try deleting cache.
Then, try php app/console doctrine:schema:update --dump-sql to see what updated.
These options with "default": false differences are not recorded sometimes. At least on PostgreSQL we use.
You might need them to run them manually.
Tip from our workflow:
1) we setup entities
2) then run php app/console doctrine:schema:update --force
3) then run migrations that add there details that are not transformed to database

Is it possible to load doctrine fixtures without deleting database

I have defined some fixtures in doctrine.
When i try to run using this
php app/console doctrine:fixtures:load then it asks me to purge the database.
is it possible to load it without purging database.
I remeber Django has fixtures which can be loaded in separate tables without purging existing database
Use the --append option
php app/console help doctrine:fixtures:load
Usage:
doctrine:fixtures:load [--fixtures[="..."]] [--append] [--em="..."] [--purge-with-truncate]
Options:
--fixtures The directory or file to load data fixtures from. (multiple values allowed)
--append Append the data fixtures instead of deleting all data from the database first.
--em The entity manager to use for this command.
--purge-with-truncate Purge data by using a database-level TRUNCATE statement
DoctrineFixtures are nice for the first init of an empty database or in development - but not in production.
Take a look at DoctrineMigrations and symfonys DcotrineMigrationsBundle, this is a good and safe way to migrate your database in production.
It is possible to update previously added data to DB (which was loaded by running app/console doctrine:fixtures:load). I used EntityManager->createQuery for that.
But I still wonder if there is or ever will be an opportunity to do it by simply running an app/console command. Something like app/console doctrine:schema:update --force, but applied to the data itself.
For now I had to write a ton of code to get my data updated and appended properly. For example to make --append work I had to write the following:
class LoadCategoryData implements FixtureInterface
{
/**
* {#inheritDoc}
*/
public function load(ObjectManager $em)
{
//a category has name and shortcut name (st)
$categories = array (
[0]=> array('name' => 'foo', 'st' = 'bar'),
...
);
foreach ($categories as $cat) {
$query = $em->createQuery(
"SELECT c
FROM MyBundle:Category c
WHERE c.st = ?1"
)->setParameter(1, $cat['st'])->getResult();
if (count($query) == 0) {
$c = new Category();
$c->setName($cat['name']);
$c->setSt($cat['st']);
$em->persist($c);
}
$em->flush();
}
}
}

Doctrine2 Duplicate key name error

I'm trying to use doctrine2 migration and schema update with symfony2's console. All my entities are being generated with no problems via:
php app/console doctrine:generate:entities myProject
Also, I am able to create my database and build my schema from scratch with no problems given the following commands:
php app/console doctrine:database:create
php app/console doctrine:schema:create
Whenever I modify, remove, or add in another entity, I always regenerate my entities with:
php app/console doctrine:generate:entities myProject
The problem however, is whenever I want to update my schema I always get the following error:
[Doctrine\DBAL\DBALException]
An exception occurred while executing 'CREATE INDEX IDX_F7129A80A76ED395 ON
user_user (user_id)':
SQLSTATE[42000]: Syntax error or access violation: 1061 Duplicate key name
'IDX_F7129A80A76ED395'
[PDOException]
SQLSTATE[42000]: Syntax error or access violation: 1061 Duplicate key name
'IDX_F7129A80A76ED395'
I have tried doing:
php app/console doctrine:schema:update --force
or
php app/console doctrine:migrations:diff
php app/console doctrine:migrations:migrate
But I always get the same error. What am I doing wrong? Is there any way for me to update my database without having to destroy and build a whole new one every time? I have over 25 entities which consist of large files with many associative mappings (OneToOne, OneToMany, ManyToMany,.. etc) which seems to be causing problems. Any help would be great! Thanks!
On a side note I have a feeling that this piece of code might be causing the problem:
/**
* #ORM\ ManyToMany(targetEntity="User", inversedBy="myFriends")
*/
protected $friendsWithMe;
/**
* #ORM\ ManyToMany(targetEntity="User", inversedBy="friendsWithMe")
* #ORM\ JoinTable(name="friends",
* joinColumns={#ORM\ JoinColumn(name="user_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\ JoinColumn(name="friend_user_id", referencedColumnName="id")}
* )
**/
protected $myFriends;
In my user entity, I wanted to build a self-referencing many to many relationship. This is how I'm building out a friends list. Maybe there is something wrong with my logic, but I took this directly off from the doctrine2 documentation on associative mapping.
Not sure it is the cause of the Exception, but there is an error in your ORM mapping. $myFriends is the owning side, $friendsWithMe is the inverse side, so inversedBy attribute should be only present on $friendsWithMe.
Try to rewrite $myFriendswith mappedBy attribute :
/**
* #ORM\ ManyToMany(targetEntity="User", mappedBy="friendsWithMe")
* #ORM\ JoinTable(name="friends",
* joinColumns={#ORM\ JoinColumn(name="user_id", referencedColumnName="id")},
* inverseJoinColumns={#ORM\ JoinColumn(name="friend_user_id", referencedColumnName="id")}
* )
**/
protected $myFriends;

Categories