Codeception & Symfony - run Doctrine migrations before tests - php

I have a Symfony 4 application and Doctrine with Doctrine migrations. I'm introducing Codeception for running API tests, and need to run migrations before the tests run. Since I'm using the Doctrine2 module I don't really want to be also including the DB module as it's not needed for the tests and would require configuring the test database in two different locations.
I am using the Symfony module currently, and I noticed that the Laravel module has a run_database_migrations configuration option.
What is the best way to handle running the Doctrine migrations command in a Symfony app prior to the tests? (bin/console doctrine:migrations:migrate -n is the specific command).
Edit I've got a solution that, although it works, is nowhere near ideal. By using Codeception Customisation I've created the following extension that basically manually execs the underlying Symfony commands.
class DatabaseMigrationExtension extends Extension
{
public static $events = [
Events::SUITE_BEFORE => 'beforeSuite',
];
public function beforeSuite(SuiteEvent $e)
{
echo(exec('bin/console doctrine:database:drop --force') . PHP_EOL);
echo(exec('bin/console doctrine:database:create') . PHP_EOL);
echo(exec('bin/console doctrine:migrations:migrate -n') . PHP_EOL);
}
}
Edit 2 The goal of this is basically to replicate similar functionality to what the Codeception DB module does, which allows you to provide an SQL dump of a database that it automatically uses in the tests, but instead use Doctrine migrations to handle the DB. - https://codeception.com/docs/modules/Db#sql-data-dump

I spent a while trying a couple of different ways to achieve this. I initially used RunProcess however this seemed to cause sporadic issues with the DB being deleted and not recreated, despite using the sleep configuration. I ended up just updating the existing extension to use the CLI module instead, and it works as desired (without having to create scripts or run multiple commands) and without having to use exec.
Final extension;
class DatabaseMigrationExtension extends Extension
{
public static $events = [
Events::SUITE_BEFORE => 'beforeSuite',
];
public function beforeSuite()
{
try {
/** #var \Codeception\Module\Cli $cli */
$cli = $this->getModule('Cli');
$this->writeln('Recreating the DB...');
$cli->runShellCommand('bin/console doctrine:database:drop --if-exists --force');
$cli->seeResultCodeIs(0);
$cli->runShellCommand('bin/console doctrine:database:create');
$cli->seeResultCodeIs(0);
$this->writeln('Running Doctrine Migrations...');
$cli->runShellCommand('bin/console doctrine:migrations:migrate --no-interaction');
$cli->seeResultCodeIs(0);
$this->writeln('Test database recreated');
} catch (\Exception $e) {
$this->writeln(
sprintf(
'An error occurred whilst rebuilding the test database: %s',
$e->getMessage()
)
);
}
}
}
and registered;
// codeception.yml
extensions:
enabled:
- \DatabaseMigrationExtension
Output (-vv or higher also displays the output of the DB & Migration commands);

I always create a bash script to do this, or a Makefile.
bash command
My ./runtests.sh scripts contains
#!/bin/bash
./bin/console command:for:migrations
./bin/phpunit
Makefile
Same with Makefile
.FOO: testsuite
testsuite:
./runtests.sh
or
.FOO: testsuite
testsuite:
./bin/console command:for:migrations
./bin/phpunit
why I prefer Makefile
Recently I added this script in my .bash_profile that allow me to autocomplete from bash all target made in makefile (very easy because you dont need anymore to remember all commands, but just make and tab).
complete -W "`grep -oE '^[a-zA-Z0-9_.-]+:([^=]|$)' Makefile | sed 's/[^a-zA-Z0-9_.-]*$//'`" make
Thus, .. you can create target like:
runtests
runtests_with_fixtures
migrations
runtests_with_migrations
...
and so on
My suggestion is to create your custom and easy way to run commands.
Here a way to run all or just one command usgin make
.FOO: dropforce
dropforce:
bin/console doctrine:database:drop --force
.FOO: dbcreate
dbcreate:
bin/console doctrine:database:create
.FOO: migrate
migrate:
bin/console doctrine:migrations:migrate
.FOO: suite
suite: dropforce dbcreate migrate

With Codeception 4 you can do it without Cli module:
$symfony = $this->getModule('Symfony');
$symfony->runSymfonyConsoleCommand('doctrine:database:drop',['--if-exists'=>true, '--force'=>true]);
$symfony->runSymfonyConsoleCommand('doctrine:database:create');
$symfony->runSymfonyConsoleCommand('doctrine:migrations:migrate', ['--no-interaction'=>true]);

Related

Laravel - DB Seeder command not working as expected

I've created a custom command functionality for my project, but there is something strange happening at the moment.
First i thought it might had something to do with the composer dump-autoload but after adding the exec command it still didn't work.
I do have the
seeds as autoload in my composer.json
$this->className = "MyDbSeeder"
$this->call('make:seeder', ['name' => $this->className]);
exec("composer dump-autoload");
sleep(5);
$this->call('db:seed', ['--class' => $this->className]);
/* Response */
// -- ReflectionException : Class MyDbSeeder does not exist --
/* But when i'm running the exactly same command but not through the `$this->call()` function it works perfectly fine. */
exec("php artisan db:seed --class {$this->className}", $output);
/* Response */
// -- "Database seeding completed successfully." --
Of course it does work with the exec command, but i don't feel like this is the solution that i need to deliver.
Anyone knows why the regular laravel command ain't working?

Symfony 4 - cannot find all migration versions

I am trying to run migrations on a Symfony 4 application with a total of 271 migrations. However, when doing so it skips the first 41 migrations (the first is Version20180921083101).
>>> php bin/console doctrine:migrations:status
== Configuration
>> Name: Application Migrations
>> Database Driver: pdo_mysql
>> Database Host: db
>> Database Name: test
>> Configuration Source: manually configured
>> Version Table Name: migration_versions
>> Version Column Name: version
>> Migrations Namespace: DoctrineMigrations
>> Migrations Directory: /var/www/src/Migrations
>> Previous Version: Already at first version
>> Current Version: 0
>> Next Version: 2018-11-14 06:38:03 (20181114063803)
>> Latest Version: 2020-01-27 05:06:49 (20200127050649)
>> Executed Migrations: 0
>> Executed Unavailable Migrations: 0
>> Available Migrations: 230
>> New Migrations: 230
I am been trying to update the schema, clear cache, and dropped/recreated the database, but without success. I have also tried to execute only the first one by running the following command:
>>> php bin/console doctrine:migrations:migrate Version20180921083101
Application Migrations
Unknown version: Version20180921083101
It seems that those versions prior to Version20181114063803 cannot be recognized.
I have been struggling with this issue for a while now and running out of things to try so any help or pointers would be highly appreciated.
Thanks!
For me the problem was that i was running
php bin/console doctrine:migrations:migrate 'DoctrineMigrations\Version20210403042222'
Instead of
php bin/console doctrine:migrations:migrate DoctrineMigrations\Version20210403042222
No single quotes, although the example in the docs uses single quotes.
So for the initial post, i think you just need to add the namespace without any quotes:
php bin/console doctrine:migrations:migrate DoctrineMigrations/Version20180921083101
If you use Symfony v.4 and want to execute one migration, you should use the next command
php bin/console doctrine:migrations:execute --up 20180921083101
You should simply pass the timestamp as argument, as example:
>>> php bin/console doctrine:migrations:migrate 20180921083101
Check also the doc here
I got the same problem after an update of the doctrine_migration bundle. In my case the Error wasn't really accurate, in fact it was more a problem of "it wasn't recognized as a valid migration" than a real not found issue so I put it there in case someone encounter the same situation.
Adding the following missing function solved my issue:
public function getDescription() : string
{
return 'Your description';
}
Problem might come from migrations executed from different branches before everything was merged together. If my memory is correct Symfony saves the last migration executed in your current database, if migrations comes from other branches with previous timestamp, they will not be executed with the doctrine:migrations:migrate command.

Doctrine commands not showing up in Symfony 3.4 console even though doctrine/doctrine-bundle is installed

I have an old Symfony 3.4 app (https://github.com/opencfp/opencfp) that needs to have Doctrine added to it so I can replace an existing auth/acl solution with Symfony Guard and then get moving towards upgrading towards Symfony 5. I've installed doctrine/doctrine-bundle and can see that the commands are in the vendor directory but when I run bin/console none of the doctrine commands show up.
Here's what I found when I searched my vendor directory for Doctrine console commands.
doctrine/doctrine-bundle/Resources/config/dbal.xml
87: <tag name="console.command" command="doctrine:database:create" />
93: <tag name="console.command" command="doctrine:database:drop" />
97: <tag name="console.command" command="doctrine:database:import" />
doctrine/doctrine-bundle/Command/Proxy/ImportDoctrineCommand.php
23: ->setName('doctrine:database:import')
doctrine/doctrine-bundle/Command/DropDatabaseDoctrineCommand.php
29: ->setName('doctrine:database:drop')
doctrine/doctrine-bundle/Command/CreateDatabaseDoctrineCommand.php
25: ->setName('doctrine:database:create')
When I run bin/console I don't see any of the commands in the doctrine namespace
Symfony 3.4.35 (kernel: OpenCFP, env: development, debug: true)
Usage:
command [options] [arguments]
Options:
-h, --help Display this help message
-q, --quiet Do not output any message
-V, --version Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n, --no-interaction Do not ask any interactive question
-e, --env=ENV The Environment name. [default: "development"]
--no-debug Switches off debug mode.
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
Available commands:
about Displays information about the current project
help Displays help for a command
list Lists commands
assets
assets:install Installs bundles web assets under a public directory
cache
cache:clear Clears the cache
cache:pool:clear Clears cache pools
cache:pool:prune Prunes cache pools
cache:warmup Warms up an empty cache
config
config:dump-reference Dumps the default configuration for an extension
debug
debug:autowiring Lists classes/interfaces you can use for autowiring
debug:config Dumps the current configuration for an extension
debug:container Displays current services for an application
debug:event-dispatcher Displays configured listeners for an application
debug:form Displays form type information
debug:router Displays current routes for an application
debug:swiftmailer Displays current mailers for an application
debug:translation Displays translation messages information
debug:twig Shows a list of twig functions, filters, globals and tests
eloquent
eloquent:make:seeder Create a new seeder class
eloquent:migrate Executes a migration.
eloquent:migrate:fresh Drop all tables and re-run all migrations.
eloquent:migrate:install Creates the migration repository.
eloquent:migrate:make Creates a new migration file
eloquent:migrate:refresh Reset and re-run all migrations
eloquent:migrate:reset Rollback all database migrations
eloquent:migrate:rollback Rollback the last database migration
eloquent:migrate:status Show the status of each migration
eloquent:seed Seed the database with records
lint
lint:twig Lints a template and outputs encountered errors
lint:xliff Lints a XLIFF file and outputs encountered errors
lint:yaml Lints a file and outputs encountered errors
router
router:match Helps debug routes by simulating a path info match
server
server:log Starts a log server that displays logs in real time
server:run Runs a local web server
server:start Starts a local web server in the background
server:status Outputs the status of the local web server
server:stop Stops the local web server that was started with the server:start command
swiftmailer
swiftmailer:email:send Send simple email message
swiftmailer:spool:send Sends emails from the spool
translation
translation:update Updates the translation file
user
user:create Creates a new user
user:demote Demote an existing user from a role
user:promote Promote an existing user to a role
I do have some custom commands in there as well.
Any help is greatly appreciated.
From what I gather from vendor/doctrine/doctrine-bundle/DependencyInjection/DoctrineExtension.php, a proper configuration for both the DBAL and the ORM are required to enable the commands:
public function load(array $configs, ContainerBuilder $container)
{
$configuration = $this->getConfiguration($configs, $container);
$config = $this->processConfiguration($configuration, $configs);
$this->adapter->loadServicesConfiguration($container);
if (! empty($config['dbal'])) {
$this->dbalLoad($config['dbal'], $container);
$this->loadMessengerServices($container);
}
if (empty($config['orm'])) {
return;
}
if (empty($config['dbal'])) {
throw new LogicException('Configuring the ORM layer requires to configure the DBAL layer as well.');
}
$this->ormLoad($config['orm'], $container);
}
The ormLoad and dbalLoad are responsible for registering the commands.
In this specific instance, Doctrine needs to be registered:
doctrine:
dbal:
url: mysql://db_user:db_password#127.0.0.1:3306/db_name
orm: ~
The above goes at the end of resources/config/config.yml, or any other file of that folder. Also, you'd need to make the proper adjustments.
doctrine
doctrine:cache:clear-collection-region Clear a second-level cache collection region
doctrine:cache:clear-entity-region Clear a second-level cache entity region
doctrine:cache:clear-metadata Clears all metadata cache for an entity manager
doctrine:cache:clear-query Clears all query cache for an entity manager
doctrine:cache:clear-query-region Clear a second-level cache query region
doctrine:cache:clear-result Clears result cache for an entity manager
doctrine:cache:contains Check if a cache entry exists
doctrine:cache:delete Delete a cache entry
doctrine:cache:flush [doctrine:cache:clear] Flush a given cache
doctrine:cache:stats Get stats on a given cache provider
doctrine:database:create Creates the configured database
doctrine:database:drop Drops the configured database
doctrine:database:import Import SQL file(s) directly to Database.
doctrine:ensure-production-settings Verify that Doctrine is properly configured for a production environment
doctrine:generate:entities [generate:doctrine:entities] Generates entity classes and method stubs from your mapping information
doctrine:mapping:convert [orm:convert:mapping] Convert mapping information between supported formats
doctrine:mapping:import Imports mapping information from an existing database
doctrine:mapping:info
doctrine:query:dql Executes arbitrary DQL directly from the command line
doctrine:query:sql Executes arbitrary SQL directly from the command line.
doctrine:schema:create Executes (or dumps) the SQL needed to generate the database schema
doctrine:schema:drop Executes (or dumps) the SQL needed to drop the current database schema
doctrine:schema:update Executes (or dumps) the SQL needed to update the database schema to match the current mapping metadata
doctrine:schema:validate Validate the mapping files

How to hide or delete the defaults available console commands?

I created a new Symfony 4 project via symfony new my_project_name.
Currently when I execute ./bin/console, the output shows
I will create some custom console commands and I want show only my custom commands when I do ./bin/console
Maybe I should create a custom executable 'console' from scratch, but I don't know how do that.
You are creating a complete Symfony application, so all the commands provided by the included packages are available.
Instead of starting from a framework and trying to trim down the parts you do not want, you need to start from further down to have a really barebones project.
Bootstrapping the project:
First, do not use symfony command, no need. Plain old composer will do the trick.
On an empty directory, execute:
composer require symfony/console
This will import the only dependency needed for a console project, and do the basic bootstrapping for your autoloader.
In composer.json, add the following:
"autoload": {
"psr-4": {
"App\\": "src/"
}
}
Command class
You'll need one or more commands to actually add to your application. Let's start with a fully fledged greeting application. Create the file src/Greet.php within your project:
declare(strict_types=1);
namespace App;
use Symfony\Component\Console\Input\InputArgument;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Input\InputOption;
use Symfony\Component\Console\Output\OutputInterface;
class Greet extends Symfony\Component\Console\Command\Command
{
protected function configure()
{
$this->addArgument('person', InputArgument::OPTIONAL, 'Name of the Entity being greeted', 'World');
$this->addOption('greeting', 'g', InputOption::VALUE_OPTIONAL, 'How to greet the entity', 'Hello');
}
protected function execute(InputInterface $input, OutputInterface $output)
{
$greeting = $input->getOption('greeting');
$entity = $input->getArgument('person');
$output->writeln("<info>$greeting $entity!</info>");
}
}
This is a basic command that will print "Hello World!" if executed without any option or argument, will accept one argument use instead of "World", and one option to set the greeting instead of "Hello".
Console Application
On the project's root, create a file app.php.
require __DIR__ . '/vendor/autoload.php';
$app = new Symfony\Component\Console\Application('Hello World App', '1.0.0');
$app->add((new App\Greet('greet')));
$app->run();
This is be a very short script, so let's go line by line
Require the autoloader. This script is the entry point for the application, so we need to execute the autoloader.
Instantiate the application. This creates a new instance of the Symfony Console application, sets a name and version for the app. This will be used in the "help" printouts for the app.
Add the command. By default, the application has no commands at all. We'll need to add the command we have just created. add() expects a Command instance. We instantiate the command we just created, and we set it to be called by the name "greet".
Run the application
Generate autoloader.
Execute composer dump-autoload so the autoloader for your application is generated.
Execute the script.
If you now execute php app.php you'll get:
Hello World App 1.0.0
Usage:
command [options] [arguments]
Options:
-h, --help Display this help message
-q, --quiet Do not output any message
-V, --version Display this application version
--ansi Force ANSI output
--no-ansi Disable ANSI output
-n, --no-interaction Do not ask any interactive question
-v|vv|vvv, --verbose Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
Available commands:
greet
help Displays help for a command
list Lists commands
Note that the only available command is greet. Which you can use like this:
# php app.php greet
Hello World!
# php app.php greet Alice
Hello Alice!
# php app.php greet Bob -g "Good morning"
Good morning Bob!
# php app.php help greet
Usage:
greet [options] [--] [<person>]
Arguments:
person Name of the Entity being greeted [default: "World"]
Options:
-g, --greeting[=GREETING] How to greet the entity [default: "Hello"]
[... default generic options omitted]

Laravel running migrations on "app/database/migrations" folder recursively

So my migrations folder looks like this since I have dozens of tables it keeps things organized and clean:
migrations/
create_user_table.php
relations/
translations/
I'm trying to do a refresh all migrations and seed but it seems like I've run into a slight hiccup where I don't know the artisan command to run migrations recursively (i.e. run migrations in the relations and translations folders as well).
I've tried adding --path="app/database/migrations/*" however it spat out an error. Does anyone know the solution to this?
The only way to do it right now is to manually go through all the migrations. That is, you have to run the migration command on each of your subfolders:
php artisan migrate --path=/app/database/migrations/relations
php artisan migrate --path=/app/database/migrations/translations
However, what you can do is easily extend the artisan system to write your own migrate command that will iterate through all folders under the migrations folder, create these commands for you and run them.
You can also simply write a shell script if you don't want to get into doing this via artisan
Edit: for Laravel >= 5.0, the correct commands to migrate migration files in sub directories would be:
php artisan migrate --path=/database/migrations/relations
php artisan migrate --path=/database/migrations/translations
This add to boot method in AppServiceProvider
$mainPath = database_path('migrations');
$directories = glob($mainPath . '/*' , GLOB_ONLYDIR);
$paths = array_merge([$mainPath], $directories);
$this->loadMigrationsFrom($paths);
Now you use can php artisan migrate and also php artisan migrate:back
You can also use a wildcard, like so:
php artisan migrate --path=/database/migrations/*
You can use the following command to do this recursively:
php artisan migrate --path=/database/migrations/**/*
**/* is also known as the globstar
Before this works you must check if your bash supports the globstar.
You can do this by executing shopt and checking for globstar.
Globstar is supported by default by most server distributions but might not work on MAC.
For more on globstar see: https://www.linuxjournal.com/content/globstar-new-bash-globbing-option
In Laravel 5 the database folder sits alongside the app folder by default. So you could run this to migrate a single folders migrations:
php artisan migrate --path=/database/migrations/users
You can try this package nscreed/laravel-migration-paths. By default all sub directories inside the migrations folder will be autoloaded. Even you can add any directories very easily.
'paths' => [
database_path('migrations'),
'path/to/custom_migrations', // Your Custom Migration Directory
],
Don't need any special command just the generic: php artisan migrate will do your tasks.
I rewrote the MigrationServiceProvider:
- registerResetCommand()
- registerStatusCommand()
- registerMigrateCommand()
There you can register your own commands:
class MigrateCommand extends Illuminate\Database\Console\Migrations\MigrateCommand
After that you just need to extend youd directories:
protected function getMigrationPaths()
Or you just register the paths on application boot.
I've already done my solution before I knwewd about '$this->loadMigrationsFrom'.
A Simple solution is to create an Artisan Command for example (migrate:all),
then inside handle function define migrate command for each sub directories as mentioned bellow.
Artisan::call('migrate', [
'--path' => '/database/migrations/employee'
]);
Only relative path works for me (in Laravel 5.7):
php artisan migrate --path=database/migrations/your-folder-with-migrations
This works at laravel 8
php artisan migrate --path=database/migrations/tenant
It is a not a "direct" solution but i suggest you to look at Modularity into your laravel project.
Modules can segment your application in several smaller "folders" and bring migration, seeds, classes, routes, controllers, commands together in easily maintainable folders.
This package is a good start : https://github.com/pingpong-labs/modules
No changes are necessary, whenever you need it, use this command:
php artisan migrate --path=database/migrations/*
If you want a shortcut, you can include that in your composer.json:
"scripts": {
"migrate": [
"#php artisan migrate --force --path=database/migrations/*"
]
}
Then just use composer migrate in the terminal.
add in app/Providers/AppServiceProvider.php the path of your migration folder, you can add a string or an array
public function register()
{
$this->registerMigrations();
}
protected function registerMigrations()
{
return $this->loadMigrationsFrom(__DIR__ . '/../../database/migrations/**/*');
}
A Simple Laravel solution is to create a gulp task (say migrate-others).
var elixir = require('laravel-elixir');
var gulp = require('gulp');
var shell = require('gulp-shell')
/*
|--------------------------------------------------------------------------
| Elixir Asset Management
|--------------------------------------------------------------------------
|
| Elixir provides a clean, fluent API for defining some basic Gulp tasks
| for your Laravel application. By default, we are compiling the Sass
| file for our application, as well as publishing vendor resources.
|
*/
elixir(function(mix) {
mix.sass('app.scss');
});
// Our Task
gulp.task('migrate-others', shell.task([
'php artisan migrate --path=/app/database/migrations/relations',
'php artisan migrate --path=/app/database/migrations/translations',
]));
Now you can simply call
gulp migrate-others
Here you go!
function rei($folder)
{
$iterator = new DirectoryIterator($folder);
system("php artisan migrate --path=" . $folder);
foreach ($iterator as $fileinfo) {
if ($fileinfo->isDir() && !$fileinfo->isDot()) {
echo $fileinfo->getFilename() . "\n";
rei($folder . $fileinfo->getFilename() . '/');
}
}
}
rei('./database/');

Categories