The decision I'm trying to make is where I want to do schema changes. I need a schema update to happen in the database, in the model definition, and I'd also like to generate a doctrine migration for that change too. I would really prefer to only have to define schema changes in one place, not three.
Right now I'm thinking of writing all schema changes only as doctrine migrations. I then have a command line tool that runs all pending migrations and does a database->model sync. Is this reliable enough to work? I'm using postgresql if it matters.
The standard flow is to generate an empty doctrine migrations, add the schema changes run the migrations and create your entities. So you'll only need to modify it at 2 places.
This works perfectly with my set-up. Never had any problems with it if you check your down statement at least.
Related
I was recently tweaking around with my DB and noticed that I made 5 migrations on top of my existing 2. So 7 in total, m1, m2, ..., m7. From these 5, I only want the latest version. So I was thinking I could remove m3, m4, m5, m6, m7 and migrate again, which will result in m3 that already contains the final version that I'm happy with. Now I have never done this before and was wondering what is the safest way to do it?
I'm using PHPStorm. Could I manually remove them (right click in PHPStorm then Delete) and then go to the command line and do php bin/console doctrine:migrations:migrate? Or do I need to do php bin/console doctrine:schema:update? Or is this really a bad idea?
No, generally you shouldn't do either of those. Let me start by clearing up what seems like a misconception: a migration doesn't contain a full database schema, only the queries necessary to bring the database from one version to the next. This system allows for database changes to be managed and versioned. So you should be starting to see some problems with this:
A migration might depend on a previous one. E.g. v1 adds an entity and v2 adds a new field. If you remove v1, v2 will be broken and needs to be removed as well.
Since a database migration usually goes hand in hand with an object model change, you will have to revert the model as well, otherwise you'll find mapping problems with non existent fields.
This might be acceptable during the design phase when you don't have any functionality yet. You can revert an individual migration using bin/console doctrine:migrations:execute --down <version>. This is usually done while testing changes if adjustments need to be made. But usually when that change hasn't been committed yet.
Doctrine keeps track of migrations using a database table called migration_versions. By naming them with a date, it can order them and apply them sequentially. Whenever a migration is executed, it adds the migration name to this table. When you roll it back, it deletes it from the table, along with the fields in the migration itself. Keep in mind that even though you can rollback a migration, it doesn't mean everything will be as it was. If a migration drops a column, the column will be recreated on rollback, but the data will be lost.
As for "can it be done"? Yes. If you really really want to, have read the documentation thoroughly and are aware of all this.
So, since your question is about merging migrations, let's tackle your actual options:
Could I manually remove them (right click in PHPStorm then Delete) and then go to the command line and do php bin/console doctrine:migrations:migrate?
No, this won't work. migrate will apply available migrations. They won't be any and, as explained, the revisions will still be in the table and its changes applied.
Or do I need to do php bin/console doctrine:schema:update?
This won't do anything either, since it will compare the model with the database and find that they match.
In any case, you will need to revert them first and then create an equivalent one. The command for that is doctrine:migrations:diff. This will compare the model with the schema and generate a migration to get the database in sync. And for this to work you will need to execute --down your migrations first, otherwise they won't be any changes, but potentially losing some data in the process.
If you work in a team, they will be seeing migrations disappear. Some might even be behind in the history and not applied all the migrations. This will soon become a management pain.
There is a rollup command that (with my understanding and never actually used it) cleans stale migrations from the table, dumps a full schema and applies it. This will be your best bet, but be aware that this most like will delete your data.
You could also combine your migrations manually. They are just classes with an up and down methods. Combine all function bodies, apply the cleanup process and call it a day.
Now, if you want to do this it shouldn't be much of a problem. Just replace your outdated versions with your new one and warn everybody about it. But if what you want is to do as they never existed at all and keep a neat commit history, then that's when your teammates will potentially want to kill you, as it will involve rewriting history. When you do that, they will have to rebase all their work.
If you want to do it:
Make backups
Introduce your changes as early as possible to avoid breaks. Is ok if there are some unused fields in the database for a while, until the model commits catch up. If you do it late and some object need it, someone might be forced to create a migration and break yours in the process.
Get it right (preferably on the first try). Don't push until you have tested extensively.
Make backups
Reference for the console commands
There isn't a "safe" way of doing it, but if you haven't deployed the migrations then you can safely throw away a series of auto-generated migrations and regenerate them. Just be careful you don't throw away manual migration SQL at the same time.
Normally during the design stage we will:
Use doctrine:schema:update --force until things are fairly stable
After they are stable, we reset the development database from a snapshot
Run doctrine:migrations:diff and then add manually migration queries where necessary
May be you need to ignore all your migrations and focus on your last state of database schema. If this is what you are wondering to have in final, then you could just hit:
php bin/console doctrine:schema:update --dump-sql to get the corresponding SQL statement, or hit php bin/console doctrine:schema:update --force to apply it actually on database.
In general yes you can.
But you should know about that the Doctrine/Migration save executed migrations in DB . So you should remove unnecessary migration files (possible combine all queries into one), after that update database.
Table name by default "doctrine_migration_version" just delete unnecessary version rows
I am using cakephp 3.
I need to run a script for updating the database schema like adding a column or altering it.
I do not wish to use Migrations as it would require me to write scripts for every change.
Is there any other way to alter schema of the database if we are neither using migrations nor making changes to the database manually using cakePHP 3?
You could use the schema system for doing this, which I think would work fine for things like adding user-defined columns. But if you're looking for an easier way to do migrations, you'd need to put that schema-related code somewhere and keep track of which changes have already been made, and then you're basically just re-inventing migrations.
I have a migration called CreateItemsTable; I ran that, I have items in that table, now I need to add a new field to the table. I can't just add a field to the migration file and migrate:refresh because I need the data that's in it.
Am I supposed to make another migration for adding a field? That's seems like mess while I'm testing things in development, I might change fields a lot. I'm not sure if migrations are cleaner than just PhpMyAdmin... or maybe I don't understand them?
Yes, each time you need to change a table in some way you'd create a new migration for it. That's the whole point of migrations. When you're developing in a collaborative environment and you pull down some changes from a remote repository, one of the things you should do (if working with a database) is run any migrations that other developers might have created. This keeps your databases in sync.
Sure you might drop and add columns occasionally but it's no big deal.
When you create a table for the first time you are probably using Schema::create(). All subsequent migrations for that table should use Scheme::table(). It accepts the same parameters except it doesn't attempt to create the table first.
What I'm trying to figure out is how to add new fields to a table, using Symfony2 with Doctrine2.
I used this to initially create the Entity:
php app/console doctrine:generate:entity --entity="MyMainBundle:ImagesTable" --fields="title:string(100) file:string(100)"
And I used this to create/update the tables on the database:
php app/console doctrine:schema:update --force
Now if I wanted to add new fields to the ImagesTable entity, is there an easy way to do it using the console, or do I have to manually edit the entity. I am just using 1 entity as an example right now, but in reality, there are many entities I'd be changing; so, there has to be an easier way to do it.
I've been manually editing them to create relationships, so if there is an easier way to do that as well, that'd be great.
I remember this being a lot easier with Symfony1.4 - all I had to do was create the database/tables using phpMyAdmin, and Symfony was able to generate the models with no issues.
I really hope I'm missing something here, because this won't work if I have to manually edit every entity for every change.
Doctrine generator commands are intended to help the developer to quickly prototype an idea. They generally don't produce production ready code, and the code needs to be checked to see if it contains what you want.
You can still create your model in phpmyadmin and use Doctrine reverse engineering tools, but it also doesn't produce production ready code, only intended to use in prototyping.
Creating database/tables beforehand doesn't really work well with Doctrine2, as the underlying relation between tables may not be the same as the relation between objects of your model. The whole point of ORM is to think in classes and letting Doctrine do the rest of the work for you.
Doctrine is not intended to write your entities for you, it gives you tools to build your data model, which you use to code your model in Php.
If you don't like to code your entities by hand (which is what all developers using doctrine does), you may want to have a look at RedbeanPHP, a zero-config ORM framework for PHP. It creates the database tables, columns, indexes on the fly depending on the data model you use.
I'm thinking about deploying Doctrine migrations in my environment to handle database changes between multiple developers. I haven't used them before, but I have done my research on the matter.
My only concern at this point is that [as far as I can tell] Doctrine migrations don't allow for fixture modifications. While I realize that migrations are for schematic changes, I think fixture changes are just as important.
I would like to have fixtures for the reference tables is my database (ie *_type, *_source, etc), and I feel that row add/delete/updates should be handled by these migrations as well, as they are just as important as any structural change.
If anybody could point me in the right direction here, it would be much appreciated.
Update
I explored the idea of simply letting SVN track my reference table fixtures, but this would be be an undeployable solution. The tables would not be able to be truncated/re-populated due to foreign key constraints.
As you pointed out, migrations are to facilitate structural changes to the database, not for manipulating your fixture data in line with them.
In my experience, using migrations when an application is in development is not necessarily the most helpful way to do it, especially if Developer A creates a new migration and doesn't commit immediately, and Developer B also creates a new migration that (unknowingly) conflicts with Developer A's migration, and then checks in immediately. Developer A checks his (or her) migration soon after, and you have two conflicting migrations and the world explodes.
I'd say the better (albeit more longwinded) way to do it would be to make your schema changes in schema.yml, apply your schema changes (either with a private migration or manually) and then doctrine:data-dump and commit your fixture data.
If you do opt for migrations in development, perhaps you should consider using the ->postUp() and ->preUp() methods of the Doctrine Migration class to transform your data in situ.