I've got a symfony2 (php framework) app for which I've written a command that can be called from the command line. This command simply checks a table in the database for a task, then executes that task. What I need is a daemon-like creature that will enable this command to be run in the background whenever a row is inserted into this (mysql) database table. I can't do this with a cron job because the tasks required need to happen as quickly as possible, and will often need to happen in parallel.
Am I looking at this all wrong?
Server: Ubuntu 11.10
PHP: 5.3
Symfony: 2.1
mysql: 5.1.66
I would assume you are using some kind of a ORM for persistence. I would then probably use a hook like postInsert() in Propel
postInsert() code executed after insertion of a new object
or postPersist() in Doctrine.
postPersist - The postPersist event occurs for an entity after the entity has been made >persistent. It will be invoked after the database insert operations. Generated primary >key values are available in the postPersist event.
Here is some more information on lifecycle events.
Even if this is a very old question...
I've found https://github.com/mac-cain13/daemonizable-command
Probably useful for what you need to do.
Related
Scenario
We want to autofill a table with new static content with doctrine, ideally with something like a fixture class or similar.
We follow the simple development life-cycle with development to staging to production. And we are using Doctrine v2.6 with Symfony v3.4. Every release step is executed by an Jenkins job.
For development and staging we use the very useful and simple doctrine-fixtures-bundle to auto-fill our database with test-datasets. The database-schema is auto-generated by doctrine:schema:update on basis of our entities.
I've tried to use the fixtures also for production but even with doctrine:fixtures:load --fixtures=src/MyBundle/DataFixture/ORM/MyFixture.php it is purging the whole database. Then I read something about the --append command to prevent doctrine from purging the database. But then it will append the datasets in every release process (?). Nevertheless, it also feels like a very bad practice.
What I'm wondering
Is it possible to truncate the table, load table records with static data loaded from a class that can be executed via a command line? Or is there a completely different (and clean) way for such a case? Is the doctrine:migration bundle the real way to go?
Thanks for your help!
You should create a Command for populate tables.
https://symfony.com/doc/current/console.html
I'm using database migrations with PHP symfony framework and I noticed that there are a lot of files, and when I build my project and every time many files (migrations) are executed.
What is best practice to manage migrations? Could I delete them and create only one database dump file, for database initiation?
I'm assuming you're using Doctrine Migrations to create these files. On dev you can always regenerate the database using the doctrine:schema:update command.
When setting up a new instance you can also use the doctrine:schema:create command which will also create the latest table definitions for you.
That brings us to your question; do you need to keep all the migrations? As long as you know that the migrations have been executed on outdated instances you can safely archive (my preferred option) or delete them as they will never be called again.
I would prefer to use migrations on production instead of doctrine:schema:update, because you can have some problems while updating process. It is always a better option IMO to keep an eye on what happens with the DB.
On development env you can easily use doctrine:schema:update command but you should use migrations on production. In some cases the command may brake the migrations (it happened to me a few times).
I am working on building an application which would be used to create new child applications which have their own databases.
I know we can use doctrine bundle to create tables from entity classes in symfony. I would like to know if there is a way to create a new database and some tables within the database programmatically or dynamically.
I know we can use php app/console doctrine:generate:database from the composer prompt, but i would like to do this from a class or a controller action.
Do let me know if there is a way possible.
You can either call exec() and drop the command in there or the better way, would be to follow the docs in symfony for calling console commands in a controller.
http://symfony.com/doc/current/cookbook/console/command_in_controller.html
What are the current best practices for testing database interaction with Symfony2? I have a simple CRUD setup and i want to make sure my testing is OK. Right now, i have 4 tests, each one making sure that create, update, delete and list actions are occurring ok.
I have two magic methods, __construct and __destruct, on my test case. Inside them, i call exec() with 'php app/console ...' in order to create the database, create the schema and later on drop the database. However, this is SLOW as hell and it happens all the time when i have more than one test case.
How should i proceed when it comes to database testing and isolating such tests?
I think it's best to always start clean to be sure that tests are fully isolated. To do that I'm simply building the database structure before each test and than I'm filling it with fixtures needed for a given test.
Note that I'm only building needed database tables and I'm only inserting needed fixtures. It's a bit faster than loading a big database dump. It's also more clean as tests don't share fixtures (which makes them easier to maintain).
I have a base test case class called KernelAwareTest which helps me in building the schema. You can find it on gist: https://gist.github.com/1319290
setUp() boots the Symfony kernel and stores a reference to it in a class property (together with references to the DIC and the entity manager). Also a call to generateSchema() is made to generate the database schema (it uses Schema Tool from Doctrine).
By default it generates the database structure for all entities known to the entity manager. You can change this behaviour in your test class by overriding the getMetadatas() method.
P.S.: I tried using in memory database (sqlite) but it wasn't perfect. Anyway it's probably better to run tests against the database you use on production.
Database testing is always slow as you need to create/drop your schema before/after each test. To avoid unnecessary operations, you could:
create schema in the 'setUpBeforeClass' method;
ensure your 4 tests are launched in one thread with the '#depend' annotation;
drop schema in the 'tearDownAfterClass' method.
The schema will be created/droped only once for your tests case.
You can also setup doctrine to use an inmemory sqlite database (which is very fast):
doctrine:
dbal:
driver: pdo_sqlite
path: :memory:
memory: true
Anyway, '_construct' and '_destruct' should never be used in phpunit test cases, instead you should use 'setUp' and 'tearDown'.
The question is pretty old but still valid today so here is my experience and how I handle it today on my Symfony projects.
I started of using an SQLite in-memory database for my tests and I rebuild the db schema + inserted fixtures before each single test case. This had two major drawbacks:
It was still way too slow :(
On dev & prod I used Mysql which soon became a problem because SQLite simply does not have all the features needed and sometimes behaves differently
Using MSQL for the tests and rebuilding the schema + inserting fixtures before each test was simply too slow. So I was looking for alternatives...
I stumbled across this blog post: http://alexandre-salome.fr/blog/Symfony2-Isolation-Of-Tests
The guy here suggests to run tests inside active database transactions and simply roll back any changes after every single test.
I took this idea and created a bundle for it: https://github.com/dmaicher/doctrine-test-bundle
The setup of the bundle is really easy and does not require changing any existing php test classes. Internally it changes the doctrine config to use custom database connections + driver.
With this bundle you can simply create your database schema + insert fixtures ONCE BEFORE running the whole testsuite (I prefer doing this in a custom phpunit bootstrap file). Using the phpunit listener all tests will run inside database transactions.
I've been using this bundle since a while already and for me it works out perfectly using SQLite, MySQL or PostgreSQL.
Since a while its also used on the symfony-demo project.
testing on local machine is pain in the ... ,so i'm started to using ci system buddy.works (there is free stand-alone version) , and for this i neeeded to resolve this issue on my own.
result is :
all tests works
tests are runing on production sql data
tests are running in separation (not in dev or production) - so i can do
anything that i want with database
all pushes to git are tested and i have notification if something is broken
all pushes/pull request to deploy branch are automatic uploaded to production
This is my solution :
second parameters.yml in config with configuration for test
on production i'm making daily sqldump
on starting test on ci this sql backup is copied via scp to test machine
to prepare all this i'm using robo.li ( my config is below)
/**
* This is project's console commands configuration for Robo task runner.
*
* #see http://robo.li/
*/
class RoboFile extends \Robo\Tasks
{
function loadDb(){
$this->taskExecStack()
->stopOnFail()
->exec(" mysql -h mariadb -u root -pqwerty -e 'create database test' ")
->exec(" mysql -h mariadb -u root -pqwerty test < test.sql ")
->run();
}
function prepareDb(){
$this->taskExecStack()
->stopOnFail()
->exec("cp app/config/parameters-test.yml app/config/parameters.yml")
->run();
$this->taskReplaceInFile('app/config/parameters.yml')
->from('database_host: 127.0.0.1')
->to("database_host: 'mariadb'")
->run();
$this->taskReplaceInFile('app/config/parameters.yml')
->from('database_user: dbuser')
->to("database_user: 'root'")
->run();
$this->taskReplaceInFile('app/config/parameters.yml')
->from('database_password: 123')
->to("database_password: 'qwerty'")
->run();
}
}
i hope it help you to create idea how to organize all this . Using stand alone ci is difficult to setup , but it's really good idea
(Make this CW if needed)
We are two developers working on a web application based (PHP5, ZF, Doctrine, MySQL5). We are working each with a local webserver and a local database. The database schema is defined in a YAML file.
What's the best way to keep our database schemas in sync?
Here's how we do it: Whenever developer "A" makes a change, he generates a migration class. Then he commits the migration file developer "B" executes the migration class.
But creating a migration class on every db change is a rather tedious process.
Do you have a better solution?
I don't know how you do in the Zend Framework with Doctrine. Here's how I would do it in Symfony with Propel. Though the exact procedure may vary, but the underlying concept is the same.
I have unit tests on my DAL.
Whenever the schema changes, we check in the yml and the generated ORM code ( You do have a source control, don't you). I set the check-in to auto-mode, meaning I will get all the check-in instantly.
If the schema changes don't affect my thing, then I would just ignore the changes. But if the schema changes break my thing, then I will rebuild my form, ORM classes and whatnot by using symfony propel build command. Rebuilding those infrastructures is just a single command line thing, so there is no problem for me.
Finally, after rebuilding, I will run my unit tests, to make sure everything is OK. If not, I better get them fixed!
I see that this question is already answered but Doctrine can migrate your databases for you without having to blow away the whole thing. We use this for every schema change where one developer changes his/her local yaml file, generates new models locally, creates a migration using Doctrine, runs that migration locally to change the db, then checks in both the new yaml file and the migration. Then other developers check out the changed yaml file and migration, then they generate new models and run the migration to sync their db. Deploying the code to our QA and production environments is pretty much the same process.
More information on Doctrine migrations can be found on the Doctrine site.