I'm testing a CRUD application written in L5 using Codeception acceptance tests. I was wondering how you guys go about this.
I originally thought I'd be able to use a CEST and use the #depends property to specify the order but it seems the auto DB cleanup is ran after each test, so after my insert the next test can't find that record in the DB.
Is there a way I can do this without specifically making up a DB dump specifically for testing?
I was hoping to be able to have the following tests in this order:
Create Item
Read Item
Update item
Delete Item
and check the DB at each stage to make sure it was successful.
Any help would be greatly appreciated.
Cheers,
Daryll
The default behaviour specified here is definitely what you want. You want the tests to be able to be performed in isolation. If you want to retest an area again, because it is broken, you don't have to go back and run your previous tests again to get it in the correct state to run the tests again.
A single broken test will give you a better idea of where the breakage is, rather than having multiple broken tests because a test that all your other tests depend on was broken, which is what the approach you are describing lends itself to.
If you think about it, say you are doing test driven development. You write a breaking test with the correct data that Creates a record. The other tests are to read, update, read again, delete the record, update a different record, create another new record, and read again. The delete fails, and the create and read tests fail because you reinsert the same index. The tests wont tell you that however, you just get 4 broken tests, and you will have to check all of your tests that are broken, and which one caused the other tests break, if that is indeed the case. You wont know.
If you do what the default behaviour tells you to do, you will have just the one broken test, and you fix the issue. Dead easy. You just add to the database as test cases arise to accommodate for them.
If you want to check what's in the database, you can do that by querying twice in the same test. Perform your update query then run your select query on what you have just done, and do and equals as another assertion. You can trust that the database has ran your query if you set it up correctly and get no errors, but you know your application better. Maybe you have triggers running, for instance.
I believe you can stop the behaviour with this sort of thing in a config, but its not what you want!:
class_name: FunctionalTester
modules:
enabled: [Filesystem, FunctionalHelper, Laravel4, Asserts]
config:
Laravel4:
environment: 'testing'
cleanup: false
I have an experience on a large scale application where at some point a decision was maid to check DB on every test. 2 Years later world of pain and useless, slow, unmanageable tests. So lessons learned from that: Do not test data in DB on unittest.
Purpose of unit tests is to test code you wrote!
When you test that proper data got to DB you test:
DBAL Driver (Larvel).
PHP DB Driver (whatever larvel uses).
DB itself (whatever DB).
That creates unnecessary complexity of test:
Every test environment should have all those components properly setup.
You need to make sure that data is consistent between tests. So easiest way is to clean data after every test and each test should create its own data(very slow).
And you repeat work done by DB creators, Driver creators and Larvel ...
As a result:
Tests getting slower, management of test environment gets harder. That leads to the point when you stop writing tests...
So solution is to write your classes to use DI and Mock DBAL on tests. There libraries that can help with DB mocking. But at the very end you should just test your code. That it calls proper functions with proper data and properly reacts to data that comes from DB.
And if you want to make sure that your DBAL works properly run it's unittest.
As for project I worked on there is an effort to move all test out of DB to mocks and speed improvements are x 1000 for tests that got changed.
Related
Reading an article Evil Unit testing, I am not exactly sure WHY the unit tests should never use DB, network or file systems. What if it an network app?
Unit tests are used to test the functionality of the smallest unit of your code. They should not have any dependency on any external resource that might change in future.
To take an example, let's say today you write a unit test that test a method that performs addition of two numbers.
public void AddNumberTest()
{
int a = 4; // Assume 4 coming from a database.
int b = 5; // Assume 5 coming from a database.
int c = a + b;
Assert.True(9, c);
}
This will run today. That's totally cool.
Let's say tomorrow you come and change the functionality of the Add method.
Now you should be able to run the test and it should be pass. But let's assume somehow the database server (or external resource ) is down. Then the test fail.
Even if someone changes the values in database, you need to be aware of the change to be made to the test.
Is that what you want??? Absolutely not. Right
That's why we write unit test cases that should be independent of external resources. That where we use mocks and stubs.
You should be able to run unit test a thousand times and it should give the same results always.
Unit testing is not intended to test the complete functionality of an application. It is meant to make sure that definable modules of code work as expected. To test an application it is not enough to use unit tests. You must also perform functional testing and regression testing. Database access falls outside the scope of unit testing, so you would not write unit tests that include database access. You would include database access testing in your functional tests. Similarly, if you have a network app, you would not include network access in your unit tests. In both cases you would unit test the code that manipulates the data that it gets from the db or the network using simulated-known data and leave the actual access out of it.
To reiterate, it is not enough to just unit test an app.
Before unit tests are run, the test DB needs to have all dependent data copied over. In this particular test I need to run, there is data spread over 12 tables that the called methods within the tests will depend on.
When the tests are run, I need to make sure the test database gets populated with the same values from production for these test cases.
Is there a streamlined efficient way of doing this or do I have to manually copy over each row from each table before running the tests?
I'm using Yii 1.1 with PHPUnit 4.6.
You can easily define a proper migration and apply them when you need.
You can use migration up and migrationdown for prepare the DB you need for all your test
Well, as in the title, I can think of three ways to manage testing the database output (I'm using ORM in my application, and PDO in unit tests). Which is the best one? How do you handle this?:
Create data set with the data I want specifically for testing, and change the code so that it reads xml instead of the ORM arrays (in tests classes).
Create setUp() method, set the attribute containing the ORM array, and work on that.
Same as the second point, but with another database, created specifically for testing
You may want to read PHPUnit's chapter on database testing.
I use PDO via my own thin wrapper that supports nested transactions via save points. In the bootstrap, I create a test database with the entire structure of production along with very basic seed data. During each setUp() and tearDown() I begin a transaction and then roll back.
Each test imports the subset of data it needs from raw SQL files. From there, the ORM is tested using real inserts, etc. But this only works because my DB driver supports nested transactions. If the tests begin/commit and check for success/failure, everything still works.
If you don't have nested transaction support, you could set up and tear down the entire database on each test, but that will be slower. And note that you don't always have to test against a real database... it depends on what sort of things you are testing.
In my tests, I use a test database.
MySQL has a few test databases on their site. I find the Sakila rather difficult, so I use the World database.
I'm in the process of testing a factory class.
One of the methods must load the data in an array for an object that another method will instantiate.
That method contains the SQL query that holds a critical condition that must be tested. ( in this case only ask for the records that are "published". Ex.: WHERE published=1 ). That distinction in the SQL Query is the only detail that makes that method differ from another one, and I want to test the query execution behaviour.
Now, I can't really mock my PDO object and ask it to return a fixed result as I would not test the execution of the query by mySQL. That would make a useless test.
That leads me to think that I'll need to set up a static database with fixed test data inside it. Am I right on this or have I missed something?
Should I separate the test requiring the "test database" from the tests that are autonomous ?
I strongly agree on the part of not mocking out the PDO. At some point i want to make sure my queries work against a real database. While this might not be a unit test anymore, technically. For me it offers so much more to know that my code that handles data storage really works against the db.
What i tend to do is creating a sort of Data Access class for each Class that needs to talk to the DB and so separating out most of the business logic from the db access code.
That way i can mock out the data access when testing the classes and after that set up a "Test Database" for each "data access class" and see if those work.
#zerkms Answer (+1) already linked http://phpunit.de/manual/current/en/database.html and the only other resource i have found to be of value when it comes to DB-Testing is the Book Real-World Solutions for Developing High-Quality PHP Frameworks and Applications that has a big chapter covering this subject.
Should I separate test requiring the "test database" from the tests that are autonomous ?
Only if your testsuite gets really really big and you have runtime issues that force you to say "hitting even a testing database just takes to long for all my tests so i run those only on a continuous integration server and not while developing.
Yes, it is the common practice.
It is also the special methods for loading fixtures: http://phpunit.de/manual/current/en/database.html
What about second question: no, you should not separate them. Your tests should test the behaviour, not the details of implementation. Join test cases into test classes by logic.
I get the idea behind unit testing however am trying to think of a clean simple way do it when it requires database functionality. For instance I have a function that return result based off a database select query. Would the database alway have to remain the same for me to properly see that only the correct results are being returned. What is the best way to perform unit testing (in PHP) when it requires database inactivity (whether it be read, write, update, or delete)?
There is a whole chapter on that in the PHPUnit manual:
http://www.phpunit.de/manual/current/en/database.html and also worth reading
http://matthewturland.com/2010/01/04/database-testing-with-phpunit-and-mysql/
It's like with everything else when Unit-Testing. Create a known state and test your code against it to see if it returns the expected results.
Personally, I create a dummy testing database and populate it with a known data set for each testing run (I do that right in the setUp functions). Then the tests run against that data set, and then it's removed on tearDown...
Now, this is more of a Integration test than an Unit test (And personally I treat it differently from a unit test, run on its own schedule along with other integration tests), but it's still quite useful.
It is not a unit test if it needs the database.