I'm currently trying to use PHPUnit to learn about Test Driven Development (TDD) and I have a question about writing reports using TDD.
First off: I understand the basic process of TDD:
But my question is this: How do you use TDD to write a report?
Say you've been tasked to write a report about the number of cars that pass by a given intersection by color, type, and weight. Now, all of the above data has been captured in a database table but you're being asked to correlate it.
How do you go about writing tests for a method that you don't know the outcome of? The outcome of the method that correlates this data is going to change based on date range and other limiting criteria that the user may provide when running the report? How do you work in the confines of TDD in this situation using a framework like PHPUnit?
You create test data beforehand that represents the type of data you will receive in production, then test your code against that, refreshing the table each time you run the test (i.e. in your SetUp() function).
You can't test against the actual data you will receive in production no matter what you're testing. You're only testing that the code works as expected for a given scenario. For example, if you load your testing table with five rows of blue cars, then you want your report to show five blue cars when you test it. You're testing the parts of the report, so that when you're done you will have tested the whole of the report automatically.
As a comparison, if you were testing a function that expected a positive integer between 1 and 100, would you write 100 tests to test each individual integer? No, you would test something within the range, then something on and around the boundaries (e.g. -1, 0, 1, 50, 99, 100, and 101). You don't test, for example, 55, because that test will go down the same code path as 50.
Identify your code paths and requirements, then create suitable tests for each one of them. Your tests will become a reflection of your requirements. If the tests pass, then the code will be an accurate representation of your requirements (and if your requirements are wrong, TDD can't save you from that anyway).
You don't use the same data when running the test suites and when running your script. You use test data. So if you want to interact with a database, a good solution is to create a sqlite database stored in your ram.
Similarly, if your function interacts with a filesystem, you can use a virtual filesystem.
And if you have to interact with objects, you can mock them too.
The good thing is you can test with all the vicious edge-case-data you think of when you write the code (hey, what if the data contains unescaped quotes?).
It is very difficult, and often unwise, to test directly against your production server, so your best bet is to fake it.
First, you create a stub, a special object which stands in for the database that allows you to have your unit tests pretend that some value came from the DB when it really came from you. If needs be, you have something which is capable of generating something which is not knowable to you, but still accessible by the tests.
Once everything is working there, you can have a data set in the DB itself in some testing schema -- basically, you connect but with different parameters so that while it thinks it is looking at PRODUCTION.CAR_TABLE it is really looking at TESTING.CAR_TABLE. You may even want to have the test drop/create table each time (though that might be a bit much it does result in more reliable tests).
Related
When running our suite of acceptance tests we would like to execute every single test on a defined database state. If one of the tests would write to the database (like create users or something else), it must of course not affect later tests.
We have thought about several options to achieve that, but copying the whole database before every single tests does not seem like the best solution (thinking of possible performance issues).
One more idea was to use MySQL transactions, but some of our tests cause many HTTP requests, so different PHP processes are spawned and they would lose the transaction too early for a clean rollback after the full test is done.
Are there better ways to guarantee a defined database state for every of our acceptance tests? We would like to keep it simpler than solutions like aufs or btrfs tackling it on system level.
You could approach this problem using PhpUnit.
It is used for automated testing with PHP. Is not the only library, but one of the most extended ones.
You could use it with database testing as well ( https://phpunit.de/manual/current/en/database.html ). Basically, it lets you accomplish exactly what you are looking for. Import initially the whole database, and then in each test suite, load what you need and then restore to the previous state. For example, you could save temporarily the current status of the table A and after you are done with all tests of the suite, simply restore it. Instead of reloading the whole database.
By the way, having a minimal Database with only the required information for testing will help a lot as well. In that case you don't have to deal with big performance issues, and you can simply restore it after each test suite.
I've been practicing and reading about test driven development.
I've been doing well so far since much of it is straightforward but I have questions as to what to test
about certain classes such as the one below.
public function testPersonEnrollmentDateIsSet()
{
//For the sake of simplicity I've abstracted the PDO/Connection mocking
$PDO = $this->getPDOMock();
$PDOStatement = $this->getPDOStatementMock();
$PDO->method('prepare')->willReturn($PDOStatement);
$PDOStatement->method('fetch')->willReturn('2000-01-01');
$AccountMapper = $this->MapperFactory->build(
'AccountMapper',
array($PDO)
);
$Person = $this->EntityFactory->build('Person');
$Account = $this->EntityFactory->build('Account');
$Person->setAccount($Account);
$AccountMapper->getAccountEnrollmentDate($Person);
$this->assertEquals(
'2001-01-01',
$Person->getAccountEnrollmentDate()
);
}
I'm a little unsure if I should even be testing this at all for two reasons.
Unit Testing Logic
In the example above I'm testing if the value is mapped correctly. Just the logic alone, mocking the connection so no database. This is awesome because I can run a test of exclusively business logic without any other developers having to install or configure a database dependency.
DBUnit Testing Result
However, a separate configuration can be run on demand to test the SQL queries themselves which is another type of test altogether. While also consolidating SQL tests to run separately from unit tests.
The test would be exactly the same as above except the PDO connection would not be mocked out and be a real database connection.
Cause For Concern
I'm torn because although it is testing for different things, it's essentially duplicate code.
If I get rid of the unit test, I introduce a required database dependency at all times. As the codebase grows, the tests will become more slow over time, no out-of-box testing; extra effort from other developers to set up a configuration.
If I get rid of the database test I can't assure the SQL will return the expected information.
My questions are:
Is this a legitimate reason to keep both tests?
Is it worth it or is it possible this may become a maintenance nightmare?
According to your comments, it seems like you want to test what PDO is doing. (Are prepare, execute etc ... being called properly and doing things I expect from them ?)
It's not what you want to do here with unit testing. You do not want to test PDO or any other library, and you are not supposed to test the logic or the behavior of PDO. Be sure that these things have been done by people in charge of PDO.
What you want to test is the correct execution of YOUR code and so the logic of YOUR code. You want to test getAccountExpirationDate, and so the results you expect from it. Not the results you expect from PDO calls.
Therefore, and if I understand your code right, you only have to test that the expiration date of the $Person you give as parameter has been set with the expected value from $result -> If you didn't use PDO in the right way, then the test will fail anyway !
A correct test could be :
Create a UserMapper object and open a DB transaction
Insert data in DB that will be retrieved in $result
Create a Person object with correct taxId and clientID to get the wanted $result from your query
Run UserMapper->getAccountExpirationDate($Person)
Assert that $Person->expirationDate equals whatever you settled as expected from $result
Rollback DB transaction
You may also check the format of the date, etc ... Anything that is related to the logic you implemented.
If you are going to test such things in many tests, use setUp and tearDown to init and rollback your DB after each test.
Is this a legitimate reason to keep both tests?
I think no one could ever be able to tell from your simple example, which test you need to keep. If you like, you can keep both. I say that because the answer depends a lot on unknown factors of your projects, including future requirements & modifications.
Personally, I don't think you need both, at least for the same function. If your function is more easier to break logically than data-sensitive, feel free to drop the database-backed ones. You will learn to use the correct type of tests over time.
I'm torn because although it is testing for different things, it's essentially duplicate code.
Yes, it is.
If I get rid of the database test I can't assure the SQL will return the expected information.
This is a valid concern if your database gets complicated enough. You are using DBUnit, so I would assume you have a sample database with sample information. Over time, a production database may gets multiple structure modifications (adding fields, removing fields, even re-working data relationship...). These things will add to your efforts to maintain your test database. However, it would also be helpful if you realize that some changes in the database aren't right.
If I get rid of the unit test, I introduce a required database dependency at all times. As the codebase grows, the tests will become more slow over time, no out-of-box testing; extra effort from other developers to set up a configuration.
What's wrong with requiring a database dependency?
Firstly, I don't see how extra efforts from other developers to setup a configuration comes to play here. You only set up one time, isn't it? In my opinion, it's a minor factor.
Second, the tests will be slow over time. Of course. You can count on in-memory database like H2 or HSQLDB to mitigate that risk, but it certainly take longer. However, do you actually need to worry? Most of these data-backed tests are integration test. They should be the responsibility of Continuous Integration Server, who runs exhausting test every now and then to prevent regression bug. For developers, I think we should only run a small set of light-weight tests during development.
In the end, I would like to remind you the number 1 rule for using unit test: Test what can break. Don't write tests for test sake. You will never have 100% coverage. However, since 20% of the code is used 80% of the time, focusing your testing effort on your main flow of business might be a good idea.
I have a PHP class that create a line with a CURRENT_TIMESTAMPS in a MySQL database when I create a new instance of it. This timestamp is then used to sort theses records (and primarily getting the last one).
In this precise case, the timestamp is the automatically generated created_at column of an Eloquent (Laravel) model. If possible I'd like a solution that is not Laravel-specific given I could encounter the same problem with other projects.
I had written tests for this class on another PC and everything was working fine, the resulting oder was always correct.
Now I copied this project on my other PC that has an SSD and that is (but not much) more powerful than the other. And now, all my tests fail, because all records end up having the exact same timestamp.
In production it would not be a an issue if two records had the same timestamps (which is very unlikely), but that's not the situation I try to replicate in my test.
To quickly address the issue without digging in the code, I added some sleep(1) functions before instantiating a new object.
But now my question is, what is the best way to deal with that ?
Here are a few possibilities I thought of, but I'm not sure which one is the best:
Hard-coding the dates in the tests: no risk of failure, but my objects currently offer no way to define a date manually. I would need to add methods for that, and they won't be used for anything else.
Empty the database before doing the next operation. The problem there is that for certain tests I need to insert two objects one after the other, so it wouldn't solve all the problems
Make the code wait. But it sounds very stupid to me, because I lose all the benefits of having a powerful machine. And of course it takes much more time to run the tests
Manipulating the system or the database current time. I have no idea of what horrible things could happen if I try that ! Maybe it is possible to isolate a process that do that without impacting anything else on the PC ?
So which method do you recommend ? Is there a better one ?
More system informations: All my dev machines are running Ubuntu 14.04 with PHP 5.5 and MySQL 5.5. My current project is using the Laravel 4 framework and phpunit for the tests
Additional informations
I'd like to add there what I said in the comments below.
Ideally my test should do its assertions with dates that are spaced by many hours, but for the moment I relied on the fact that two records had at least one second difference, which is no more the case with this other PC config.
And, additional informations, in production, the older records get normally deleted before a new one is inserted or the db queried (say, 90% of the time) so even if two were created quickly, there would be a chance that one is removed before I do my logic on the database. But for my test, I need to have at least two records in database, spaced by some time, which is not important but greater than zero.
I've found out a good solution that apply to Eloquent only.
This is the same that is used for the Database Seeds.
We can add Eloquent::unguard() before instantiating the object in the test:
Eloquent::unguard();
Model::create(array('created_at' => '2014-12-17'));
So we can manually set the created_at field of the model, without exposing it to "everybody" who use the class.
You can use Carbon::setTestNow($now) for mocking now time
foreach ($list as $index => $test) {
Carbon::setTestNow(now()->addDays($index + 1));
// functionCreateModel();
}
so far, it's "easy" to test something, but this time I need to test an algorithm based on database source. The database should be filled for that, but is there a good, working way to do it?
What you are describing is really an integration test.
You would need to ensure that your test can set up the required data and clean up after itself to keep the tests repeatable. I normally create the database / tables as part of the test set-up, then drop them when I'm done, which is easier than trying to get a table back into a particular state.
I've created a basic unit test suite in PHP. It shows whether or not a test passed and the time it took to execute the test. It works great, but I've run into a problem and I'm not sure how to address it. Each initial unit test of each unit test class always appears to take more time to execute than the others in that class, usually by a significant factor.
As an example:
TestClass1
Test1.1 (0.042395)
Test1.2 (0.000392)
Test1.3 (0.000325)
Test1.4 (0.000645)
TestClass2
Test2.1 (0.042395)
Test2.2 (0.000392)
Test2.3 (0.000325)
Test2.4 (0.000645)
I'm assuming that the initial test time is also factoring in the loading of resources, but I can't seem to make the first test consistent with the other times. Each test has setUp and tearDown methods that are run. I've also tried repeating the exact same test under a different name and it runs in the expected shorter time, so the first test always has some sort of overhead.
PHP doesn't have any sort of unloadClass or unloadFile function, so I'm not sure if I can recreate a consistent test environment for each test if the first test has the overhead of loading resources into the memory. I've tried preloading files and an initial class instantiation, but neither seems to make an impact.
Are there other possibilities that I might be missing that could be affecting the initial test time? What is the proper way to address this?
Note: I'm aware of existing PHP unit testing frameworks, but I'm trying to do something a little different, so please don't say, "Just use PHPUnit", as it's not helpful in helping me solve this issue.
Any benchmarks you're doing in PHP code are going to be skewed. I would suggest looking into using Xdebug's profiling features to get more accurate benchmarks: http://xdebug.org/docs/profiler