Reading an article Evil Unit testing, I am not exactly sure WHY the unit tests should never use DB, network or file systems. What if it an network app?
Unit tests are used to test the functionality of the smallest unit of your code. They should not have any dependency on any external resource that might change in future.
To take an example, let's say today you write a unit test that test a method that performs addition of two numbers.
public void AddNumberTest()
{
int a = 4; // Assume 4 coming from a database.
int b = 5; // Assume 5 coming from a database.
int c = a + b;
Assert.True(9, c);
}
This will run today. That's totally cool.
Let's say tomorrow you come and change the functionality of the Add method.
Now you should be able to run the test and it should be pass. But let's assume somehow the database server (or external resource ) is down. Then the test fail.
Even if someone changes the values in database, you need to be aware of the change to be made to the test.
Is that what you want??? Absolutely not. Right
That's why we write unit test cases that should be independent of external resources. That where we use mocks and stubs.
You should be able to run unit test a thousand times and it should give the same results always.
Unit testing is not intended to test the complete functionality of an application. It is meant to make sure that definable modules of code work as expected. To test an application it is not enough to use unit tests. You must also perform functional testing and regression testing. Database access falls outside the scope of unit testing, so you would not write unit tests that include database access. You would include database access testing in your functional tests. Similarly, if you have a network app, you would not include network access in your unit tests. In both cases you would unit test the code that manipulates the data that it gets from the db or the network using simulated-known data and leave the actual access out of it.
To reiterate, it is not enough to just unit test an app.
Related
I have read in another document that classes which have methods with connections to a database have to be tested by integration tests. I tried to find an example, but I didn't find any, only theory. I've very little experience with testing. Can you give me an example? a link?, something (instead of only theory, because I read a lot), with something like this?. I think this is not an strange case. If its possible with php and sql or similar.
If you dont know about integration tests, how do you test this kinds of problems?
class database{
public $server;
public $database;
public $password;
public $user;
public $descriptor;
function __construct(){
$this->server="localhost";
$this->database="mydb";
$this->user="stackoverflow";
$this->password="1234";
$this->descriptor=mysql_connect($this->server,$this->user,$this->password);
mysql_select_db($this->database,$this->descriptor);
}
function make_request($query){
$response=mysql_query($query,$this->descriptor);
return $response;
}
function close_connection(){
mysql_close($this->descriptor);
}
function number_rows($result_query){
return mysql_num_rows($result_query);
}
}
Look into PHPUnit for unit testing. You can then use mock objects to simulate the dependencies (DB calls) to return dummy data. Because you're trying to test some unit of logic and not that your database layer, it should be acceptable and give you a reasonable amount of confidence that your code is working.
Another option is to use integration tests, but this requires that you put a lot of effort into the setup and teardown of your tests. You need to make sure any data required for your tests exists within your database. It's common to use transactions for these types of tests which you can then rollback after your finished running those tests.
Which code is it you're trying to test? Is it the code that uses your database class? If so, you don't need to introduce PDO, as your database class already provides an abstraction (well, it would be much safer if public $descriptor was private). So, you should start by mocking your database class and then testing the code that uses it. PHPUnit has a sophisticated mocking system.
Another thing you might want to be testing is this database class itself? In that case you'll have to accept the MySQL database dependency, and the speed hit that gives you. Set up a completely dedicated MySQL database for that testing. If you don't mind the speed hit, consider creating the database in a setUp() function and removing it after each test in a tearDown() function.
The third thing you might want to be testing is that your website (or whatever) works as a whole, and connects to the database, etc. This is functional testing (also called integration testing). If your system offers a web services API, then you could use that to test it. Otherwise you could use Selenium (still using PHPUnit) to test by remote control of a real browser. If your tests will only read from the database, you could use your live system; but it would be much more sensible to assume a write will happen at some point, so test against a test version of the website, with its own dedicated test database.
The advantage of functional testing is it high level, testing for all interactions between components, testing the system as the end user will use it. The downside is when a test fails it only tells you something is wrong, not exactly which component is to blame. So we have unit tests to test each component in isolation.
I know you this is more text, less code, than you wanted! But you have actually asked quite a theoretical question! Just try setting up some unit tests of this code, watch what problems occur over the first month of using them, and you'll understand why and where to use mocks.
I am trying to implement some unit tests into a legacy PHP application.
There have been a number of challenges with this, but in particular for this question, I am currently looking at a small class that manages the app config.
The class interface is pretty simple; it does the following:
The constructor calls a Populate method, that uses our Recordset class to load the config for the requested module from the database.
Get method, which returns a specified config value.
Set method, which sets the config value in memory.
Save method, which writes the config update(s) back to the DB.
Testing the Get/Set methods is straightforward; they map directly to private array, and work pretty much as you'd expect.
The problem I have is with testing the database handling. The class uses a number of fields on the config table (module name, language, etc) to determine which config items to load and in what priority. In order to do so, it constructs a series of elaborate SQL strings, and then makes direct calls to the DB to get the correct config data.
I have no idea how to go about writing a unit test for this. Aside from the Get/Set methods, the class consists pretty much entirely of building SQL strings and running them.
I can't see a way to test it sensibly without actually running it against a real DB, and all the issues that go with that -- if nothing else, the complexity of the config loader would mean I'd need at least seven or eight test databases populated with slightly different config. It seems like it would be unmanageable and fragile, which would defeat the point somewhat.
Can anyone suggest how I should proceed from here? Is it even possible to unit test this kind of class?
Many thanks.
I must say I'm not sure I agree that unit tests would be somewhat pointless without hitting the database here. My goal would be to get the business logic that produces the SQL under test, without involving your database. Here is an example of what I'm talking about:
class Foo {
// ... Getters and setters for your config ...
public function doSomeBusinessLogicThenHitDb()
{
$sql = 'SELECT * FROM mytable WHERE ';
$sql .= $this->_doSomethingComplicatedThatInvolvesParsingTheConfig();
$this->_queryDb($sql);
}
protected function _queryDb($sql)
{
// Do something with a PDO or whatever
}
}
Having abstracted the _queryDb() bit to a separate function, you can then write this test:
public function testMyClassUnderSomeCircumstances()
{
// Set up config
$exampleConfig = // whatever
// Set up expected result
$whatTheSqlShouldLookLikeForThisConfig = 'SELECT ... WHERE ...';
// Set up a partial mock that doesn't actually hit the DB
$myPartialMockObject = $this->getMock('Foo', array('_queryDb'), array(), '');
$myPartialMockObject->expects($this->once())
->method('_queryDb')
->with($whatTheSqlShouldLookLikeForThisConfig);
// Exercise the class under test
$myPartialMockObject->setConfig($exampleConfig);
$myPartialMockObject->doSomeBusinessLogicThenHitTheDb();
}
The point of this test is to test the business logic that produces your SQL - not to test your database itself. By putting the expectation in place that the resulting SQL must look like whatever it must look like, you are ensuring that your tests will fail if innocent refactoring of _doSomethingComplicatedThatInvolvesParsingTheConfig() accidentally breaks your code, by making it produce different SQL from what it used to.
If testing the whole application, including its database, is your goal, try a proper integration testing suite like Selenium. Unit tests monitor individual classes and tell you when they've stopped behaving as they're supposed to. You will face problems with speed of execution, and with error localisation (i.e. is the bug even in the code, let alone the class under test, or is it a DB thing?), if you let them overreach.
One straight forward to better test these things is to give your config class an object to access the database, so you can run any config with the real database or just some mock of it that writes to memory instead or even lightweight files if you need persistence for example.
That can be done with creating an adapter with a defined interface. The first adapter you write is for your database.
When the config object is created, you pass in the adapter. As it has a defined interface, the config class can work with any adapter that has that interface. First of all the database.
Then you either mock an adapter or write an adapter of it's own for your tests. Inside your tests, you don't use the database adapter, but the test adapter.
You then can unit-test the config class independent of the database.
I'm writing a PHPUnit test case for an API (so not exactly a unit test) and I'm thinking about having a test that all other tests will depend on.
The tests in the test case make API requests. Most of these requests require a user. The test in question will create that user that the other tests will use.
Would that be a horrible idea?
I think that the best way for unit tests is to eliminate the dependencies first.
You can abstract the end point with your own local version that will return predictible results. This way you can test that your requests are correct.
You can abstract the data providers (database, filesitem, etc...) with your stubs that will also return predictible data (username, etc..).
After that you just test your request and see they are correct..
The second part is to actualy test the data providers, with different tests, so you know that the good username will be given.
And then you can test the API connectivity, etc..
EDIT. If you have dependencies in your code, and it's difficult to abstract the providers or the end point web service, you may need to adjust your code so that it will accept references to those objects as parameters. Than in your tests you change the objects passed with your own stub objects. In production you pass the correct references, so that you will not need to change your code for testing.
I hope i have been clear. If not, ask me and i can explain better, maybe i did not understand your question well
I am quite new to unit testing and testing in general. I am developing with phpUnit, but as my question is more general / a design question, the actual environment shouldn't be of too much importance.
I assume, that it is a good practice, to write your testcases as specific as possible. For example (the later the better):
assertNotEmpty($myObject); // myObject is not Null
assertInternalType('array', $myObject); // myObject is an array
assertGreaterThan(0, count($myObject)); // myObject actually has entries
If that is correct, here is my question:
Is it a accepted practice to write some flow control inside a testcase, if the state of the object one is testing against depends on external sources (i.e. DB) - or even in general?
Like:
if (myObject !== null) {
if (count(myObject) > 0) {
// assert some Business Logic
}
else {
// assert some different Business Logic
}
}
Is this kind of flow control inside a testcase admissible or is a "code smell" and should be circumvented? If it is ok, are there any tips or practices, which should be kept in mind here?
Paul's answer addresses test method scope and assertions, but one thing your question's code implied is that you would test A if the returned object had value X but test B if it had value Y. In other words, your test would expect multiple values and test different things based on what it actually got.
In general, you will be a happier tester if each of your tests has a single, known, correct value. You achieve this by using fixed, known test data, often by setting it up inside the test itself. Here are three common paths:
Fill a database with a set of fixed data used by all tests. This will evolve as you add more tests and functionality, and it can become cumbersome to keep it up-to-date as things move. If you have tests that modify the data, you either need to reset the data after each test or rollback the changes.
Create a streamlined data set for each test. During setUp() the test drops and recreates the database using its specific data set. It can make writing tests easier initially, but you still must update the datasets as the objects evolve. This can also make running the tests take longer.
Use mocks for your data access objects when not testing those DAOs directly. This allows you to specify in the test exactly what data should be returned and when. Since you're not testing the DAO code, it's okay to mock it out. This makes the tests run quickly and means you don't need to manage data sets. You still have to manage the mock data, however, and write the mocking code. There are many mocking frameworks depending on your preference, including PHPUnit's own built-in one.
It's okay to have SOME control flow within your testcases, but in general, understand that your unit tests will work out best if they are disjoint, that is, they each test for different things. The reason this works out well is that when your test cases fail, you can see precisely what the failure is from the testcase that failed, as opposed to needing to go deeper inside a larger test case to see what went wrong. The usual metric is a single assertion per unit test case. That said, there are exceptions to every rule, and that's certainly one of them; there's nothing necessarily wrong with having a couple of assertions in your test case, particularly when the setup / teardown of the test case scenario is particularly difficult. However, the real code smell you want to avoid is the situation where you have one "test" which tests all the code paths.
I get the idea behind unit testing however am trying to think of a clean simple way do it when it requires database functionality. For instance I have a function that return result based off a database select query. Would the database alway have to remain the same for me to properly see that only the correct results are being returned. What is the best way to perform unit testing (in PHP) when it requires database inactivity (whether it be read, write, update, or delete)?
There is a whole chapter on that in the PHPUnit manual:
http://www.phpunit.de/manual/current/en/database.html and also worth reading
http://matthewturland.com/2010/01/04/database-testing-with-phpunit-and-mysql/
It's like with everything else when Unit-Testing. Create a known state and test your code against it to see if it returns the expected results.
Personally, I create a dummy testing database and populate it with a known data set for each testing run (I do that right in the setUp functions). Then the tests run against that data set, and then it's removed on tearDown...
Now, this is more of a Integration test than an Unit test (And personally I treat it differently from a unit test, run on its own schedule along with other integration tests), but it's still quite useful.
It is not a unit test if it needs the database.