is it better to fake dependencies (for example Doctrine) for unit-tests or to use the real ones?
In a unit test, you use only ONE real instance of a class, and that is the class that you want to test.
ALL dependencies of that class should be mocked, unless there is a reason not to.
Reasons not to mock would be if data objects are being used that have no dependencies itself - you can use the real object and test if it received correct data afterwards.
Another reason not to mock would be if the configuration of the mock is too complicated - in that case, you have a reason to refactor the code instead, because if mocking a class is too complicated, the API of that class might be too complicated, too.
But the general answer: You want to always mock every dependency, every time.
I'll give you an example for the "too-complicated-so-refactor" case.
I was using a "Zend_Session_Namespace" object for internal data storage of a model. That instance got injected into the model, so mocking was not an issue.
But the internal implementation of the real "Namespace" class made me mock all the calls to __set and __get in the correct order of how they were used in the model. And that sucked. Because every time I decided to reorder the reading and writing of a value in my code, I had to change the mocking in the tests, although nothing was broken. Refactoring in the code should not lead to broken tests or force you to change them.
The refactoring added a new object that separates the "Zend_Session_Namespace" from the model. I created an object that extends "ArrayObject" and contains the "Namespace". On creation, all the values got read from the Namespace and added to the ArrayObject, and on every write, the value also gets passed to the Namespace object as well.
I now had the situation that I could use a real extended ArrayObject for all my tests, which in itself only needed an unconfigured mocked instance of "Zend_Session_Namespace", because I did not need to test whether the values were correctly stored in the session when I tested the model. I only needed a data store that gets used inside the model.
To test that the session gets correctly read and written, I have tests for that ArrayObject itself.
So in the end I am using a real instance of the model, and a real instance of the data store together with a mocked instance of "Zend_Session_Namespace" which does nothing. I deliberately chose to separate "model stuff" and "session save stuff" which had been mixed into the model class before -> "single responsibility principle".
The testing really got easier that way. And I'd say that this is also a code smell: If creating and configuring the mock classes is complicated, or needs a lot of changes when you change the tested class, it is time to think about refactoring. There is something wrong there.
Mocking should be done for a reason. Good reasons are:
You can not easily make the depended-on-component (DOC) behave as intended for your tests.
Does calling the DOC cause any non-derministic behaviour (date/time, randomness, network connections)?
The test setup is overly complex and/or maintenance intensive (like, need for external files)
The original DOC brings portability problems for your test code.
Does using the original DOC cause unnacceptably long build / execution times?
Has the DOC stability (maturity) issues that make the tests unreliable, or, worse, is the DOC not even available yet?
For example, you (typically) don't mock standard library math functions like sin or cos, because they don't have any of the abovementioned problems.
Why is it recommendable to avoid mocking where unnecessary?
For one thing, mocking increases test complexity.
Secondly, mocking makes your tests dependent on the inner workings of your code, namely on how the code interacts with the DOCs. Would be acceptable for white box tests where the implemented algorithm is tested, but not desirable for black box tests.
Related
In my example I'm using the PHP framework Yii2 but I think this applies to most OO languages.
I have an ActiveRecord base class which most of my business objects extend from e.g. Project.
At the moment if I want a Project instance I call
Project::findOne(['id' => $id]);
findOne is a static method of ActiveRecord (which is part of the Yii2 framework). So this is bad form because I can't easily mock/stub the return of this call when writing unit tests.
But what's the best way to get around this?
I could create a class CActiveRecord that inherits from ActiveRecord and wrap the static call in a non-static call and use that everywhere - but then I would have to instantiate a throw-away Project object in order to get the actual instance. What if the Project object needed some heavy config to be instantiated - I would be passing random nonsense into the constructor just to get an instance.
Summary:
Simply changing statics to non-statics seems wrong - shouldn't I also move the functions somewhere else? If so, where?
The issue with static calls is the hard coupling to a specific other piece of code. Just wrapping that in a "dynamic" call doesn't make this any better:
$c = new CProject;
$c->findOne(); // Calls Project::findOne()
That's pretty darn pointless. The issue is not the syntax of -> vs. ::, the issue is that this particular code references a specific other class and that you cannot easily exchange this class for something else. You're building rigid, hardcoded dependencies between your classes/objects, which makes it hard to take them apart, which makes your code hard to test, and which makes it harder to adapt code to different situations.
The alternative is dependency injection:
function foo(Project $project) {
$p = $project->findOne();
}
This function is not coupled to any one specific Project class, but to a class which simply offers an interface akin to Project. In fact, Project could even be simply an interface. Which specific class and method is getting called here then is decided somewhere completely different, like your dependency injection container; or simply the caller of this code.
This makes it a lot easier to take this code apart and put it back together in different ways, as necessary for the situation at hand. That's not to say it can't work and that you should never use static calls at all, but you really need to be aware of what cross-dependencies you're establishing with every hardcoded class name, and whether that may or may not cause a problem down the line. For even moderately complex and/or growing software projects, it will almost certainly cause friction in some form or another eventually.
See How Not To Kill Your Testability Using Statics for a longer in-depth article.
In my example I'm using the PHP framework Yii2 but I think this applies to most OO languages.
I have an ActiveRecord base class which most of my business objects extend from e.g. Project.
At the moment if I want a Project instance I call
Project::findOne(['id' => $id]);
findOne is a static method of ActiveRecord (which is part of the Yii2 framework). So this is bad form because I can't easily mock/stub the return of this call when writing unit tests.
But what's the best way to get around this?
I could create a class CActiveRecord that inherits from ActiveRecord and wrap the static call in a non-static call and use that everywhere - but then I would have to instantiate a throw-away Project object in order to get the actual instance. What if the Project object needed some heavy config to be instantiated - I would be passing random nonsense into the constructor just to get an instance.
Summary:
Simply changing statics to non-statics seems wrong - shouldn't I also move the functions somewhere else? If so, where?
The issue with static calls is the hard coupling to a specific other piece of code. Just wrapping that in a "dynamic" call doesn't make this any better:
$c = new CProject;
$c->findOne(); // Calls Project::findOne()
That's pretty darn pointless. The issue is not the syntax of -> vs. ::, the issue is that this particular code references a specific other class and that you cannot easily exchange this class for something else. You're building rigid, hardcoded dependencies between your classes/objects, which makes it hard to take them apart, which makes your code hard to test, and which makes it harder to adapt code to different situations.
The alternative is dependency injection:
function foo(Project $project) {
$p = $project->findOne();
}
This function is not coupled to any one specific Project class, but to a class which simply offers an interface akin to Project. In fact, Project could even be simply an interface. Which specific class and method is getting called here then is decided somewhere completely different, like your dependency injection container; or simply the caller of this code.
This makes it a lot easier to take this code apart and put it back together in different ways, as necessary for the situation at hand. That's not to say it can't work and that you should never use static calls at all, but you really need to be aware of what cross-dependencies you're establishing with every hardcoded class name, and whether that may or may not cause a problem down the line. For even moderately complex and/or growing software projects, it will almost certainly cause friction in some form or another eventually.
See How Not To Kill Your Testability Using Statics for a longer in-depth article.
I'm designing a class that has two dependencies. One of the dependency classes has been written and tested. The other has not yet been written.
It has occurred to me because the remaining dependency will be written to facilitate the class that will use it, that I should write the latter first, and design the interface of the former as I go along, learning what it should do.
That seems to me to be a great way to make code. After all, as long as the main class gets a mock in its constructor, I can write it and test it without it being aware that its dependency doesn't exists, then I can create the dependency once I am sure I know what I need.
So: how do I do this? Create a skeleton class that I modify as I go along. Perhaps something like:
class NonExistantSkeleton
{
public function requiredMethod1()
{
}
public function newlyDiscoveredRequirement()
{
}
}
and then mock it using PHPUnit, and setting up stubs, etc, to keep my class under development happy?
Is this the way to go?
It seems like a nice way to develop code - and seems to me to make more sense than developing a dependency, without really knowing for sure how it's going to be used.
In short:
Yes. At least thats what I'm doing right now.
Longer Version:
If the expected collaborators of your class don't exist at the point in time where you need them in your tests for the class you are building you have a few options:
Mock non existing classes (which phpunit can do)
Create class skeletions and mock those
Just create interfaces and get mocks for those (which phpunit can do too)
Maybe you don't need any of the above depending on the object
If you programm against an interface anyways than all you need to do is to create that interface and tell PHPUnit to create a stub/mock from it
+No new class without a test
+Using interfaces when appropriate is considered nicer/better than just hinting against classes
When mocking non existing classes you get some drawbacks that I don't like:
-High mock maintenance cost
-Chaning the methods on that classes is slow and tedious
-If you created the class you should rework the mocks again
so I'd advice against that.
The middle way would be to just create the empty class skeleton with its method and use those for mocking.
I quite like that way in cases where there is no interface to hint against as It is fast and creates stable test code.
Having barebone classes with public apis, for me, is no violation of TDD.
There are classes you don't need to mock.
Data transfer objects and Value Objects can always be created anywhere using the new in your production code so your tests also can just the the real objects.
It helps to keep your tests a little cleaner as you don't need to mock/expect a lot of getter/setter methods and so on.
If you follow a test-driven development methodology then the usual approach is as follows:
Figure out what your classes are meant to do, and what their public-facing APIs should be.
Implement "empty" classes that consist of nothing but the public methods signitures with empty bodies (as you have done in the code example you gave).
Work out an implementation strategy. This means working out which classes are dependant on each other and implementing them in an order that means that dependant classes aren't implemented until the classes it depends on are finished, or at least sufficiently functional to develop against. This means doing the classes with no dependencies first, then the classes that depend only on the completed classes, and so on.
Write your tests. It's possible to write the tests now because you know what the black box for your classes look like, what they need to take as input and what they're supposed to return as output.
Run the tests. You should get 0% success, but also 100% code coverage. This is now your baseline.
Start to implement your classes according to your implementation strategy. Run your unit tests from time to time during this process, say once you've got a class completed, to make sure that it meets its specification as laid down in the unit test. Ideally, each test should show an increase in test passes whilst maintaining 100% code coverage.
EDIT: As edorian pointed out, PHP interfaces are a huge help here because PHPUnit can generate mocks and stubs from interfaces as well as from classes. They're also an excellent tool in reducing coupling and improving substitutability in general. They allow you to substitute any class that implements the expected interface, instead of just subclasses of the expected class.
Given the following methods:
public function setFoo($foo) {
$this->_foo = $foo;
return $this;
}
public function getFoo() {
return $this->_foo;
}
Assuming, they may be changed to be more complex in the future:
How would you write unit tests for those methods?
Just one test method?
Should I skip those tests?
What about code coverage?
How about #covers annotation?
Maybe some universal test method to implement in the abstract test case?
(I use Netbeans 7)
This seems like a waste of time, but I wouldn't mind if IDE would generate those test methods automatically.
To qoute from the comment of Sebastian Bergman's blog:
(it's like testing getters and setters -- fail!). In any case, if they were to fail; wouldn't the methods that depend on on them fail?
So, what about the code coverage?
If you do TDD you should write a test for getter and setter. too. Do not
write a single line of code without a test for it - even if your code is
very simple.
Its a kind of religious war to use a tandem of getter and setter for
your test or to isolate each by accessing protected class members using
your unit test framework capabilities. As a black box tester i prefer to
tie my unit test code to the public api instead of tie it to the
concrete implementation details. I expect change. I want to encourage
the developers to refactor existing code. And class internals should not
effect "external code" (unit tests in this case). I don want to break
unit tests when internals change, i want them to break when the public
api changes or when behavior changes. Ok, ok, in case of a failing unit
test do not pin-point to the one-and-only source of problem. I do have
to look in the getter AND the setter to figure out what caused the
problem. Most of the time your getter is very simple (less then 5 lines
of code: e.g. a return and an optional null-check with an exception). So
checking this first is no big deal and not time consuming. And checking
the happy path of the setter is most of the time only a little more
complex (even if you have some validation checks).
Try to isolate your test cases - write a test for a SUT (Subject under
test) that validates its correctness without reley on other methods
(except my example above). The more you isolate the test, the more your
tests spot the problem.
Depending on your test strategy you may be want to cover happy path only
(pragmatic programmer). Or sad pathes, too. I prefer to cover all
execution pathes. When i think i discovered all execution pathes i check
code coverage to identify dead code (not to identify if there are
uncovered execution pathes - 100% code coverage is a missleading indicator).
It is best practice for black box testers to use phpunit in strict mode
and to use #covers to hide collateral coverage.
When you write unit test your test on class A should be executed independent from class B. So your unit tests for class A should not call / cover method of class B.
If you want to identify obsolete getter/setter and other "dead" methods (which are not used by production code) use static code analysis for that. The metric you are interested in is called "Afferent coupling at method level (MethodCa)". Unfortunately this metric (ca) is not available at method-level in PHP Depend (see: http://pdepend.org/documentation/software-metrics/index.html and http://pdepend.org/documentation/software-metrics/afferent-coupling.html). If you realy need it, feel free to contribute it to PHP Depend. An option to exclude calls from the same class would be helpful to get a result without "collateral" calls. If you identify a "dead method" try to figure out if it is meant to be used in near future (the counterpart for an other method that has a #depricated annotation) else remove it. In case it is used in the same class only, make it privat / protected. Do not apply this rule to library code.
Plan B:
If you have acceptance tests (integration test, regression test, etc.) you can run that test without running unit tests at the same time and without phpunits strict mode. This can result in a very similar code coverage result as if you had analysed your production code. But in most cases your non-unit tests are not as strong as your production code is. It depends on your discipline if this plan B is "equal enought" to production code to get a meaningful result.
Further reading:
- Book: Pragmatic Programmer
- Book: Clean Code
Good Question,
i usually try not to test getters&setters directly since i see a greater benefit in testing only the methods that actually do something.
Especially when not using TDD this has the added benefit of showing me setters that i don't use in my unittests showing me that ether my tests are incomplete or that the setter is not used/needed. "If i can execute all the "real" code without using that setter why is it there."
When using fluent setter i sometimes write a test checking the 'fluent' part of the setters but usually that is covered in other tests.
To answer your list:
Just one test method?
That is my least favorite option. All or none. Testing only one is not easy for other people to understand and looks 'random' or needs to be documented in a way.
Edit after comment:
Yes, for "trivial" get/set testing I'd only use one method per property maybe depending on the case even only one method for the whole class (for value objects with many getters and setters I don't want to write/maintain many tests)
How would you write unit tests for those methods?
Should I skip those tests?
I wouldn't skip them. Maybe the getters depending on how many you have (i tend to write only getters i actually need) but the task of having a class completely covered shouldn't fail because of getters.
What about code coverage?
How about #covers annotation?
With #covers my take is always "use it everywhere or don't use it at all". Mixing the two 'styles' of testing takes away some of the benefits of the annotation and looks 'unfinished' to me.
Maybe some universal test method to implement in the abstract test case?
For something like value objects that could work nicely. It might break (or gets more complicated) once you pass in objects / array with type hinting but I'd presonally prefer it over writing manual tests for 500 getters and setters.
This is a common question but strangely can't find a dupe on SO.
You could write unit tests for accessors but the majority of practioners do not. i.e. if the accessors do not have any custom logic, I would not write unit tests to verify if field access works.
Instead I would rely on the consumers of these accessors to ensure that the accessors work. e.g. If getFoo and setFoo don't work, the callers of these method should break. So by writing unit tests for the calling methods, the accessors get verified.
This also means that code coverage should not be a problem. If you find accessors that are not covered after all test suites are run, maybe they are redundant / unused. Delete them.
Try to write a test that illustrates a scenario where a client will use that accessor. e.g. The below snippet shows how the Tooltip (property) for the Pause Button toggles based on its current mode.
[Test]
public void UpdatesTogglePauseTooltipBasedOnState()
{
Assert.That(_mainViewModel.TogglePauseTooltip, Is.EqualTo(Strings.Main_PauseAllBeacons));
_mainViewModel.TogglePauseCommand.Execute(null);
Assert.That(_mainViewModel.TogglePauseTooltip, Is.EqualTo(Strings.Main_ResumeAllBeacons));
_mainViewModel.TogglePauseCommand.Execute(null);
Assert.That(_mainViewModel.TogglePauseTooltip, Is.EqualTo(Strings.Main_PauseAllBeacons));
}
I'm stuck with a general OOP problem, and can't find the right way to phrase my question.
I want to create a class that gives me an object which I can write to once, persist it to storage and then be unable to change the properties. (for example: invoice information - once written to storage, this should be immutable). Not all information is available immediately, during the lifecycle of the object, information is added.
What I'd like to avoid is having exceptions flying out of setters when trying to write, because it feels like you're offering a contract you don't intend to keep.
Here are some ideas I've considered so far:
Pass in any write-information in the constructor. Constructor throws exception if the data is already present.
Create multiple classes in an inheritance tree, with each class representing the entity at some stage of its lifecycle, with appropriate setters where needed. Add a colletive interface for all the read operations.
Silently discarding any inappropriate writes.
My thoughts on these:
1. Makes the constructor highly unstable, generally a bad idea.
2. Explosion of complexity, and doesn't solve the problem completely (you can call the setter twice in a row, within the same request)
3. Easy, but same problem as with the exceptions; it's all a big deception towards your clients.
(Just FYI: I'm working in PHP5 at the moment - although I suspect this to be a generic problem)
Interesting problem. I think your best choice was #1, but I'm not sure I'd do it in the constructor. That way the client code can choose what it wants to do with the exception (suppress them, handle them, pass them up to the caller, etc...). And if you don't like exceptions, you could move the writing to a write() method that returns true if the write was successful and false otherwise.