I have a function, which someone else wrote, that creates a cURL wrapper object inside the function. Simplified version below
public function getCodes()
{
//do some stufff
$communicator = new Communicator();
$result = $communicator->call($this->API_KEY);
//do some stuff with $result
}
I was tasked with learning PHPUnit and writing tests for this type of code. In doing so I found that it's really hard to test a function like this when the object is created inside of the function and also that tests shouldn't require any outside communication to work.
We wanted to push our tests to git as many projects do but we didn't want to accidentally or intentionally push our API credentials to git.
So my solution was to keep getCodes() public but make it a wrapper for a private function that accepts a Communicator object as a parameter. Then I could test the private method with a mock Communicator object.
But this would mean that getCodes is never tested (my boss wants 100% code coverage) and I also read that you shouldn't be writing tests for private functions in most circumstances.
So my question is basically, how do I write a test for a function like this with an API call.
I would really suggest rewriting the code to inject the Communicator object via constructor.
If you already see there is a big issue with writing tests for something it is a very strong signal to re-thing the current implementation.
Another thing is that you shouldn't test your privates. Sebastian Bergmann wrote a post on his blog about this some time ago and the conclusion is - it is possible just not good (https://sebastian-bergmann.de/archives/881-Testing-Your-Privates.html).
Completely different thing is that I think your tests shouldn't go outside of the boundaries of your system. That is - mock everything that connects to outside systems. Such tests may fail for various of reasons non of witch are valid from the sole perspective of running tests.
You also mentioned coverage. Unfortunately this is something where, I hope, everyone will agree - you cannot have it the moment you start using native PHP resources (with some small exception like FS). You have to understand that things like curl, ssh, ftp and so on cannot be unit tested.
Related
I recently tried to improve my unit testing skills and read quite some literature about unit testing and I am also trying to realize what I learned in a php-project I am currently developing with phpunit. But I still have a in my opinion very fundamental question how to unit test methods which interact with objects of other classes or even with other methods of the same class.
Is there some rule of thumb or some help how I can decide what dependencies I should stub/mock and for what dependencies I should simply use a normal object? To clarify my question, here is an example code, with different scenarios:
interface DependencyInterface {
public method dependentMethod() { ... }
}
class Dependency implements DependencyInterface {...}
class ClassUnderTest {
private $dependency
public __construct(DependencyInterface $dependency) {
$this->dependency = dependency;
}
public function methodUnderTest() {
...
$result1 = $this->dependency->dependentMethod();
...
$result2 = $this->otherMethod();
...
$result3 = $this->usedInMultiplePublicMethods();
}
public function otherMethod() {...}
private function usedInMultiplePublicMethods() {...}
}
So my questions are now for a unit test which tests the functionality of the method methodUnderTest, should I:
stub the interface DependencyInterface and inject it in the constructor, or should I simply use an instance of the implementation Dependency?
partially stub the class ClassUnderTest itself to deliver a fixed result for otherMethod, since this maybe very complex method has already its own complete unit tests?
I decided to not unit tests private methods, since they are not part of the interface of the class (I know this is a controversial topic and is not the scope of my question). Do I have now to cover for each public method which uses the private method usedInMultiplePublicMethods all possible effects which may occur in the private method? Or should I only test all possible effects in one of the public methods which uses it and stub the private method in the tests for all other public methods?
I am quite not sure about when to use stubbing/mocking and when not.
The why of mocking is to be able to write unit test that means a test which is: fast, isolated, repeatable, self validating and Thorough and Timely (F.I.R.S.T)
To be able to test a unit/module in isolation you may need to mock/stub any external module (database access, api call, logging system...).
For your points 1 & 2 , rad's answer points to main underlying principles to keep in mind e.g. if you are going to test a logic which is using a database service to fetch data and then doing computations on fetched data, would you mock that database service or use real database service ?
As is clear from objective - you are unit testing logic itself and not database service data fetch so you would mock database service and would assume that database service is giving correct data & you would simply focus on testing logic on computed data. You will have separate tests for database fetching service and that isolation property is all about.
The word unit is significant in that sense that your these tests should be very focused on current logic only by limiting your scope & by not cluttering everything else into it.
This answer is mainly for your points # 3. Its OK to not explicitly test private methods but if you go by basic purpose of unit tests - you wouldn't be much worried about something being private or public. Somewhere down the line, unit testing is for developer self satisfaction too & it would simply make your code more robust if unit tests are written for private methods too.
Just because access level is private doesn't change the basic concept that its a piece of logic that needs testing. Code coverage wise , you might be OK from one public method but I am of view that you should treat calls from different public methods as distinct.
Never forget the basic purpose of unit tests - that you are trying to find errors in your logic, trying to cover all boundary cases & trying to make your code more robust.
I am new to PHPUnit, infact I started today. And, as far as I have been reading, I came to understand only what this script does.
class UserTest extends PHPUnit_Framework_TestCase
{
protected $user;
// test the talk method
protected function setUp() {
$this->user = new User();
$this->user->setName("Tom");
}
protected function tearDown() {
unset($this->user);
}
public function testTalk() {
$expected = "Hello world!";
$actual = $this->user->talk();
$this->assertEquals($expected, $actual);
}
}
For this class:
<?php
class User {
protected $name;
public function getName() {
return $this->name;
}
public function setName($name) {
$this->name = $name;
}
public function talk() {
return "Hello world!";
}
}
Ok, so I have established that the test returns a Ok/Fail statement based on the equality of the test, but what I am looking for is more. I need an actually ways to test a more complex class that, its outcome, un-like in this example can not be guessed easily.
Say, I write a script that does polling. How would, or in what ways can I test if the methods/classes can work? The above code only shows if a methods outcome will only be 'Hello World' But, that is too easy thing to test, because I need to test complex things and there aren't much tutorials for that.
Testing a class in a unit test is supposed to NOT be complex.
And the reason for this to be true is that in a well designed system, you can test that class in isolation, and do not add the complexity of the system behind that class - because that system does not exist during the test, it is mocked.
The classic example for this is that a User class usually asks a database for info, but in the test case, that database is really hard to set up, prepare with data, and destroy afterwards. And using a real database also slows things down. So you design the User class to accept a database object from the outside, and in the test case you give to it a mock object of the database.
That way, it is really simple to simulate all kinds of return values from the database, and additionally you can check whether the database object gets the right parameters without the complexity of dealing with a real database.
You can only do proper mock object injection if your classes are designed to allow dependency injection. That is a principle to NOT create objects inside of other objects, but require the outside world to supply them. Just have a quick look at some video for explanation:
Dependency Injection - Programming With Anthony (Jan 2013)
And remember that creating good tests needs some experience. Good for you to start experimenting.
But, that is too easy thing to test, because I need to test complex things and there aren't much tutorials for that.
The more complex, the more complex the tests are. It's a bit like a snake biting in its own end. You normally want to prevent that, so to make it golden: Write simple tests to ensure that a complex software is tested and runs.
This does not always work 100% but it works better than without tests. PHPUnit has been designed for Unit-Tests (Xunit test patterns) but you can also use it to run different tests. For that you group tests. This is done differently by different people depending on different things. For example:
Small tests
Medium tests
Large tests
or (not equivalent):
Unit tests
Integration tests
Acceptance tests
or (perhaps equivalent TestPyramid):
Unit tests
Service tests
UI tests
and what not. When you start with testing, it's probably good to start with the unit-tests and as Sven already answered, keep them simple. Get comfortable with TDD, read some slides and books. And welcome to the world of automating tests.
What is Unit test, Integration Test, Smoke test, Regression Test?
P.S. Yes, simple getter setters are too easy to test. If all a setter does is storing to a private member and the getter gets it back, you can trust that PHP is working, writing a test for that is a waste of time and will only lead to cruft. It clearly shows that somebody wrote the test after the code has been written. Instead write first the test and see it fail (red), than hack together the code as quick as possible to get the test to pass (green). You can improve the code later on as the test already shows it's working.
I am writing a Unit Test for a PHP library that uses SOAP. The test case has two tests in it.
The library is falling over on the second unit test with the following message (I am using PHPUnit):
SoapClient::__call(): supplied argument is not a valid sdl resource
/test/SOAPLib.php:186
/test/SOAPLibTest.php:56
Line 186 of the library I am testing is:
$issue = $client->getIssueById($auth_token, $id);
This does not fail when called in the usual fashion from any other script - It is only failing when used from the second unit test. This implies that there is some kind of authentication or token error with the global variables in my second SOAP call.
Line 56 of the UnitTest file is:
list($summary, $pri) = \SOAPLibTest\get_issue_by_id($id);
I have looked up SoapClient::_call(): in the PHP manual, which says that function is deprecated, and I should be using SoapClient::_soapCall instead. The problem is that I don't know how to change it to that.
Attempting to update php-soap reports that I am already up-to-date.
I really am stumped on this one, and there doesn't appear to be any immediate help from Google.
Many thanks,
ns
I managed to find the root of the problem - It was actually a PHPUnit problem with some global variables I had in my SOAP library. I will change the title of this question to reflect PHPUnit, not php-soap as I originally thought.
This blog post by Sebastian Bergmann (The creator of PHPUnit) was really helpful:
http://sebastian-bergmann.de/archives/797-Global-Variables-and-PHPUnit.html
Some important statements he included were:
It is hard to test code that uses singletons. The same is true for code that uses global variables."
Typically, the code you want to test is coupled strongly with a global variable and you cannot control its creation. An additional problem is the fact that one test's change to a global variable might break another test.
The implementation of the backup and restore operations for the global and super-global variables uses serialize() and unserialize(). Objects of some classes that are provided by PHP itself, such as PDO, cannot be serialized and the backup operation will break when such an object is stored in the $GLOBALS array, for instance.
The backup and restore operations for the global and super-global variables can be disabled like this:
<?php
class MyTest extends PHPUnit_Framework_TestCase
{
protected $backupGlobals = FALSE;
// ...
}
?>
I hope this helps some other people.
Regards,
ns
Context
I recently inherited the development and maintenance of a well-programmed PHP application (sarcasm). The application is based on a commercial software (which I will not name), and there is a layer of customization (ours) that is built on top of it.
Unfortunately, this application uses a ton of globals and singletons (pun-intended). I have built test cases for all the things we've overridden. However, a lot of things relies on some global state of something, which can cause race conditions and all sorts of weird stuff.
Randomizing the tests
In order to catch most of these weird-o-tons (I like to call them that), I have built a PHPUnit TestDecorator, [as documented in the manual][1]. This one:
class PHPUnit_Extensions_Randomizer extends PHPUnit_Extensions_TestDecorator
{
public function __construct(PHPUnit_Framework_Test $test)
{
$tests = $test->tests();
$shuffle = array();
foreach ($tests as $t) {
if ($t instanceof PHPUnit_Framework_TestSuite) {
$shuffle = array_merge($shuffle, $t->tests());
} else {
$shuffle[] = $t;
}
}
shuffle($shuffle);
$suite = new PHPUnit_Framework_TestSuite();
foreach ($shuffle as $t) $suite->addTest($t);
parent::__construct($suite);
}
}
It basically randomizes the tests order to make sure a test doesn't depend on a global state that may or may not be correct.
The question
The problem arose when came the time to test my custom decorator. I have not found anywhere in the manual, Google, or Stack Overflow how to load it.
When analyzing the code, I saw that PHPUnit itself was instantiating the RepeatedTest decorator in the TextUI_TestRunner::doRun() method. I know I can subclass the TestRunner, override doRun(), arrange for my decorator to be created and then just call the parent::doRun() with my decorator instance as the argument, override TextUI_Command and create a new CLI script to use my stuff instead of the built-in stuff.
Before I (re-)invent the wheel, I was just wondering, is it possible do load a custom decorator without subclassing the TestRunner?
Thanks.
With current PHPUnit Versions there is no easy way to plug in in your own code for the most part. The only thing that offers interchangeability is the TestRunner and what you described makes sense to me.
I'm not aware of any other way to provide test decorators or change out most of the other classes phpunit uses.
The way you want to change the test execution order seems to work out even so I'm not sure how well it would shuffle suits.
Another way to achieve this thats maybe less hassle would be to create a subclass of PHPUnit_Framework_TestSuite randomly addTest your code.
If that doesn't work out another option would be to use the xml configuration file and build the test suite from <file> tags and before every execution have some code shuffle those tags. Afaik phpunit doesn't sort them in any way so execution will be random
Are you looking to see if every tests on its on really works and want to find interdependent tests?
Or do you really want to see if something breaks horribly when you do a lot of stuff that should not change anything in the wrong order?
I'm asking just in case you haven't considered --process-isolation yet. (Which i assume you have but asking isn't going to hurt and might save you some time and effort :) )
When you run every test with a fresh set of globals you will at least find all test interdependencies and thats just one cli switch away to make sure every test in your suite works fine on its own.
I am developing an API documentation system, and want to dynamically check that each command has documentation attached.
The easiest way to do this is dynamically loop through each command and check for existing documentation to match it.
My code looks like this:
public function testMissingDocs()
{
foreach ($aCommands as $sKey => $aOptions)
{
$this->assertNotNull($oDocs->get($sKey));
}
}
The problem with this is the StopOnFailure/Error feature of PHPUnit which stops the test after the first assertion fails. I understand the reasons for this functionality and I want to keep it on for the majority of my test cases, but for dynamic assertions/tests it makes things a bit hard.
Is there a way to disable it on a per-test basis so I can check each command in this test?
You can use a data provider to split the single test into as many tests as you have commands.
/**
* #dataProvider getDocsForAllCommands
*/
public function testEveryCommandHasDocs($sKey)
{
$this->assertNotNull($oDocs->get($sKey));
}
public function getKeysForAllCommands()
{
return array_keys($aCommands);
}
If the documentation for a particular class or method is missing, that would represent a problem with that class, not with the method to retrieve the documentation.
Although it is probably easier to combine all of the documentation into a single test, that does not follow unit testing best practices (and hence is why the PHPUnit framework is working against you rather than for you).
I would suggest one of two approaches to rectify the issue:
Refactor your tests so that each class and/or method has its own documentation check (there are a few ways you can run multiple tests in one execution).
Use a tool such as PHP_CodeSniffer instead, as lack of documentation could be considered a convention – rather than a functional – defect.