I have an unusual problem regarding PhpUnit/unit testing and code coverage.
Background
Background is important in this question. The code is to be tested and 100% coverage is required as part of an audit process. I am aware of points such as "state X could never happen" or "you're not testing this usefully", or other similar points. This is a paper exercise and a project requirement.
Current problems/items
As an example using a php function (random_int)which illustrates the issue:
The function random_int($min, $max) is a core php function.
If this is wrapped (simplified for illustatrative purposes):
class Foo
{
public function Bar(int $min, int $max)
{
return random_int($min, $max)
}
}
Please note that this is not the entire purpose of Foo.. I'm simply removing easy to test things in this example.
This is fine. However, this throws an \Exception and part of the code specification is that we should return exceptions in our own namespace/Exception classes (also including suitably verbose responses etc).
So, if the class FooException is defined in an appropriate namespace, this can be set up as follows:
class Foo
{
public function bar(int $min, int $max)
{
try {
$response = random_int($min, $max);
} catch (Exception $e) {
throw new FooException('Foo::bar() threw an Exception: ' . $e->getMessage());
}
}
}
This effectively ensures that it's almost (but not definitely) impossible to trigger this. (random_int throws an \Exception if it was not possible to gather sufficient entropy... which is unlikely)
Mocking Attempt
In order to remedy this, it's fairly simple to create a call that simply wraps only the function
class Foo
{
public function bar(int $min, int $max)
{
try {
$response = $this->randomInt($min, $max);
} catch (Exception $e) {
throw new FooException('Foo::bar() threw an Exception: ' . $e->getMessage());
}
}
protected function randomInt(int $min, int $max)
{
return random_int($min, $max);
}
}
In the unit test, these can then be mocked using a partial mock:
class FooTest extends TestCase
{
public function testRandomIntegerException()
{
// mock to prove an exception:
$example_class_instance = $this->createPartialMock(Foo::class, ["randomInt"]);
$expected_exception = new FooException();
$example_class_instance
->expects($this->any())
->method("randomInt")
->willThrowException($expected_exception);
// Trigger exception
$this->expectException(GeneratorException::class);
$example_class_instance->randomInteger(1, 5);
}
This works as expected for Unit Tests, and the FooException is correctly generated.
However, this results in the code coverage not counting that test; the unit test is against a mock object, not an instance of the class.
Using namespace overloading
It's possible to use namespace overloading:
// overwrite the base functions in the same namespace as the Generator
namespace FooSpace\;
function random_int($min, $max): int
{
throw new \Exception('Test exception');
}
namespace FooSpaceTests;
class FooTest extends TestCase
{
public function testRandomInteger()
{
$foo = new Foo();
// Trigger exception
$this->expectException(FooException::class);
$foo->randomInteger();
}
}
This also works in terms of a test. It also ensures that the code coverage is correctly set up. However, this function overwrite is permanent as the classes are parsed. This messes other testing up, even if its in a different file/test suite.
I don't think it's possible to "unset" or "undefine" function after these tests.
TL;DR
For a paper/requirements testing suite that should give 100% code coverage
How do you correctly test a class method that includes core php functions?
Thanks in advance
Related
I am writing phpunit tests for my app and one of the units under test uses the following function:
private function createRandomString(): string {
try {
return bin2hex(random_bytes(32));
} catch (Exception) {
$this->logger->warning('Unable to created random binary bytes.');
}
return substr(str_shuffle('0123456789abcdefABCDEF'), 10, 64);
}
The random_bytes function that is included in PHP 7+, may throw an exception, so this method has the "fallback" random string generation in case of problems. I would like to test the problem scenario, so I need the exception to be thrown, but none of the methods I found properly mock this function (I also tried to do the same for bin2hex, but again with no positive result). I have already tried:
MockBuilder
$builder = new MockBuilder();
$builder->setNamespace(__NAMESPACE__)
->setName('random_bytes')
->setFunction(fn() => throw new Exception());
$mock = $builder->build();
$mock->enable();
FunctionMock
$mock = $this->getFunctionMock(__NAMESPACE__, 'random_bytes');
$mock->expects($this->once())->willThrowException(new Exception());
Spy
$spy = new Spy(__NAMESPACE__, 'random_bytes', fn() => throw new Exception());
$spy->enable();
Is there any way of mocking this function? I use pure phpunit and php-mock (not Mockery), because I needed this configuration for other tests.
Ok, I found some solution/workaround.
I decided to do the Dependency Injection. I created another service that was easy to mock in the typical way.
class RandomizerService {
private const AVAILABLE_CHARS = '0123456789abcdefABCDEF';
/**
* #throws Exception
*/
public function getRandomBytes(int $length): string {
return random_bytes($length);
}
public function getRandomString(int $length): string {
return substr(str_shuffle(self::AVAILABLE_CHARS), 10, 64);
}
}
And used it as follows:
private function createRandomString(): string {
try {
return bin2hex($this->randomizerService->getRandomBytes(32));
} catch (Exception) {
$this->logger->warning('Unable to created random binary bytes.');
}
return $this->randomizerService->getRandomString(64);
}
Then I only mocked it:
$randomizerService->method('getRandomBytes')
->willThrowException(new Exception());
$randomizerService->method('getRandomString')
->willReturn('abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh');
Or if I wanted no mocks, I created it instance:
$randomizerService = new RandomizerService();
And passed the object or mock as a constructor parameter.
Now everything works and it's covered in 100%.
Say you have a method which ultimately boils down to
class Pager
{
private $i;
public function next()
{
if ($this->i >= 3) {
throw new OutOfBoundsException();
}
$this->i++;
}
}
How would you unit test this class. I.e. test whether the exception gets thrown on the third call of next() using PHPUnit? I've added my attempt as an answer, but I'm not sure whether this really is the way to go.
What about testing for null on the first two calls and also test for the exception being thrown like follows:
class PagerTest
{
public function setUp()
{
$this->pager = new Pager();
}
public function testTooManyNextCalls()
{
$this->assertNull($this->pager->next());
$this->assertNull($this->pager->next());
$this->assertNull($this->pager->next());
$this->setExpectedException('OutOfBoundsException');
$this->pager->next();
}
}
It's very important when unit-testing to avoid testing implementation details. Instead, you want to limit yourself to testing only the public interface of your code. Why? Because implementation details change often, but your API should change very rarely. Testing implementation details means that you'll constantly have to rewrite your tests as those implementations change, and you don't want to be stuck doing this.
So what does this mean for the OP's code? Let's look at the public Pager::next method. Code that consumes the Pager class API doesn't care how Pager::next determines if an exception should be thrown. It only cares that Pager::next actually throws an exception if something is wrong.
We don't want to test how the method arrives at it's decision to throw an OutOfBoundsException -- this is an implementation detail. We only want to test that it does so when appropriate.
So to test this scenario we simulate a situation in which the Pager::next will throw. To accomplish this we simply implement what's called a "test seam." ...
<?php
class Pager
{
protected $i;
public function next()
{
if ($this->isValid()) {
$this->i++;
} else {
throw new OutOfBoundsException();
}
}
protected function isValid() {
return $this->i < 3;
}
}
In the above code, the protected Pager::isValid method is our test seam. It exposes a seam in our code (hence the name) that we can latch onto for testing purposes. Using our new test seam and PHPUnit's mocking API, testing that Pager::next throws an exception for invalid values of $i is trivial:
class PagerTest extends PHPUnit_Framework_TestCase
{
/**
* #covers Pager::next
* #expectedException OutOfBoundsException
*/
public function testNextThrowsExceptionOnInvalidIncrementValue() {
$pagerMock = $this->getMock('Pager', array('isValid'));
$pagerMock->expects($this->once())
->method('isValid')
->will($this->returnValue(false));
$pagerMock->next();
}
}
Notice how this test specifically doesn't care how the implementation method Pager::isValid determines that the current increment is invalid. The test simply mocks the method to return false when it's invoked so that we can test that our public Pager::next method throws an exception when it's supposed to do so.
The PHPUnit mocking API is fully covered in Test Doubles section of the PHPUnit manual. The API isn't the most intuitive thing in the history of the world, but with some repeated use it generally makes sense.
Here's what I have at the moment, but I was wondering if there were any better ways of doing this.
class PagerTest
{
public function setUp()
{
$this->pager = new Pager();
}
public function testTooManyNextCalls()
{
for ($i = 0; $i < 10; $i++) {
try {
$this->pager->next();
} catch(OutOfBoundsException $e) {
if ($i == 3) {
return;
} else {
$this->fail('OutOfBoundsException was thrown unexpectedly, on iteration ' . $i);
}
}
if ($i > 3) {
$this->fail('OutOfBoundsException was not thrown when expected');
}
}
}
}
You could use something like:
class PagerTest extends PHPUnit_Framework_TestCase {
/**
* #expectedException OutOfBoundsException
*/
public function testTooManyNextCalls() {
$this->pager = new Pager();
$this->pager->next();
$this->pager->next();
$this->pager->next();
$this->assertTrue(false);
}
}
If an exception is thrown in the 3rd method call, the always-failing assert-statement should never be reached and the test should pass. on the other hand if no exception gets thrown, the test will fail.
You may pass the value $this->i to the exception instantiation, which will then be the message of the exception.
class Pager
{
private $i;
public function next()
{
if ($this->i >= 3) {
throw new OutOfBoundsException($this->i);
}
$this->i++;
}
}
$a=new Pager();
$a->next();
$a->next();
$a->next();
$a->next();
//outputs: "Exception: 3"
i want to use PHP Unit inside a Zend Framework application. I need to make several database writing operations inside the tests.
I want to start an MySQL transaction in the setUpBeforeClass() method. That is possible, but if I try to rollback the transaction in the tearDownAfterClass() method he throws an exception with the message 'There is no active transaction'. And the test methods does the writing operations in the database.
But if i start the transaction in the test method itselfs. It works like I want.
I don't understand why it reacts like this. Knows anyone an explanation?
<?php
class ConferenceControllerTest
extends PHPUnit_Framework_TestCase
{
/**
* A database connection.
* #var Zend_Db_Adapter_Pdo_Mysql
*/
protected static $hostDb = null;
public static function setUpBeforeClass()
{
static::$hostDb = Zend_Registry::get('db_host');
static::$hostDb->beginTransaction();
// The transaction for the Adapter is activated. But not inside the tests anymore.
}
public function testTest1()
{
// At this position teh transaction is not setted anymor? Why?
static::$hostDb->beginTransaction();
$sql = 'INSERT INTO test(test) VALUES(5);';
static::$hostDb->exec($sql);
}
public static function tearDownAfterClass()
{
try
{
static::$hostDb->rollBack();
}
catch(Exception $exception)
{
$message = $exception->getMessage();
Zend_Debug::dump($message);
}
}
}
I think you may be running into phpUnit's feature to backup statics and other globals between each unit test, see the "globals" section of the Fixtures chapter of the manual
The quick-hack fix would be to add this line, in the comments just above your class:
* #backupStaticAttributes disabled
That still leaves you with the xUnit Patterns book would call a nasty smell. I'm assuming you have a few testXXX functions that you expect to run in a certain order, each depending on the result of the previous one? That requires using #depends on each function, and it is easy to get wrong.
The alternative approach is a single long unit test function, and put the DB code in setUp() and tearDown():
public function setUp()
{
$this->db = Zend_Registry::get('db_host');
$this->db->beginTransaction();
}
public function tearDown()
{
try
{
$this->db->rollBack();
}
catch(Exception $exception)
{
$message = $exception->getMessage();
Zend_Debug::dump($message);
}
public function testTestVariousDBActions()
{
$sql = 'INSERT INTO test(test) VALUES(5);';
$this->db->exec($sql);
//another DB action
$this->assertEquals(...)
//another DB action
$this->assertEquals(...)
//...
}
The advantage of this is that if any assert fails none of the subsequent ones will be attempted. But tearDown() will always be called so the database is restored.
The disadvantage could be you get a Very Long Function. Use refactoring to deal with this (e.g. if you really want each action and its test in its own function, refactor so it looks like that and have testTestVariousDBActions() call each of them in turn).
I have div() in Cal class in my CalTest class has following methods to test the div().
public fucnction div($a,$b){
if($b == 0){
throw new Exception("Divided by zero");
}
return $a/$b
}
I can pass only testDiv() but testDiv2().
I want to catch check wheather the exeption thrown correctly using PHPUnit. What am I missing here?? Your help is highly appreciated. Thank you!
Your 2nd screenshot (the one with the error) has
"#expectedException Exception"
while the third has
#expectedException InvalidArgumentException
Do you really still get the error? Did you save the file?
Works for me:
Foo.php
<?php
class Foo
{
static function t()
{
throw new InvalidArgumentException('Hi there');
}
}
?>
FooTest.php
<?php
require_once 'Foo.php';
class FooTest extends PHPUnit_Framework_TestCase
{
/**
* #expectedException InvalidArgumentException
*/
public function testT()
{
Foo::t();
}
}
?>
Result
$ phpunit .
PHPUnit 3.6.10 by Sebastian Bergmann.
.
Time: 0 seconds, Memory: 5.25Mb
OK (1 test, 1 assertion)
Just came across the same issue. For some reason PHPUnit will not allow you to set the expectedException to a generic exception and I am not sure why. Personally I opt to throw custom Exception codes rather than have to create a new exception class every time I want to differentiate exceptions.
Here is how I got around it:
/**
* #expectedException Test_Exception
*/
public function testDivByZero()
{
try {
// Fyi you don't need to do an assert test here, as we are only testing the exception, so just make the call
$result = $this->object->div(1,0);
} catch (Exception $e) {
if ('Exception' === get_class($e)) {
throw new Test_Exception($e->getMessage(), $e->getCode());
}
}
}
// Test_Exception.php
class Test_Exception extends Exception
{
public function __construct($message = null, $code = 0, Exception $previous = null)
{
parent::__construct($message, $code, $previous);
}
}
This will allow you to engineer your code the way you want, and throw "generic" exceptions. Basically it just tests the Exception class and if it's generic, re-wrap it as a different exception; Test_Exception.
-- Update --
Found out yesterday that they have removed the generic exception restriction in the current "master" branch, which will be 3.7. Apparently the lead engineer has no desire of patching 3.6.
This problem has been fixed in PHPUnit 3.7.x
You can use the following Generic Exception in PHPUnit 3.7.x
/**
* #expectedException Exception
*/
It says so in the exception thrown by PHPUnit:
You must not except the generic exception class.
Make your class throw more detailed Exceptions, and further specify the exception type your expecting in your unit test.
you can do something like this
write your unit test so you inject the values required to trigger the exception then
//assert
$this->assertTrue($this->setExpectedException('PHPUnit_Framework_ExpectationFailedException'));
I create a parent class method to do it. In fact, this class came from the laravel but is valid in any context.
The cool part of this method is using anonymous functions into PHP
<?php
class TestCase
/* if you are not using laravel, you dont need this
extends Illuminate\Foundation\Testing\TestCase */ {
protected function assertThrows( $function , $classException = "/Exception" )
{
try
{
// Anonymous functions FTW
$function();
}
catch( /Exception $objException )
{
// the assertInstanceOf should solve from here but
// it is not working that great
// #see http://stackoverflow.com/questions/16833923/phpunit-assertinstanceof-not-working
if( $objException instanceof $classException ) {
return $this->assertTrue( true );
} else {
// this will throw a error with a cool message
return $this->assertEquals( $classException , get_class( $objException ));
}
}
// no exception happened.
return $this->fail( "Exception " . $classException . " expected." );
}
}
class LanguageTest extends TestCase {
/**
* Test the translation
*
* #return void
*/
public function testTranslation()
{
// this test may be ok
$this->assertThrows(
function(){ throw new Full\Path\Exception(":("); },
"Full\Path\Exception" );
// this test may fail
$this->assertThrows(
function(){ return 1 + 1; },
"Some\Excepted\Exception" );
// this test may work
$this->assertThrows(
function(){ throw new Exception( "sad" ); }
);
}
}
With PHPUnit, I am testing a sequence of method calls using ->at(), like so:
$mock->expects($this->at(0))->method('execute')->will($this->returnValue('foo'));
$mock->expects($this->at(1))->method('execute')->will($this->returnValue('bar'));
$mock->expects($this->at(2))->method('execute')->will($this->returnValue('baz'));
How can I set up the mock so that, in the above scenario, if execute() is called four or more times, it will immediately fail? I tried this:
$mock->expects($this->at(3))->method('execute')->will($this->throwException(new Exception('Called too many times.')));
But this also fails if execute() is not called four times. It needs to fail immediately, otherwise the system under test will produce errors of its own, which causes the resulting error message to be unclear.
I managed to find a solution in the end. I used a comination of $this->returnCallback() and passing the PHPUnit matcher to keep track of the invocation count. You can then throw a PHPUnit exception so that you get nice output too:
$matcher = $this->any();
$mock
->expects($matcher)
->method('execute')
->will($this->returnCallback(function() use($matcher) {
switch ($matcher->getInvocationCount())
{
case 0: return 'foo';
case 1: return 'bar';
case 2: return 'baz';
}
throw new PHPUnit_Framework_ExpectationFailedException('Called too many times.');
}))
;
For special cases like this, I typically use something like the following:
public function myMockCallback() {
++$this -> _myCounter;
if( $this -> _myCounter > 3 ) {
// THROW EXCEPTION OR TRIGGER ERROR
}
... THEN YOUR CASE STATEMENT OR IF/ELSE WITH YOUR CHOICE OF RETURN VALUES
}
... INSIDE TEST FUNCTION ....
$mockObject ->expects($this->any())
->method('myMethod')
->will($this->returnCallback( array ($this, 'myMockCallback' )));
You could separate test to 2 dependent methods, using #depends annotation.
In this case your first test only tests that there are exact 3 method executions, and second - other logic.
What about using data providers?
class MyTest extends PHPUnit.... {
/**
* #var const how much till throwing exception
*/
const MAX_EXECUTE_TILL_EXCEPTION = 3;
public function setUp(){}
public function tearDown(){}
/**
* #dataProvider MyExecuteProvider
*/
pulbic function testMyExecuteReturnFalse($data){
$mock = //setup your mock here
//if using "$ret" doesn't work you cant just call another private helper that will decide if you need to
// return value or throwing exception
if (self::MAX_EXECUTE_TILL_EXCEPTION == $data){
$ret = $this->throwException(new Exception('Called too many times.'));
} else {
$ret = $this->returnValue('foo');
}
$mock->expects($this->at($data))->method('execute')->will($ret);
}
public function MyExecuteProvider(){
return array(
0,1,2,3
)
}
}
This is just another idea, and I think that zerkms suggested very good idea as well