I have a simple unit test case (extensive question here) on a configuration class that by design triggers PHP errors on type mismatches and undefined configuration settings. In addition to the error, the method is exited returning false.
In my test case, I want to have a number of tests that fail. Trouble is, I can't do a simple 'assert' on the return value: Every error thrown during running a test will be converted into a PHPUnit_Framework_Error exception.
Now I can make PHPUnit aware that I am expecting an error:
/**
* #expectedException PHPUnit_Framework_Error
*/
public function testSetInvalidKey()
{
$this->assertEquals(true, conf::set("invalid_key", "12345"));
}
This works fine, but what I don't understand is that any additional assertion within that function will not be evaluated. Whether it would fail or not, PHPUnit seems to only wait for the exception to happen, and be satisfied with the whole function when it has happened.
To wit, this test will run OK:
/**
* #expectedException PHPUnit_Framework_Error
*/
public function testSetInvalidKey()
{
// The error will be triggered here
$this->assertEquals(true, conf::set("invalid_key", "12345"));
$this->assertEquals(12345, 67810); // Huh?
$this->assertEquals("abc", "def"); // Huh?
$this->assertEquals(true, false); // Huh?
}
Why? Is this the intended behaviour?
I realize you would just separate the assertions into different functions, but I would like to understand the behaviour.
Since conf::set() is executed within the method testSetInvalidKey() the corresponding catch block must be outside. Once caught and logged as the expected exception, I don't see how PHP could resume execution after the first assertion.
Extreme psuedo-code:
class Tester
{
public function run()
{
try {
$test->testSetInvalidKey();
}
catch (PHPUnit_Framework_Error $e) {
// Expected exception caught! Woohoo!
// How can I continue to run the above method where I left off?
}
}
}
This type of behavior would be a great proponent to those who believe in the 1 assertion per test axiom.
php unit has the ability to test for exceptions. Take a look at:
http://www.phpunit.de/manual/3.2/en/writing-tests-for-phpunit.html
Related
I know both versions are correct but I would like to know which is "better".
The problem with expectException() method is it is written before you type the method which launches the exception.
My question is, should I put them at the beginning of the method (to make them more visible) or otherwise only just before the method which causes the exception (I think it has more sense)?
Option A)
/** #test */
public function shouldThrowsAnException(): void
{
$this->expectException(RuntimeException::class);
$this->expectExceptionMessage('Error message');
$foo = new Foo();
$foo->bar(); // <-- This method launches the exception!!
}
Option B)
/** #test */
public function shouldThrowsAnException(): void
{
$foo = new Foo();
$this->expectException(RuntimeException::class);
$this->expectExceptionMessage('Error message');
$foo->bar(); // <-- This method launches the exception!!
}
When writing unit tests, each unit should be testing only one thing. So your expectException should be as specific as possible, as you have in your code, looking for a particular exception and a particular message.
If your constructor happened to throw the same exception, then Option B would be the better option, so you don't catch the constructor exception by accident. If the constructor does throw exceptions, you would then write other test cases that flush out the constructor.
If the constructor does not throw any exceptions, the two unit test are the same as far as execution.
If you were asking from a readability standpoint, then Option B is more readable, because it is clear that the expectation is that your method Bar will be throwing the exception.
I witnessed some strange behaviour regarding PHP's exception handling in a recent project. Case goes as follows.
In my app, I use namespaces. All classes are in individual source code files. The code relevant to this particular case, is spread over 3 classes.
The "outermost" class is a dispatcher (or router), which wraps the dispatch call inside a try-catch block. The dispatched request, calls a method in a third class, which runs code (wrapped in a try-catch block), which causes an exception.
Because I had omitted a use Exception; statement in the class where the error happens, the thrown exception trickles all the way back to the outermost layer (the dispatcher), where it is caught - causing me to scratch my head why the catch around the code causing the error isn't working.
To me this seems strange. Logically, PHP should in this situation (IMO) throw a Class not found exception/error, leading me to the actual error in my code, instead of trying to "stay alive" as long as possible.
Should this be filed as a bug, or is this expected behaviour?
Edit: Code example
File: class-a.php
<?php
namespace hello\world;
class classA {
protected $b;
public function __construct() {
$this->b = new \hello\world\classB();
}
public function doSomething() {
try {
$this->b->throwException();
} catch (Exception $e) {
}
}
}
File: class-b.php
<?php
namespace hello\world;
class classB
{
public function throwException() {
throw new \Exception("bar closed");
}
}
File: run.php
<?php
include 'class-a.php';
include 'class-b.php';
$a = new \hello\world\classA();
$a->doSomething();
ClassB throws an \Exception in ClassB::doSomething(), for which ClassA has a catch-clause, but because ClassA doesn't declare use Exception or catch (\Exception), the catch doesn't match and execution ends with a Uncaught exception error1. But in my opinion, it should cause a Class not found error.
I might be expecting too much of the permissive PHP compiler, but it would help in tracking down silly errors that should be easy for the compiler to spot.
1 If the $a->doSomething() in run.php was surrounded by a try..catch clause, the Exception would (or at least could) be caught there, since it trickles down the stack.
PHP's exception catching mechanism does not validate that the class you catch actually exists.
It exhibits the same behavior when using typehinting in functions, so I suspect it merely converts the exception/function type hint into a string or something and compares that with the type of the relevant object.
Whether this is a bug or not is questionable. Personally I think it should be classified as a bug, but PHP has all sorts of wonky behaviors :D
I have a set of tests, and I want to test that my classes throw exceptions at the right time. In the example, my class uses the __get() magic method, so I need to test that an exception is thrown when an invalid property is retrieved:
function testExceptionThrownWhenGettingInvalidProperty() {
$obj = new MyClass();
$this->setExpectedException("Property qwerty does not exist");
$qwerty = $obj->qwerty;
}
The class throws an error as it should, but instead of just getting a pass, the exception isn't caught!
There was 1 error:
1) QueryTest::testExceptionThrownWhenGettingInvalidProperty
Exception: Property qwerty does not exist
I was using SimpleTest before, and $this->expectException(new Exception("Property qwerty does not exist")); worked just fine. I know there are other methods (#expectedException and try-catch), but this one should work and it looks a lot cleaner. Any ideas how I can make this work?
It's not looking for the text in the exception, it's looking for the name of the exception class... Docs
$this->setExpectedException('Exception');
It's quite handy when you're using SPL Exceptions, or custom exception classes...
Adding to ircmaxell's answer, there is actually a simpler way of doing this:
/**
* #expectedException MyExceptionClass
* #expectedExceptionMessage Array too small
*/
public function testSomething()
{
}
The #expectedException the class name of the exception to expect, and #expectedExceptionMessage is a substring of the exception message to expect (that's right, you don't have to have the entire message).
If you prefer to not use docblock annotations, both of these are actually available as methods on the test case.
Essentially I have a method of a class called killProgram, which is intended to send a hTTP redirect and then kill PHP.
How am I supposed to test this? When I run phpunit it doesn't return anything for that test, and closes completely.
Right now I'm considering having the killProgram function throw an exception which shouldn't get handled, which would allow me to assert that an exception was thrown.
Is there a better way?
It's obviously an old question but my suggestion would be to move the code that die()'s into a separate method that you can then mock.
As an example, instead of having this:
class SomeClass
{
public function do()
{
exit(1);
// or
die('Message');
}
}
do this:
class SomeClass
{
public function do()
{
$this->terminate(123);
// or
$this->terminate('Message');
}
protected function terminate($code = 0)
{
exit($code);
}
// or
protected function terminate($message = '')
{
die($message);
}
}
That way you can easily mock the terminate method and you don't have to worry about the script terminating without you being able to catch it.
Your test would look something like this:
class SomeClassTest extends \PHPUnit_Framework_TestCase
{
/**
* #expectedExceptionCode 123
*/
public function testDoFail()
{
$mock = $this->getMock('SomeClass');
$mock->expects($this->any())
->method('terminate')
->will($this->returnCallback(function($code) {
throw new \Exception($code);
}));
// run to fail
$mock->do();
}
}
I haven't tested the code but should be pretty close to a working state.
As every tests are run by the same PHPUnit process, if you use exit/die in your PHP code, you will kill everything -- as you noticed ^^
So, you have to find another solution, yes -- like returning instead of dying ; or throwing an exception (you can test if some tested code has thrown an expected exception).
Maybe PHPUnit 3.4 and it's --process-isolation switch (see Optionally execute each test using a separate PHP process) might help (by not having everything dying), but you still wouldn't be able to get the result of the test, if PHPUnit doesn't get the control back.
I've had this problem a couple of times ; solved it by returning instead of dying -- even returning several times, if needed, to go back "high enough" in the call stack ^^
In the end, I suppose I don't have any "die" anymore in my application... It's probably better, when thinking about MVC, btw.
There's no need to change the code just to be able to test it, you can simply use set_exit_overload() (provided by test_helpers from same author as PHPUnit).
I realise you've already accepted an answer for this and it's an old question, but I figure this might be useful for someone, so here goes:
Instead of using die(), you could use throw new RuntimeException() (or an exception class of your own), which will also halt program execution (albeit in a different fashion) and use PHPUnit's setExpectedException() to catch it. If you want your script to die() when that exception is encountered, printing absolutely nothing up at level of the user, take a look at set_exception_handler().
Specifically, I'm thinking of a scenario in which you'd place the set_exception_handler()-call into a bootstrap file that the tests don't use, so the handler won't fire there regardless of scenario, so nothing interferes with PHPUnit's native exception handling.
This relates to set of issues I've been having getting some legacy code to pass a test. So I've come up with a Testable class like this...
class Testable {
static function exitphp() {
if (defined('UNIT_TESTING')) {
throw new TestingPhpExitException();
} else {
exit();
}
}
}
Now I simply replace calls to exit() with Testable::exitphp().
If it's under test I just define UNIT_TESTING, in production I don't. Seems like a simple Mock.
You can kill the script or throw an exception, depending on the value of an environmental variable...
So you kill in production or throw an exception in test environment.
Any call to die or exit, Kills the whole process...
This was supposed to be a comment but I can't comment with the level of my reputation points.
This question is specific to using PHPUnit.
PHPUnit automatically converts php errors to exceptions. Is there a way to test the return value of a method that happens to trigger a php error (either built-in errors or user generated errors via trigger_error)?
Example of code to test:
function load_file ($file)
{
if (! file_exists($file)) {
trigger_error("file {$file} does not exist", E_USER_WARNING);
return false;
}
return file_get_contents($file);
}
This is the type of test I want to write:
public function testLoadFile ()
{
$this->assertFalse(load_file('/some/non-existent/file'));
}
The problem I am having is that the triggered error causes my unit test to fail (as it should). But if I try to catch it, or set an expected exception any code that after the error is triggered never executes so I have no way of testing the return value of the method.
This example doesn't work:
public function testLoadFile ()
{
$this->setExpectedException('Exception');
$result = load_file('/some/non-existent/file');
// code after this point never gets executed
$this->assertFalse($result);
}
Any ideas how I could achieve this?
There is no way to do this within one unit test. It is possible if you break up testing the return value, and the notice into two different tests.
PHPUnit's error handler catches PHP errors and notices and converts them into Exceptions--which by definition stops program execution. The function you are testing never returns at all. You can, however, temporarily disable the conversion of errors into exceptions, even at runtime.
This is probably easier with an example, so, here's what the two tests should look like:
public function testLoadFileTriggersErrorWhenFileNotFound()
{
$this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Or whichever exception it is
$result = load_file('/some/non-existent/file');
}
public function testLoadFileRetunsFalseWhenFileNotFound()
{
PHPUnit_Framework_Error_Warning::$enabled = FALSE;
$result = load_file('/some/non-existent/file');
$this->assertFalse($result);
}
This also has the added bonus of making your tests clearer, cleaner and self documenting.
Re: Comment:
That's a great question, and I had no idea until I ran a couple of tests. It looks as if it will not restore the default/original value, at least as of PHPUnit 3.3.17 (the current stable release right now).
So, I would actually amend the above to look like so:
public function testLoadFileRetunsFalseWhenFileNotFound()
{
$warningEnabledOrig = PHPUnit_Framework_Error_Warning::$enabled;
PHPUnit_Framework_Error_Warning::$enabled = false;
$result = load_file('/some/non-existent/file');
$this->assertFalse($result);
PHPUnit_Framework_Error_Warning::$enabled = $warningEnabledOrig;
}
Re: Second Comment:
That's not completely true. I'm looking at PHPUnit's error handler, and it works as follows:
If it is an E_WARNING, use PHPUnit_Framework_Error_Warning as an exception class.
If it is an E_NOTICE or E_STRICT error, use PHPUnit_Framework_Error_Notice
Else, use PHPUnit_Framework_Error as the exception class.
So, yes, errors of the E_USER_* are not turned into PHPUnit's *_Warning or *_Notice class, they are still transformed into a generic PHPUnit_Framework_Error exception.
Further Thoughts
While it depends exactly on how the function is used, I'd probably switch to throwing an actual exception instead of triggering an error, if it were me. Yes, this would change the logic flow of the method, and the code that uses the method... right now the execution does not stop when it cannot read a file. But that's up to you to decide whether the requested file not existing is truly exceptional behaviour. I tend to use exceptions way more than errors/warnings/notices, because they are easier to handle, test and work into your application flow. I usually reserve the notices for things like depreciated method calls, etc.
Use a phpunit.xml configuration file and disable the notice/warning/error to Exception conversion. More details in the manual. It's basically something like this:
<phpunit convertErrorsToExceptions="false"
convertNoticesToExceptions="false"
convertWarningsToExceptions="false">
</phpunit>
Instead of expecting a generic "Exception", what about expecting a "PHPUnit_Framework_Error" ?
Something like this might do :
/**
* #expectedException PHPUnit_Framework_Error
*/
public function testFailingInclude()
{
include 'not_existing_file.php';
}
Which, I suppose, might also be written as :
public function testLoadFile ()
{
$this->setExpectedException('PHPUnit_Framework_Error');
$result = load_file('/some/non-existent/file');
// code after this point never gets executed
$this->assertFalse($result);
}
For more informations, see Testing PHP Errors
Especially, it says (quoting) :
PHPUnit_Framework_Error_Notice and
PHPUnit_Framework_Error_Warning represent
PHP notices and warning, respectively.
Looking at the /usr/share/php/PHPUnit/TextUI/TestRunner.php file I have on my system, I see this (line 198 and following) :
if (!$arguments['convertNoticesToExceptions']) {
PHPUnit_Framework_Error_Notice::$enabled = FALSE;
}
if (!$arguments['convertWarningsToExceptions']) {
PHPUnit_Framework_Error_Warning::$enabled = FALSE;
}
So maybe you'll have to pass some kind of parameter to activate that behaviour ? But it seems to be enabled by default...
Actually there is a way to test both the return value and the exception thrown (in this case an error converted by PHPUnit).
You just have to do the following:
public function testLoadFileTriggersErrorWhenFileNotFound()
{
$this->assertFalse(#load_file('/some/non-existent/file'));
$this->setExpectedException('PHPUnit_Framework_Error_Warning'); // Or whichever exception it is
load_file('/some/non-existent/file');
}
Notice that to test for the return value you have to use the error suppression operator on the function call (the # before the function name). This way no exception will be thrown and the execution will continue. You then have to set the expected exception as usual to test the error.
What you cannot do is test multiple exceptions within a unit test.
This answer is a bit late to the party, but anyhow:
You can use Netsilik/BaseTestCase (MIT License) to test directly for triggered Notices/Warnings, without ignoring them or converting them to Exceptions. Because the notices/warnings they are not converted to an Exception, the execution is not halted.
composer require netsilik/base-test-case
Testing for an E_USER_NOTICE:
<?php
namespace Tests;
class MyTestCase extends \Netsilik\Testing\BaseTestCase
{
public function test_whenNoticeTriggered_weCanTestForIt()
{
$foo = new Foo();
$foo->bar();
self::assertErrorTriggered(E_USER_NOTICE, 'The notice message');
}
}
Hope this helps someone in the future.