So I playing around with PHPUnit and would like to get some insight to the output that PHPUnit generates when I try to test for an Exception. I am confused as to why I am getting a failed test. Here is my test:
class ConfigTest extends PHPUnit_Framework_Testcase
{
public function testTrueIfJobGivenExists()
{
$conf = Config::getInstance('test1.php', new Database());
$setup = $conf->getConfig();
$this->assertTrue($setup);
}
/**
* #expectedException Exception
*/
public function testExceptionIfJobGivenNotExists()
{
$conf = Config::getInstance('test.php', new Database());
$setup = $conf->getConfig();
}
}
In here I am not mocking the Database class (I have not learned how to do that yet) but basically the code looks for and entry for a job called test.php and pulls the config col for that. If the job does not exists it throws a new Exception. Here is my output:
PHPUnit 4.1.0 by Sebastian Bergmann.
.F
Time: 26 ms, Memory: 3.50Mb
There was 1 failure:
1) ConfigTest::testExceptionIfJobGivenNotExists
Failed asserting that exception of type "Exception" is thrown.
FAILURES!
Tests: 2, Assertions: 2, Failures: 1.
Here to me seems that the test is failing but looking at the PHPUnit documentation about testing exception the output looks similar. Is my test working?
EDIT: New test fail
Using Mockery I created my test like:
class ConfigTest extends PHPUnit_Framework_Testcase
{
public function tearDown()
{
Mockery::close();
}
public function testTrueIfConfigForGivenJobExists()
{
$dbJSON = array( array(
'jobConfig' => '{
"config": {
"aquisition": {
"type": "xx",
"customerKey": "xxxxx",
"login":"xxxx",
"password":"xxxxx",
"host":"xxxxxx",
"account":"",
"email":""
}
}
}'
) );
$database = Mockery::mock('Database');
$database->shouldReceive('select->where->runQuery->fetch')->andReturn($dbJSON);
$conf = Config::getInstance('getLoadsPE.php', $database);
$setup = $conf->getConfig();
$this->assertTrue($setup);
}
/**
* #expectedException Exception
*/
public function testExceptionIfJobGivenNotExists()
{
$database = Mockery::mock('Database');
$database->shouldReceive('select->where->runQuery->fetch')->andReturn(null);
$conf = Config::getInstance('getLoadsPE.php', $database);
$setup = $conf->getConfig();
$this->assertTrue($setup);
}
}
and I get
PHPUnit 4.1.0 by Sebastian Bergmann.
.F
Time: 39 ms, Memory: 4.75Mb
There was 1 failure:
1) ConfigTest::testExceptionIfJobGivenNotExists
Failed asserting that exception of type "Exception" is thrown.
FAILURES!
Tests: 2, Assertions: 3, Failures: 1
With this I dont know where the 3rd assertion is coming from. Also I dont get why Im getting the Fail test. If I comment the first test then the second passes. Any thoughts anyone?
FYI
This is what getConfig() looks like:
public function getConfig()
{
if ($this->flag) {
// Config has already been set
return true;
}
$data = self::$database->select('configs', ['jobConfig'])
->where('jobName', self::$jobName)
->runQuery()
->fetch();
if (empty($data)) {
throw new Exception("Config Exception: No config available for " . self::$jobName, 1);
}
if (count($data) > 1) {
throw new Exception("Config Exception: More than one config for same job!!!", 1);
}
$arr = json_decode($data[0]['jobConfig'], true);
// maybe threre is a better way of doing this
if (array_key_exists('aquisition', $arr['config'])) {
$this->aquisition = $arr['config']['aquisition'];
}
if (array_key_exists('ftpSetup', $arr['config'])) {
$this->ftpSetup = $arr['config']['ftpSetup'];
}
if (array_key_exists('fileSetup', $arr['config'])) {
$this->fileSetup = $arr['config']['fileSetup'];
}
if (array_key_exists('fileMaps', $arr['config'])) {
$this->fileMaps = $arr['config']['fileMaps'];
}
if (array_key_exists('fileRows', $arr['config'])) {
$this->fileRows = $arr['config']['fileRows'];
}
$this->flag = true;
return true;
}
}
#expectedException Exception is not a good idea here. If an exception is thrown in your test setup (e.g. first line of you test) your test will still pass.
You could use:
//given
$conf = ...;
try {
//when
$conf->getConfig();
$this->fail("YourException expected");
//then
} catch (YourException $e) {}
But it's messy and will not work with Exception (because phpunit fail also throws Exception). So you would have to use a custom exception.
You can try CatchException from ouzo goodies:
//given
$conf = ...;
//when
CatchException::when($conf)->getConfig();
//then
CatchException::assertThat()->isInstanceOf("Exception");
The obvious answer to your question is that there actually is a job in your database at the time the test is run, and therefore your code is not throwing the exception you expect.
This may or may not be correct, but from your code there is no way for us to know that; in fact, from your code, there is no way for the test to adequately test for that case because you don't know what's going to be in the database at the time the test is run. This is why mocking dependencies like the database is so important: you won't actually test your code the way you think you're testing it unless all the external stuff is set up exactly the way you want it to be.
I personally find it hard to wrap my head around testing and when I've been away from it for a few days (like on a Monday morning), I find it helpful to stick to a formula to get me back into the groove. I like to use the "given, when, then" pattern when writing my tests. I picked this up from Jeffrey Way's book "Laravel Testing Decoded". Here's what it looks like:
public function testSomething()
{
// given
// ... set up everything for the test here
// when
// ... execute the code that's being tested
// then
// ... assert
}
So in your case, it might look something like this:
/**
* #expectedException Exception
*/
public function testExceptionIfJobGivenNotExists()
{
// given
// I don't know what your database interface looks like, so I'm
// making stuff up here
$database = Mockery::mock("Database");
$database->shouldReceive("query")->with("SELECT something")->andReturn(null);
// set up the config object
$conf = Config::getInstance('test.php', $database);
// when
// execute your code here
$setup = $conf->getConfig();
// then
// normally you'd assert something here, but in this case, the exception
// handler will take care of it
}
(I've assumed you're using Mockery for mocking. I've also assumed that your database class will return null when you query for a job and one doesn't exist.)
Don't sweat the mocking - it's really not that hard. All I've done here is replace your new Database() with a "fake" database object (ie. mocked). The shouldReceive method simply tells Mockery that the query method should be called with some arguments and return a value. Mocking can be much more complicated than this, but to start out, this is pretty simple.
Why is this important?
This test is supposed to tell you what happens when there's no job in the database. In your test, you have no way of knowing whether your test is actually testing that, because you really have no way of knowing what the database class is returning when you run the query. In my example, on the other hand, I've "faked" the database class and ensured that it's going to return exactly what I want it to return, every time the test is run.
You expect exception to be thrown but it is not thrown. That's why your test fails. Check if your code with dataset test.php should really throw an exception.
Related
I want to test that a method called foo() throws an exception. The problem is that I can't get PHPUnit expectException() to catch the exception.
foo() looks something like this:
public function foo()
{
$params = $this->readAndFormatConfig();
// exception actually gets thrown in this method
$this->method->throws->exception($params);
}
If I catch the exception manually it works fine, like this:
public function testFoo()
{
$badConfig = new Config([]);
$driver = new bar($badConfig);
$exceptionThrown = false;
try {
$driver->foo();
} catch (Exception $e) {
$exceptionThrown = true;
}
$this->assertTrue($exceptionThrown);
}
If I catch it using expectException, like this:
public function testFoo()
{
$badConfig = new Config([]);
$driver = new bar($badConfig);
$this->expectException(Exception::class);
$driver->foo();
}
the test fails and I get this exception:
MyTestClass::testFoo Invalid argument supplied for foreach()
The output of get_class($e) is PHPUnit_Framework_Error_Warning which surprised me, but explains why the first test works but the second doesn't.
I'd like to either ignore the warning and wait until a real exception is thrown, or get the original warning, not PHPUnit_Framework_Error_Warning.
I'm using php 5.6.32 and PHPUnit 5.7.15
Adding the following to boostrap.php converts warnings to exceptions.
function warningToException($errno, $errstr, $errfile, $errline)
{
throw new Exception($errstr . " on line " . $errline . " in file " . $errfile);
}
set_error_handler("warningToException", E_WARNING);
This allowed the following test pass.
public function testFoo()
{
$badConfig = new Config([]);
$driver = new bar($badConfig);
$this->expectException(Exception::class);
$driver->foo();
}
I think a better method would be to expect PHPUnit_Framework_Error_Warning in the test as ishegg suggested.
What I ended up actually doing was making sure the object I was passing to foreach was traversable before entering the loop.
public function foo()
{
if (is_array($x) || $x instanceof \Traversable) {
// do stuff
} else {
// return false;
}
}
I think it makes more sense to refactor the application code in this case. What I'm really trying to test is that the function foo returns false if the application is misconfigured, so handling the misconfiguration properly seems like the right path in my opinion.
If you stumble upon this question for some reason and really need to convert warnings to an exception class other than PHPUnit_Framework_Error_Warning this is how I did it.
Thanks to ishegg for pointing me in the right direction.
For our PHPUnit testing, we sometimes write custom assertions. Today I found a custom assertion that wasn't asserting quite what it ought to have been. It seems that this problem could have been avoided if I had written a unit test for the assertion itself.
The only problem I see is that I'm not quite sure how to handle writing tests for an assertion that it ought to fail, without having that lead to the test itself failing. In other words, for a test that expects a string, 'foo', I want to do something like:
public function testAssertFoo()
{
$var = 'bar';
$callable = array( $this, "assertFoo" );
$this->assertTestFails( $callable, $var );
}
Of course, there is no assertTestFails assertion. But is there a clean way to do something like that?
Assuming that assertFoo uses PHPUnit's built-in assertions such as assertEquals, you can simply catch the PHPUnit_Framework_ExpectationFailedException that is thrown when the assertion fails.
function testAssertFoo() {
try {
$this->assertFoo('bar');
self::fail("assertFoo should fail for 'bar'");
}
catch (PHPUnit_Framework_ExpectationFailedException $e) { /* test passed */ }
}
function assertFoo($value) {
self::assertEquals('foo', $value);
}
FWIW I'm using SimpleTest 1.1alpha.
I have a singleton class, and I want to write a unit test that guarantees that the class is a singleton by attempting to instantiate the class (it has a private constructor).
This obviously causes a Fatal Error:
Fatal error: Call to private FrontController::__construct()
Is there any way to "catch" that Fatal Error and report a passed test?
No. Fatal error stops the execution of the script.
And it's not really necessary to test a singleton in that way. If you insist on checking if constructor is private, you can use ReflectionClass:getConstructor()
public function testCannotInstantiateExternally()
{
$reflection = new \ReflectionClass('\My\Namespace\MyClassName');
$constructor = $reflection->getConstructor();
$this->assertFalse($constructor->isPublic());
}
Another thing to consider is that Singleton classes/objects are an obstacle in TTD since they're difficult to mock.
Here's a complete code snippet of Mchl's answer so people don't have to go through the docs...
public function testCannotInstantiateExternally()
{
$reflection = new \ReflectionClass('\My\Namespace\MyClassName');
$constructor = $reflection->getConstructor();
$this->assertFalse($constructor->isPublic());
}
You can use a concept like PHPUnit's process-isolation.
This means the test code will be executed in a sub process of php. This example shows how this could work.
<?php
// get the test code as string
$testcode = '<?php new '; // will cause a syntax error
// put it in a temporary file
$testfile = tmpfile();
file_put_contents($testfile, $testcode);
exec("php $tempfile", $output, $return_value);
// now you can process the scripts return value and output
// in case of an syntax error the return value is 255
switch($return_value) {
case 0 :
echo 'PASSED';
break;
default :
echo 'FAILED ' . $output;
}
// clean up
unlink($testfile);
I'm testing some legacy code that extends the default php exception object. This code prints out a custom HTML error message.
I would like to mock this exception object in such a way that when the tested code generates an exception it will just echo the basic message instead of giving me the whole HTML message.
I cannot figure out a way to do this. It seems like you can test for explicit exceptions, but you can't change in a general way the behavior of an exception, and you also can't mock up an object that extends a default php functionality. ( can't think of another example of this beyond exceptions... but it would seem to be the case )
I guess the problem is, where would you attach the mocked object?? It seems like you can't interfere with 'throw new' and this is the place that the object method is called....
Or if you could somehow use the existing phpunit exception functionality to change the exception behavior the way you want, in a general way for all your code... but this seems like it would be hacky and bad....
EDIT: here is some code to make things clearer:
class FooTest extends PHPUnit_Framework_TestCase{
public function testBar(){
include '/path/to/file.php'; //generates exception
$this->assertTrue($baz);
}
}
...
//overridden exception class
class Foo_Exception extends ErrorException{
...
so, my question, is there a way to deal with this overriden class, without doing it on a case by case basis? what if I'm not testing the behavior of the exception, just the code that causes the exception?
I would first write a test that captures the exception generation behavior:
include '/path/to/file.php'; //generates exception
public function testCatchFooException() {
try {
$this->assertTrue($baz);
}
catch (Exception $expected) {
$this->assertEquals('This is expected html from exception', $expected->getMessage());
return;
}
$this->fail('An expected Exception has not been raised Foo_Excpetion.');
}
Now you can do several things with this coverage test. You can either fix up the exception, or fix the code that causes the exception.
Another thing you can do is wrap the entire file.php in a class:
class FooClass {
function runFoo() {
include '/path/to/file.php'; //generates exception
}
}
Then add tests while using extract method until you isolate exception.
[EDIT]
Here is some serious procedural legacy code:
<?php
require_once 'helper.php'; //helper file
function countNewMessages($user_id) {
}
function countNewOrders() {
}
function countNewReturns() {
}
function getDB($init = NULL) {
}
function getDisplay() {
}
getDisplay();
?>
And here is the wrapped class:
<?php
require_once ''; //helper file
class Displayer {
function countNewMessages($user_id) {
}
function countNewOrders() {
}
function countNewReturns() {
}
function getDB($init = NULL) {
}
function getDisplay() {
}
}
?>
And now I can test it:
function testGetDisplay() {
$display = new Displayer();
$this->assertEquals('html code', $display->getDisplay());
}
And test the individual functions in it. And if I can further sprout methods on it.
The above test would be considered a coverage test. There may be bugs in it, but that is what it does. So as I sprout methods the get more code coverage from tests by sprouting that I can make sure I don't break the output.
The extened PHP exception object "prints" a costum HTML error page? You mean its error message is an entire HTML page? That's not very clever...
What you can do about it is to replace the default exception handler (see this function), call getMessage on the exception and parse the HTML error page to extract the message. Then you can print the error message and kill the script. Like this (in PHP 5.3):
set_exception_handler(
function (Exception $e) {
die(parse_html_error_page($e->getMessage()));
}
);
OK, I misunderstood the question. If the script you're testing catches the error and then echoes an error page, then this has nothing to do with exceptions. You can use the ob_ family:
ob_start();
include $file;
$contents = ob_get_contents();
if (result_is_error($contents))
die(extract_error_from_result($contents));
else
echo $contents;
ob_end_clean();
If I am correct, SimpleTest will allow you to assert a PHP error is thrown. However, I can't figure out how to use it, based on the documentation. I want to assert that the object I pass into my constructor is an instance of MyOtherObject
class Object {
public function __construct(MyOtherObject $object) {
//do something with $object
}
}
//...and in my test I have...
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$notAnObject = 'foobar';
$object = new Object($notAnObject);
$this->expectError($object);
}
Where am I going wrong?
Type hinting throws E_RECOVERABLE_ERROR which can be caught by SimpleTest since PHP version 5.2. The following will catch any error containing the text "must be an instance of". The constructor of PatternExpectation takes a perl regex.
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$notAnObject = 'foobar';
$this->expectError(new PatternExpectation("/must be an instance of/i"));
$object = new Object($notAnObject);
}
PHP has both errors and exceptions, which work slightly different. Passing a wrong type to a typehinted function will raise an exception. You have to catch that in your test case. Eg.:
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$notAnObject = 'foobar';
try {
$object = new Object($notAnObject);
$this->fail("Expected exception");
} catch (Exception $ex) {
$this->pass();
}
}
or simply:
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$this->expectException();
$notAnObject = 'foobar';
$object = new Object($notAnObject);
}
But note that this will halt the test after the line where the exception occurs.
Turns out, SimpleTest doesn't actually support this. You can't catch Fatal PHP errors in SimpleTest. Type hinting is great, except you can't test it. Type hinting throws fatal PHP errors.
you have to expect the error before it happens, then SimpleTest will swallow it and count a pass, if the test gets to the end and there is no error then it will fail. (there's expectError and expectException that act in the same way, for PHP (non-fatal) errors and Exceptions, respectively.)