For our PHPUnit testing, we sometimes write custom assertions. Today I found a custom assertion that wasn't asserting quite what it ought to have been. It seems that this problem could have been avoided if I had written a unit test for the assertion itself.
The only problem I see is that I'm not quite sure how to handle writing tests for an assertion that it ought to fail, without having that lead to the test itself failing. In other words, for a test that expects a string, 'foo', I want to do something like:
public function testAssertFoo()
{
$var = 'bar';
$callable = array( $this, "assertFoo" );
$this->assertTestFails( $callable, $var );
}
Of course, there is no assertTestFails assertion. But is there a clean way to do something like that?
Assuming that assertFoo uses PHPUnit's built-in assertions such as assertEquals, you can simply catch the PHPUnit_Framework_ExpectationFailedException that is thrown when the assertion fails.
function testAssertFoo() {
try {
$this->assertFoo('bar');
self::fail("assertFoo should fail for 'bar'");
}
catch (PHPUnit_Framework_ExpectationFailedException $e) { /* test passed */ }
}
function assertFoo($value) {
self::assertEquals('foo', $value);
}
Related
The exception is thrown correctly in the program, but the test does not detect it. Why??
public function null_user_send()
{
$message = 'Hi';
$response = $this->post(route('dialogSend'), ['to' => -1, 'message' => $message]);
$this->assertDatabaseMissing('messages', ['to' => -1, 'message' => $message]);
$this->expectException(Exception::class);
$response->assertStatus(500);
}
MailController:
/** #throws \Exception */
public function dialogSend(Request $request)
{
$handler = app(MailHandler::class);
if ($request->input('to') <= 0) {
throw new \Exception('Параметр "to" имел отрицательное значение', 500);
}
...
}
Test response: 'Failed asserting that exception of type "Exception" is thrown.'
And it doesn't matter what class of exceptions, nothing works. Pls help
Check the namespace of the exception you are testing
$this->expectException(Exception::class);
probably should be
$this->expectException(\Exception::class);
This is because your test itself sits in its own namespace (depend on the type of test). What your test is may actually be seeing is
$this->expectException(\Tests\Exception::class);
which will of course never be thrown since it doesn't exist.
One of the reasons you are having trouble diagnosing this is the nature of your tests. There are a few things there, while not strictly wrong, are going to make your testing that much harder to determine the cause of your problem.
You're trying to test too much at once. Separate each concern into its own test so that you can first determine if the database is empty, then if it throws an exception, then if it returns a 500 error. You are essentially testing three different aspects of your application here.
public function null_user_send()
{
// Arrange
$message = 'Hi';
// Act
$response = $this->post(route('dialogSend'), ['to' => -1, 'message' => $message]);
// Assert
$this->assertDatabaseMissing('messages', ['to' => -1, 'message' => $message]);
}
public function does_an_exception()
{
// Arrange, Act
...
// Assert
$this->expectException(Exception::class);
}
public function returns_server_erro()
{
// Arrange, Act
...
// Assert
$response->assertStatus(500);
}
Give your tests meaning in their names. This will help you immensely in the future when you can't remember what something is meant to do verus what it is actually doing.
public function it_should_not_have_a_message_if_the_input_is_wrong()
{
// database test
}
public function it_should_throw_an_exception_if_the_input_is_wrong()
{
// exception test
}
public function it_should_return_a_500_error_if_the_input_is_wrong()
{
// http code test
}
A good test should be able to answer the following question:
If I set up my application like this, and then I do that, then my application should now look like this.
This is where the Arrange (organise my application), Act (do something) and Assert (check my application) comes from in the code above.
There are couple of other things you might to look at, although this will be up you depending on your needs.
You look like you're doing validation on input here. You should probably throw a 400 or 422 error if your input is wrong
The 500 error code on your exception probably won't really do anything, it won't be related to the 500 HTTP status code, unless you have something in your error handling that does that.
In the PHP project I'm working on there are several methods that make use of individual try/catch blocks. Here is one example:
public function getListData()
{
$clauses['filter'] = '';
$clauses['sort'] = 'CAST(propertyID AS INT) DESC';
try
{
return $this->getModel()->getListData($clauses);
}
catch (Exception $e)
{
// create an Error() object, send $e->getMessage() to it
}
}
Now, keeping in mind there are several similar methods, it would seem more efficient to write a method in the Model class that would look like this:
public function run($method)
{
try
{
return $this->$method;
}
catch (Exception $e)
{
//create an Error() object, send $e->getMessage() to it
}
}
The problem is calling it. This does not work:
public function getListData()
{
return $this->getModel()->run('getListData($clauses)');
}
The error is:
Undefined property:
classes\utility\Model::$getListData($clauses).
Is there a way to get this to work?
I'm going to assume that the first and second getListData() methods are in separate classes, otherwise you are calling a loop, since getListData would call getListData...which would call, you get it.
However, the way you are calling the method is incorrect in the run() method. It should be called using call_user_func. It is a callback to the method, not a call to the property, of the class.
You could call it statically using
public function run($method, $data)
{
try
{
return call_user_func(array($this, $method), $data);
}
catch (Exception $e)
{
//create an Error() object, send $e->getMessage() to it
}
}
public function getListData()
{
return $this->getModel()->run('getListData', $clauses);
}
There are several problems with this approach:
It prevents you from listening for custom exceptions
You can throw exceptions other than Exception, but this type of wrapper will make it much more difficult to do so.
It is difficult to follow the execution flow
When you pass method names and parameters around as strings, it becomes much harder for humans, IDEs and code analysis tools to understand what the program will do at runtime.
Try/catch blocks are cheap
The code required to catch exceptions is very simple and easy to use. This wrapper adds more complexity and more cost (an extra function call).
Consider just using try/catch blocks where needed instead of using the wrapper. If you have fifty similar methods as described in your comment above, you may gain more efficiency by eliminating the duplicate business logic and combining those methods.
You could simply convert errors to exceptions using this code:
set_error_handler(function ($errno, $errstr, $errfile, $errline)
{
if ((error_reporting() & $errno) === $errno)
throw new \Exception("$errstr ($errfile: $errline)", (int) $errno);
}, -1);
After it any error would be converted to exception.
So I playing around with PHPUnit and would like to get some insight to the output that PHPUnit generates when I try to test for an Exception. I am confused as to why I am getting a failed test. Here is my test:
class ConfigTest extends PHPUnit_Framework_Testcase
{
public function testTrueIfJobGivenExists()
{
$conf = Config::getInstance('test1.php', new Database());
$setup = $conf->getConfig();
$this->assertTrue($setup);
}
/**
* #expectedException Exception
*/
public function testExceptionIfJobGivenNotExists()
{
$conf = Config::getInstance('test.php', new Database());
$setup = $conf->getConfig();
}
}
In here I am not mocking the Database class (I have not learned how to do that yet) but basically the code looks for and entry for a job called test.php and pulls the config col for that. If the job does not exists it throws a new Exception. Here is my output:
PHPUnit 4.1.0 by Sebastian Bergmann.
.F
Time: 26 ms, Memory: 3.50Mb
There was 1 failure:
1) ConfigTest::testExceptionIfJobGivenNotExists
Failed asserting that exception of type "Exception" is thrown.
FAILURES!
Tests: 2, Assertions: 2, Failures: 1.
Here to me seems that the test is failing but looking at the PHPUnit documentation about testing exception the output looks similar. Is my test working?
EDIT: New test fail
Using Mockery I created my test like:
class ConfigTest extends PHPUnit_Framework_Testcase
{
public function tearDown()
{
Mockery::close();
}
public function testTrueIfConfigForGivenJobExists()
{
$dbJSON = array( array(
'jobConfig' => '{
"config": {
"aquisition": {
"type": "xx",
"customerKey": "xxxxx",
"login":"xxxx",
"password":"xxxxx",
"host":"xxxxxx",
"account":"",
"email":""
}
}
}'
) );
$database = Mockery::mock('Database');
$database->shouldReceive('select->where->runQuery->fetch')->andReturn($dbJSON);
$conf = Config::getInstance('getLoadsPE.php', $database);
$setup = $conf->getConfig();
$this->assertTrue($setup);
}
/**
* #expectedException Exception
*/
public function testExceptionIfJobGivenNotExists()
{
$database = Mockery::mock('Database');
$database->shouldReceive('select->where->runQuery->fetch')->andReturn(null);
$conf = Config::getInstance('getLoadsPE.php', $database);
$setup = $conf->getConfig();
$this->assertTrue($setup);
}
}
and I get
PHPUnit 4.1.0 by Sebastian Bergmann.
.F
Time: 39 ms, Memory: 4.75Mb
There was 1 failure:
1) ConfigTest::testExceptionIfJobGivenNotExists
Failed asserting that exception of type "Exception" is thrown.
FAILURES!
Tests: 2, Assertions: 3, Failures: 1
With this I dont know where the 3rd assertion is coming from. Also I dont get why Im getting the Fail test. If I comment the first test then the second passes. Any thoughts anyone?
FYI
This is what getConfig() looks like:
public function getConfig()
{
if ($this->flag) {
// Config has already been set
return true;
}
$data = self::$database->select('configs', ['jobConfig'])
->where('jobName', self::$jobName)
->runQuery()
->fetch();
if (empty($data)) {
throw new Exception("Config Exception: No config available for " . self::$jobName, 1);
}
if (count($data) > 1) {
throw new Exception("Config Exception: More than one config for same job!!!", 1);
}
$arr = json_decode($data[0]['jobConfig'], true);
// maybe threre is a better way of doing this
if (array_key_exists('aquisition', $arr['config'])) {
$this->aquisition = $arr['config']['aquisition'];
}
if (array_key_exists('ftpSetup', $arr['config'])) {
$this->ftpSetup = $arr['config']['ftpSetup'];
}
if (array_key_exists('fileSetup', $arr['config'])) {
$this->fileSetup = $arr['config']['fileSetup'];
}
if (array_key_exists('fileMaps', $arr['config'])) {
$this->fileMaps = $arr['config']['fileMaps'];
}
if (array_key_exists('fileRows', $arr['config'])) {
$this->fileRows = $arr['config']['fileRows'];
}
$this->flag = true;
return true;
}
}
#expectedException Exception is not a good idea here. If an exception is thrown in your test setup (e.g. first line of you test) your test will still pass.
You could use:
//given
$conf = ...;
try {
//when
$conf->getConfig();
$this->fail("YourException expected");
//then
} catch (YourException $e) {}
But it's messy and will not work with Exception (because phpunit fail also throws Exception). So you would have to use a custom exception.
You can try CatchException from ouzo goodies:
//given
$conf = ...;
//when
CatchException::when($conf)->getConfig();
//then
CatchException::assertThat()->isInstanceOf("Exception");
The obvious answer to your question is that there actually is a job in your database at the time the test is run, and therefore your code is not throwing the exception you expect.
This may or may not be correct, but from your code there is no way for us to know that; in fact, from your code, there is no way for the test to adequately test for that case because you don't know what's going to be in the database at the time the test is run. This is why mocking dependencies like the database is so important: you won't actually test your code the way you think you're testing it unless all the external stuff is set up exactly the way you want it to be.
I personally find it hard to wrap my head around testing and when I've been away from it for a few days (like on a Monday morning), I find it helpful to stick to a formula to get me back into the groove. I like to use the "given, when, then" pattern when writing my tests. I picked this up from Jeffrey Way's book "Laravel Testing Decoded". Here's what it looks like:
public function testSomething()
{
// given
// ... set up everything for the test here
// when
// ... execute the code that's being tested
// then
// ... assert
}
So in your case, it might look something like this:
/**
* #expectedException Exception
*/
public function testExceptionIfJobGivenNotExists()
{
// given
// I don't know what your database interface looks like, so I'm
// making stuff up here
$database = Mockery::mock("Database");
$database->shouldReceive("query")->with("SELECT something")->andReturn(null);
// set up the config object
$conf = Config::getInstance('test.php', $database);
// when
// execute your code here
$setup = $conf->getConfig();
// then
// normally you'd assert something here, but in this case, the exception
// handler will take care of it
}
(I've assumed you're using Mockery for mocking. I've also assumed that your database class will return null when you query for a job and one doesn't exist.)
Don't sweat the mocking - it's really not that hard. All I've done here is replace your new Database() with a "fake" database object (ie. mocked). The shouldReceive method simply tells Mockery that the query method should be called with some arguments and return a value. Mocking can be much more complicated than this, but to start out, this is pretty simple.
Why is this important?
This test is supposed to tell you what happens when there's no job in the database. In your test, you have no way of knowing whether your test is actually testing that, because you really have no way of knowing what the database class is returning when you run the query. In my example, on the other hand, I've "faked" the database class and ensured that it's going to return exactly what I want it to return, every time the test is run.
You expect exception to be thrown but it is not thrown. That's why your test fails. Check if your code with dataset test.php should really throw an exception.
I'm testing some legacy code that extends the default php exception object. This code prints out a custom HTML error message.
I would like to mock this exception object in such a way that when the tested code generates an exception it will just echo the basic message instead of giving me the whole HTML message.
I cannot figure out a way to do this. It seems like you can test for explicit exceptions, but you can't change in a general way the behavior of an exception, and you also can't mock up an object that extends a default php functionality. ( can't think of another example of this beyond exceptions... but it would seem to be the case )
I guess the problem is, where would you attach the mocked object?? It seems like you can't interfere with 'throw new' and this is the place that the object method is called....
Or if you could somehow use the existing phpunit exception functionality to change the exception behavior the way you want, in a general way for all your code... but this seems like it would be hacky and bad....
EDIT: here is some code to make things clearer:
class FooTest extends PHPUnit_Framework_TestCase{
public function testBar(){
include '/path/to/file.php'; //generates exception
$this->assertTrue($baz);
}
}
...
//overridden exception class
class Foo_Exception extends ErrorException{
...
so, my question, is there a way to deal with this overriden class, without doing it on a case by case basis? what if I'm not testing the behavior of the exception, just the code that causes the exception?
I would first write a test that captures the exception generation behavior:
include '/path/to/file.php'; //generates exception
public function testCatchFooException() {
try {
$this->assertTrue($baz);
}
catch (Exception $expected) {
$this->assertEquals('This is expected html from exception', $expected->getMessage());
return;
}
$this->fail('An expected Exception has not been raised Foo_Excpetion.');
}
Now you can do several things with this coverage test. You can either fix up the exception, or fix the code that causes the exception.
Another thing you can do is wrap the entire file.php in a class:
class FooClass {
function runFoo() {
include '/path/to/file.php'; //generates exception
}
}
Then add tests while using extract method until you isolate exception.
[EDIT]
Here is some serious procedural legacy code:
<?php
require_once 'helper.php'; //helper file
function countNewMessages($user_id) {
}
function countNewOrders() {
}
function countNewReturns() {
}
function getDB($init = NULL) {
}
function getDisplay() {
}
getDisplay();
?>
And here is the wrapped class:
<?php
require_once ''; //helper file
class Displayer {
function countNewMessages($user_id) {
}
function countNewOrders() {
}
function countNewReturns() {
}
function getDB($init = NULL) {
}
function getDisplay() {
}
}
?>
And now I can test it:
function testGetDisplay() {
$display = new Displayer();
$this->assertEquals('html code', $display->getDisplay());
}
And test the individual functions in it. And if I can further sprout methods on it.
The above test would be considered a coverage test. There may be bugs in it, but that is what it does. So as I sprout methods the get more code coverage from tests by sprouting that I can make sure I don't break the output.
The extened PHP exception object "prints" a costum HTML error page? You mean its error message is an entire HTML page? That's not very clever...
What you can do about it is to replace the default exception handler (see this function), call getMessage on the exception and parse the HTML error page to extract the message. Then you can print the error message and kill the script. Like this (in PHP 5.3):
set_exception_handler(
function (Exception $e) {
die(parse_html_error_page($e->getMessage()));
}
);
OK, I misunderstood the question. If the script you're testing catches the error and then echoes an error page, then this has nothing to do with exceptions. You can use the ob_ family:
ob_start();
include $file;
$contents = ob_get_contents();
if (result_is_error($contents))
die(extract_error_from_result($contents));
else
echo $contents;
ob_end_clean();
If I am correct, SimpleTest will allow you to assert a PHP error is thrown. However, I can't figure out how to use it, based on the documentation. I want to assert that the object I pass into my constructor is an instance of MyOtherObject
class Object {
public function __construct(MyOtherObject $object) {
//do something with $object
}
}
//...and in my test I have...
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$notAnObject = 'foobar';
$object = new Object($notAnObject);
$this->expectError($object);
}
Where am I going wrong?
Type hinting throws E_RECOVERABLE_ERROR which can be caught by SimpleTest since PHP version 5.2. The following will catch any error containing the text "must be an instance of". The constructor of PatternExpectation takes a perl regex.
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$notAnObject = 'foobar';
$this->expectError(new PatternExpectation("/must be an instance of/i"));
$object = new Object($notAnObject);
}
PHP has both errors and exceptions, which work slightly different. Passing a wrong type to a typehinted function will raise an exception. You have to catch that in your test case. Eg.:
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$notAnObject = 'foobar';
try {
$object = new Object($notAnObject);
$this->fail("Expected exception");
} catch (Exception $ex) {
$this->pass();
}
}
or simply:
public function testConstruct_ExpectsAnInstanceOfMyOtherObject() {
$this->expectException();
$notAnObject = 'foobar';
$object = new Object($notAnObject);
}
But note that this will halt the test after the line where the exception occurs.
Turns out, SimpleTest doesn't actually support this. You can't catch Fatal PHP errors in SimpleTest. Type hinting is great, except you can't test it. Type hinting throws fatal PHP errors.
you have to expect the error before it happens, then SimpleTest will swallow it and count a pass, if the test gets to the end and there is no error then it will fail. (there's expectError and expectException that act in the same way, for PHP (non-fatal) errors and Exceptions, respectively.)