I am trying to set test expectations on a mock object that is created in a data provider and passed to my test method. This is useful because I can reused my data provider across different test cases and have the tests define what to expect on the mock. However, phpunit marks this test as risky when the case passes, but correctly fails the test when it does not pass. Is this a known thing that cannot be done?
I am using phpunit v9.3
Here is a contrived example to show the problem:
<?php
use PHPUnit\Framework\MockObject\MockObject;
use PHPUnit\Framework\TestCase;
class Test extends TestCase
{
public function provideMock(): array
{
return [
[$this->createMock(\DateTime::class)],
];
}
/** #dataProvider provideMock */
public function testMockPasses(MockObject $mock): void
{
$mock->expects($this->once())->method('format')->with('Y-m-d');
$mock->format('Y-m-d');
}
/** #dataProvider provideMock */
public function testMockFails(MockObject $mock): void
{
$mock->expects($this->once())->method('format')->with('Y-m-d');
$mock->format('Y-m-');
}
}
I would expect this to work fine as I am just passing the object to the method - all internal php stuff.
Try running PHPUnit with --verbose key - it may tell more about the reason of marking the test as risky.
Related
I have a Symfony WebTestCase-extending PHP Test class and a DRY-trait. The test functions in my class #depend on the test functions in my trait, however I can't manage to execute the trait tests before the class tests. Is that possible?
trait TestTrait
{
/**
* #beforeClass
*/
public function testBeforeAnythingElseHappens()
{
/*...*/
}
/* more functions */
}
Test Class
class Test extends WebTestCase
{
use TestTrait;
/**
* #depends testBeforeAnythingElseHappens
*/
function testClassSpecificStuff()
{
/* ... */
}
}
First, see PHPUnit Manual - Appendix B. Annotations:
#beforeClass
The #beforeClass annotation can be used to specify static methods that should be called before any test methods in a test class are run to set up shared fixtures.
So, you should not use #beforeClass here - it is intended to allow setting up fixtures. In addition, the methods annotated as such should be static.
Second, see PHPUnit Manual - Appendix B. Annotations:
#depends
PHPUnit supports the declaration of explicit dependencies between test methods. Such dependencies do not define the order in which the test methods are to be executed but they allow the returning of an instance of the test fixture by a producer and passing it to the dependent consumers.
So yes, you could use the #depends annotation to declare dependencies between tests. Just remove the #beforeClass annotation:
trait TestTrait
{
public function testBeforeAnythingElseHappens()
{
}
}
class Test extends WebTestCase
{
use TestTrait;
/**
* #depends testBeforeAnythingElseHappens
*/
public function testClassSpecificStuff()
{
}
}
Third, a few words of advice (from my own experience):
Do not require tests to be run in a certain order.
If you arrange something in one test and need the same thing in another, just do it multiple times, do not worry too much about repeating yourself.
For reference, see:
http://blog.ploeh.dk/2011/05/24/DesignSmellTemporalCoupling/
What does “DAMP not DRY” mean when talking about unit tests?
I've been reading this manual, but I'm not understanding why the $stack loose the values after each test function.
Here's my code:
require_once BASE . 'Transaction.php';
class TransactionTest extends PHPUnit_Framework_TestCase
{
protected $stack;
protected function setUp()
{
Database::getInstance()->connect();
$this->stack = new Transaction(123456789);
}
public function testInsert()
{
$data['name'] = 'Omega';
$this->stack->set($data);
$this->assertTrue($this->stack->save());
}
public function testUpdate()
{
$object = PHPUnitReflectionClass::getInstance($this->stack);
$this->assertEquals(array('name' , 'Omega'), $object->getProperty('name'));
}
}
At the "testUpdate" function, I don't have the name.
If I just copy all the PHPUnit example, all tests will run perfectly, but I'm not able to see the values using var_dump at the next function. And this is another thing that I don't get it.
The problems are:
the setup method is called before each test method invocation;
your test are depends each other.
For the first problem you can use the setUpBeforeClass method. From the doc:
The setUp() and tearDown() template methods are run once for each test
method (and on fresh instances) of the test case class.
In addition, the setUpBeforeClass() and tearDownAfterClass() template
methods are called before the first test of the test case class is run
and after the last test of the test case class is run, respectively.
For the second problem, is a bad practice to have tests that depends each otherbut PHPUnit supports the declaration of explicit dependencies between test methods, so you can use the #depends annotation to express dependencies: if a test fail the other is not executed.
So your test class can be, as example:
require_once BASE . 'Transaction.php';
class TransactionTest extends PHPUnit_Framework_TestCase
{
protected static $stack;
public static function setUpBeforeClass()
{
Database::getInstance()->connect();
$this->stack = new Transaction(123456789);
}
public function testInsert()
{
$data['name'] = 'Omega';
$this->stack->set($data);
$this->assertTrue($this->stack->save());
}
/**
* #depends testInsert
*/
public function testUpdate()
{
$object = PHPUnitReflectionClass::getInstance($this->stack);
$this->assertEquals(array('name' , 'Omega'), $object->getProperty('name'));
}
}
Hope this help
Unit tests should not depend on other tests, it's considered bad practice. The point of unit tests is that they run in a controlled and isolated environment. The setUp() and tearDown() methods are called respectively before and after each test method. And thus your $stack property is overwritten before each test. This is also pointed out in the document you linked to (just below example 4.1):
The setUp() and tearDown() template methods are run once for each test method (and on fresh instances) of the test case class.
(You're also making the assumption that the methods in your test class are run in the same order that they are written, but that isn't necessarily the case. Most of the time they do run in the same order, but you can't depend on it.)
I know it's possible to test private/protected methods with PHPUnit using reflection or other workarounds.
But most sources tell me that it's not best practice to write tests for private methods inside of a class.
You are supposed to test the class as if it were a "black box" — you just test for expected behavior by comparing inputs with outputs disregarding the internal mechanics. Writing tests for classes should also notify you to unused private methods, by showing lack of code coverage.
When I test my class and generate an HTML report, it shows the private methods as not covered by tests, even though the lines from which they are called are absolutely executed/covered. I know that the private methods are executed, because if they weren't the assertions on my class would not pass.
Is this expected behavior in PHPUnit? Can I strive for 100% coverage, while still testing private methods only indirectly?
Some simplified example code (using RestBundle in Symfony2):
class ApiController extends FOSRestController {
/*
* #REST\View()
* #REST\Get("/api/{codes}")
*/
public function getCodesAction($codes) {
$view = new View();
$view->setHeader('Access-Control-Allow-Origin', '*');
$view->setData(array('type' => 'codes','data' => $this->_stringToArray($codes)));
$view->setFormat('json')->setHeader('Content-Type', 'application/json');
return $this->handleView($view);
}
private function _stringToArray($string){
return explode('+',$string);
}
The public function shows as "covered", the private function is indirectly covered but shows colored red in PHPUnit reports.
Test:
class ApiControllerTest extends WebTestCase {
public function test_getCodesAction(){
$client = static::createClient();
$client->request('GET', '/api/1+2+3');
$this->assertContains('{"type": "codes", "data": [1,2,3]}', $client->getResponse()->getContent());
}
}
This is just a silly example of course, I could just as well include the explode() right there in the public function; But the controllers I'm writing tests for contain much more intricate and re-usable private functions which transform data in more complex ways (but are still side-effect free).
In Phpunit you can specify the Covered Methods with special annotation, as descrived in the doc.
You can do something like this:
class ApiControllerTest extends WebTestCase {
/**
* #covers ApiController::getCodesAction
* #covers ApiController::_stringToArray
*/
public function test_getCodesAction(){
$client = static::createClient();
$client->request('GET', '/api/1+2+3');
$this->assertContains('{"type": "codes", "data": [1,2,3]}', $client->getResponse()->getContent());
}
}
Hope this help
My setup is something like this:
class MyTest extends PHPUnit_Framework_TestCase
{
// More tests before
public function testOne()
{
// Assertions
return $value;
}
/**
* #depends testOne
*/
public function testTwo($value)
{
// Assertions
}
// More tests after
}
I'd like to focus on testTwo but when I do phpunit --filter testTwo I get message like this:
This test depends on "MyTest::testOne" to pass.
No tests executed!
My question: Is there a way to run one test with all its dependencies?
There's not out of the box way to run automatically all the dependencies. You can however put your tests in groups with the #group annotation and then run phpunit --group myGroup.
I know, this is also not much convenient, but you can try
phpunit --filter 'testOne|testTwo'
According to phpunit docs we can use regexps as filter.
Also you may consider using data provider to generate your value for the second test. But be aware that data provider method will always be executed before all tests so it may slow down the execution if it has any heavy processing.
One more approach is to create some helper method or object that will do some actual job and cache results to be used by various tests. Then you won't need to use dependencies and your data will be generated on request and cached to be shared by different tests.
class MyTest extends PHPUnit_Framework_TestCase
{
protected function _helper($someParameter) {
static $resultsCache;
if(!isset($resultsCache[$someParameter])) {
// generate your $value based on parameters
$resultsCache[$someParameter] = $value;
}
return $resultsCache[$someParameter];
}
// More tests before
public function testOne()
{
$value = $this->_helper('my parameter');
// Assertions for $value
}
/**
*
*/
public function testTwo()
{
$value = $this->_helper('my parameter');
// Get another results using $value
// Assertions
}
// More tests after
}
use regex
phpunit --filter='/testOne|testTwo/'
I have something like the following set up in Laravel:
In /app/controllers/MyController.php:
class MyController extends BaseController {
const MAX_FILE_SIZE = 10000;
// ....
}
In /app/tests/MyControllerTest.php:
class MyControllerTest extends TestCase {
public function myDataProvider() {
return [
[ MyController::MAX_FILE_SIZE ]
];
}
/**
* #dataProvider myDataProvider
*/
public function testMyController($a) {
// Just an example
$this->assertTrue(1 == 1);
}
}
However, when I run vendor/bin/phpunit I get the following error:
PHP Fatal error: Class 'Controller' not found in /home/me/my-app/app/controllers/BaseController.php on line 3
Fatal error: Class 'Controller' not found in /home/me/my-app/app/controllers/BaseController.php on line 3
If I remove the reference to the MyController class in myDataProvider() and replace it with a literal constant then the test completes successfully.
In addition, I can place references to MyController::MAX_FILE_SIZE inside the actual testMyController() method, and the test also completes successfully.
It appears that the autoloading setup for Laravel framework classes isn't being set up until after the data provider method is being called, but before the actual test methods are called. Is there any way around this so that I can access Laravel framework classes from within a PHPUnit data provider?
NOTE: I'm calling PHPUnit directly from the command line and not from within an IDE (such as NetBeans). I know some people have had issues with that, but I don't think that applies to my problem.
As implied in this answer, this appears to be related to the order that PHPUnit will call any data providers and the setUp() method in any test cases.
PHPUnit will call the data provider methods before running any tests. Before each test it will also call the setUp() method in the test case. Laravel hooks into the setUp() method to call $this->createApplication() which will add the controller classes to the 'include path' so that they can be autoloaded correctly.
Since the data provider methods are run before this happens then any references to controller classes inside a data provider fail. It's possible work around this by modifying the test class to something like this:
class MyControllerTest extends TestCase {
public function __construct($name = null, array $data = array(), $dataName = '') {
parent::__construct($name, $data, $dataName);
$this->createApplication();
}
public function myDataProvider() {
return [
[ MyController::MAX_FILE_SIZE ]
];
}
/**
* #dataProvider myDataProvider
*/
public function testMyController($a) {
// Just an example
$this->assertTrue(1 == 1);
}
}
This will call createApplication() before the data provider methods are run, and so there is a valid application instance that will allow the appropriate classes to be autoloaded correctly.
This seems to work, but I'm not sure if it's the best solution, or if it is likely to cause any issues (although I can't think of any reasons why it should).
The test will initialize much faster if you create the application right within the dataProvider method, especially if you have large set of items to test.
public function myDataProvider() {
$this->createApplication();
return [
[ MyController::MAX_FILE_SIZE ]
];
}
Performance warning for the other solutions (especially if you plan to use factories inside your dataProviders):
As this article explains:
The test runner builds a test suite by scanning all of your test
directories […] When a
#dataProvider annotation is found, the referenced data provider is
EXECUTED, then a TestCase is created and added to the TestSuite for
each dataset in the provider.
[…]
if you use factory methods in your data providers, these
factories will run once for each test utilizing this data provider
BEFORE your first test even runs. So a data provider […] that is used by ten tests
will run ten times before your first
test even runs. This could drastically slow down the time until your
first test executes. Even […] using phpunit --filter,
every data provider will still run multiple times. Filtering occurs after the test
suite has been generated and therefore after any
data providers have been executed.
The above article proposes to return a closure from the dataProvider and execute that in your test:
/**
* #test
* #dataProvider paymentProcessorProvider
*/
public function user_can_charge_an_amount($paymentProcessorProvider)
{
$paymentProcessorProvider();
$paymentProcessor = $this->app->make(PaymentProviderContract::class);
$paymentProcessor->charge(2000);
$this->assertEquals(2000, $paymentProcessor->totalCharges());
}
public function paymentProcessorProvider()
{
return [
'Braintree processor' => [function () {
$container = Container::getInstance();
$container->bind(PaymentProviderContract::class, BraintreeProvider::class);
}],
...
];
}
You can adjust this behaviour of PHPUnit by adding your custom bootstrapper to your projects phpunit.xml like this (look at 3rd line):
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
bootstrap="tests/bootstrap.php" ← ← ← THIS
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd">
...
</phpunit>
Then create a bootstrap.php file in your tests folder (i.e. the path you denoted above), and paste this:
<?php
use Illuminate\Contracts\Console\Kernel;
require __DIR__ . '/../vendor/autoload.php';
$app = require __DIR__.'/../bootstrap/app.php';
$app->make(Kernel::class)->bootstrap();
You can now use Laravel functionality in your data providers, just keep in mind they still run after your setUp methods.