Description
I have a TestSuite which I need to mark as skipped (the entire test suite - not the specific test cases within the suite).
class AllTests
{
public static function suite()
{
// this does not work same as within TestCase:
// throw new \PHPUnit_Framework_SkippedTestError("Out of order");
$Suite = new \PHPUnit_Framework_TestSuite(__NAMESPACE__);
$Suite->addTestSuite(translators\AllTests::cls());
$Suite->addTestSuite(TlScopeTest::cls());
$Suite->addTestSuite(TlNsTest::cls());
$Suite->addTestSuite(TlElementTest::cls());
$Suite->addTestSuite(TlItemTest::cls());
$Suite->addTestSuite(LangCodeTest::cls());
$Suite->addTestSuite(TlElemClassTagTest::cls());
return $Suite;
}
}
As you can see throwing the PHPUnit_Framework_SkippedTestError exception does not work. It is not caught by the PHPUnit, and is breaks the execution as any uncaught exception (which is understandable, as the suite() method is invoked while building tests hierarchy, before actually running the tests).
I've seen an exception class named PHPUnit_Framework_SkippedTestSuiteError, but have no clue how to take advantage of it. Any ideas?
Motivation
I have a TestSuite, which aggregates many test cases as well as other test suites. Almost every test in this fails, becouse of a change which I made in the core of my code.
The problem is that this package is not crutial, and is scheduled to be fixed later. Until then I have to run tests for every other package, but when I do the PHPUnit output becomes flooded with the errors coming from the package in question. This forces me to check every time if any of the failures is coming from any other package.
This, as you might suspect, is very susceptible to human error, i.e. I could miss a failure, which actually is important.
I could run only the test suite on which I am currently working, but I lose control of whether or not my changes in one package causes a failure in other package.
I do not want to comment out that test suite, because I'm afraid that I (or someone who will take over the code after me) could forget about it entirely.
Ok, so I'll put it together:
The AllTests class has to be refactored to extend PHPUnit_Framework_TestSuite.
This makes the class a fully valuable TestSuite and allows to implement a setUp method on the suite level.
The setUp method is called by the test runner (not by the builder), so it is safe to throw a SkippedTestError exception.
The corresponding method to do just that within a test suite is called markTestSuiteSkipped (notice the Suite in the method name).
The entire class would look like this:
class AllTests extends \PHPUnit_Framework_TestSuite
{
protected function setUp()
{
$this->markTestSuiteSkipped("zzz");
}
public static function suite()
{
$Suite = new self(__NAMESPACE__);
$Suite->addTestSuite(translators\AllTests::cls());
$Suite->addTestSuite(TlScopeTest::cls());
$Suite->addTestSuite(TlNsTest::cls());
$Suite->addTestSuite(TlElementTest::cls());
$Suite->addTestSuite(TlItemTest::cls());
$Suite->addTestSuite(LangCodeTest::cls());
$Suite->addTestSuite(TlElemClassTagTest::cls());
return $Suite;
}
}
The output is a pretty block of S letters, which definetly indicate, that we skipped a lot of tests. This cannot escape our attention and yet allows our tests to pass.
You could mark test as skipped.
Related
I have following method:
MyClass {
public function __construct(array $data, ?SomeObject $object): void
}
I have following test:
public function testWrongDataTypeThrowsErrorException(): void
{
$this->expectException(TypeError::class);
//works as expected
new MyClass(null, new SomeObject);
//this is never called
new MyClass('string', new SomeObject);
}
And what happens is it only runs 1 test/assertion with the null parameter. The second time when I try to create MyClass with string, it never gets there. Why is that?
The same logic works if the code throws regular Exception.
I can write 2 separate methods instead, but thats very inconvenient.
Or is it stupid to test for this? My idea is that I want to test that the method accept array and only array. So if somebody would remove it from the method the test would fail.
Place the expectExeception... or exepectError... only directly before the action you want to assert it for.
If you need multiple, write multiple tests methods - for each exception and error individually.
The suggestion above is how it works with Phpunit out of the box and normally keeps your tests clean.
There is however also the option to just make use of PHP here and handle it like the following:
try {
new MyClass(null, new SomeObject);
$this->fail('an expected exception was not thrown');
} catch (TypeError $e) {
$this->addToAssertionCount(1);
}
Then the test-method is not interrupted and you can continue (you have to remove the earlier TypeError expectation as Phpunit will not detect it any longer).
If you feel this is more convenient than separated tests, go for it.
You can even add a helper function and pass a closure in so you can wrap it. Normally it's better to stick with clean tests IMHO, if you really must test for that.
From what you write in your question, I'd say the problem is solved in PHP since the array type-hint back in PHP 5.??? and I would not need to test for that (or you would also need to write tests for your tests, because also the tests can be changed).
If someone changes the method signature, they should have a valid reason for that. If they make a mistake, they perhaps didn't cover their refactoring with tests so the problem lies not in the code of the unit and is not to test for with unit tests.
Keep in mind, that a test should only have one reason to fail. So it's good to aim for that when writing a (unit) test.
Is it OK to have multiple asserts in a single unit test?
I created a simple test for my new Laravel 7 application. But when I run php artisan test I get the following error.
Target [Illuminate\Contracts\View\Factory] is not instantiable.
The error doesn't appear when I go to the page in the browser.
$controller = new HomeController();
$request = Request::create('/', 'GET');
$response = $controller->home($request);
$this->assertEquals(200, $response->getStatusCode());
Although "Just write feature tests" may seem like a cop-out ("They're not unit tests!"), it is sound advice if you do not want to get bogged down by framework-specific knowledge.
You see, this is one of those problems that come from using facades, globals, or static methods. All sorts of things happen outside of your code (and thus your test code) in order for things to work.
The problem
To understand what is going on, you first need to know how Laravel utilizes Containers and Factories in order to glue things together.
Next, what happens is:
Your code (in HomeController::home() calls view() somewhere.
view() calls app() to get the factory that creates Views1
app() calls Container::make
Container::make calls Container::resolve1
Container::resolve decides the Factory needs to be built and calls Container::build to do so
Finally Container::build (using PHP's ReflectionClass figures out that \Illuminate\Contracts\View\Factory can not be Instantiated (as it is an interface) and triggers the error you see.
Or, if you're more of a visual thinker:
The reason that the error is triggered is that the framework expects the container to be configured so that a concrete class is known for abstracts (such as interfaces).
The solution
So now we know what is going on, and we want to create a unit-test, what can we do?
One solution might seem to not use view. Just inject the View class yourself! But if you try to do this, you'll quickly find yourself going down a path that will lead to basically recreating loads of framework code in userland. So not such a good idea.
A better solution would be to mock view() (Now it is really a unit!). But that will still require recreating framework code, only, within the test code. Still not that good.[3]
The easiest thing is to simply configure the Container and tell it which class to use. At this point, you could even mock the View class!
Now, purists might complain that this is not "unit" enough, as your tests will still be calling "real" code outside of the code-under-test, but I disagree...
You are using a framework, so use the framework! If your code uses glue provided by the framework, it makes sense for the test to mirror this behavior. As long as you don't call non-glue code, you'll be fine![4]
So, finally, to give you an idea of how this can be done, an example!
The example
Lets say you have a controller that looks a bit like this:
namespace App\Http\Controllers;
class HomeController extends \Illuminate\Routing\Controller
{
public function home()
{
/* ... */
return view('my.view');
}
}
Then your test[5] might look thus:
namespace Tests\Unit\app\Http\Controllers;
use App\Http\Controllers\HomeController;
use Illuminate\Contracts\View\Factory;
class HomeControllerTest extends \PHPUnit\Framework\TestCase
{
public function testHome()
{
/*/ Arange /*/
$mockFactory = $this->createMock(Factory::class);
app()->instance(Factory::class, $mockFactory);
/*/ Assert /*/
$mockFactory->expects(self::once())
->method('make')
->with('my.view')
;
/*/ Act /*/
(new HomeController())->home();
}
}
A more complex example would be to also create a mock View and have that be returned by the mock factory, but I'll leave that as an exercise to the reader.
Footnotes
app() is asked for the interface Illuminate\Contracts\View\Factory, it is not passed a concrete class name
The reason Container::make does nothing other than call another function is that the method name make is defined by PSR-11 and the Laravel container is PSR compliant.
Also, the Feature test logic provided by Laravel already does all of this for you...
Just don't forget to annotate the test with #uses for the glue that is needed, to avoid warnings when PHPUnit is set to strict mode regarding "risky" tests.
Using a variation of the "Arrange, Act, Assert" pattern
This is not how you test endpoints in Laravel. You should let Laravel instantiate the application as it is already setup in the project, the examples you can see here.
What you already wrote can be rewritten to something like this.
$response = $this->call('GET', route('home')); // insert the correct route
$response->assertOk(); // should be 200
For the test to work, you should extend the TestCase.php, that is located in your test folder.
If you're finding this in The Future and you see #Pothcera's wall of text, here's what you need to know:
The ONLY reason he's doing any of that and the ONLY reason you're seeing this in the first place in a Unit test is because he and you haven't changed from PHPUnit\Framework\TestCase to Tests\TestCase in the test file. This exception doesn't exist when you extend the test case that includes app().
My advice would be to simply extend the correct base test case and move on with your life.
I started using unit and functionality test with this project and because of this I have some questions:
Im working with the symfony php framework. And I have a doctrine like LDAP ORM service.
Furthermore I have a user repository (as a service) which depends on the LDAP ORM service, the logger and a validation service.
Now I want to write a unit test for the addUser function of the UserRepo.
It will internally call: getNewUidNumber, userToEntities, doesUserExist and getUserByUid.
My question is:
Should I mock all these internal function to just test the addUser function? Would that be against the unit test idea (just test the API).
Or should I just mock the LDAP ORM service, the Logger, and the validation service, so that the class calls all internal functions? But this would cause a huge test function with a lot of mocking because I have to mock the repositories for all internal repositories calls.
Or should I start the symfony kernel and use the ServiceContainer to use the ORM LDAP service with a real test database. But wouldn't that be a functionally test and not a unit test?
I heard that its bad to have so many dependencies in a test. So I thought it would be bad to use the whole serviceContainer.
Adduser:
public function addUser(User $user)
{
$pbnlAccount = $this->userToEntities($user);
if(!$this->doesUserExist($user)) {
$pbnlAccount->setUidNumber($this->getNewUidNumber());
$this->ldapEntityManager->persist($pbnlAccount);
$this->ldapEntityManager->flush();
}
else {
throw new UserAlreadyExistException("The user ".$user->getUid()." already exists.");
}
return $this->getUserByUid($user->getUid());
}
For more code, like the internal functions:
https://gist.github.com/NKPmedia/4a6ee55b6bb96e8af409debd98950678
Thanks
Paul
First, I would like to rewrite the method a tiny bit, if I may.
public function addUser(User $user)
{
if ($this->doesUserExist($user)) {
throw new UserAlreadyExistException("The user ".$user->getUid()." already exists.");
}
// ... shortened for brevity
$pbnlAccount = $this->userToEntities($user);
$this->ldapEntityManager->persist($pbnlAccount);
}
The other relevant method is:
private function doesUserExist(User $user)
{
$users = $this->ldapRepository->findByUid($user->getUid());
return count($users) === 1;
}
Immediately we can see that we basically have two tests:
We test that the method throws when the user exists
We test that the method persists a PbnlAccount if the user does not exist.
If you do not see why we have these two tests, note that there are 2 possible "flows" in this method: one where the block inside the if statement is executed, and one where it is not executed.
Lets tackle the first one:
public function testAddUserThrowsWhenUserExistsAlready()
{
$user = new User();
$user->setUid('123');
$ldapRepositoryMock = $this->createMock(LdapRepository::class);
$ldapRepositoryMock
->method('findByUid')
->expects($this->once())
->with('123')
->willReturn(new PbnlAccount());
$userRepository = new UserRepository($ldapRepositoryMock);
$this->expectException(UserAlreadyExistException::class);
$userRepository->addUser($user);
}
The second test is left as an exercise for the reader :)
Yes you will have to do some mocking in your case. You wil need to mock the LdapRepository and LdapEntityManager both in this case.
Note 1: this code probably is not runnable, since I do not know the exact details of your code base (and I wrote this off the top of my head), but that is beside the point. The point is that you want to test for the exception.
Note 2:
I would rename your function to createNewPbnlAccountForUser(User $user) which is longer, but more descriptive of what it actually does.
Note 3:
I am not sure why you are returning $this->getUserByUid() since that seems redundant (you already have the User right there), so I am ommitting that case.
You need to mock ldapEntityManager and all repository services but not the internal function.And as you said, don't boot kernel in unit test. So, you should test all cases with success and throwing exception (make sure to check all behaviour)
If you want to perform a unit test, you should mock all collaborators.
Now, entity managers, ldap services and so on should not be mocked (read more here).
Moreover, if you happen to be in a situation where the Arrange part (set mocks, stubs, and so on) is painful and take "a lot" of the test, maybe this is a smell that your class has too many responsibility (is doing too much things).
That said, when I do unit test, I would like the test to fail only for an internal (to the class) reason, not because I've changed a collaborator line that messes up all my tests.
How do you write a unit test for a method that calls other methods of the same class, but doesn't return a value? (Let's say with PHPUnit.)
For example, let's say that I have the following class:
class MyClass {
public function doEverything() {
$this->doA();
$this->doB();
$this->doC();
}
public function doA() {
// do something, return nothing
}
public function doB() {
// do something, return nothing
}
public function doC() {
// do something, return nothing
}
}
How would you test doEverything()?
EDIT:
I'm asking this because from what I've read it seems like pretty much every method should have its own dedicated unit test. Of course, you also have functional and integration tests, but those target specific routines, so to speak (not on a per method level necessarily).
But if pretty much every method needs its own unit test, I'm thinking it would be "best practice" to unit test all of the above methods. Yes/no?
Okay! I've figured it out! As might be expected, mocking is what I need in this situation--and mocking a sibling method is called partial mocking. There's some pretty great info about PHPUnit mocking in this article by Juan Treminio.
So to test doEverything() in the above class, I would need to do something like this:
public function testDoEverything()
{
// Any methods not specified in setMethods will execute perfectly normally,
// and any methods that ARE specified return null (or whatever you specify)
$mock = $this->getMockBuilder('\MyClass')
->setMethods(array('doA', 'doB', 'doC'))
->getMock();
// doA() should be called once
$mock->expects($this->once())
->method('doA');
// doB() should be called once
$mock->expects($this->once())
->method('doB');
// doC() should be called once
$mock->expects($this->once())
->method('doC');
// Call doEverything and see if it calls the functions like our
// above written expectations specify
$mock->doEverything();
}
That's it! Pretty easy!
BONUS: If you use Laravel and Codeception...
I'm using the Laravel Framework as well as Codeception, which made it a little bit trickier to figure out. If you use Laravel and Codeception you'll need to do a little bit more to get it working, since the Laravel autoloading doesn't by default connect into the PHPUnit tests. You'll basically need to update your unit.suite.yml to include Laravel4, as shown below:
# Codeception Test Suite Configuration
# suite for unit (internal) tests.
class_name: UnitTester
modules:
enabled: [Asserts, UnitHelper, Laravel4]
Once you've updated your file, don't forget to call php codecept.phar build to update your configuration.
While your mocking test does achieve your goal, I would argue that you've decreased confidence in the code. Compare the original trivial method to the complicated method that tests it. The only way the method under test can fail is by forgetting to add one of the method calls or mistype a name. But you're now doubly-likely to do that with all that additional code, and it doesn't have any tests!
Rule: If your test code is more complicated than the code under test, it needs its own tests.
Given the above, you're better off finding another way to test the original code. For the method as written--three method calls with no parameters--inspection by eyeball is sufficient. But I suspect that the method does have some side-effects somewhere, otherwise you could delete it.
Unit testing is about testing the class as a unit, not each method individually. Testing each method alone is a good indication that you're writing your tests after the code. Employing Test Driven Development and writing your tests first will help you design a better class that is more-easily testable.
class MyClass {
private $numeric;
public function MyMethod($numeric)
{
if (! is_numeric($numeric)) {
throw new InvalidArgumentException
}
$this->numeric = $numeric;
}
}
1 - It is necessary to test whether the class exists?
The PHPUnit has several methods that run automatically, such as assertPreConditions, setUp, and others. It is necessary within these methods check whether the class exists using the assertTrue and class_exists? Example:
protected function assertPreConditions()
{
$this->assertTrue(class_exists("MyClass"), "The class does not exists.");
}
2 - It is necessary to check if a method exist? If yes, this test should
be a separate test or within each unit test?
Suppose we have a method that accepts only numeric type parameters, so we have two tests, one test with the correct parameter and another with incorrect method expecting an exception, right? The correct way to write this method would be ...
This way:
public function testIfMyMethodExists()
{
$this->assertTrue(method_exists($MyInstance, "MyMethod"), "The method does not exists.");
}
/**
* #depends testIfMyMethodExists
* #expectedException InvalidArgumentExcepiton
*/
public function testMyMethodWithAValidArgument()
{
//[...]
}
/**
* #depends testIfMyMethodExists
* #expectedException InvalidArgumentExcepiton
*/
public function testMyMethodWithAnInvalidArgument()
{
//[...]
}
Or this way?
public function testMyMethodWithAValidArgument()
{
$this->assertTrue(method_exists($MyInstance, "MyMethod"), "The method does not exists.");
}
/**
* #expectedException InvalidArgumentExcepiton
*/
public function testMyMethodWithAnInvalidArgument()
{
$this->assertTrue(method_exists($MyInstance, "MyMethod"), "The method does not exists.");
//[...]
}
And why?
3 - What is the real purpose of the #covers and the #coversNothing?
I was reading a document that Sebastian Bergmann the creator of PHPUnit wrote as good practice we should always write #covers and #coversNothing in the methods and class and add these options in the xml:
mapTestClassNameToCoveredClassName="true"
forceCoversAnnotation="true"
and in the whitelist:
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php"></directory>
</whitelist>
But what is the real need for it?
4 - What is the correct way to test a constructor that calls another method?
It seems everything is fine, but not on tests.
Even if I do tests with valid arguments and invalid arguments expecting an exception on the method "MyMethod", that will not happen if I enter an incorrect value in the constructor (The test fails).
And if I test with valid argument, the code coverage does not result in 100%.
public function __construct($numeric)
{
$this->MyMethod($numeric);
}
I write tests for methods that should exist, to test that the code does what it is expected of it. I also test the InstanceOf() the class (and inherited class definitions) to ensure the object did create what was suppose to be created. If FOO() extends BAR(), then I test that my object created is an InstanceOf(FOO) and InstanceOf(BAR). If the class gets changed to inherit from something else, or has the extends removed, my test will once again inform the developer to check the code to ensure this change is desired. Potentially some inherited functions are being called on FOO, and without the extends from BAR, this code will break.
I write tests on the various code paths that are there to be executed. Therefore, if I expect that the function should throw an exception when bad data is passed, I write a test for that. I do this to also help document our source code. The tests show the expected behavior. If someone removes the exception (new functionality to accept a different parameter type) then the test should be updated to show that this is allowed. Potentially, the change to the parameters could cause a problem elsewhere. Testing this way ensures that years later, I know that code required this to be a number, and I should definitely re-factor the code carefully if I am changing the parameter types.
Using Test Driven Development (TDD) may cause you to not write the code to throw the exception, as you write a test, then the code to make the test pass. As such, you might not be testing all the parameters and their types or values, but I try to do this as best as I can to validate reasonable data input to try to avoid the Garbage In/Garbage Out (GIGO) problem.
All these tests also give me a good code coverage metric, as the majority of the code base is tested, and the code does step through all the lines in the class files. However, testing to this level, and trying to achieve a high code coverage metric, is really a choice for your team to make if this is desired or not.
I can't see any reason nor advantage of writing this kind of tests. If the class or the method doesn't exist your tests will fail anyway. So you don't have any profit from writing them.
Maybe the one exception from above (point 1) could be a situation where you always build SUT using PHPUnit mock framework (theoretically a situation where your SUT is a mock with mocked other method you don't need in particular test is possible, but I can't imagine real situation which leads to it).
In my opinion if the test covers flow path which is completely included in another test - this means that the test is redundant and unnecessary.
edit:
ad 3. Because you want to know which exactly flow path was invoked by particular unit test. Let's say you have two methods A and B. Method B invokes method A. If you don't have #covers annotation test for method B could generate code coverage for method A, and you can't say if your unit tests for A covers code in 100%.