I have my test class like this:
/**
* #runTestsInSeparateProcesses
*/
class ProfileTest extends TestCase
{
public function testFalsePositive()
{
$this->assertFalse(true);
}
}
And the weird thing is, this test class passes successfully.
If I remove annotation "runTestsInSeparateProcesses" - then it gives me correct result (failing test).
Another weird thing is, that in my other test cases, there should be "call method on null" exception, and even though phpunit cheerfully informed me, that tests passed correctly.
I'm sure, that phpunit command catches my testclass.
I'm using PHPUnit v.7.4.1.
Can anybody tell me what's going on?
Related
PHPUnit has two special annotations to indicate a test should run in a background process: a class annotation #runTestsInSeparateProcesses that affects all tests in the current class, and a test annotation #runInSeparateProcess that affects the current test.
This is useful in some cases when the tested class needs to print some output, so it would not mess up with PHPUnit's own output.
So, here's my question: is the TestCase class aware of this fact? Does it know when it's running on background? If so, is there a method to detect it?
I'm looking for something like $this->isRunningOnBackground().
I did not find anything about this in the documentation.
Browsing PHPUnit's source code, I found a boolean protected property in TestCase class $runTestInSeparateProcess that is set to true when the annotation is present. There's a $runClassInSeparateProcess as well, but it's set to private, so you won't access it in a child class context.
We can detect if a test is running on background by just checking if ($this->runTestInSeparateProcess) {...} inside setUp() or tearDown().
In a Yii project, I am using phpunit with getMockbuilder. When I run unit tests on the whole file, they all pass. However, when I do phpunit --filter testMyFunction, I get the following error. "Call to undefined method Mock_Account_3a811374::__construct() ..."
After doing a little more checking I see that if the --filter ends up including a test that does not use the mock in addition to the one that does, then it works fine.
Anyone have any ideas on how to fix it?
Here's some of my code (simplified) ...
use components\Account;
class UtilsTest extends CDbTestCase
{
...
public function testMyFunction()
{
$accountStub = $this->getMockBuilder('Account')
->disableOriginalConstructor()
->setMethods(array('methodToStub'))
->getMock();
$accountStub->expects($this->any())
->method('methodToStub')
->will($this->returnValue(false));
$accountStub->__construct();
...
}
}
I'm confused by what are are trying to do with $accountStub->__construct() when you have specified that you don't want to call the original constructor with ->disableOriginalConstructor()?
You can call the constructor yourself and pass parameters with setConstructorArgs(array $args). So this would look something like this:
$accountStub = $this->getMockBuilder('Account')
->setConstructorArgs($args)
->getMock();
However it wouldn't make sense to call this at the same time as disableOriginalConstructor(). I think you probably want to do one or the other.
I believe the error message you are getting is just PHP telling you that you are trying to do something that doesn't make sense. You are trying to call the constructor on an object that has already been constructed. What's more, you have even specifically told the mock to skip the call to the constructor method. PHP is just telling you that this method does not exist.
I think you probably just need to remove the following line and test again:
$accountStub->__construct();
Description
I have a TestSuite which I need to mark as skipped (the entire test suite - not the specific test cases within the suite).
class AllTests
{
public static function suite()
{
// this does not work same as within TestCase:
// throw new \PHPUnit_Framework_SkippedTestError("Out of order");
$Suite = new \PHPUnit_Framework_TestSuite(__NAMESPACE__);
$Suite->addTestSuite(translators\AllTests::cls());
$Suite->addTestSuite(TlScopeTest::cls());
$Suite->addTestSuite(TlNsTest::cls());
$Suite->addTestSuite(TlElementTest::cls());
$Suite->addTestSuite(TlItemTest::cls());
$Suite->addTestSuite(LangCodeTest::cls());
$Suite->addTestSuite(TlElemClassTagTest::cls());
return $Suite;
}
}
As you can see throwing the PHPUnit_Framework_SkippedTestError exception does not work. It is not caught by the PHPUnit, and is breaks the execution as any uncaught exception (which is understandable, as the suite() method is invoked while building tests hierarchy, before actually running the tests).
I've seen an exception class named PHPUnit_Framework_SkippedTestSuiteError, but have no clue how to take advantage of it. Any ideas?
Motivation
I have a TestSuite, which aggregates many test cases as well as other test suites. Almost every test in this fails, becouse of a change which I made in the core of my code.
The problem is that this package is not crutial, and is scheduled to be fixed later. Until then I have to run tests for every other package, but when I do the PHPUnit output becomes flooded with the errors coming from the package in question. This forces me to check every time if any of the failures is coming from any other package.
This, as you might suspect, is very susceptible to human error, i.e. I could miss a failure, which actually is important.
I could run only the test suite on which I am currently working, but I lose control of whether or not my changes in one package causes a failure in other package.
I do not want to comment out that test suite, because I'm afraid that I (or someone who will take over the code after me) could forget about it entirely.
Ok, so I'll put it together:
The AllTests class has to be refactored to extend PHPUnit_Framework_TestSuite.
This makes the class a fully valuable TestSuite and allows to implement a setUp method on the suite level.
The setUp method is called by the test runner (not by the builder), so it is safe to throw a SkippedTestError exception.
The corresponding method to do just that within a test suite is called markTestSuiteSkipped (notice the Suite in the method name).
The entire class would look like this:
class AllTests extends \PHPUnit_Framework_TestSuite
{
protected function setUp()
{
$this->markTestSuiteSkipped("zzz");
}
public static function suite()
{
$Suite = new self(__NAMESPACE__);
$Suite->addTestSuite(translators\AllTests::cls());
$Suite->addTestSuite(TlScopeTest::cls());
$Suite->addTestSuite(TlNsTest::cls());
$Suite->addTestSuite(TlElementTest::cls());
$Suite->addTestSuite(TlItemTest::cls());
$Suite->addTestSuite(LangCodeTest::cls());
$Suite->addTestSuite(TlElemClassTagTest::cls());
return $Suite;
}
}
The output is a pretty block of S letters, which definetly indicate, that we skipped a lot of tests. This cannot escape our attention and yet allows our tests to pass.
You could mark test as skipped.
class MyClass {
private $numeric;
public function MyMethod($numeric)
{
if (! is_numeric($numeric)) {
throw new InvalidArgumentException
}
$this->numeric = $numeric;
}
}
1 - It is necessary to test whether the class exists?
The PHPUnit has several methods that run automatically, such as assertPreConditions, setUp, and others. It is necessary within these methods check whether the class exists using the assertTrue and class_exists? Example:
protected function assertPreConditions()
{
$this->assertTrue(class_exists("MyClass"), "The class does not exists.");
}
2 - It is necessary to check if a method exist? If yes, this test should
be a separate test or within each unit test?
Suppose we have a method that accepts only numeric type parameters, so we have two tests, one test with the correct parameter and another with incorrect method expecting an exception, right? The correct way to write this method would be ...
This way:
public function testIfMyMethodExists()
{
$this->assertTrue(method_exists($MyInstance, "MyMethod"), "The method does not exists.");
}
/**
* #depends testIfMyMethodExists
* #expectedException InvalidArgumentExcepiton
*/
public function testMyMethodWithAValidArgument()
{
//[...]
}
/**
* #depends testIfMyMethodExists
* #expectedException InvalidArgumentExcepiton
*/
public function testMyMethodWithAnInvalidArgument()
{
//[...]
}
Or this way?
public function testMyMethodWithAValidArgument()
{
$this->assertTrue(method_exists($MyInstance, "MyMethod"), "The method does not exists.");
}
/**
* #expectedException InvalidArgumentExcepiton
*/
public function testMyMethodWithAnInvalidArgument()
{
$this->assertTrue(method_exists($MyInstance, "MyMethod"), "The method does not exists.");
//[...]
}
And why?
3 - What is the real purpose of the #covers and the #coversNothing?
I was reading a document that Sebastian Bergmann the creator of PHPUnit wrote as good practice we should always write #covers and #coversNothing in the methods and class and add these options in the xml:
mapTestClassNameToCoveredClassName="true"
forceCoversAnnotation="true"
and in the whitelist:
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php"></directory>
</whitelist>
But what is the real need for it?
4 - What is the correct way to test a constructor that calls another method?
It seems everything is fine, but not on tests.
Even if I do tests with valid arguments and invalid arguments expecting an exception on the method "MyMethod", that will not happen if I enter an incorrect value in the constructor (The test fails).
And if I test with valid argument, the code coverage does not result in 100%.
public function __construct($numeric)
{
$this->MyMethod($numeric);
}
I write tests for methods that should exist, to test that the code does what it is expected of it. I also test the InstanceOf() the class (and inherited class definitions) to ensure the object did create what was suppose to be created. If FOO() extends BAR(), then I test that my object created is an InstanceOf(FOO) and InstanceOf(BAR). If the class gets changed to inherit from something else, or has the extends removed, my test will once again inform the developer to check the code to ensure this change is desired. Potentially some inherited functions are being called on FOO, and without the extends from BAR, this code will break.
I write tests on the various code paths that are there to be executed. Therefore, if I expect that the function should throw an exception when bad data is passed, I write a test for that. I do this to also help document our source code. The tests show the expected behavior. If someone removes the exception (new functionality to accept a different parameter type) then the test should be updated to show that this is allowed. Potentially, the change to the parameters could cause a problem elsewhere. Testing this way ensures that years later, I know that code required this to be a number, and I should definitely re-factor the code carefully if I am changing the parameter types.
Using Test Driven Development (TDD) may cause you to not write the code to throw the exception, as you write a test, then the code to make the test pass. As such, you might not be testing all the parameters and their types or values, but I try to do this as best as I can to validate reasonable data input to try to avoid the Garbage In/Garbage Out (GIGO) problem.
All these tests also give me a good code coverage metric, as the majority of the code base is tested, and the code does step through all the lines in the class files. However, testing to this level, and trying to achieve a high code coverage metric, is really a choice for your team to make if this is desired or not.
I can't see any reason nor advantage of writing this kind of tests. If the class or the method doesn't exist your tests will fail anyway. So you don't have any profit from writing them.
Maybe the one exception from above (point 1) could be a situation where you always build SUT using PHPUnit mock framework (theoretically a situation where your SUT is a mock with mocked other method you don't need in particular test is possible, but I can't imagine real situation which leads to it).
In my opinion if the test covers flow path which is completely included in another test - this means that the test is redundant and unnecessary.
edit:
ad 3. Because you want to know which exactly flow path was invoked by particular unit test. Let's say you have two methods A and B. Method B invokes method A. If you don't have #covers annotation test for method B could generate code coverage for method A, and you can't say if your unit tests for A covers code in 100%.
I was reading my project's code coverage report, and I noticed something strange: a line was uncovered, but I was sure that line got executed during the tests. So, I added a var_dump() before it and this is what I got when executing the tests:
bool(true)
PHPUnit 3.5.5 by Sebastian Bergmann.
...
This is weird. How is it possible that a line is executed before PHPUnit's initialization? I believe this is the reason why code coverage says that line is uncovered.
Any hints?
EDIT: Here's some code. It's an IRC framework that makes use of the Doctrine Common library to read annotations and also uses the ClassLoader and EventDispatcher Symfony components. This is the incriminated method:
/**
* Returns this module's reflection.
*
* #return \ReflectionClass
* #access public
*/
static public function getReflection()
{
// The var_dump() displaying bool(false) is executed before PHPUnit, while the other
// ones are correctly executed.
var_dump(is_null(self::$reflection));
if (null === self::$reflection) {
// This line is reported as uncovered, but it must be executed since I'm
// accessing the reflection!
self::$reflection = new \ReflectionClass(get_called_class());
}
return self::$reflection;
}
What do you think?
Then, why are all the others var_dump() (that method gets executed many times in the application) shown after PHPUnit's output? And why isn't that line reported in code coverage even if it's executed?
I assume (but that's just a guess as it's hard to say since you have not shown the code), that it's related to code that gets executed on file-inclusion, rather after actual test functions are executed or testcases get instantiated.
The accessor must be getting called outside of a unit test which initializes self::$reflection. After that, all further calls to getReflection() skip the if block so it'll never be counted as covered. PHPUnit instantiates all of the test case classes--one per test method, data provider method, and data provider argument array--before running any of the tests or tracing code coverage. Look for a test case that calls getReflection() from its constructor or outside the class itself where the code is executed upon loading.
I forget if the test cases are instantiated before PHPUnit outputs its version and cannot check now, but I believe this is the case. The other possibility is that you're calling getReflection() from bootstrap.php, but you probably already checked for that.