I have a large number of tests in a PHP Unit test suite.
All tests will pass when running the complete suite, however when run individually, certain tests may pass or fail depending on how I call PHP Unit:
$ php phpunit --configuration phpunit.xml --filter FooIntegrationTest
PHPUnit 5.7.19 by Sebastian Bergmann and contributors.
................ 16 / 16 (100%)
Time: 3.97 seconds, Memory: 109.50MB
OK (16 tests, 36 assertions)
vs.
$ php phpunit --configuration phpunit.xml tests/Integration/FooIntegrationTest.php
PHPUnit 5.7.21 by Sebastian Bergmann and contributors.
..........F.F... 16 / 16 (100%)
Time: 3.73 seconds, Memory: 111.75MB
FAILURES!
Tests: 16, Assertions: 36, Failures: 2.
The failures in question are unexpected results (i.e. the code runs just fine), no PHP errors or Exceptions.
The majority of tests will pass both ways, and there doesn't seem to be anything special about those tests that produce the above results.
Solved.... namespace conflict from an unrelated test class
phpunit ... FooIntegrationTest.php only opens & runs the single class file specfied
whereas phpunit ... --filter FooIntegrationTest will open every file in the suite to look for matching tests; which lead to a globally namespaced function in another file causing false pass.
Lesson learned: don't declare globally namespaced functions in unit test classes!
Related
Unable to run tests for the specific file, cause PhpStorm adds namespace to the final command
Testing started at 14:36 ...
[sftp://inf#127.0.0.1:22]:/usr/bin/php /mnt/d/MP/mp.kz/phpunit.phar --bootstrap /mnt/d/MP/mp.kz/vendor/autoload.php --configuration /mnt/d/MP/mp.kz/phpunit.xml MP\\Tests\\Auction\\AuctionsListPublicSearchServiceTest /mnt/d/MP/mp.kz/tests/Auction/AuctionsListPublicSearchServiceTest.php --teamcity --cache-result-file=/mnt/d/MP/mp.kz/.phpunit.result.cache
PHPUnit 9.0.1 by Sebastian Bergmann and contributors.
Cannot open file "MP\\Tests\\Auction\\AuctionsListPublicSearchServiceTest".
Process finished with exit code 1
PhpStorm adds namespace to the command line.
If I remove namespace from command line then it works OK:
[SANDBOX]inf#NM0-MP:~$ /usr/bin/php /mnt/d/MP/mp.kz/phpunit.phar --bootstrap /mnt/d/MP/mp.kz/vendor/autoload.php --no-configuration /mnt/d/MP/mp.kz/tests/Auction/AuctionsListPublicSearchServiceTest.php
PHPUnit 9.0.1 by Sebastian Bergmann and contributors.
. 1 / 1 (100%)
Time: 1.48 seconds, Memory: 18.00 MB
OK (1 test, 3 assertions)
[SANDBOX]inf#NM0-MP:~$
The fix is going to be available tomorrow in 2020.1 EAP: https://youtrack.jetbrains.com/issue/WI-50201
https://www.jetbrains.com/phpstorm/nextversion/
I run my PHP unit tests using PHPUnit. When a dataprovider fails to load, it results in a warning, instead of a failure.
Up until PHPUnit version 4, when a dataprovider fails to load, a failure is shown in the PHPUnit output.
Starting with PHPUnit 5, a warning is being issued instead. This is a problem for when running the tests using a script (for example in continuous integration) so I don't see the results.
Another difference is when running the code with PHP 5 vs PHP 7. When running the tests with PHP 5, instead of seeing the PHPUnit output I immediately get a PHP Fatal Error. With PHP 7 it only shows the failure/warning when PHPUnit gets to the failed test. This leads me to believe that this has something to do with the error_handler set by PHPUnit, that can catch the error that PHP7 throws but PHP5 doesn't.
Here is my PHP code:
class TestTest extends PHPUnit_Extensions_Database_TestCase
{
/**
* #dataProvider provider_sample
*/
public function test_sample($foo)
{
$this->assertString($foo);
}
public function provider_sample()
{
return [
[ClassName::string] // no such class, this should fail
];
}
public function getDataset()
{
return new ArrayDataSet([]);
}
}
Here are the results of running PHPUnit 4:
PHPUnit 4.8.36 by Sebastian Bergmann and contributors.
Runtime: PHP 7.2.5
Configuration: /home/some-path/phpunit.xml
Warning: The Xdebug extension is not loaded
No code coverage will be generated.
F
Time: 120 ms, Memory: 14.00MB
There was 1 failure:
1) Warning
The data provider specified for TestTest::test_sample is invalid.
Class 'ClassName' not found
FAILURES!
Tests: 1, Assertions: 0, Failures: 1.
And here is the results of running the same code with PHPUnit 5:
PHPUnit 5.7.21 by Sebastian Bergmann and contributors.
Runtime: PHP 7.2.5
Configuration: /home/some-path/phpunit.xml
Error: No code coverage driver is available
W 1 / 1 (100%)
Time: 99 ms, Memory: 14.00MB
There was 1 warning:
1) Warning
The data provider specified for TestTest::test_sample is invalid.
Class 'ClassName' not found
WARNINGS!
Tests: 1, Assertions: 0, Warnings: 1.
The results of PHPUnit 4 is what I expect and want.
Is there a way to configure PHPUnit 5 and higher to behave the same?
I don't think you can do exactly what you're asking for, but if you want to integrate PHPUnit with a continuous integration server (for example, Jenkins), then you can spawn a JUnit-compatible report with warnings reported as errors instead and tell Jenkins to use that instead of PHPUnit report.
To do that, you will need to extend JUnit logger shipped with PHPUnit:
use PHPUnit\Util\Log\JUnit;
class MyJUnit extends Junit {
public function addWarning(Test $test, Warning $e, float $time): void {
$this->addError($test, $e, $time);
}
}
And register it as a listener in phpunit.xml:
<listeners>
<listener class="MyJUnit" file="MyJUnit.php">
<arguments>
<!--
Print the output to the file instead of stdout
which is the default.
-->
<string>junit.xml</string>
</arguments>
</listener>
</listeners>
Finally, go to Jenkins settings and add a JUnit job specifying the location of output file.
check the file name and import the
require_once 'ClassName.php';
I am trying to mock a class for phpunit. Php unit fails with the error Could not load mock ... class already exists. This is the only test I'm running, so it can't be the case that the class is mocked already.
Any suggestion would be appreciated.
Here is the error case:
namespace Tests\Feature;
use Tests\TestCase;
class DeactivateACSTest extends TestCase
{
public function testDeactivateAcs()
{
$deviceController = \Mockery::mock('overload:App\Http\Controllers\Cloud\DeviceController');
$deviceController
->shouldReceive('deactivateACS')
->andReturn('hilfehilfehilfe');
$devCon = new \App\Http\Controllers\Cloud\DeviceController();
$this->assertEquals('hilfehilfehilfe', $devCon->deactivateACS());
}
}
When running it without --code-coverage it works:
[13:10:15] vagrant#homestead [~/Code/ekp] $ phpunit --filter DeactivateACS
PHPUnit 6.5.10 by Sebastian Bergmann and contributors.
==> Tests\Feature\DeactivateACSTest ✓
Time: 1.08 seconds, Memory: 16.00MB
OK (1 test, 3 assertions)
However, when running it with --code-coverage it fails:
[13:10:23] vagrant#homestead [~/Code/ekp] $ phpunit --coverage-html coverage --coverage-text=code_coverage.txt --filter DeactivateACSTest
PHPUnit 6.5.10 by Sebastian Bergmann and contributors.
==> Tests\Feature\DeactivateACSTest ⚈
Time: 5.79 seconds, Memory: 44.00MB
There was 1 error:
1) Tests\Feature\DeactivateACSTest::testDeactivateAcs
Mockery\Exception\RuntimeException: Could not load mock \App\Http\Controllers\Cloud\DeviceController, class already exists
/home/vagrant/Code/ekp/vendor/mockery/mockery/library/Mockery/Container.php:220
/home/vagrant/Code/ekp/vendor/mockery/mockery/library/Mockery.php:116
/home/vagrant/Code/ekp/tests/Feature/DeactivateACSTest.php:11
ERRORS!
Tests: 1, Assertions: 0, Errors: 1.
Generating code coverage report in HTML format ... done
You should add these annotations before the functions that are mocking this class.
/**
* #runInSeparateProcess
* #preserveGlobalState disabled
*/
For reference you can check out the phpunit documentation.
https://phpunit.de/manual/current/en/appendixes.annotations.html#appendixes.annotations.runInSeparateProcess
https://phpunit.de/manual/current/en/appendixes.annotations.html#appendixes.annotations.preserveGlobalState
I ran into the same issue and fixed like this:
There was another test in my unit tests (not mockery test) which had require_once on the PHP file that had the class I was mocking. I've removed that line.
I've added processIsolation="true" in test suite
I am using the cakephp test (php unit test) with xdebug to run the server side code and monitor the output from the terminal rather than using it for validation.
So, every time, i want to run some particular lib/controller/model method from the terminal and see the output, i have to comment out the other test case functions.
I know this might not be the right approach,
but i want to know, if there is a way, i can build up a wrapper around the cake test, that will take the argument of the method name which i want to run?
There is no need to write a wrapper script
To run one test method
Use the filter option:
-> phpunit --help
PHPUnit 4.4.1 by Sebastian Bergmann.
Usage: phpunit [options] UnitTest [UnitTest.php]
phpunit [options] <directory>
...
Test Selection Options:
--filter <pattern> Filter which tests to run.
For example:
-> phpunit --debug --filter testValidationDefault tests/TestCase/Model/Table/PostsTableTest.php
PHPUnit 4.4.1 by Sebastian Bergmann.
Configuration read from /var/www/cakephp.dev/phpunit.xml.dist
Starting test 'App\Test\TestCase\Model\Table\PostsTableTest::testValidationDefault'.
I
Time: 130 ms, Memory: 9.75Mb
OK, but incomplete, skipped, or risky tests!
Tests: 1, Assertions: 0, Incomplete: 1.
www-data # dev [ /var/www/cakephp.dev ]
->
To run only one test use the filter option in the cake command line tool:
cake test app Model/Post --filter testGetAllPostsByAuthor
Works on CakePhp 2.2
I have created a laravel test file using the command
php artisan make:test EventUserRelations
When I run phpunit, it is as if the file does not even exist. I tried to fail the test just to make sure it is being tested. Here is the code in the file:
class EventUserRelations extends TestCase
{
public function testExample()
{
$this->fail();
}
}
And here is the result.
phpunit --debug
PHPUnit 5.4.6 by Sebastian Bergmann and contributors.
Starting test 'ExampleTest::testBasicExample'.
. 1 / 1 (100%)
Time: 119 ms, Memory: 20.75MB
In Laravel test files need to end with ..test.php. So change EventUserRelations.php to EventUserRelationsTest.php and it should work.