Currently we have written some unit test for our php laravel 5.5 application using codeception. just for additional info, our laravel code base have around 200k LOC. For normal unit test run, we noticed that it is really fast in which we can get more than 200 tests to finish within 1 hour.
But the main issue is that when we enabled codecoverage in codeception which is using xdebug by default, we noticed the execution time increased drastically.
Now it already took 1 week but the whole codecoverage execution not even finished yet.
I am not sure whether this is the problem from codeception or xdebug itself but if anybody have experiences running php codecoverage on a huge codebase, it would be nice if you can share how you achieve it. Would appreciate it also if somebody can suggest any other tools to look into. Currently we are considering switching to phpunit but are still open to other tools to explore.
Replacing Codeception with PHPUnit will be a lot of work for little gain, because Codeception uses PHPUnit and its PHP-Code-Coverage library under the hood.
There is a new code coverage extension, called pcov which is supposedly much faster than xdebug.
https://github.com/krakjoe/pcov/blob/develop/INSTALL.md
I haven't tried to use it, but be aware that it requires PHPUnit 8, which is only available on PHP 7.2 or later versions.
Recently I have seen code coverage sped up by replacing xdebug with phpdbg - I can't give exact numbers as the code base has extensive functional tests in its test run (and the speed-up was only for unit tests), but a 2+ hour test and coverage run has been reduced to around 50 minutes.
Note that xdebug and phpdbg can differ in their code coverage (it looked like xdebug better dealt with opcache optimisations).
edit:
Since replacing xdebug with phpdbg, I have seen further speed improvements by replacing phpdbg with pcov.
Related
I've tried opcache but it even makes it lower, in some cli programs there is a watch option, the cli keeps running that on changes it will redo something, I couldn't find that for phpunit.
Just to be clear, the tests are pretty fast, what's slow is the bootstrap because the codebase is rather big, the code base runs fine on a webserver thanks to opcache, but for phpunit opcache (which only should work with file cache I'd suppose?) seems to be hopeless...
I'm using PHP 8 and PHPUnit 9
I'm running test on PHPUnit using Xdebug for generating coverage and it is very slow.
I tried using PHPdbg but that leads to memory errors.
I was told that I can create a filter file, and that should help. Can anyone explain to me how that would work.
The documentation for PHPUnit has information on speeding up code coverage with Xdebug. For more background information I recommend this article.
That being said, I recommend having a look at PCOV for even faster code coverage data collection for PHP and PHPUnit.
I want to allow my testers to use the development website "as usual" and collect code coverages of every "run", combined everything and be able to say "after 4 hours of tests, here are the 75% of the code that were executed".
I use the php-code-coverage library (https://github.com/sebastianbergmann/php-code-coverage) and everything is working fine except that with xdebug as the tool used for code coverage it's way too slow (10 times slower that without activating php-code-coverage).
I've compiled my own version of php 7.2 with "--enable-phpdbg" and with the help of the command "update-alternatives" I'm able to run in cli :
$ php index.php
and get the code coverage I need, and it's only two times slower (every call to "php" is calling "phpdbg").
But I can't find a way to make it work with apache so that when I'm loading my website it's the executable "phpdbg" and not "php" that is used.
Even if I'm compiling my own ".so" it will still be "php" that will be executed.
I'm currently using Zend Framework in conjunction with PHPUnit to conduct unit testing on an application. When Hudson executes the PHPUnit shell command, the maximum PHP memory limit is reached sometime during code coverage generation. I currently have a total of 41 tests with 334 assertions.
I have successfully eliminated this error by raising the memory_limit setting to 768M using the -d memory_limit=768M switch; however, I am worried that as the complexity increases along with the total number of tests/assertions, I will not have enough memory to generate the HTML for code coverage statistics.
OS: CentOS 5.5
Control Panel: WHM/cPanel
CI Server: Hudson
/usr/local/bin/phpunit
--verbose
-d memory_limit=512M
--log-junit ../../build/logs/phpunit.xml
--coverage-clover ../../build/logs/coverage/clover.xml
--coverage-html ../../build/logs/coverage-html/
Fatal error: Allowed memory size of
536870912 bytes exhausted
Before committing my changes and letting Hudson handle the rest, I use Windows 7 for development. The memory usage never exceeded 340MB while running the same command within W7.
By reducing the number of files included within code coverage, as well as increasing the overall memory limit in PHP, I was able to basically kill this error. The entire Zend Framework was being included within code coverage, which is very large.
Do you have xdebug profiling enabled, if so try disabling it. I've experienced this problem before, and it came down to extensions in php (specifically xdebug profiling and/or Inclued heirarchy viewer)
As of 2019, you can use the PCOV driver with PHPUnit to generate your code coverage report. In my experience, it's only marginally less performant than running a plain PHPUnit suite.
Read Speed up PHPUnit Code Coverage Analysis for some good benchmark comparing XDebug, PHPDebug and PCOV. It also has instructions on how to enable PCOV on PHPUnit 8.
Read Setup PHP PCOV for 5 times faster PHPUnit code coverage for instructions on setting up PCOV on PHPUnit7 and below.
I've been working on writing unit tests for my PHP code. PHPUnit is what I'm using for this.
I have some classes that work great until...
I throw XDebug into the mix.
At that point, PHPUnit doesn't crash or anything, but setExpectedException never triggers.
In fact, the code never goes beyond that point.
Anyone run across this and know what the solution is?
Edit: This is with the PHP5.2.6, Latest versions of XUnit and XDebug. This is on linux fedora 7
I've had some problems before with one combination of Xdebug & PHPUnit (as did others, from a report I say on devzone.zend.com at the time), but that was a while ago. PHPUnit does explicitly support Xdebug though - and I'm using it to produce code coverage reports for my own systems.
I'd suggest the fairly standard debugging techniques, updating any versions you can (maybe even PHP?) commenting out the #setExpectedException, or throwing it explicitly - and of course, making sure that you have full warnings and errors being shown with error_reporting(E_ALL|E_STRICT); set on.
Don't forget to stop and restart the Apache server to make sure that the Xdebug module is loaded as well.