PHPUnit code coverage doesn't include certain lines - php

I'm having a strange issue with PHPUnit. I'm working on a project with a team of developers, all of which must use PHPUnit 3.6.5. For the most part, my PHPUnit works great and yields the same results as on other developers' machines. However, I will occasionally find that a certain line isn't covered on my machine, but is on other developers' machines. Here is a recent scenario:
Bob is checking in a new file that he just updated. He made sure the unit test passed for this file and had 95% code coverage. After performing this check, he checks in the file, where it is built and the unit test is run. If the unit test fails on the server, our build goes red and nobody can check in.
The server runs the phpunit test and it passes.
Other developers, including myself, get the latest from the perforce repository
After making my own set of changes, I run all of the phpunit tests for the application, but I notice one of the files does not have enough code coverage. It's the file that Bob just checked in. I check with some of the other developers on my team and the file passes code coverage without any problems for them. When I check the code coverage report, I see the following:
array_walk(
$variable1,
function($val,$key) use(&$variableData, &$variable2)
{
$variableData[$variable2][$key] = 1;
}
);
$variable1,
function($val,$key) use(&$variableData, &$variable2)
These two lines are not covered! I don't understand why my phpunit doesn't want to cover these lines. There must be a setting that I am not aware of that is causing this as I have the same version of PHPUnit as other developers and it passes fine on their machine but not on mine. I'm also seeing the same problem in a statement like this:
if ($var === 1) {
echo 'yes';
} else {
echo 'nope';
}
Now, the unit test will cover both conditions, however, it doesn't cover the following line:
} else {
Does this make any sense at all? The strangest part about it is that it works most of the time, but there are a few locations that have this behavior, though there is no pattern to indicate what it causing it. I've had other developers look at this issue on my machine and they are baffled and cannot explain what is causing it. The problem is, since it is failing for me locally, even though I haven't touched Bob's file, it is preventing me from checking in my changes. Has anyone ran into this problem before? I am going to try uninstalling php unit and re-installing it, but other than that, I am at a loss for words. Any help would be greatly appreciated! Here are my specs:
PHPUnit: Version 3.6.5
Operating system: OS X 10.6.8
PHP: PHP 5.3.8-ZS5.5.0 (cli) (built: Aug 24 2011 11:03:29)
Zend Debugger: Turned off
XDebug: Turned on
Zend Optimizer+: off
Regards,
Will

Executable and executed lines are reported by xdebug to PHPUnit. Different versions of xdebug may vary in the lines reported as covered.
If you feel that they are incorrect, provide a simple testcase and open a bug at xdebug's issue tracker.

Related

PhpStorm Xdebug Exiting Method Prematurely

I am working on an Ubuntu 18.04 system with PhpStorm version #PS-222.3345.135. I am on PHP 7.4.20 and Xdebug 2.9.8.
First off, when I am not running Xdebug on my web requests they work perfectly fine. When I attempt to step through execution with Xdebug, methods are exiting prematurely as I will show in the screenshots below:
First screenshot:
Execution is at the fist line of the launch() method. As such, pressing the Step Over button should bring execution to line 112 ($this->_state->setAreaCode($areaCode)); Instead, pressing Step Over prematurely exits the launch() method and brings the execution thread back out to the Bootstrap::run() method as shown below:
Note that there is no $response variable in the run() local scope as the launch() method was not allowed to return a value due to being exited prematurely. Pressing Step Over again will result in an exception being thrown by the $response->sendResponse(), as shown in the screenshot below:
To summarize, the issue here is that Xdebug is not stepping through code correctly. This issue didn't start occurring until this past week, and my path mappings for the project are the same as they were before.
This is happening for multiple PhpStorm projects, not just this one codebase. However I do have one PhpStorm project (with a much smaller codebase) in which stepping through code works fine (I upgraded the PhpStorm memory limit to 2GB as a result but that did not resolve anything).
Does anyone know why this would be happening?
It's PhpStorm 2022.2 issue (WI-67891 ticket).
https://youtrack.jetbrains.com/issue/WI-67891/Step-Over-broken-for-Xdebug-30-for-static-methods-and-private-properties-from-parent-classes#focus=Comments-27-6367485.0-0
Reason of the bug is found and fixed. Fix will be available in next minor update. Below is a technical summary of the issue for ones who curious about what happened:
Xdebug had a bug which is basically breaks further step_over commands by passing invalid eval commands. This bug is fixed in 3.1, but present in all older versions.
In 2022.2 We've implemented feature WI-56517 that shows __toString representation of the object in debug panel. This feature is implemented by passing eval((string)expression) command on start of debug sessions
It turned out that there are number of invalid inputs that happened to be produced by this command:
Private properties from parent class. Xdebug returns such properties in the format that is not suitable for plain evaluation https://github.com/xdebug/xdebug/blob/5af18430eb0b2c9574e903e6a9b9ef40c0ea7296/src/lib/var_export_xml.c#L441
:: instead of $this in case of static method (issue WI-67023)
Combination of these factors resulted into broken step_over. Both cases are fixed and will be available in next minor update. Sorry for the inconvenience and thank you for reporting the issue.
The aforementioned WI-67891 ticket is marked as fixed for 2022.2.1. You can try 2022.2.1 Preview now.
At the same time there are some reports that there are still some scenarios (not-yet-covered cases) where the debug still does not work as in previous version. Sadly there are more rather useless "me too" comments than "here is a code sample where it is still does not work" (as it requires time to make it, to see what may be causing it, be able to spot the difference in setup/environment etc).
Current solutions:
Upgrade to Xdebug 3.1 or newer if you can (the latest version right now is 3.1.5).
Disable Settings/Preferences | PHP | Debug | Evaluation | Enable '__toString' object view until confirmed as properly resolved. Do not forget to revert it back (this feature seems convenient for a quick-glance when debugging).
Stay on previous stable 2022.1.4 until the issue is confirmed as properly resolved.

How can i improve Codeception Code Coverage speed

Currently we have written some unit test for our php laravel 5.5 application using codeception. just for additional info, our laravel code base have around 200k LOC. For normal unit test run, we noticed that it is really fast in which we can get more than 200 tests to finish within 1 hour.
But the main issue is that when we enabled codecoverage in codeception which is using xdebug by default, we noticed the execution time increased drastically.
Now it already took 1 week but the whole codecoverage execution not even finished yet.
I am not sure whether this is the problem from codeception or xdebug itself but if anybody have experiences running php codecoverage on a huge codebase, it would be nice if you can share how you achieve it. Would appreciate it also if somebody can suggest any other tools to look into. Currently we are considering switching to phpunit but are still open to other tools to explore.
Replacing Codeception with PHPUnit will be a lot of work for little gain, because Codeception uses PHPUnit and its PHP-Code-Coverage library under the hood.
There is a new code coverage extension, called pcov which is supposedly much faster than xdebug.
https://github.com/krakjoe/pcov/blob/develop/INSTALL.md
I haven't tried to use it, but be aware that it requires PHPUnit 8, which is only available on PHP 7.2 or later versions.
Recently I have seen code coverage sped up by replacing xdebug with phpdbg - I can't give exact numbers as the code base has extensive functional tests in its test run (and the speed-up was only for unit tests), but a 2+ hour test and coverage run has been reduced to around 50 minutes.
Note that xdebug and phpdbg can differ in their code coverage (it looked like xdebug better dealt with opcache optimisations).
edit:
Since replacing xdebug with phpdbg, I have seen further speed improvements by replacing phpdbg with pcov.

XDebug and PHPUnit getting Code Coverage Wrong

I am using php 5.5.9 on Ubuntu 14.04. I have upgraded Xdebug to the version that its Wizard recommends, that is version 2.2.6, and I have used apt-get to make sure my Ubuntu system is up to date. The problem is, however, that my code coverage reports lines as executed when they are clearly not.
Here are the offending lines:
if(!$filter->isValid()) {
throw new Exception\InvalidInput($this->formatFilterErrors($filter));
}
I know that the conditional code inside the if statement is not being executed, since the filtered data is always valid. I can prove this by putting echo or die statements in the code block (above the throw statement) and they are never executed.
So why is code coverage telling me that the throw statement is being executed, numerous times?
Although I only have one example of incorrect reporting I really need to sort it out as a priority since I have a lot of testing to do on this application. I could try upgraded PHP (but that has other consequences, of course). Apart from that, my only option is to raise a but with the developer.
Have you come across this bug? I have heard of others having problems with Xdebug on Ubuntu 14.04, but upgrading Xdebug has sorted their problems out.

Phpunit behaves differently on different machines

I wrote a few PHPUnit tests for my PHP application. I am working on two development machines that synchronize the entire development filesystem, and there is an automatic testing system that runs the tests on a clean svn checkout.
Now, something weird is going on. One of the development machines runs Ubuntu linux 10.10, PHP 5.3.3, phpunit 3.4.6 which I later upgraded to 3.4.14. When I run phpunit on each single test, all tests pass. When I run phpunit on my xml configuration file that just runs them all, using phpunit --configuration=phpunit.cfg, the last test case fails, and a Class myClass not found error is returned. This class is inside a file that is loaded using require_once. Upgrading phpunit made no difference.
edit: I have the feeling that the list of files PHP includes using require_once is not reset on my linux box, but the classes are actually lost, that is, it thinks it has
already included myclass.php while the class inside is not loaded.
Now, the other machine is running Mac OS X Snow Leopard 10.6.5, PHP 5.3.3 and phpunit 3.4.3. If I run the unit tests there individually, no errors occur, just as on the linux box, but when I run the tests using the xml configuration, no errors occur either. We also have an auto testing environment running linux 10.4 LTS, PHP 5.2.6 and phpunit 3.4.9. Also no errors.
All systems use exactly the same files, so there is no difference in the source code or tests. Also the phpunit.xml is the same.
I feel completely in the dark here. How can I figure out what the problem may be?
Both the linux development machine and the macbook are running PHP 5.3.3. The autotest machine is runing 5.2.6
Two things to watch out for:
The include path in the PHP config on both machines
That there isn't another copy of the class file sitting around in your homedir or the cwd of the script. Usually . will be in the include path for PHP and rogue copies of files can cause problems like this
Is it possible that it isn't PHPUnit that is acting up, but a local php configuration issue causing problems in your system under test? Diff your php.ini on the 2 machines and you will likely find some differences.

Will PHPUnit and XDebug work together?

I've been working on writing unit tests for my PHP code. PHPUnit is what I'm using for this.
I have some classes that work great until...
I throw XDebug into the mix.
At that point, PHPUnit doesn't crash or anything, but setExpectedException never triggers.
In fact, the code never goes beyond that point.
Anyone run across this and know what the solution is?
Edit: This is with the PHP5.2.6, Latest versions of XUnit and XDebug. This is on linux fedora 7
I've had some problems before with one combination of Xdebug & PHPUnit (as did others, from a report I say on devzone.zend.com at the time), but that was a while ago. PHPUnit does explicitly support Xdebug though - and I'm using it to produce code coverage reports for my own systems.
I'd suggest the fairly standard debugging techniques, updating any versions you can (maybe even PHP?) commenting out the #setExpectedException, or throwing it explicitly - and of course, making sure that you have full warnings and errors being shown with error_reporting(E_ALL|E_STRICT); set on.
Don't forget to stop and restart the Apache server to make sure that the Xdebug module is loaded as well.

Categories