To contextualise, in similar tools, PHPMD and PHPCS, one can specify a custom formatter for result output, eg:
PHPMD:
vendor/bin/phpmd test \\my\\namespace\\renderers\\phpmd\\AdamFormat phpmd.xml
PHPCS:
vendor/bin/phpcs --standard=phpcs.xml --report=./src/renderers/phpcs/AdamFormat.php
I'm looking to do the same thing for PHPUnit, but have thusfar drawn a blank (investigation, googling, searching here). Looking at the code of PHPUnit, it all seems a bit hard-codey to me:
Code coverage handler:
if (isset($arguments['coverageClover'])) {
$this->printer->write(
"\nGenerating code coverage report in Clover XML format ..."
);
try {
$writer = new CloverReport;
Logging:
if (isset($arguments['testdoxHTMLFile'])) {
$result->addListener(
new HtmlResultPrinter(
I have not spotted anywhere in the docs that suggest otherwise. Seems like an odd shortfall to me.
So two questions:
Am I reading things right? PHPUnit doesn't support this?
Presuming that's "yes: not supported", has anyone had any success with a tactic to circumventing this in a non Heath Robinson manner?
I realise that one can use the --coverage-php option to output the results as PHP variables for another process to then utilise to do [whatever], but that seems like an inside out approach to me, and falls into the Heath Robinson category.
Related
I'm using a framework which has method defined something like
class Abc {
public function doThis($what) {
...
}
}
Since I'm using PHP 7 and also fan of PHP codesniffer, it tells me to define function argument types, that said I have wrote class in my code:-
class Pqr extends Abc {
public function doThis(string $what) {
...
}
}
This code gives me warning Declaration of Pqr::doThis(string $what) should be compatible with Abc::doThis($what)
It seems PHP is treating $what in Abc class differently (not as string). Since Abc is part of framework and I cannot do anything about it. I do not want to remove argument types in my code and want to keep cngode more strict. Disabling all warnings would be bad idea.
Anything better we have to fix this issue ?
Code Sniffer may well be telling you to do something, and you may want to follow its advice, but if your framework isn't doing it then you may not be able to do it either. You can't dicatate the code rules to the framework; you have to live with what it imposes on you, even if that goes against Code Sniffer's rules.
My advice is to simply ignore this issue. Code Sniffer is a great tool, and its advice is worth following, but there are times when you simply can't do so.
If your goal is to get your system to show zero Code Sniffer warnings, then you can do so by explicitly adding markers to your code telling Code Sniffer to ignore specific rules at various points in your code. Code Sniffer has the ability to ignore sections of code; this is described in it's Advanced Usage documentation page.
I would like to generate code coverage from functional testing in Silex App via PHPUnit. I created sandbox where you could reproduce.
The question is: Why Controller::indexAction() method is marked as Not Executed code in code coverage report?
Thank you!
No time for testing.
What i have seen:
You are setting your test array for the first test in app.php
return new \Symfony\Component\HttpFoundation\JsonResponse(['foo' => 'bar']);
Why? And did the test fail if you remove that? Maybe here the Controller isn't tested.
Then you are testing the 2 methods not in the same way.
Maybe that leads to the solution of the problem.
When I'm trying to execute a functional unittest of a module within my Yii code, I keep receiving the following error:
CException: CHttpRequest is unable to determine the request URI.
At first, I though it was because it couldn't find the module. However, If I change the url to a wrong one, I get a correct error,s tating it couldn't find the view.
This is how my testing code looks like
public function testViewControllerModule()
{
ob_start();
Yii::app()->runController('module/controller/view');
}
Any ideas on what I might be missing?
bool.devs answer works so far.
This blog post explains the origin of the exception pretty well:
http://mattmccormick.ca/2012/09/14/unit-testing-url-routes-in-yii-framework/
In my case, I generalized the solution and have set the following variables in /www/protected/tests/bootstrap.php:
...
$_SERVER['SCRIPT_FILENAME'] = 'index-test.php';
$_SERVER['SCRIPT_NAME'] = '/index-test.php';
$_SERVER['REQUEST_URI'] = 'index-test.php';
Yii::createWebApplication($config);
Consider using 'index-test.php' instead of 'index.php' because it contains the config 'test.php' which is responsible for fixtures and maybe other test relevated configurations.
If someone has better suggestions feel free to comment :)
Kind regards
I think it's because you haven't set any server variables, i.e $_SERVER and you might be doing something like this in your controller:
Yii::app()->request ....
So before you run your test, make sure you use a fixture for the server variables also. I think this should suffice for now:
$_SERVER=array(
'REQUEST_URI'=>'index.php', // the other fields should follow
);
However to run functional tests i would recommend using SeleniumRC, you won't have to do these workarounds then, and can simulate user clicks also, i think.
Read the initial guide to Functional Testing , read the selenium rc phpunit guide, and also the CWebTestCase documentation.
Notes: You might still have to use fixtures for some variables, and i don't have much experience in testing(which is bad), so i'm not very sure if i am completely correct about selenium.
I've been in the process of creating a test suite for a project, and while I realize getting 100% coverage isn't the metric one should strive to, there is a strange bit in the code coverage report to which I would like some clarification.
See screenshot:
Because the last line of the method being tested is a return, the final line (which is just a closing bracket) shows up as never being executed, and as a consequence the whole method is flagged as not executed in the overview. (Either that, or I'm not reading the report correctly.)
The complete method:
static public function &getDomain($domain = null) {
$domain = $domain ?: self::domain();
if (! array_key_exists($domain, self::$domains)) {
self::$domains[$domain] = new Config();
}
return self::$domains[$domain];
}
Is there a reason for this, or is it a glitch?
(Yes, I read through How to get 100% Code Coverage with PHPUnit, different case although similar.)
Edit:
Trudging on through the report, I noticed the same is true for a switch statement elsewhere in the code. So this behaviour is at least to some extent consistent, but baffling to me none the less.
Edit2:
I'm running on: PHPUnit 3.6.7, PHP 5.4.0RC5, XDebug 2.2.0-dev on a OS X
First off: 100% code coverage is a great metric to strive for. It's just not always achievable with a sane amount of effort and it's not always important to do so :)
The issue comes from xDebug telling PHPUnit that this line is executable but not covered.
For simple cases xDebug can tell that the line is NOT reachable so you get 100% code coverage there.
See the simple example below.
2nd Update
The issue is now fixed xDebug bugtracker so building a new version of xDebug will solve those issues :)
Update (see below for issues with php 5.3.x)
Since you are running PHP 5.4 and the DEV version of xDebug I've installed those and tested it. I run into the same issues as you with the same output you've commented on.
I'm not a 100% sure if the issue comes from php-code-coverage (the phpunit module) for xDebug. It might also be an issue with xDebug dev.
I've filed a bug with php-code-coverage and we'll figure out where the issue comes from.
For PHP 5.3.x issues:
For more complex cases this CAN fail.
For the code you showed all I can say is that "It works for me" (complex sample below).
Maybe update xDebug and PHPUnit Versions and try again.
I've seen it fail with current versions but it depends on how the whole class looks sometimes.
Removing ?: operators and other single-line multi-statement things might also help out.
There is ongoing refactoring in xDebug to avoid more of those cases as far as I'm aware. xDebug once wants to be able to provide "statement coverage" and that should fix a lot of those cases. For now there is not much one can do here
While //#codeCoverageIgnoreStart and //#codeCoverageIgnoreEnd will get this line "covered" it looks really ugly and is usually doing more bad than good.
For another case where this happens see the question and answers from:
what-to-do-when-project-coding-standards-conflicts-with-unit-test-code-coverage
Simple example:
<?php
class FooTest extends PHPUnit_Framework_TestCase {
public function testBar() {
$x = new Foo();
$this->assertSame(1, $x->bar());
}
}
<?php
class Foo {
public function bar() {
return 1;
}
}
produces:
phpunit --coverage-text mep.php
PHPUnit 3.6.7 by Sebastian Bergmann.
.
Time: 0 seconds, Memory: 3.50Mb
OK (1 test, 1 assertion)
Generating textual code coverage report, this may take a moment.
Code Coverage Report
2012-01-10 15:54:56
Summary:
Classes: 100.00% (2/2)
Methods: 100.00% (1/1)
Lines: 100.00% (1/1)
Foo
Methods: 100.00% ( 1/ 1) Lines: 100.00% ( 1/ 1)
Complex example:
<?php
require __DIR__ . '/foo.php';
class FooTest extends PHPUnit_Framework_TestCase {
public function testBar() {
$this->assertSame('b', Foo::getDomain('a'));
$this->assertInstanceOf('Config', Foo::getDomain('foo'));
}
}
<?php
class Foo {
static $domains = array('a' => 'b');
static public function &getDomain($domain = null) {
$domain = $domain ?: self::domain();
if (! array_key_exists($domain, self::$domains)) {
self::$domains[$domain] = new Config();
}
return self::$domains[$domain];
}
}
class Config {}
produces:
PHPUnit 3.6.7 by Sebastian Bergmann.
.
Time: 0 seconds, Memory: 3.50Mb
OK (1 test, 2 assertions)
Generating textual code coverage report, this may take a moment.
Code Coverage Report
2012-01-10 15:55:55
Summary:
Classes: 100.00% (2/2)
Methods: 100.00% (1/1)
Lines: 100.00% (5/5)
Foo
Methods: 100.00% ( 1/ 1) Lines: 100.00% ( 5/ 5)
Much of the problem here is the insistence on getting 100% execution coverage of "lines".
(Managers like this idea; it is a simple model they can understand). Many lines aren't "executable" (whitespace, gaps between function declarations, comments, declarations, "pure syntax" e.g., the closing "}" of a switch or class declaration, or complex statements split across multiple source lines).
What you really want to know is, "is all the executable code covered?" This distinction seems silly yet leads to a solution. XDebug tracks what gets executed, well, by line number and your XDebug-based scheme thus reports ranges of executed lines. And you get the troubles discussed in this thread, including the klunky solutions of having to annotate the code with "don't count me" comments, putting "}" on the same line as the last executable statement, etc. No programmer is really willing to do that let alone maintain it.
If one defines executable code as that code which which can be called or is controlled by a conditional (what the compiler people call "basic blocks"), and the coverage tracking is done that way, then the layout of the code and the silly cases simply disappear. A test coverage tool of this type collects what is called "branch coverage", and you can get or not get 100% "branch coverage" literally by executing all the executable code. In addition, it will pick up those funny cases where you have a conditional within a line (using "x?y:z") or in which you have two conventional statements in a line (e.g.,
if (...) { if (...) stmt1; else stmt2; stmt3 }
Since XDebug tracks by line, I beleive it treats this as one statment, and considers it coverage if control gets to the line, when in fact there are 5 parts to actually test.
Our PHP Test Coverage tool implements these ideas. In particular, it understands that code following a return statement isn't executable, and it will tell you that you haven't executed it, if it is non-empty. That makes the OP's original problem just vanish. No more playing games to get "real" coverage numbers.
As with all choices, sometimes there is a downside. Our tool has a code instrument component that only runs under Windows; instrumented PHP code can run anywhere and the processing/display is done by a platform independent Java program. So this might be awkward for OP's OSX system. The instrumenter works fine across NFS-capable file systems, so he could arguably run the instrumenter on a PC and instrument his OSX files.
This particular problem was raised by someone trying to push his coverage numbers up; the problem was IMHO artificial and can be cured by stepping around the artificiality. There's another way to push up your numbers without writing more tests, and that's finding and removing duplicate code. If you remove duplicates, there's less code to test and testing one (non)copy in effects tests the (now nonexistent other copy) so it is easier to get higher numbers. You can read more about this here.
With regards to your switch statement code coverage issue, simply add a "default" case which doesn't do anything and you'll get full coverage.
Here is what to do to get switch statement 100% coverage:
Ensure there is at least one test that sends a case that doesn't exist.
So, if you have:
switch ($name) {
case 'terry':
return 'blah';
case 'lucky':
return 'blahblah';
case 'gerard':
return 'blahblah';
}
ensure at least one of your tests sends a name that is neither terry nor lucky nor gerard.
I'm looking for a simpler test framework. I had a look at a few PHPUnit and SimpleTest scripts and I find the required syntactic sugar appalling. SnapTest sounded nice, but was as cumbersome. Apache More::Test was too procedural, even for my taste. And Symfony lime-test was ununique in that regard.
BDD tools like http://everzet.com/Behat/#basics are very nice, but even two abstraction levels higher than desired.
Moreover I've been using throwaway test scripts till now. And I'm wondering if instead of throwing them away, there is a testing framework/tool which simplifies using them for automated tests. Specifically I'd like to use something that:
evaluates output (print/echo), or even return values/objects
serializes and saves it away as probe/comparison data
allows to classify that comparison output as passed test or failure
also collects headers, warning or error messages (which might also be expected output)
in addition to a few $test->assert() or test::fail() states
Basically I'm too lazy to do the test frameworks work, manually pre-define or boolean evaluate and classify the expected output. Also I don't find it entertaining to needlessly wrap test methods into classes, plain include scripts or functions should suffice. Furthermore it shouldn't be difficult to autorun through the test scripts with a pre-initialized base and test environment.
The old .phpt scripts with their --expect-- output come close, but still require too much manual setup. Also I'd prefer a web GUI to run the tests. Is there a modern rehersal of such test scripts? (plus some header/error/result evalation and eventually unit test::assert methods)
Edit, I'll have to give an example. This is your typical PHPUnit test:
class Test_Something extends PHPUnit_Test_Case_Or_Whatever {
function tearUp() {
app::__construct(...);
}
function testMyFunctionForProperResults() {
$this->assertFalse(my_func(false));
$this->assertMatch(my_func("xyzABC"), "/Z.+c/");
$this->assertTrue(my_func(123) == 321);
}
}
Instead I'd like to use plain PHP with less intermingled test API:
function test_my_function_for_proper_results() {
assert::false(my_func(false));
print my_func("xyz_ABC");
return my_func(123);
}
Well, that's actually three tests wrapped in one. But just to highlight: the first version needs manual testing. What I want is sending/returning the test data to the test framework. It's the task of the framework to compare results, and not just spoon-feeded booleans. Or imagine I get a bloated array result or object chain, which I don't want to manually list in the test scripts.
For the record, I've now discovered Shinpuru.
http://arkanis.de/projects/shinpuru/
Which looks promising for real world test cases, and uses PHP5.3-style anonymous functions instead of introspection-class wrappers.
Have to say - it isn't obvious how your example of a simplified test case would be possible to implement. Unfortunately the convolutedness is - more or less - something that has to be lived with. That said, I've seen cases where PHPUnit is extended to simplify things, as well as adding web test runners, tests for headers, output etc (thinking SilverStripe here - they're doing a lot of what you want with PHPUnit). That might be your best bet. For example:
evaluates output (print/echo):
enable output buffering and assert against the buffer result
collect headers, warning or error messages
register your own handler that stores the error message
wget against urls and compare the result (headers and all)
Etc.