I'm trying to write PHPUnit tests for a class that writes to error_log but seem to be having trouble with PHPUnit's output buffering.
Because of the potential volatility of error_log given an unknown SUT php.ini configuration, I ultimately decided upon writing to stdout after having limited success with overloading via namespace.
Unfortunately, there appears to be some quirk in how PHPUnit is buffering output in that calling expectOutputString returns an empty string as "Actual" rather than what was written to stdout.
Here's a quick-and-dirty test class I wrote to eliminate anything that might be giving me unintended grief ("ObTest.php"):
<?php
namespace Some\Test;
class ObTest
{
public function writeToErrorLog($message)
{
error_log($message);
}
}
And here's the test I wrote:
<?php
namespace Some\Test;
include "ObTest.php";
class ObTestTest extends \PHPUnit_Framework_TestCase
{
public static $error_log_config;
public static function setUpBeforeClass()
{
self::$error_log_config = ini_get("error_log");
ini_set("error_log", "/dev/stdout");
}
public static function tearDownAfterClass()
{
ini_set("error_log", self::$error_log_config);
}
public function testWriteToErrorLogOutputs()
{
$ob_test = new ObTest();
$this->expectOutputString("Test");
$ob_test->writeToErrorLog("Test");
}
}
And finally the output:
PHPUnit 5.4-g9b5b99b by Sebastian Bergmann and contributors.
[06-May-2016 01:05:51 UTC] Test
Failed asserting that two strings are equal.
Expected :'Test'
Actual :''
Note that I do realize that the test will fail anyway due to the timestamp, but writing a test with expectOutputRegex is unnecessary to demonstrate the blank string being returned as "Actual"; I can confirm that I get the same result using the regex option.
I'm running PHP 5.6.20-1+deb.sury.org~trusty+1 if it's relevant.
Update 2016-05-22
Per the comments, I have tried several variants of setting error_log to write to output, including the STDOUT constant and php://output stream, all with no success.
For anyone interested or encountering a similar issue, I also have since logged an issue on GitHub thinking that this may be some sort of bug, but it hasn't yet received any attention as of this update. The issue notes that I've also tried this on the latest stable PHPUnit version (5.3.2) without success.
Related
I was programming an application using php and pthreads and I noticed that it was't working correctly. In order to isolate the error I reduced the code, and now I have this piece of code, and I've checked that it behaves differently when executes it through the console than trought a web browser, and I can't find why, I'm here for help. The code is:
<?php
echo "beforethread<br/>";
class AsyncOperation extends Thread
{
private $sum;
public function __construct()
{
$this->sum = 5;
}
public function run()
{
echo "insidethread<br/>";
}
}
$thread = new AsyncOperation();
$thread->start();
$thread->join();
?>
through console I get:
beforethread
insidethread
and through a web browser:
insidethread
beforethread
I can't understand if php executes line by line why through web browser the code doesn't behave correctly. Any help will be appreciated. Thank you.
You should not write standard output in a web environment from any thread other than the one responding to the request. This is not safe, at least you will experience garbled output and at worst segmentation faults (since you are filling the buffer for stdout from another thread, which is not allowed, zend provides no way to abstract this).
Even at the console, if you want to make sense of your output you should use a mutex when writing to stdout.
Description
I have a TestSuite which I need to mark as skipped (the entire test suite - not the specific test cases within the suite).
class AllTests
{
public static function suite()
{
// this does not work same as within TestCase:
// throw new \PHPUnit_Framework_SkippedTestError("Out of order");
$Suite = new \PHPUnit_Framework_TestSuite(__NAMESPACE__);
$Suite->addTestSuite(translators\AllTests::cls());
$Suite->addTestSuite(TlScopeTest::cls());
$Suite->addTestSuite(TlNsTest::cls());
$Suite->addTestSuite(TlElementTest::cls());
$Suite->addTestSuite(TlItemTest::cls());
$Suite->addTestSuite(LangCodeTest::cls());
$Suite->addTestSuite(TlElemClassTagTest::cls());
return $Suite;
}
}
As you can see throwing the PHPUnit_Framework_SkippedTestError exception does not work. It is not caught by the PHPUnit, and is breaks the execution as any uncaught exception (which is understandable, as the suite() method is invoked while building tests hierarchy, before actually running the tests).
I've seen an exception class named PHPUnit_Framework_SkippedTestSuiteError, but have no clue how to take advantage of it. Any ideas?
Motivation
I have a TestSuite, which aggregates many test cases as well as other test suites. Almost every test in this fails, becouse of a change which I made in the core of my code.
The problem is that this package is not crutial, and is scheduled to be fixed later. Until then I have to run tests for every other package, but when I do the PHPUnit output becomes flooded with the errors coming from the package in question. This forces me to check every time if any of the failures is coming from any other package.
This, as you might suspect, is very susceptible to human error, i.e. I could miss a failure, which actually is important.
I could run only the test suite on which I am currently working, but I lose control of whether or not my changes in one package causes a failure in other package.
I do not want to comment out that test suite, because I'm afraid that I (or someone who will take over the code after me) could forget about it entirely.
Ok, so I'll put it together:
The AllTests class has to be refactored to extend PHPUnit_Framework_TestSuite.
This makes the class a fully valuable TestSuite and allows to implement a setUp method on the suite level.
The setUp method is called by the test runner (not by the builder), so it is safe to throw a SkippedTestError exception.
The corresponding method to do just that within a test suite is called markTestSuiteSkipped (notice the Suite in the method name).
The entire class would look like this:
class AllTests extends \PHPUnit_Framework_TestSuite
{
protected function setUp()
{
$this->markTestSuiteSkipped("zzz");
}
public static function suite()
{
$Suite = new self(__NAMESPACE__);
$Suite->addTestSuite(translators\AllTests::cls());
$Suite->addTestSuite(TlScopeTest::cls());
$Suite->addTestSuite(TlNsTest::cls());
$Suite->addTestSuite(TlElementTest::cls());
$Suite->addTestSuite(TlItemTest::cls());
$Suite->addTestSuite(LangCodeTest::cls());
$Suite->addTestSuite(TlElemClassTagTest::cls());
return $Suite;
}
}
The output is a pretty block of S letters, which definetly indicate, that we skipped a lot of tests. This cannot escape our attention and yet allows our tests to pass.
You could mark test as skipped.
I'm calling phpunit with the argument being a directory (bonus questions: why can't it accept a list of files?) and it's now complaining about a class being declared more than once because of a file included in the previous test!
If i run phpunit firstTest.php; phpunit secondTest.php everything works
But phpunit ./ fails with PHP Fatal error: Cannot redeclare class X
my tests are basically:
include 'class_to_be_tested.php'
class class1Test extends...
and nothing else. And i'm using the option --process-isolation. I could add require_once on my classes, but that's not what i want to be able to test them individually.
shouldn't phpunit follow best testing practices and run one test, clear whatever garbage it have, run another test on a clean state? or am i doing something wrong?
Since you have include rather than include_once and there is no other code shown in your question, the cannot redeclare error could also be that you are including the file again somewhere in the code under test.
Assuming that is not the case, there some behind the scenes things that happen with --process-isolation that can keep the global class declarations. This blog post gives more detail: http://matthewturland.com/2010/08/19/process-isolation-in-phpunit/
Basically, you will want to create your own base TestCase and override the run() method to set the preserveGlobalState to false. This should properly allow all your tests to run together.
The base class would look similar to this (taken from the blog post I referred to):
class MyTestCase extends PHPUnit_Framework_TestCase
{
public function run(PHPUnit_Framework_TestResult $result = NULL)
{
$this->setPreserveGlobalState(false);
return parent::run($result);
}
}
The way phpUnit works means that all your tests are run in the context of a single php program. phpUnit aims to isolate the tests from each other, but they are all run within the same program execution.
PHP's include statement will include the requested file regardless of whether it has been included before. This means that if you include a given class twice, you will get an error the second time. This is happening in your tests because each test is including the same file, but without any consideration to whether it's already been included by one of the other tests.
Solutions:
Wrap your include calls with a if(class_exists('classname')) so that you don't include the file if the class has already been defined.
Include the files in a phpUnit bootstrap file instead of in the tests.
Use include_once (or even require_once) instead of include.
Stop including files arbitrarily, and start using an autoloader.
Change:
include 'class_to_be_tested.php';
class class1Test extends...
to be:
include_once 'class_to_be_tested.php';
class class1Test extends...
In PHP you need to have a Really Good Reason to use the former.
Regarding why can't it accept a list of files?, I think the design decision is that you generally don't need to. However you can do it by creating a test suite in the phpunit.xml.dist file, see
http://www.phpunit.de/manual/current/en/organizing-tests.html#organizing-tests.xml-configuration
I was reading my project's code coverage report, and I noticed something strange: a line was uncovered, but I was sure that line got executed during the tests. So, I added a var_dump() before it and this is what I got when executing the tests:
bool(true)
PHPUnit 3.5.5 by Sebastian Bergmann.
...
This is weird. How is it possible that a line is executed before PHPUnit's initialization? I believe this is the reason why code coverage says that line is uncovered.
Any hints?
EDIT: Here's some code. It's an IRC framework that makes use of the Doctrine Common library to read annotations and also uses the ClassLoader and EventDispatcher Symfony components. This is the incriminated method:
/**
* Returns this module's reflection.
*
* #return \ReflectionClass
* #access public
*/
static public function getReflection()
{
// The var_dump() displaying bool(false) is executed before PHPUnit, while the other
// ones are correctly executed.
var_dump(is_null(self::$reflection));
if (null === self::$reflection) {
// This line is reported as uncovered, but it must be executed since I'm
// accessing the reflection!
self::$reflection = new \ReflectionClass(get_called_class());
}
return self::$reflection;
}
What do you think?
Then, why are all the others var_dump() (that method gets executed many times in the application) shown after PHPUnit's output? And why isn't that line reported in code coverage even if it's executed?
I assume (but that's just a guess as it's hard to say since you have not shown the code), that it's related to code that gets executed on file-inclusion, rather after actual test functions are executed or testcases get instantiated.
The accessor must be getting called outside of a unit test which initializes self::$reflection. After that, all further calls to getReflection() skip the if block so it'll never be counted as covered. PHPUnit instantiates all of the test case classes--one per test method, data provider method, and data provider argument array--before running any of the tests or tracing code coverage. Look for a test case that calls getReflection() from its constructor or outside the class itself where the code is executed upon loading.
I forget if the test cases are instantiated before PHPUnit outputs its version and cannot check now, but I believe this is the case. The other possibility is that you're calling getReflection() from bootstrap.php, but you probably already checked for that.
Which action/method is in PHPUnit equal to in simpleTest:
$this->UnitTestCase('message .....')
Edit: Sorry for my mistake
I think what I asking about not exist in simple test directly its just our extended class.
But this method display message in begining of test- how its done with PHPUnit?
Thanks
I'm not a SimpleTest expert, but as far as I can tell that's the constructor for the UnitTestCase class. The equivalent in PHPUnit is PHPUnit_Framework_TestCase; you create your own tests by subclassing that and defining test methods. See the PHPUnit docs on writing tests for a quick howto and more info, but briefly, this is a complete PHPUnit test:
class MyTest extends PHPUnit_Framework_TestCase {
public function testSomething {
$this->assertTrue(MyClass::getSomethingTrue());
}
}
Update: to answer the revised question, the primary way to display messages in PHPUnit is on assertion failure. Every assert* function comes with an optional $message argument at the end, which you can use to display a custom message when that assertion fails.
If you want to always display a message, without having to fail an assertion, then you might try a straightforward print statement. It'll be interspersed with the test output, so this may not be the best (or nicest-looking) way of accomplishing what you want, but it'll certainly dump text to a console, which is what you seem to be asking.
If you're looking for some advanced debugging during unit-testing, you may also want to consider a logging framework of some kind (or even just a custom function that opens a file, prints to it, and closes the file again). That way, you preserve the integrity of the test output, but still get extra custom messages wherever you want them during your tests.