assert dilemma in unit testing class - php

Question

I would like to use PHP's assert function in my unit testing framework. It has the advantage of being able to see the expression being evaluated (including comments) within the error message.
The problem is that each method containing tests may have more than one assert statement, and I would like to keep track of how many actual assert statements have been run. assert does not give me a way to count how many times it has been run, only how many times it has failed (within the failure callback).
I tried to abstract the assert statement into a function so that I can add a counting mechanism.
private function assertTrue($expression) {
$this->testCount++;
assert($expression);
}
This does not work however because any variables within the expression are now out of scope.
$var = true;
$this->assertTrue('$var == true'); // fails
Any advice on how I can use assert in my unit testing while being able to count the number of actual tests?
The two ideas I have come up with are to make users count themselves
$this->testCount++;
assert('$foo');
$this->testCount++;
assert('$bar');
or make users put only one assert in each test method (I could then count the number of methods run). but neither of these solutions is very enforcable, and make coding more difficult. Any ideas on how to accomplish this? Or should I just strip assert() from my testing framework?

Answer 1

In PHPUnit, all of the assert*() methods take an additional $message parameter, which you can take advantage of:
$this->assertTrue($var, 'Expected $var to be true.');
If the assertion fails, the message is output with the failure in the post-test report.
This is more useful generally than outputting the actual expression because then you can comment on the significance of the failure:
$this->assertTrue($var, 'Expected result of x to be true when y and z.');

Answer 2

A bit of a cheeky answer here, but open vim and type:
:%s/assert(\(.+\));/assert(\1) ? $assertSuccesses++ : $assertFailures++;/g
(In principle, replace all assert() calls with assert() ? $success++ : $fail++;)
More seriously, providing a mechanism to count tests is really a responsibility a bit beyond the scope of the assert() function. Presumably you want this for an "X/Y tests succeeded" type indicator. You should be doing this in a testing framework, recording what each test is, its outcome and any other debug information.

Answer 3

You are restricted by the fact assert() must be called in the same scope the variables you are testing lie. That leaves -- as far as I can tell -- solutions that require extra code, modify the source before runtime (preprocessing), or a solution that extends PHP at the C-level. This is my proposed solution that involves extra code.
class UnitTest {
// controller that runs the tests
public function runTests() {
// the unit test is called, creating a new variable holder
// and passing it to the unit test.
$this->testAbc($this->newVarScope());
}
// keeps an active reference to the variable holder
private $var_scope;
// refreshes and returns the variable holder
private function newVarScope() {
$this->var_scope = new stdClass;
return $this->var_scope;
}
// number of times $this->assert was called
public $assert_count = 0;
// our assert wrapper
private function assert($__expr) {
++$this->assert_count;
extract(get_object_vars($this->var_scope));
assert($__expr);
}
// an example unit test
private function testAbc($v) {
$v->foo = true;
$this->assert('$foo == true');
}
}
Downfalls to this approach: all variables used in unit testing must be declared as $v->* rather than $*, whereas variables written in the assert statement are still written as $*. Secondly, the warning emitted by assert() will not report the line number at which $this->assert() was called.
For more consistency you could move the assert() method to the variable holder class, as that way you could think about each unit test operating on a test bed, rather than having some sort of magical assert call.

Answer 4

That's not something which unit-testing is intended to do (remember it originated in compiled langs).
And PHPs semantics do not help much with what you are trying to do either.
But you could accomplish it with some syntactic overhead still:
assert('$what == "ever"') and $your->assertCount();
Or even:
$this->assertCount(assert('...'));
To get the assertion string for succeeded conditions still, you could only utilize debug_backtrace and some heuristic string extraction.
This is not enforced much either (short of running a precompiler/regex over the test scripts). But I would look at this from the upside: not every check might be significant enough to warrant recording. A wrapper method thus allows opting out.

Answer 5

It's hard to give an answer without knowing how your framework has been built, but I'll give it a shot.
Instead of directly call the methods of your unit testing class ( methods like assertTrue() ), you could use the magic method of PHP __call(). Using this, you could increase an internal counter everytime assertTrue() method is called. Actually, you can do whatever you want, every time any method is called.
Remember that __call() is invoked if you try to call a method that does not exist. So you would've to change all your methods names, and call them internally from __call(). For instance, you'd have a method called fAssertTrue(), but the unit testing class would use assertTrue(). So since assertTrue() is not defined, __call() method would be invoked, and there you would call fAssertTrue().

Answer 6

Since you're passing the expression already (which might lead, correct me if I'm wrong, to quoting hell):
$this->assertTrue('$var == true'); // fails with asset($expression);
Why not add a tiny extra layer of complexity, and avoid the quoting hell, by using a closure instead?
$this->assertTrue(function() use ($var) {
return $var == true;
}); // succeeds with asset($expression());

Answer 7

Simple:
$this->assertTrue($var == true);
(without quotes!)
It will be evaluated in caller space, so assertTrue() will be passed just false or true.
As others have pointed out, this might not be the best way of testing, but that's another question entirely... ;)