Context
I recently inherited the development and maintenance of a well-programmed PHP application (sarcasm). The application is based on a commercial software (which I will not name), and there is a layer of customization (ours) that is built on top of it.
Unfortunately, this application uses a ton of globals and singletons (pun-intended). I have built test cases for all the things we've overridden. However, a lot of things relies on some global state of something, which can cause race conditions and all sorts of weird stuff.
Randomizing the tests
In order to catch most of these weird-o-tons (I like to call them that), I have built a PHPUnit TestDecorator, [as documented in the manual][1]. This one:
class PHPUnit_Extensions_Randomizer extends PHPUnit_Extensions_TestDecorator
{
public function __construct(PHPUnit_Framework_Test $test)
{
$tests = $test->tests();
$shuffle = array();
foreach ($tests as $t) {
if ($t instanceof PHPUnit_Framework_TestSuite) {
$shuffle = array_merge($shuffle, $t->tests());
} else {
$shuffle[] = $t;
}
}
shuffle($shuffle);
$suite = new PHPUnit_Framework_TestSuite();
foreach ($shuffle as $t) $suite->addTest($t);
parent::__construct($suite);
}
}
It basically randomizes the tests order to make sure a test doesn't depend on a global state that may or may not be correct.
The question
The problem arose when came the time to test my custom decorator. I have not found anywhere in the manual, Google, or Stack Overflow how to load it.
When analyzing the code, I saw that PHPUnit itself was instantiating the RepeatedTest decorator in the TextUI_TestRunner::doRun() method. I know I can subclass the TestRunner, override doRun(), arrange for my decorator to be created and then just call the parent::doRun() with my decorator instance as the argument, override TextUI_Command and create a new CLI script to use my stuff instead of the built-in stuff.
Before I (re-)invent the wheel, I was just wondering, is it possible do load a custom decorator without subclassing the TestRunner?
Thanks.
With current PHPUnit Versions there is no easy way to plug in in your own code for the most part. The only thing that offers interchangeability is the TestRunner and what you described makes sense to me.
I'm not aware of any other way to provide test decorators or change out most of the other classes phpunit uses.
The way you want to change the test execution order seems to work out even so I'm not sure how well it would shuffle suits.
Another way to achieve this thats maybe less hassle would be to create a subclass of PHPUnit_Framework_TestSuite randomly addTest your code.
If that doesn't work out another option would be to use the xml configuration file and build the test suite from <file> tags and before every execution have some code shuffle those tags. Afaik phpunit doesn't sort them in any way so execution will be random
Are you looking to see if every tests on its on really works and want to find interdependent tests?
Or do you really want to see if something breaks horribly when you do a lot of stuff that should not change anything in the wrong order?
I'm asking just in case you haven't considered --process-isolation yet. (Which i assume you have but asking isn't going to hurt and might save you some time and effort :) )
When you run every test with a fresh set of globals you will at least find all test interdependencies and thats just one cli switch away to make sure every test in your suite works fine on its own.
Related
I am trying to test a class that has other classes passed to it (dependency injection), which are stored as class properties. (A settings class, for example that is the result of a Factory).
So, in psuedo code, this is what I am dealing with:
//Get settings for this user.
$settings = SettingsFactory::GetSettings();
//Create widget that uses settings
$widget = new Widget($settings);
Now, obviously, if we refactor code and break SettingsFactory::Getsettings(), the constructor for Widget will fail. But, not necessarily because anything is wrong with Widget.
So, I want to test these components IN ORDER, skipping "later" tests when "early" tests fail.
Consider my current structure for tests:
tests\vendor\Settings\SettingsTest.php
tests\vendor\Widget\WidgetTest.php
PHPUnit needs to test SettingsTest.php (and have everything pass) before running WidgetTest.php will be meaningful.
I think I need to use #depends, but that seems to be limited to a single class scope.
This question seems to hold a piece of the puzzle, but I am not sure how to implement.
How do I write (structure) these tests?
One of the big benefits of DI is that you can avoid issues like this, by injecting a mock $settings object that behaves the way you want. This gives you a true unit test of Widget without worrying about the implementation details of Settings:
$mockSettings = $this->createMock(Settings::class);
$mockSettings->method('someMethod')->willReturn('something');
$widget = new Widget($mockSettings);
// assertions here
I have a function, which someone else wrote, that creates a cURL wrapper object inside the function. Simplified version below
public function getCodes()
{
//do some stufff
$communicator = new Communicator();
$result = $communicator->call($this->API_KEY);
//do some stuff with $result
}
I was tasked with learning PHPUnit and writing tests for this type of code. In doing so I found that it's really hard to test a function like this when the object is created inside of the function and also that tests shouldn't require any outside communication to work.
We wanted to push our tests to git as many projects do but we didn't want to accidentally or intentionally push our API credentials to git.
So my solution was to keep getCodes() public but make it a wrapper for a private function that accepts a Communicator object as a parameter. Then I could test the private method with a mock Communicator object.
But this would mean that getCodes is never tested (my boss wants 100% code coverage) and I also read that you shouldn't be writing tests for private functions in most circumstances.
So my question is basically, how do I write a test for a function like this with an API call.
I would really suggest rewriting the code to inject the Communicator object via constructor.
If you already see there is a big issue with writing tests for something it is a very strong signal to re-thing the current implementation.
Another thing is that you shouldn't test your privates. Sebastian Bergmann wrote a post on his blog about this some time ago and the conclusion is - it is possible just not good (https://sebastian-bergmann.de/archives/881-Testing-Your-Privates.html).
Completely different thing is that I think your tests shouldn't go outside of the boundaries of your system. That is - mock everything that connects to outside systems. Such tests may fail for various of reasons non of witch are valid from the sole perspective of running tests.
You also mentioned coverage. Unfortunately this is something where, I hope, everyone will agree - you cannot have it the moment you start using native PHP resources (with some small exception like FS). You have to understand that things like curl, ssh, ftp and so on cannot be unit tested.
I am quite new to OOP, so this is really a basic OOP question, in the context of a Laravel Controller.
I'm attempting to create a notification system system that creates Notification objects when certain other objects are created, edited, deleted, etc. So, for example, if a User is edited, then I want to generate a Notification regarding this edit. Following this example, I've created UserObserver that calls NotificationController::store() when a User is saved.
class UserObserver extends BaseObserver
{
public function saved($user)
{
$data = [
// omitted from example
];
NotificationController::store($data);
}
}
In order to make this work, I had to make NotificationController::store() static.
class NotificationController extends \BaseController {
public static function store($data)
{
// validation omitted from example
$notification = Notification::create($data);
}
I'm only vaguely familiar with what static means, so there's more than likely something inherently wrong with what I'm doing here, but this seems to get the job done, more or less. However, everything that I've read indicates that static functions are generally bad practice. Is what I'm doing here "wrong," per say? How could I do this better?
I will have several other Observer classes that will need to call this same NotificationController::store(), and I want NotificationController::store() to handle any validation of $data.
I am just starting to learn about unit testing. Would what I've done here make anything difficult with regard to testing?
I've written about statics extensively here: How Not To Kill Your Testability Using Statics. The gist of it as applies to your case is as follows:
Static function calls couple code. It is not possible to substitute static function calls with anything else or to skip those calls, for whatever reason. NotificationController::store() is essentially in the same class of things as substr(). Now, you probably wouldn't want to substitute a call to substr by anything else; but there are a ton of reasons why you may want to substitute NotificationController, now or later.
Unit testing is just one very obvious use case where substitution is very desirable. If you want to test the UserObserver::saved function just by itself, because it contains a complex algorithm which you need to test with all possible inputs to ensure it's working correctly, you cannot decouple that algorithm from the call to NotificationController::store(). And that function in turn probably calls some Model::save() method, which in turn wants to talk to a database. You'd need to set up this whole environment which all this other unrelated code requires (and which may or may not contain bugs of its own), that it essentially is impossible to simply test this one function by itself.
If your code looked more like this:
class UserObserver extends BaseObserver
{
public function saved($user)
{
$data = [
// omitted from example
];
$this->NotificationController->store($data);
}
}
Well, $this->NotificationController is obviously a variable which can be substituted at some point. Most typically this object would be injected at the time you instantiate the class:
new UserObserver($notificationController)
You could simply inject a mock object which allows any methods to be called, but which simply does nothing. Then you could test UserObserver::saved() in isolation and ensure it's actually bug free.
In general, using dependency injected code makes your application more flexible and allows you to take it apart. This is necessary for unit testing, but will also come in handy later in scenarios you can't even imagine right now, but will be stumped by half a year from now as you need to restructure and refactor your application for some new feature you want to implement.
Caveat: I have never written a single line of Laravel code, but as I understand it, it does support some form of dependency injection. If that's actually really the case, you should definitely use that capability. Otherwise be very aware of what parts of your code you're coupling to what other parts and how this will impact your ability to take it apart and refactor later.
My code is located here: https://github.com/maniator/SmallFry
Should I make it so that that the App class does not have to use static functions but at the same time be able to set and set variables for the app from anywhere?
Or should I keep it how it is now with App::get and App::set methods?
What are the advantages and disadvantages of both?
How would I accomplish that 1st task if I was to undertake it?
Related Question
Sample code:
//DEFAULT TEMPLATE
App::set('APP_NAME', 'SmallVC');
//END DEFAULT TEMPLAT
//
//DEFAULT TEMPLATE
App::set('DEFAULT_TEMPLATE', 'default');
//END DEFAULT TEMPLATE
//DEFAULT TITLE
App::set('DEFAULT_TITLE', 'Small-VC');
//END DEFAULT TITLE
//LOGIN SEED
App::set('LOGIN_SEED', "lijfg98u5;jfd7hyf");
//END LOGIN SEED
App::set('DEFAULT_CONTROLLER', 'AppController');
if(App::get('view')){
$template_file = $cwd.'/../view/'.App::get('view').'/'.App::get('method').'.stp';
if(is_file($template_file)){
include $template_file;
}
else {
include $cwd.'/../view/missingview.stp'; //no such view error
}
}
else {
App::set('template', 'blank');
include $cwd.'/../view/missingfunction.stp'; //no such function error
}
I think you have a feeling that static is bad. What I am posting may seem fairly crazy as it is a massive change. At the very least hopefully it presents a different idea of the world.
Miško Hevery wrote static methods are a death to testability.
I like testing, so for that reason I don't use them. So, how else can we solve the problem? I like to solve it using what I think is a type of dependency injection. Martin Fowler has a good but complicated article on it here.
For each object at construction I pass the objects that are required for them to operate. From your code I would make AppController become:
class AppController
{
protected $setup;
public function __construct(array $setup = array())
{
$setup += array('App' => NULL, 'Database' => NULL);
if (!$setup['App'] instanceof App)
{
if (NULL !== $setup['App'])
{
throw new InvalidArgumentException('Not an App.');
}
$setup['App'] = new App();
}
// Same for Database.
// Avoid doing any more in the constructor if possible.
$this->setup = $setup;
}
public function otherFunction()
{
echo $this->setup['App']->get('view');
}
}
The dependancies default to values that are most likely (your default constructions in the if statements). So, normally you don't need to pass a setup. However, when you are testing or want different functionality you can pass in mocks or different classes (that derive from the right base class). You can use interfaces as an option too.
Edit The more pure form of dependency injection involves further change. It requires that you pass always pass required objects rather than letting the class default one when the object isn't passed. I have been through a similar change in my codebase of +20K LOC. Having implemented it, I see many benefits to going the whole way. Objects encapsulation is greatly improved. It makes you feel like you have real objects rather than every bit of code relying on something else.
Throwing exceptions when you don't inject all of the dependencies causes you to fix things quickly. With a good system wide exception handler set with set_exception_handler in some bootstrap code you will easily see your exceptions and can fix each one quickly. The code then becomes simpler in the AppController with the check in the constructor becoming:
if (!$setup['App'] instanceof App)
{
throw new InvalidArgumentException('Not an App.');
}
With every class you then write all objects would be constructed upon initialisation. Also, with each construction of an object you would pass down the dependencies that are required (or let the default ones you provide) be instantiated. (You will notice when you forget to do this because you will have to rewrite your code to take out dependencies before you can test it.)
It seems like a lot of work, but the classes reflect the real world closer and testing becomes a breeze. You can also see the dependencies you have in your code easily in the constructor.
Well, if it was me, I would have the end goal of injecting the App dependency into any class (or class tree) that needs it. That way in testing or reusing the code you can inject whatever you want.
Note I said reuse there. That's because it's hard to re-use code that has static calls in it. That's because it's tied to the global state so you can't really "change" the state for a subrequest (or whatever you want to do).
Now, on to the question at hand. It appears that you have a legacy codebase, which will complicate things. The way I would approach it is as follows:
Create a non-static version of the app class (name it something different for now) that does nothing but proxy its get/set calls to the real app class. So, for example:
class AppProxy {
public function set($value) {
return App::set($value);
}
}
For now, all it has to do is proxy. Once we finish getting all the code talking to the proxy instead of the static app, we'll make it actually function. But until then, this will keep the application running. That way you can take your time implementing these steps and don't need to do it all in one big sweep.
Pick a main class (one that does a lot for the application, or is important) that you easily control the instantiation of. Preferably one that you instantiate in only one place (in the bootstrap is the easiest). Change that class to use Dependency Injection via the constructor to get the "appproxy".
a. Test this!
Pick another class tree to work on, based on what you think will be most important and easiest.
a. Test!!!
If you have more calls to App::, Go to #3
Change the existing App class to be non-static.
a. Test!!!!!!!!!!
Remove the AppProxy and replace with App in the dependency injectors. If you did it right, you should only have one place to change to make this switch.
Pat yourself on the back and go get a drink, cause you're done.
The reason that I segmented it out like this is that once a step is completed (any step), you can still ship working software. So this conversion could take literally months (depending on the size of your codebase) without interrupting business as usual...
Now, once you're done, you do get some significant benefits:
Easy to test since you can just create a new App object to inject (or mock it as needed).
Side effects are easier to see since the App object is required wherever it could be changed.
It's easier to componentize libraries this way since their side effects are localized/
It's easier to override (polymorphism) the core app class if it's injected than if it's static.
I could go on, but I think it's pretty easy to find resources on why statics are generally bad. So that's the approach I would use to migrate away from a static class to an instance...
If you don't want to have static functions but global access from everywhere WITHOUT passing the object to the places where it is actually needed then you pretty much can only use one thing:
A global variable
So you are not really better of doing that. But that is the only thing i can think of that would fulfill your requirements.
If you App object is something like an application config a first possible step would be to pass it to the objects that need it:
class Login {
public function __construct() {
$this->_login_seed = App::get('LOGIN_SEED');
self::$_ms = Database::getConnection();
}
changes into:
class Login {
public function __construct(App $app) {
$this->_login_seed = $app->get('LOGIN_SEED');
self::$_ms = Database::getConnection();
}
I am developing an API documentation system, and want to dynamically check that each command has documentation attached.
The easiest way to do this is dynamically loop through each command and check for existing documentation to match it.
My code looks like this:
public function testMissingDocs()
{
foreach ($aCommands as $sKey => $aOptions)
{
$this->assertNotNull($oDocs->get($sKey));
}
}
The problem with this is the StopOnFailure/Error feature of PHPUnit which stops the test after the first assertion fails. I understand the reasons for this functionality and I want to keep it on for the majority of my test cases, but for dynamic assertions/tests it makes things a bit hard.
Is there a way to disable it on a per-test basis so I can check each command in this test?
You can use a data provider to split the single test into as many tests as you have commands.
/**
* #dataProvider getDocsForAllCommands
*/
public function testEveryCommandHasDocs($sKey)
{
$this->assertNotNull($oDocs->get($sKey));
}
public function getKeysForAllCommands()
{
return array_keys($aCommands);
}
If the documentation for a particular class or method is missing, that would represent a problem with that class, not with the method to retrieve the documentation.
Although it is probably easier to combine all of the documentation into a single test, that does not follow unit testing best practices (and hence is why the PHPUnit framework is working against you rather than for you).
I would suggest one of two approaches to rectify the issue:
Refactor your tests so that each class and/or method has its own documentation check (there are a few ways you can run multiple tests in one execution).
Use a tool such as PHP_CodeSniffer instead, as lack of documentation could be considered a convention – rather than a functional – defect.