On our development team, we decided to give Unit Testing a try. We use Simpletest. However, it's been a tough road. After a week, I only have created 1 unit test that tests a certain helper file. That's it. The rest (controllers, models, views, libraries) don't have unit tests yet. And I plan to not test a majority of them. Views, for example, are too trivial to test, so I pass up on testing that. Next, the controllers. I plan my controllers to not do complex stuff, so that it only does passing of information between the models and the views. I'd move those more complex stuff to libraries or helpers.
Now for my questions:
1) Am I doing it wrong? So far, there's nothing more that I can see that can be erroneous so it would need a unit test. Most of the stuff (right now) are just CRUD.
2) Do we really need to unit test controllers? Since a controller's job is just minor processing of data passed between View and Model, I find very little initiative in unit testing it.
3) If I use WebTestCase to test for controllers, would that be still considered a Unit Test? Or is it already an integration test?
4) Suppose you got me to test my controller, how would I test it? As far as I know, CI follows the Front Controller pattern through index.php, so how would I handle (mock?) that?
If you are still open with suggestion to another unit test for CodeIgniter, I suggest you try Toast. I found it easy to use, and not interfere much with my development process.
I only use unit test to test my library, helper, and model. My controller not have much code, only get post and uri parameter, sanitizing it using trim or intval, pass it to library or model, then pass the result to view.
View almost have none code to test, since it display everything to browser. Mostly, it just need css and js debugging.
Model almost always need testing, since it dealing with data. Without unit test, I found it difficult to trace some bug, specially with a complex computation.
Library and helper do a repetitive task, so it need a unit test to make sure the logic in it is doing the work correctly.
I hope this help.
Are you doing something wrong? I don't think so.
Do we really need to unit test controllers? I don't. Maybe I should. Seems like a lot of work, though.
If I use WebTestCase to test for controllers, would that be still considered a Unit Test? Or is it already an integration test? WebTestCase would be an interesting approach to testing controllers if some meaningful output could be detected; for example, detecting that no error has occurred when calling /some/specific/path.
Suppose you got me to test my controller, how would I test it? That's a tough one. You would presumably need to initialize part of the application environment in order to do anything worthwhile.
Most articles/books tell you to define your tests before you start coding. Maybe I've tried that, but I'm usually too impatient. It seems to get in the way of quick prototyping, but maybe defining the unit tests is a way of quick prototyping.
I've found that deciding what to test with PHP is a challenge. I think you have to pick your battles. If it's critical that a method return a value of specific type, then test for that. If a lot of things happen automatically when instantiating an object, you could test for that too.
In general, what I do -- right or wrong -- is get everything working, then create some basic tests and then add tests as needed based on any problems I encounter. The idea is to never have a repeat problem and each test will insure that the application behaves the same through its life-cycle.
As for specifics, I'm using Zend Framework, but it will be similar in CodeIgniter. I also use SimpleTest (but with my own class wrapper). I may or may not unit test models and I have never implemented tests for controllers or views; it's seems like too much work and too little benefit. Most frameworks "fail early" and so issues are typically obvious. But any common library code make for easier targets and bugs here -- especially logic bugs -- are harder to detect. Set up your tests to make sure things work as expected so that your framework-specific code encounters few problems.
Related
With tests, using the #depends you can make a test method depend on another one, that way, should the first test fail, the test that depends on it will be ignored, is there a way to do that with Test Classes
For example, say i have a page which i test the layout of, checking that the images are showing up, if links are correct, this will be one Test Class, on this page is a link to a form, for that page i would create a new test class checking its layout, validation ect.
what i want to try and pull of is if any test of the first Test Class fails then the second one should be skipped since the first page should be correct as it will be the page a user will see before entering the first (unless they type the second page's url in but have to assume users are stupid and as such don't know how the address bar work)
i should also note that all the tests are being stored using TFS so while the team will have the same tests we may have different phpunit.xml files (one person apparently uses php.xml.dist and wont change from it because it is mentioned once in Magento TAF even though i use .xml and have had no problems), as such, trying to enforce an order in phpunit.xml wont be all that useful (also enforcing the order in the .xml wont do this dependency thing)
Just because we can do a thing, does not mean we should. Having tests that depend on other tests' successful execution is a bad idea. Every decent text on testing tells you this. Listen to them.
I don't think phpUnit has a way to make one test class depend on another. (I started writing down some hacks that spring to mind, but they are so ugly and fragile and I deleted them again.)
I think the best approach is one big class for all your functional tests. Then you can use #depends as much as you need to. <-- That is where my answer to your actual question ends :-)
In your comment on Ross's answer you say: "I was taught that if you have a large number of (test) methods in one class then you should break it into separate classes" To see why we are allowed to break this rule you have to go below the surface to why that is usually a good idea: a lot of code in one class suggests the class is doing too much, making it harder to change, and harder to test. So you use Extract Class refactoring to split classes up into finer functionality. But never mechanically: each class should still be a nice, clean abstraction of something.
In unit testing the class is better thought of as a way to collect related tests together. When one test depends on another then obviously they are related, so they should be in the same class.
If a 2000-line file make you unhappy, one thing you can do, and should do, is Extract Parent Class. Into your parent class goes all the helper functions and custom asserts. You will leave all the actual tests in the derived class, but go through each of them and see what common functionality could be moved to a shared function, and then put that shared function in the parent class.
Responding to Ross's suggestion that #depends is evil, I prefer to think of it as helping you find the balance between idealism and real-world constraints. In the ideal world you want all your tests to be completely independent. That means each test needs to do its own fixture creation and tear down. If using a database, it should be creating its own database (a unique name, so in the future they can be run in parallel), then creating the tables, filling them with data, etc. (Use the helper functions in your parent class to share this common fixture code.)
On the other hand, we want our tests to finish in under 100 milliseconds, so that they don't interrupt our creative flow. Fixture sharing helps speed up tests at the cost of removing the independence.
For functional tests of a website I would recommend using #depends for obvious things like login. If most of your tests will first log into the site, then it makes a lot of sense to make a loginTest(), and have all other tests #depend on that. If login is not working, you know for sure that all your other tests are going to fail... and are wasting huge amounts of the most valuable programmer resource in the process.
When it is not so clear-cut, I'd err on the side of idealism, and come back and optimize later, if you need to.
I've started from scratch a new application using TDD and PHPUnit. It is going to be a MVC application. I've started my tests from the Model. That was fun and i didn't have too much problems.
Now I want to create my views/models. But the question is: How do I test my controllers? What about views? My controller will use my tested models and my future views.
Thanks.
There are three broad categories of testing for each of the layer in the MVC. The unit tests for models, functional tests for the controllers and the UI testing for the views.
Unit tests are the easiest to write. They are cheap in terms of time and do not require too many dependencies to be stubbed/mocked.
Functional tests on the other hand are a bit expensive compared to an unit test. In a given scenario, if you had covered enough in your application for the models using unit test, you can relax a bit in the functional testing part. However, you should still be having a good code coverage - 100% to be ideal for your controller methods as well.
The last is the UI testing which is the costliest of all tests. You can probably use Selenium for browser based testing and try to automate it with your language. You may have to have a RC sever running in the background to achieve this. But trust me, if you cover the first two parts - unit and functional, this can be made optional if not no.
And it is advised to have a CI - Continious Integration setup with a code coverage utility that gives a trending on the % of code covered through tests.
When you are running the tests , you should be using only the class under test. Other objects should be replaced with mocks or other fake structures.
You do this, because ( for example ) when you write a test for controllers action , you will have provide some values for said action, and then check if the correct data is passed to the view and models.
This is one of the reasons why you should avoid use of global state ( in form of static calls or global variables ) in your code.
Some links you might find useful
The Clean Code Talks - Unit Testing
The Clean Code Talks - Global State and Singletons
The Clean Code Talks - Don't Look For Things!
I'm writing unit tests for a project (written in PHP, using PHPUnit) that have its entire environment (loaded components, events, configuration, cache, per-environment singletons, etc) held in an object which all the components use to interact with each other (using a mediator pattern).
In order to make the unit tests run faster, I'm sharing the environment object and some other objects (for example, in my test case for view object [as in the V of MVC], the view manager object [which acts as a factory for view objects and responsible for the actual rendering]) among tests in the same test case (using PHPUnit's setUpBeforeClass() and static properties).
Even though, to the best of my knowledge, the objects I share shouldn't effect the integrity of the tests (in the views case, for example, the environment and view manager object are shared, but a separate view object is created for every test - which is the object that's actually being tested by the test case), it just feels increasingly wrong to me.
I would prefer it if each test used a completely isolated environment and couldn't effect other tests in the same test case in any way. However, that would make the tests run much slower and it feels like a big price for something that I can't really pinpoint the downside of and mainly just "feels wrong".
What do you think? Can you pinpoint any downsides so I can convince myself its worth the longer execution time? Or am I just over reacting and its completely fine?
I share your feelings so maybe i just state my goals and my solution when i faced that issue:
Devs should have a test suite that runs very very fast
At least single test cases should execute in less than a second
I really want to be sure i don't have interdependencies in my test cases
I'm going to assume you have a Continuous Integration Server running. If not a cronjob might do but consider setting up jenkins, it's really really easy.
For normal usage:
Just share as much fixtures as you need to get the speed you need. It might not be pretty and there might be better solutions along the way but if you have something that is expensive to create just do it once.
I'd suggest helper methods getFoo() { if(!self::$foo) .... create ... return $foo;} over setUpBeforeClass because it can make sharing easier but mainly because of the following point.
Once a night:
Run your test suite with --process-isolation and in that bootstrap recreate your complete database and everything.
It might run 6 hours (disable code coverage for that!) but how cares. Your fixtures will be recreated for every single test case since it's a new php process and the static vars don't exist.
Using this way you can be sure that you don't have created dependent once a day. Thats good enough to remember what you did (and you can run with --filter and --process-isolation if you need to fix something).
Like writing "normal" code, when you write test cases, it's fine to rely on knowledge of how fixture objects work.
If a given factory method is documented as generating new instances every time, then I see only downside in creating the factory method anew each time, especially if the factory creation is itself expensive.
It helps to keep in mind a key goal around writing unit tests. You want to know within 5-10 minutes whether you broke the build. That way you can go out to lunch, go to a meeting, go home, etc. after you get the "all clear". If you know that some part of the fixture is reusable without creating interactions, then you should use that knowledge to make your tests even more comprehensive within that 5-10 minute window. I understand the purist impulse here, but it buys you nothing in terms of test independence, and it unnecessarily limits what your test suite will accomplish for you.
Perhaps this is blasphemous, but if you can still manage a significant threshold of code coverage and you can also guarantee that no state pollution can exist (and that is your real issue -- are you making sure that each test will not be effected by the data left over from the test before), I see no problem with leaving things the way they are.
Let the tests run quickly, and when a bug is found (which is inevitable in integration testing) then you have reason to invest the time in localizing one particular test or set of tests. If you have a toolkit which works for the general case, however, I would leave it as is.
I'm looking for a good unit testing for my web development. I'm using CodeIgniter for PHP, and I found this class on CodeIgniter's documentation, so here is my doubt:
Should I use 'Unit Testing Class' to test my code, or is going to be better to use PHPUnit or TOAST? Which is better and why?
I hope someone experienced can help me. Thanks!
In my opinion, I think it's better to use the standard PHPUnit for unit-testing.
Aside from that it has many nice features, your tests will be more consistent and it wont depend on CakePHP's unit testing class.
If in the future you decide that CakePHP does not suite your needs, you'll lose your unit tests. (or lets say, you'll have to migrate them, which is time)
If you are testing code, that depends on CakePHP(Controller, Model, etc), there's no problem to use CakePHP's tests, but for any other case, i think you should bet on the more-popular unit testing suite that can be used in every project (PHPUnit)
Happy testing :)
I haven't done any test with codeignitor's build in unit test class. but it looks like that this class provide only very simple assertion function (maybe it is because of the pursuit of lightweight) and the worst thing is, you have to mess up the testing code with your production code.
For unit test, it offers much more assertion functions, more features: like code coverage, selenium RC...etc. so it can extend your ability to ensure code quality. one more benefit is, it does not depend on any of your framework code, so you can easily migrate.
Unit Testing Class is quite simple test suite. If you need full unit testing of your code use PHPUnit.
Codeigniter has its own unit testing feature
Please go to http://codeigniter.com/user_guide/libraries/unit_testing.html
Thanks
When one starts searching for PHP unit tests, one usually stumbles upon:
PHPUnit.
SimpleTest.
A plethora of blogs explaining how to use PHPUnit and SimpleTest.
StackOverflow questions about PHPUnit and SimpleTest...
...and I think you get the idea.
I was wondering: How does one go about unit testing with plain ol' PHP? Is that even a wise endeavor?
I suppose I would have to build my own little framework. I'm interested because I want to have a better understanding of what's going on in my unit tests. I'm also interested because I think a lightweight, tailored solution would run my tests faster.
Bonus question: Who's the underdog? Is there a third framework worth looking into (for acedemic purposes)?
Unit testing is basically a set of assertions.
Consider the following PHPUnit test case:
class MyTest extends PHPUnit_Framework_TestCase {
public function testFoo() {
$obj = new My;
$this->assertEquals('bar', $obj->foo());
}
}
You can have a similar test case without using PHPUnit:
class MyTest {
public function testFoo() {
$obj = new My;
assert("$obj->foo() == 'bar'");
}
}
However, by doing it without a framework, you will have to manually create an instance of the test case (MyTest) and call each test method manually (MyTest::testFoo, etc.).
The framework (e.g.: PHPUnit) is nothing more than a set of "helpers" to make it easier and faster: by automatically generating skeleton; with built-in mock objects, command lines scripts, etc.
You can still unit test without a framework, but in the end, you'll probably save more time using one, because after all, that's generally what frameworks are for.
My view would be what's so wrong with the standard wheel (PHPUnit in all probability) that warrants the development of a replacement? After all, Isn't a proven solution the wiser choice in this instance?
Additionally, the output of the standard tools will be easier to use if you ever want to take advantage of a Continuous Integration server further down the line. (You could of course make your own tool output in an identical format, but that leads me back to the "why re-invent the wheel" argument.)
I've used SimpleTest about a year ago and it's very lightweight. There really wasn't much to it except providing you a way to auto-load test packages and assert.
Essentially it included all classes starting with "test" and it would call all methods starting with "test" inside that class. Inside the methods you could assert the results are what you expected.
If you were to build your own framework, I would strongly recommend looking into SimpleTest before. It's very lightweight compared to PHP Unit (if that's what scares you).
You can write tests in any language without a framework, but the code you write that you share in your tests will end up being a kind of mini-framework anyhow.
A good unit testing framework will have the common stuff that you need to make testing easier, such as assertions, mocks, stubs, exception handling and code coverage.
Bonus question: Who's the underdog? Is there a third framework worth looking into (for acedemic purposes)?
There are underdogs in the PHP unit testing space, for example Enhance PHP, which is a unit-testing framework with mocks and stubs built in. It has some advantages over the big players - it has good documentation and is very quick to get started on.