I'm looking for a good unit testing for my web development. I'm using CodeIgniter for PHP, and I found this class on CodeIgniter's documentation, so here is my doubt:
Should I use 'Unit Testing Class' to test my code, or is going to be better to use PHPUnit or TOAST? Which is better and why?
I hope someone experienced can help me. Thanks!
In my opinion, I think it's better to use the standard PHPUnit for unit-testing.
Aside from that it has many nice features, your tests will be more consistent and it wont depend on CakePHP's unit testing class.
If in the future you decide that CakePHP does not suite your needs, you'll lose your unit tests. (or lets say, you'll have to migrate them, which is time)
If you are testing code, that depends on CakePHP(Controller, Model, etc), there's no problem to use CakePHP's tests, but for any other case, i think you should bet on the more-popular unit testing suite that can be used in every project (PHPUnit)
Happy testing :)
I haven't done any test with codeignitor's build in unit test class. but it looks like that this class provide only very simple assertion function (maybe it is because of the pursuit of lightweight) and the worst thing is, you have to mess up the testing code with your production code.
For unit test, it offers much more assertion functions, more features: like code coverage, selenium RC...etc. so it can extend your ability to ensure code quality. one more benefit is, it does not depend on any of your framework code, so you can easily migrate.
Unit Testing Class is quite simple test suite. If you need full unit testing of your code use PHPUnit.
Codeigniter has its own unit testing feature
Please go to http://codeigniter.com/user_guide/libraries/unit_testing.html
Thanks
Related
I'm somewhat new to testing in PHP. I've done quite a bit of testing on Rails using Cucumber, RSpec, Capybara and Factory Girl, but I've barely done any testing with PHP.
I'll ask a question about my current challenge in the most general way possible, since I've gone down a few specific paths and been met only with frustration.
I want to write a functional test for signing a user in. What's a good way to create the test user object that I would need in order to try to sign in?
Even more generally, what's the de facto standard for creating test objects in Symfony2? Fixtures? Some kind of factory?
In Ruby, I would use Factory Girl, since it lets you handle any object's dependencies in a clean, DRY way. There seems to be a Factory Girl equivalent in PHP in Phactory, but unfortunately it seems that that tool is not widely used and no longer maintained.
Fixtures on a test environment configuration are probably the way to go!
About the fixtures being application-wide comment, I disagree.
If you were unit testing, maybe that would make a lot of sense.
Functional tests, however, are supposed to be application-wide.
I just write test data builders myself, because there is almost no boilerplate anyway. BTW, Factory Girl is an implementation of that very pattern.
You can find a good discussion of the pattern and a lot of other useful stuff in the Growing Object-Oriented Software, Guided by Tests book.
And I recommend against fixtures, because they are application-wide. Each test better prepare only the data it needs for itself.
I use fixtures for functional tests but i disagree that they are application wide only. Im using it bundle-wide, defined in proper dirs in bundles. On test setup i run console fixtures:load method from proper directory and maybe not best solution but it works good for me. If anyone have better solution i would happily read about it :)
I've started from scratch a new application using TDD and PHPUnit. It is going to be a MVC application. I've started my tests from the Model. That was fun and i didn't have too much problems.
Now I want to create my views/models. But the question is: How do I test my controllers? What about views? My controller will use my tested models and my future views.
Thanks.
There are three broad categories of testing for each of the layer in the MVC. The unit tests for models, functional tests for the controllers and the UI testing for the views.
Unit tests are the easiest to write. They are cheap in terms of time and do not require too many dependencies to be stubbed/mocked.
Functional tests on the other hand are a bit expensive compared to an unit test. In a given scenario, if you had covered enough in your application for the models using unit test, you can relax a bit in the functional testing part. However, you should still be having a good code coverage - 100% to be ideal for your controller methods as well.
The last is the UI testing which is the costliest of all tests. You can probably use Selenium for browser based testing and try to automate it with your language. You may have to have a RC sever running in the background to achieve this. But trust me, if you cover the first two parts - unit and functional, this can be made optional if not no.
And it is advised to have a CI - Continious Integration setup with a code coverage utility that gives a trending on the % of code covered through tests.
When you are running the tests , you should be using only the class under test. Other objects should be replaced with mocks or other fake structures.
You do this, because ( for example ) when you write a test for controllers action , you will have provide some values for said action, and then check if the correct data is passed to the view and models.
This is one of the reasons why you should avoid use of global state ( in form of static calls or global variables ) in your code.
Some links you might find useful
The Clean Code Talks - Unit Testing
The Clean Code Talks - Global State and Singletons
The Clean Code Talks - Don't Look For Things!
When one starts searching for PHP unit tests, one usually stumbles upon:
PHPUnit.
SimpleTest.
A plethora of blogs explaining how to use PHPUnit and SimpleTest.
StackOverflow questions about PHPUnit and SimpleTest...
...and I think you get the idea.
I was wondering: How does one go about unit testing with plain ol' PHP? Is that even a wise endeavor?
I suppose I would have to build my own little framework. I'm interested because I want to have a better understanding of what's going on in my unit tests. I'm also interested because I think a lightweight, tailored solution would run my tests faster.
Bonus question: Who's the underdog? Is there a third framework worth looking into (for acedemic purposes)?
Unit testing is basically a set of assertions.
Consider the following PHPUnit test case:
class MyTest extends PHPUnit_Framework_TestCase {
public function testFoo() {
$obj = new My;
$this->assertEquals('bar', $obj->foo());
}
}
You can have a similar test case without using PHPUnit:
class MyTest {
public function testFoo() {
$obj = new My;
assert("$obj->foo() == 'bar'");
}
}
However, by doing it without a framework, you will have to manually create an instance of the test case (MyTest) and call each test method manually (MyTest::testFoo, etc.).
The framework (e.g.: PHPUnit) is nothing more than a set of "helpers" to make it easier and faster: by automatically generating skeleton; with built-in mock objects, command lines scripts, etc.
You can still unit test without a framework, but in the end, you'll probably save more time using one, because after all, that's generally what frameworks are for.
My view would be what's so wrong with the standard wheel (PHPUnit in all probability) that warrants the development of a replacement? After all, Isn't a proven solution the wiser choice in this instance?
Additionally, the output of the standard tools will be easier to use if you ever want to take advantage of a Continuous Integration server further down the line. (You could of course make your own tool output in an identical format, but that leads me back to the "why re-invent the wheel" argument.)
I've used SimpleTest about a year ago and it's very lightweight. There really wasn't much to it except providing you a way to auto-load test packages and assert.
Essentially it included all classes starting with "test" and it would call all methods starting with "test" inside that class. Inside the methods you could assert the results are what you expected.
If you were to build your own framework, I would strongly recommend looking into SimpleTest before. It's very lightweight compared to PHP Unit (if that's what scares you).
You can write tests in any language without a framework, but the code you write that you share in your tests will end up being a kind of mini-framework anyhow.
A good unit testing framework will have the common stuff that you need to make testing easier, such as assertions, mocks, stubs, exception handling and code coverage.
Bonus question: Who's the underdog? Is there a third framework worth looking into (for acedemic purposes)?
There are underdogs in the PHP unit testing space, for example Enhance PHP, which is a unit-testing framework with mocks and stubs built in. It has some advantages over the big players - it has good documentation and is very quick to get started on.
On our development team, we decided to give Unit Testing a try. We use Simpletest. However, it's been a tough road. After a week, I only have created 1 unit test that tests a certain helper file. That's it. The rest (controllers, models, views, libraries) don't have unit tests yet. And I plan to not test a majority of them. Views, for example, are too trivial to test, so I pass up on testing that. Next, the controllers. I plan my controllers to not do complex stuff, so that it only does passing of information between the models and the views. I'd move those more complex stuff to libraries or helpers.
Now for my questions:
1) Am I doing it wrong? So far, there's nothing more that I can see that can be erroneous so it would need a unit test. Most of the stuff (right now) are just CRUD.
2) Do we really need to unit test controllers? Since a controller's job is just minor processing of data passed between View and Model, I find very little initiative in unit testing it.
3) If I use WebTestCase to test for controllers, would that be still considered a Unit Test? Or is it already an integration test?
4) Suppose you got me to test my controller, how would I test it? As far as I know, CI follows the Front Controller pattern through index.php, so how would I handle (mock?) that?
If you are still open with suggestion to another unit test for CodeIgniter, I suggest you try Toast. I found it easy to use, and not interfere much with my development process.
I only use unit test to test my library, helper, and model. My controller not have much code, only get post and uri parameter, sanitizing it using trim or intval, pass it to library or model, then pass the result to view.
View almost have none code to test, since it display everything to browser. Mostly, it just need css and js debugging.
Model almost always need testing, since it dealing with data. Without unit test, I found it difficult to trace some bug, specially with a complex computation.
Library and helper do a repetitive task, so it need a unit test to make sure the logic in it is doing the work correctly.
I hope this help.
Are you doing something wrong? I don't think so.
Do we really need to unit test controllers? I don't. Maybe I should. Seems like a lot of work, though.
If I use WebTestCase to test for controllers, would that be still considered a Unit Test? Or is it already an integration test? WebTestCase would be an interesting approach to testing controllers if some meaningful output could be detected; for example, detecting that no error has occurred when calling /some/specific/path.
Suppose you got me to test my controller, how would I test it? That's a tough one. You would presumably need to initialize part of the application environment in order to do anything worthwhile.
Most articles/books tell you to define your tests before you start coding. Maybe I've tried that, but I'm usually too impatient. It seems to get in the way of quick prototyping, but maybe defining the unit tests is a way of quick prototyping.
I've found that deciding what to test with PHP is a challenge. I think you have to pick your battles. If it's critical that a method return a value of specific type, then test for that. If a lot of things happen automatically when instantiating an object, you could test for that too.
In general, what I do -- right or wrong -- is get everything working, then create some basic tests and then add tests as needed based on any problems I encounter. The idea is to never have a repeat problem and each test will insure that the application behaves the same through its life-cycle.
As for specifics, I'm using Zend Framework, but it will be similar in CodeIgniter. I also use SimpleTest (but with my own class wrapper). I may or may not unit test models and I have never implemented tests for controllers or views; it's seems like too much work and too little benefit. Most frameworks "fail early" and so issues are typically obvious. But any common library code make for easier targets and bugs here -- especially logic bugs -- are harder to detect. Set up your tests to make sure things work as expected so that your framework-specific code encounters few problems.
I've got a big project written in PHP and Javascript. The problem is that it's become so big and unmaintainable that changing some little portion of the code will upset and probably break a whole lot of other portions.
I'm really bad at testing my own code (as a matter of fact, others point this out daily), which makes it even more difficult to maintain the project.
The project itself isn't that complicated or complex, it's more the way it's built that makes it complex: we don't have predefined rules or lists to follow when doing our testing. This often results in lots of bugs and unhappy customers.
We started discussing this at the office and came up with the idea of starting to use test driven development instead of the develop like hell and maybe test later (which almost always ends up being fix bugs all the time).
After that background, the things I need help with are the following:
How to implement a test
framework into an already existing
project? (3 years in the
making and counting)
What kind of frameworks are there
for testing? I figure I'll need one
framework for Javascript and
one for PHP.
Whats the best approach for testing
the graphical user interface?
I've never used Unit Testing before so this is really uncharted territory for me.
G'day,
Edit: I've just had a quick look through the first chapter of "The Art of Unit Testing" which is also available as a free PDF at the book's website. It'll give you a good overview of what you are trying to do with a unit test.
I'm assuming you're going to use an xUnit type framework. Some initial high-level thoughts are:
Edit: make sure that everyone is is agreement as to what constitutes a good unit test. I'd suggest using the above overview chapter as a good starting point and if needed take it from there. Imagine having people run off enthusiastically to create lots of unit tests while having a different understanding of what a "good" unit test. It'd be terrible for you to out in the future that 25% of your unit tests aren't useful, repeatable, reliable, etc., etc..
add tests to cover small chunks of code at a time. That is, don't create a single, monolithic task to add tests for the existing code base.
modify any existing processes to make sure new tests are added for any new code written. Make it a part of the review process of the code that unit tests must be provided for the new functionality.
extend any existing bugfix processes to make sure that new tests are created to show presence and prove the absence of the bug. N.B. Don't forget to rollback your candidate fix to introduce the bug again to verify that it is only that single patch that has corrected the problem and it is not being fixed by a combination of factors.
Edit: as you start to build up the number of your tests, start running them as nightly regression tests to check nothing has been broken by new functionality.
make a successful run of all existing tests and entry criterion for the review process of a candidate bugfix.
Edit: start keeping a catalogue of test types, i.e. test code fragments, to make the creation of new tests easier. No sense in reinventing the wheel all the time. The unit test(s) written to test opening a file in one part of the code base is/are going to be similar to the unit test(s) written to test code that opens a different file in a different part of the code base. Catalogue these to make them easy to find.
Edit: where you are only modifying a couple of methods for an existing class, create a test suite to hold the complete set of tests for the class. Then only add the individual tests for the methods you are modifying to this test suite. This uses xUnit termonology as I'm now assuming you'll be using an xUnit framework like PHPUnit.
use a standard convention for the naming of your test suites and tests, e.g. testSuite_classA which will then contain individual tests like test__test_function. For example, test_fopen_bad_name and test_fopen_bad_perms, etc. This helps minimise the noise when moving around the code base and looking at other people's tests. It also has then benefit of helping people when they come to name their tests in the first place by freeing up their mind to work on the more interesting stuff like the tests themselves.
Edit: i wouldn't use TDD at this stage. By definition, TDD will need all tests present before the changes are in place so you will have failing tests all over the place as you add new testSuites to cover classes that you are working on. Instead add the new testSuite and then add the individual tests as required so you don't get a lot of noise occurring in your test results for failing tests. And, as Yishai points out, adding the task of learning TDD at this point in time will really slow you down. Put learning TDD as a task to be done when you have some spare time. It's not that difficult.
as a corollary of this you'll need a tool to keep track of the those existing classes where the testSuite exists but where tests have not yet been written to cover the other member functions in the class. This way you can keep track of where your test coverage has holes. I'm talking at a high level here where you can generate a list of classes and specific member functions where no tests currently exist. A standard naming convention for the tests and testSuites will greatly help you here.
I'll add more points as I think of them.
HTH
You should get yourself a copy Working Effectively with Legacy Code. This will give you good guidance in how to introduce tests into code that is not written to be tested.
TDD is great, but you do need to start with just putting existing code under test to make sure that changes you make don't change existing required behavior while introducing changes.
However, introducing TDD now will slow you down a lot before you get back going, because retrofitting tests, even only in the area you are changing, is going to get complicated before it gets simple.
Just to add to the other excellent answers, I'd agree that going from 0% to 100% coverage in one go is unrealistic - but that you should definitely add unit tests every time you fix a bug.
You say that there are quite a lot of bugs and unhappy customers - I'd be very positive about incorporating strict TDD into the bugfixing process, which is much easier than implementing it overall. After all, if there really is a bug there that needs to be fixed, then creating a test that reproduces it serves various goals:
It's likely to be a minimal test case to demonstrate that there really is an issue
With confidence that the (currently failing) test highlights the reported problem, you'll know for sure if your changes have fixed it
It will forever stand as a regression test that will prevent this same issue recurring in future.
Introducing tests to an existing project is difficult and likely to be a long process, but doing them at the same time as fixing bugs is such an ideal time to do so (parallel to introducing tests gradually in a "normal" sense) that it would be a shame not to take that chance and make lemonade from your bug reports. :-)
From a planning perspective, I think you have three basic choices:
take a cycle to retrofit the code with unit tests
designate part of the team to retrofit the code with unit tests
introduce unit tests gradually as you work on the code
The first approach may well last a lot longer than you anticipate, and your visible productivity will take a hit. If you use it, you will need to get buy-in from all your stakeholders. However, you might use it to kickstart the process.
The problem with the second approach is that you create a distinction between coders and test writers. The coders will not feel any ownership for test maintenance. I think this approach is worth avoiding.
The third approach is the most organic, and it gets you into test-driven development from the get go. It may take some time for a useful body of unit tests to accumulate. The slow pace of test accumulation might actually be an advantage in that it gives you time to get good at writing tests.
All things considered, I think I'd opt for a modest sprint in the spirit of approach 1, followed by a commitment to approach 3.
For the general principles of unit testing I recommend the book xUnit Test Patterns: Refactoring Test Code by Gerard Meszaros.
I've used PHPUnit with good results. PHPUnit, like other JUnit-derived projects, requires that code to be tested be organized into classes. If your project is not object-oriented, then you'll need to start refactoring non-procedural code into functions, and functions into classes.
I've not personally used a JavaScript framework, though I would image that these frameworks would also require that your code be structured into (at least) callable functions if not full-blown objects.
For testing GUI applications, you may benefit from using Selenium, though a checklist written by a programmer with good QA instincts might work just fine. I've found that using MediaWiki or your favorite Wiki engine is a good place to store checklists and related project documentation.
Implementing a framework is in most cases a complex task because you kinda start rebuilding your old code with some new solid framework parts. Those old parts must start to communicate with the framework. The old parts must receive some callbacks and returnstates, the old parts must then somehow point that out to the user and in fact you suddenly have 2 systems to test.
If you say that you application itself isn't that complex but it has become due to lack of testing it might be a better option to rebuild the application. Put some common frameworks like Zend to the test, gather your requirements, find out if the tested framework suits for the requirements and decide if it's usefull to start over.
I'm not very sure of unit testing, but NetBeans has a built-in unit testing suite.
If the code is really messy, it's possible that it will be very hard to do any unit testing. Only sufficiently loosely coupled and sufficiently well designed components can be unit tested easily. However, functional testing may be a lot easier to implement in your case. I would recommend taking a look at Selenium. With this framework, you will be able to test your GUI and the backend at the same time. However, most probably, it won't help you catch the bugs as well as you could with unit testing.
Maybe this list will help you and your mates to re-structure everything:
Use UML, to design and handle exceptions (http://en.wikipedia.org/wiki/Unified_Modeling_Language)
Use BPMS, to design your work-flow so you won't struggle (http://en.wikipedia.org/wiki/Business_process_management)
Get a list of php frameworks which also support javascript backends (e.g. Zend with jQuery)
Compare these frameworks and take the one, which matches the most to your project desgin and the coding structure used before
You should may be consider using things like ezComponents and Dtrace for debugging and testing
Do not be afraid of changes ;)
For GUI testing you may want to take a look at Selenium (as Ignas R pointed out already) OR you may wanna take a look at this tool as well: STIQ.
Best of luck!
In some cases, doing automatic testing may not be such a good idea, especially when the code base is dirty and PHP mixes it's behavior with Javascript.
It could be better to start with a simple checklist (with links, to make it faster) of the tests that should be done (manually) on the thing before each delivery.
When coding in a 3 years old minefield, better protect yourself with many error checking. The 15 minutes spent writing THE proper error message for each case will not be lost.
Use the bridging method: bridge ugly lengthy function fold() with a call to fnew() which is a wrapper around some clean classes, call both fold and fnew and compare result, log differences, throw the code into production and wait for your fishes. When doing this, always use one cycle for refactoring, anOTHER cycle for changing result (do not even fix bugs in old behavior, just bridge it).
I agree with KOHb, Selenium is a must !
Also have a look at PHPure,
Their software records the inputs and outputs from a working php website, and then automatically writes phpunit tests for functions that do not access external sources (db, files etc..).
It's not a 100% solution, but it's a great start