I do some PHP with Kohana 3 (IDE:Netbeans), and got excited about idea of writing tests for code. It sounds pretty cool thing to do, but i have few complications and worries.
Why using Kohana unittest module in browser is like 5 times faster then running tests in Netbeans or command Line ?
How could i exclude all kohana internal tests? In the PHPUnit .xml configuration file ?
Why when run any test i've got in Netbeans panel two entries for it - one with yellow triangle (it says 'file x skipped'), and entry with normal test result. I do get that double entries for every test, also those native from Kohana. I don't mind but it's strange.
All over the Web i see examples, tutorials and screencasts of PHPUnit with sample classes and methods that add two numbers or displays name or do some other trivial things. I've learnt to do those kind of assertions, but how could i test my code in Kohana? My Models are 90% ORM stuff. Controllers? How? Any 'How-tos' and examples are welcome.
I've seen in Ruby tutorial about Rspec a way to test DB by using testing enviroment Databse and rollbacks after finisning tests. Also user actions like clicking links were simulated. Is it possible with PHPUnit ?
There always has been a lot of discussion on what has to be tested and what has not to be tested. Generally my opinion is that you shouldn't test things that should work, like the database driver and connection, this has little to do with your code. Some then argue that you should be able to test it anyway, but in most environment this isn't an easy thing to do and usually a big hassle.
Generally controller actions should be tested as well as any helpers or modules you've written. Usually one uses the paradigm of a mocking framework to get around the database. The good thing about this is a gigantic speed increase in your testing. There are several PHP mocking frameworks as well I suppose.
Another great thing to keep in mind is that you also have user testing. This cannot be simulated with the kind of tests you write in kohana. For this it is interesting to look at http://seleniumhq.org/
It's probably better to split such a rambling question into multiple SO questions.
No idea. Perhaps there's an invocation overhead for NetBeans to invoke phpunit, compared to apache passing the request to PHP.
That might be possible, or you could find a way to set the following option: --exclude-group kohana
No idea sorry.
AFAIK PHPUnit can't do client-interaction tests. How to do system behaviour testing could be a question on its own.
Related
I just started to learn the concept of BDD.
I learn PHPSpec and Behat for that, but it is not clear to my why do I need to use both. I understand that behat is for the functional/acceptance testing and PHPSpec is mainly for unit testing, but the articles and videos I found on this is basicly testing the the code twice: once with behat (with scenarios) and once with with phpspec. Can someone explain to my with easy examples what is the difference and when do I need to use behat and whan phpspec?
Thanks the anwers in advance,
Br.
Well, before starting with the answer, I would like to point out that what follows is a more general answer than a "phpspec and behat" one.
As you correctly stated, phpspec is a tool designed to write unit tests whereas behat is something for other kind of tests (let's say from integration to e2e/smoke tests). So far so good. So now we can abstract and distinct between unit test tools and other testing tools.
Let's start defining what a unit test is and what is not. A unit test is a kind of test done against a "small" part (the unit) of a system. Typically it's focus is on a single class or method (whereas not always true). Unit tests promote fast refactor as they run fast and in isolation. Please, focus on fast and isolation, we'll be back at them sooner.
Other kind of tests are more cumbersome and oriented to test an interaction between some components, or a whole function, or the "whole" system as the user would use it. Why cumbersome? Because you may have to setup a database, you may have to run a webserver, you may have need for a browser simulator like selenium, and so on. Because of this, those kind of test are much slower than unit ones. Moreover, when an error arise from other tests but unit, as you have a "whole" funcionality, it will be more painful to hunt down the bug whereas with unit tests, at least, you know what class (or group of them) are causing the bug.
Having said that, remeber the statement about unit test speed and isolation? Well, speed helps you to "fail faster" (you don't need to wait the whole system bootstrap, whatever it means for your project) and fail in a more "localized" way (isolation).
My suggestion is to follow the test pyramid: a lot of unit tests (all possibile permutation of I/O, for instance, of a method) and only what's valuable for integration and above. Just to make an example: you can test a repository (that iteracts with DB, so something you can not do with unit) for a particular query, you can test for the login page or homepage to be reached (as a status for the application health) and so on.
My answer is just a short summary; hope that drives you in the right direction.
We have two CodeIgniter instances in production which I've just finished migrating to CodeIgniter 3. A large gap has been left in our code base since it was not designed with testability in mind. As the applications have evolved over 4 years throwing a test harness around it is not so simple.
There is code in controllers that just render views and code in views that just post to controllers and getting between the two is maddening.
I've tried to use PHPUnit to achieve testing as well as the CodeIgniter Unit Test class, but I always wind up back at this conundrum. Is there any advice people in similar situations can offer?
Well, it's quite hard to achieve testing in a legacy application in Codeigniter... What I ended up doing is:
Libraries and classes that can be isolated, are tested via PHPUnit
Controllers and rendering are tested via BDD. I use Codeception and Selenium (with Mozilla browser) to perform Acceptance tests, that at least makes sure that all is working alright (rendering, placing of elements and so on).
I would also like hearing from others experience. I personally use Codeception all the time even for Unit test (at the end, is built over PHPUnit). Main page: http://codeception.com/
I am currently creating a project using symfony2, i am using PHPUnit vendor to functional test my code.
I am facing a problem that the functional test time increases as the projects gets bigger.
Note: i am using Fixtures to be loaded for every functional test function.
My question here? what is the best practice to implement functional test that run in a reasonable time.
on another hand, Is there any bundle or tool that i can use to speedup this process?
Your question is complex and it is very hard to say exactly problem of delay.
First of all you can divide your fixtures that possible to load before your test suit and other part that can be loaded via setUp method.
2nd advice that can really speedup you tests process is multi-threads run. I use "fastest" lib that allow me to speed up my tests in about 3-4 times (on i7) in compare with one thread run.
https://github.com/liuggio/fastest
find tests/ -name "*Test.php" | ./bin/fastest "bin/phpunit -c app {};"
Just some addition about arguments for unit/functional test. As I see the topic is about "functional" tests and if the tool is PhpUnit, it doesn't mean that tests must be "unit", because it is possible to make functional tests with it too, but I believe that it is not the best choice of the tool for functional tests.
I will recommend you take a look to other a great tool for functional tests like Behat (http://behat.org). It will allow you to not only work with functional tests more comfortable, but use BDD(Behavior Driven Development) approach for your development process
Just small example how we use it for functional testing of shopping cart http://www.youtube.com/watch?v=XIUmDGaaZWM
EDIT: to clarify - I am asking for advice on both unit testing and user interface testing.
Currently, I don't use TDD. While I am developing an application I am constantly testing what I am working on. Testing iteration could be anything from minor function changes to entire models. I try not to code too much before I test. I like instant feedback.
Of course, with experience I can see potential problems or bugs occurring as I'm coding.
BUT, after an application is complete I will usually go through the entire app on the frontend and ensure all functions are working as expected. This means literally everything. Every add/edit/delete, sort, filter, even broken links and such.
This can take a lot of time sometimes but it does ensure my work hardly ever contains bugs after deployment.
However, I'm looking for a more standard solution. What do experienced developers do? Assume for a moment I am a single developer and so do not have a testing department etc
Do you hire beta testers (no good if app is sensitive to public use)?
Is it viable to build a series of 'general' unit tests which can for example test ALL sorting, filter functions. One for testing ALL add/edit/delete functions.
Love to hear your feedback. Will be changing the way I develop based on suggestions.
Of course as David said : Unit-testing for models and helpers, of course.
and I need to add Selenium
Selenium is a robust set of tools that supports rapid development of
test automation for web-based
applications. Selenium provides a rich
set of testing functions specifically
geared to the needs of testing of a
web application. These operations are
highly flexible, allowing many options
for locating UI elements and comparing
expected test results against actual
application behavior.
which is amazing :
watch the 2 min intro
http://seleniumhq.org/movies/intro.mov
How Selenium Works
Testing with PHPUnit and Selenium
Unit-testing for models and helpers, of course.
But you can do unit-testing on "page-requests", as well. See, for example:
Content with Style - Unit testing controllers with Zend Framework
That's a step towards integration testing. But for issues of layout and visual aesthetics, you're pretty much stuck with walking-through each request with your browser.
Usually what you code works is not that the problem, very nice if you also write tests for it. The problem is that you need to test the integration of your changes every time you make commit.
If you like instant feedback,as I do, probably you should have a look at Continuous Integration.
I've started using Hudson as CI server and I am not regretting it!
Which unit testing framework do you use for Symfony?
Lime or PHPUnit? What are the pros and cons of using them?
In my opinion, here are a few things that come to my mind :
PHPUnit is more integrated with other tools, like, for instance,
Selenium (PHPUnit can use it to open true real browsers to test your site)
phpUnderControl for continuous-integration
PHPUnit works well with Xdebug, to generate code-coverage reports
PHPUnit is more widely used ; which probably means more support
But note I don't work with symfony, nor lime...
Still, I've never heards anyone speak about it, except for those working with symfony -- that not a good thing, for the day you'll have to work with another framework (yes, this happens ^^ )
One thing that's not in PHPUnit :
"false" browser (being able to do HTTP Requests to the application, without using Selenium to open a real browser)
But some frameworks (Zend Framework does, with it's Zend_Test component) integrate with PHPUnit (or use it), while allowing injection of data into the MVC and fetching of the response, without having to issue any HTTP Request.
I don't know if symfony allows that, but that's a nice thing with ZF/PHPUnit ^^
(Yes, not a symfony-specific answer ; but of the things I said must still be valid with that framework)
Lime is a much more simple testing framework, which can be a good or a bad thing depending on how you want to use it.
The symfony library itself uses its own testing framework, Lime, to test its code base. From the symfony book:
It is based on the Test::More Perl
library, and is TAP compliant, which
means that the result of tests is
displayed as specified in the Test
Anything Protocol, designed for better
readability of test output.
I cannot vouch for the statement that the lime framework is "more lightweight" than other PHP testing frameworks as the symfony docs claim, but I do really like that it's built right into your symfony project and works well with the symfony command line tools without any additional configuration. One thing that is especially cool is that the lime tests within symfony are set to run within your "test" environment which has it's own database, symfony cache (which gets cleared out during each test session), and environment variables. This comes in handy when you want to do functional testing (checking server response and your html output in your modules/actions, versus basic unit testing). I also like that lime is super easy to pick up and understand since it's so simple. You also have the ability to put your tests into YAML configuration file rather than write the tests by hand.
Pascal is entirely right that PHPUnit is much more widely used and you'd be able to use it in non-symfony projects. There is even a plugin for it, PHPUnit symfony plugin. My best advice would be to use lime if you just wanted to jump right into writing simple tests while you develop your symfony app. But, if you have the time and hope to use these testing skills outside of the symfony world, or bring in pre-existing PHPUnit tests into your symfony code, it'd be worth your time to check out the plugin and give it a spin.