EDIT: to clarify - I am asking for advice on both unit testing and user interface testing.
Currently, I don't use TDD. While I am developing an application I am constantly testing what I am working on. Testing iteration could be anything from minor function changes to entire models. I try not to code too much before I test. I like instant feedback.
Of course, with experience I can see potential problems or bugs occurring as I'm coding.
BUT, after an application is complete I will usually go through the entire app on the frontend and ensure all functions are working as expected. This means literally everything. Every add/edit/delete, sort, filter, even broken links and such.
This can take a lot of time sometimes but it does ensure my work hardly ever contains bugs after deployment.
However, I'm looking for a more standard solution. What do experienced developers do? Assume for a moment I am a single developer and so do not have a testing department etc
Do you hire beta testers (no good if app is sensitive to public use)?
Is it viable to build a series of 'general' unit tests which can for example test ALL sorting, filter functions. One for testing ALL add/edit/delete functions.
Love to hear your feedback. Will be changing the way I develop based on suggestions.
Of course as David said : Unit-testing for models and helpers, of course.
and I need to add Selenium
Selenium is a robust set of tools that supports rapid development of
test automation for web-based
applications. Selenium provides a rich
set of testing functions specifically
geared to the needs of testing of a
web application. These operations are
highly flexible, allowing many options
for locating UI elements and comparing
expected test results against actual
application behavior.
which is amazing :
watch the 2 min intro
http://seleniumhq.org/movies/intro.mov
How Selenium Works
Testing with PHPUnit and Selenium
Unit-testing for models and helpers, of course.
But you can do unit-testing on "page-requests", as well. See, for example:
Content with Style - Unit testing controllers with Zend Framework
That's a step towards integration testing. But for issues of layout and visual aesthetics, you're pretty much stuck with walking-through each request with your browser.
Usually what you code works is not that the problem, very nice if you also write tests for it. The problem is that you need to test the integration of your changes every time you make commit.
If you like instant feedback,as I do, probably you should have a look at Continuous Integration.
I've started using Hudson as CI server and I am not regretting it!
Related
I've got a PHP project to test. I've already (almost) finished coding the project and I didn't have any testing plan at the start. Being new to PHP, I thought I'd finish the coding and start testing afterwards. As unit testing needs to be done as you go along, what is my other alternative for a test? I want something like a complete system test rather than unit by unit.
I've got around 30 PHP script files, 5 of which are class scripts.
NB It needs to be a formal way of testing, rather than trying to hack it.
I believe your aim behind the test is more "academic" than practical. So testing by friends doesn't really help much. Having said that you have already completed coding the project so if you wanted to unit test using PHPUnit or SimpleTest you will have to go back and start coding the unit test cases.
I think the best option here would be to create a "formal" test document. There are tons of resources out there for how to do this. Basically it will have:
Url
Action
Expected Result
Observed Result
If your expected and observed results match, then that test case is passed.
You can create as many test cases as you want for your project based on these basic guidelines.
Testing has it's different role in SDLC. There are various aspect on which testing could be done. Like:
Smoke Testing
Functionality Testing
System Testing
Retesting
Regration Testing
Preformance Testing
and many more testing technique are there for testing a web application.
Just upload the project and ask someone to test it.You have to test it or get it tested.There is no other way
You can also run a simple scanner(like burp or acunetix) to test whether website can be hacked or not.
You can use a web app scanner,
There are some F/OSS out there such as;
Skipfish - CLI Windows/Linux
Netsparker Community Edition - Windows - Light version of a commercial scanner
List of web app security scanners - commercial and free
I do some PHP with Kohana 3 (IDE:Netbeans), and got excited about idea of writing tests for code. It sounds pretty cool thing to do, but i have few complications and worries.
Why using Kohana unittest module in browser is like 5 times faster then running tests in Netbeans or command Line ?
How could i exclude all kohana internal tests? In the PHPUnit .xml configuration file ?
Why when run any test i've got in Netbeans panel two entries for it - one with yellow triangle (it says 'file x skipped'), and entry with normal test result. I do get that double entries for every test, also those native from Kohana. I don't mind but it's strange.
All over the Web i see examples, tutorials and screencasts of PHPUnit with sample classes and methods that add two numbers or displays name or do some other trivial things. I've learnt to do those kind of assertions, but how could i test my code in Kohana? My Models are 90% ORM stuff. Controllers? How? Any 'How-tos' and examples are welcome.
I've seen in Ruby tutorial about Rspec a way to test DB by using testing enviroment Databse and rollbacks after finisning tests. Also user actions like clicking links were simulated. Is it possible with PHPUnit ?
There always has been a lot of discussion on what has to be tested and what has not to be tested. Generally my opinion is that you shouldn't test things that should work, like the database driver and connection, this has little to do with your code. Some then argue that you should be able to test it anyway, but in most environment this isn't an easy thing to do and usually a big hassle.
Generally controller actions should be tested as well as any helpers or modules you've written. Usually one uses the paradigm of a mocking framework to get around the database. The good thing about this is a gigantic speed increase in your testing. There are several PHP mocking frameworks as well I suppose.
Another great thing to keep in mind is that you also have user testing. This cannot be simulated with the kind of tests you write in kohana. For this it is interesting to look at http://seleniumhq.org/
It's probably better to split such a rambling question into multiple SO questions.
No idea. Perhaps there's an invocation overhead for NetBeans to invoke phpunit, compared to apache passing the request to PHP.
That might be possible, or you could find a way to set the following option: --exclude-group kohana
No idea sorry.
AFAIK PHPUnit can't do client-interaction tests. How to do system behaviour testing could be a question on its own.
Here's the setup at our shop:
1 VERY large PHP app (Kohana 2) with
many dev's and lots of infrastructure
Multiple (4-5 and growing) small PHP
apps with 1-2 dev's working on these
Issues:
no testing
no documentation
a fragile and tedious deployment
process
I'm being moved from the single large app over to the side of the house with the multiple smaller apps. The lack of testing and a proper deployment process in our shop makes me nervous that I'll spend more time fixing bugs and deploying fixes than actually writing code for new features.
Solution A:
Introduce PHPUnit and Selenium
Move us over to Phing and Dbdeploy
Problem with A:
Setting up PHPUnit has been relatively easy, but functional testing with Selenium has been a total pain. Our VM's work great for dev, but Selenium pegs the needle, plus a few simple tests take forever. I don't doubt I could get all of these technologies playing well together, but it all seems like a mess and the complexity of these working together seems fragile.
Solution B:
Switch to Rails
Use integrated testing and/or
Rspec/Cucumber (integration of the
latter seems simple)
Use integrated DB migrations
Use Capistrano for deployments
Based on the major issues of testing, I began to look into Rails. Based on the nature of these other sites we manage, I think Rails may be a good solution. Built-in testing, great community, lots of great tools, and fast development.
Problem with B:
Every app we have right now is on Kohana 2 (PHP framework) and no one in the organization knows Rails. The downside to introducing a new technology would be fracturing the teams. If I migrate the sites to Rails, then get hit by a bus, we're kind of screwed.
Bottom line:
Based on our pain points (deployments, testing, documentation, DB migrations), is it worth the cost to switch to Rails? Or should we stay put in Kohana and continue to try and get the other tools built?
Any suggestions? Anyone gone through anything similar? Management has already told me that they're open to hearing about Rails and simply want to use the best tool possible -- whatever that is. Our lead architect, however, will need some convincing if I decide to switch frameworks on our smaller projects.
I think you might get a lot of different answers depending on what type of developers we are.
Personally, I think you should stick to PHP but move to Kohana 3. And then introduce better management and development techniques (documentation, testing, etc). Just my opinion, not really a solution.
There are a lot of factors that will influence your decision.
If you switch to rails, keep in mind that it will take a while for you and your team to learn the framework/language and can easily slow adding features for a while. It really just depends on your team, time constraints and many other factors.
Maybe try 1 of the small projects with rails and see if you and your team actually like rails (I dont).
The answer will be different for every team. I would have a group meeting and discuss the pros and cons of both decisions. Then have a vote.
Moving to Rails (or any other language) will probably cost you a lot in at least one of the following ways:
Time investment. Your entire team will have to learn Rails, while continuing to work with PHP.
Server cost. You'll have to a separate set of servers for Rails and PHP.
Human cost. Some of your team may not want to switch and you will have to hire new people.
My recommendation is that you look into phpUnderControl and start commenting your code. You don't have to write piles of documentation, but make sure that every method is commented.
And finally, my completely biased opinion is that you should give Kohana 3 a try. Even if you can't migrate your existing apps, it could save you some frustration with new apps.
Which unit testing framework do you use for Symfony?
Lime or PHPUnit? What are the pros and cons of using them?
In my opinion, here are a few things that come to my mind :
PHPUnit is more integrated with other tools, like, for instance,
Selenium (PHPUnit can use it to open true real browsers to test your site)
phpUnderControl for continuous-integration
PHPUnit works well with Xdebug, to generate code-coverage reports
PHPUnit is more widely used ; which probably means more support
But note I don't work with symfony, nor lime...
Still, I've never heards anyone speak about it, except for those working with symfony -- that not a good thing, for the day you'll have to work with another framework (yes, this happens ^^ )
One thing that's not in PHPUnit :
"false" browser (being able to do HTTP Requests to the application, without using Selenium to open a real browser)
But some frameworks (Zend Framework does, with it's Zend_Test component) integrate with PHPUnit (or use it), while allowing injection of data into the MVC and fetching of the response, without having to issue any HTTP Request.
I don't know if symfony allows that, but that's a nice thing with ZF/PHPUnit ^^
(Yes, not a symfony-specific answer ; but of the things I said must still be valid with that framework)
Lime is a much more simple testing framework, which can be a good or a bad thing depending on how you want to use it.
The symfony library itself uses its own testing framework, Lime, to test its code base. From the symfony book:
It is based on the Test::More Perl
library, and is TAP compliant, which
means that the result of tests is
displayed as specified in the Test
Anything Protocol, designed for better
readability of test output.
I cannot vouch for the statement that the lime framework is "more lightweight" than other PHP testing frameworks as the symfony docs claim, but I do really like that it's built right into your symfony project and works well with the symfony command line tools without any additional configuration. One thing that is especially cool is that the lime tests within symfony are set to run within your "test" environment which has it's own database, symfony cache (which gets cleared out during each test session), and environment variables. This comes in handy when you want to do functional testing (checking server response and your html output in your modules/actions, versus basic unit testing). I also like that lime is super easy to pick up and understand since it's so simple. You also have the ability to put your tests into YAML configuration file rather than write the tests by hand.
Pascal is entirely right that PHPUnit is much more widely used and you'd be able to use it in non-symfony projects. There is even a plugin for it, PHPUnit symfony plugin. My best advice would be to use lime if you just wanted to jump right into writing simple tests while you develop your symfony app. But, if you have the time and hope to use these testing skills outside of the symfony world, or bring in pre-existing PHPUnit tests into your symfony code, it'd be worth your time to check out the plugin and give it a spin.
I'm curious to see how other developers go about testing their web sites. PHP specifically in my case, but this probably spans multiple languages. I've been working on a site for over a year now, and I'd really like to automate a lot of the regression testing I do between versions.
This specific site is in CodeIgniter, so I have some tests for my models. I'd like to move beyond just testing those though. However, this is an issue even non-MVC developers have had to tackle I'm sure.
Edit: I think the functionality that would satisfy a lot of my test desires is the ability to assert that paramters have a specific value at the end of the script processing. In my case a lot of logic is in the controller, and that's the main area I'd like to test.
For actual unit testing without testing the UI, you should just test the functions in the model. Most of your functionality should be in there anyways.
You might want to have a look at Selenium for testing the UI of your site. It can record your actions and play them back, or you can edit the scripting directly.
(source: seleniumhq.org)
Have you tried Fitnesse ?
It helps on creating Acceptance tests. They are specially useful for websites, which doing this kind of tests are a pain.
There are a couple of videos from unclebob inside the webpage too. The good thing is that Fitnesse is not restricted for website testing, so your knowledge about using it can be used with other apps too.
The project I'm working on is a Desktop APP written in c++ that uses Fitnesse tests.
But if you meant unit testing the models (which I think you didn't), they can be create using the phpunit lib. I think the ZEND framework has a similar lib for that.
You might want to check out PHPUnit
http://www.phpunit.de/manual/current/en/
I have started using it on my PHP projects and it's very easy to work with and very powerful. In particular, learn and use mocks:
http://www.phpunit.de/manual/3.0/en/mock-objects.html
Mocking is especially important when unit testing applications that do database operations.
Take a look at TOAST. It's build specially for CodeIgniter. It uses CI infrastructure, so you can run all test tests via a browser and results are displayed back as a web page (HTML). It's very simple to use.
I suggest you test your Controllers as well. Testing model is ok, but model is just the DB storage. Controllers contain all the "business logic" and are the place where most things go wrong.
One of the best ideas I've heard of, as far as testing web apps go, was to create a script that would go over all the pages in the site and check them for differences from the previous scan, letting you accept changes and fix regressions.
Generally speaking, automatic testing of GUI applications (websites are GUI apps) is difficult and usually unnecessary. Unit tests work best with simple libraries.
I use Canoo WebTest. It is the best free web site unit test framework out there. It is entirely scriptable with XML and requires no browser so it can run from a build server.
We modified Waiter (Ruby). It plays back "scripts" of URLs and Form Filling to IE and we have added a script "command" to take a Screen Capture; the screen capture image is compared against a Known-Good-Image (i.e. a Master Image) and if that image is different it is logged (basically a Web page of such results is prepared) and "a human" does a review of the Master / Test image. Obviously there are two outcomes at that point - "The difference is intentional" or "There is an incorrect change". In the first instance the Master image is replaced with the New Image; in the second we go fix the bug, and the change will be included in the next test run