I was wondering if anyone that has experience in both this stuff can shed some light on the significant difference between the two if any?
Any specific strength of each that makes it suitable for any specific case?
This question is quite dated but as it is still getting traffic and answers I though I state my point here again even so I already did it on some other (newer) questions.
I'm really really baffled that SimpleTest still is considered an alternative to phpunit. Maybe i'm just misinformed but as far as I've seen:
PHPUnit is the standard; most frameworks use it (like Zend Framework (1&2), Cake, Agavi, even Symfony is dropping their own Framework in Symfony 2 for phpunit).
PHPUnit is integrated in every PHP IDE (Eclipse, Netbeans, Zend Stuide, PHPStorm) and works nicely.
Simpletest has an eclipse extension for PHP 5.1 (a.k.a. old) and nothing else.
PHPUnit works fine with every continuous integration server since it outputs all standard log files for code coverage and test reports.
Simpletest does not. While this is not a big problem to start with it will bite you big time once you stop "just testing" and start developing software (Yes that statement is provocative :) Don't take it too seriously).
PHPUnit is actively maintained, stable and works great for every codebase, every scenario and every way you want to write your tests.
(Subjective) PHPUnit provides much nicer code coverage reports than Simpletest
With PHPUnit you also get these reports inside your IDE (Netbeans, Eclipse, ...)
Also there are a couple of suggestings for a web interface to phpunit tests.
I've yet to see any argument in favor of SimpleTest. It's not even simpler to install since PHPUnit is available via pear:
pear channel-discover pear.phpunit.de
pear install phpunit/PHPUnit
and the "first test" looks pretty much the same.
As of PHPUnit 3.7 it's even easier to install it by just using the PHAR Archive
wget http://pear.phpunit.de/get/phpunit.phar
chmod +x phpunit-3.7.6.phar
or for windows just downloading the phar and running:
php phpunit-.phar
or when using the supported composer install ways like
"require-dev": {
"phpunit/phpunit": "3.7.*"
}
to your composer.json.
For everything you want to test PHPUnit will have a solution and you will be able to find help pretty much anywhere (SO, #phpunit irc channel on freenode, pretty much every php developer ;) )
Please correct me if I've stated something wrong or forgot something :)
Overview of PHP Testing tools
Video: http://conference.phpnw.org.uk/phpnw11/schedule/sebastian-bergmann/
Slides: http://www.slideshare.net/sebastian_bergmann/the-php-testers-toolbox-osi-days-2011
It mentions stuff like Atoum which calls its self: "A simple, modern and intuitive unit testing framework for PHP!"
Full disclosure
I've originally written this answer Jan. 2011 where I had no affiliation with any PHP Testing project. Since then I became a contributor to PHPUnit.
I prefer PHPUnit now, but when I started out I used SimpleTest as I didn't always have access to the command line. SimpleTest is nice, but the only thing it really has over PHPUnit, in my opinion, is the web runner.
The reasons I like PHPUnit are that it integrates with other PHP developer tools such as phing (as does SimpleTest), phpUnderControl, and Xinc. As of version 3.0 it has mocking support, is being actively developed, and the documentation is excellent.
Really the only way to answer this question for yourself is to try both out for a time, and see which fits your style better.
EDIT: Phing now integrates with SimpleTest as well.
I could NOT understand how to download and install PHPUnit.
I could, however, easily understand how to install SimpleTest.
(As far as i can remember the instructions for PHPUnit said something along the lines of "install it via PEAR and we won't give any instructions on how to do it any other way")
see:
http://www.phpunit.de/manual/current/en/installation.html
For SimpleTest, just download it and point to it from your code.
So Simpletest won for me.
Half of the mentioned points in the accepted answer are simply not true:
SimpleTest has
the easier setup (extract to folder, include and run)
simply check the folder into version control (try to do that with phpunit nowadays :))
less dependencies and lots of extensions (webtester, formtester, auth)
a good code coverage reporter, which is easy to extend (dots, function names, colors)
a code coverage summary (finally landed in PHPUnit 4.x)
a decent web runner and an ajax web runner, with groups and single file executions
still better diff tool (with no whitespace or newline problems)
an adapter/wrapper to run SimpleTests by phpUnit and vice versa
compatibility PHP5.4+
The downside:
not industry standard (PHPUnit)
not actively maintained
Baphled has a nice article on SimpleTest vs PHPUnit3.
I found SimpleTest was even easier than PHPUnit to set up. Just extract it and you are good to go. A benefit of this is if you are working at more than one machine, since you can store the whole testing framework the same way as your source code, and thereby know that you are using the same framework code. Especially if you modify it in any way.
So, I would say that a strength of SimpleTest is that it is very light weight and portable.
SimpleTest also ships with a very simple HTML GUI, which is quite easy to extend if you want to. As far as I know, PHPUnit does not include a HTML GUI, but there are GUI:s available to download, such as Cool.
Well I made a phpUnit web based UI test case runner and made it available on sourceforge. Uses ajax and has quite cool interface as well if you want to give it a shot check it at sourceforge. The project name is phpunitwebui and the website is http://phpunitwebui.sourceforge.net/
As it has been pointed, it's mostly a preference choice, as both will run the tests you write for it and report back the results.
The Simpletest web UI is very useful, but it can also sometimes get cumbersome. In my current project, I would have had to put more work into a system to make my application (an API) work with the web interface (set up apache correctly, copy files to the public_html root, etc.) than it would have been to simply run phpunit from the eclipse workspace. Therefore I choose PHPUnit. Also, the use of PEAR was a big plus since you don't need to manually track updates. Simply run pear upgrade once in a while and PHPUnit will be kept up-to-date.
This is from the point of view of a very casual PHP developer:
It took me two days to grasp PHPUnit, mostly trying to debug under Eclipse that I finally gave up.
It took me two hours to setup Simpletest including debug under Eclipse.
Maybe I will find the shortfalls of Simpletest in the future but so far it does well what I need: TestClasses, Mock objects, test-code debugging, and web interface for a quick snapshot of the situation.
Again: This from the point of view of a very casual PHP user (not even developer :-)
I haven't checked Simple Test for a while, last time it had an eclipse plugin, which is a major factor for me, but it hasn't been updated for a long time.
Sebastian Bergmann is still very actively working on PHPUnit, but it still lacks a good plugin for eclipse - but it is included for the new Zend Studio.
This question is old, but I want to add my experience: PHPUnit seems to be the standard now, but if you work with a legacy system that uses lots and lots of global variables, you may get stuck from the get go. It seems like there is no good way to do tests with global vars in PHPUnit, you seem to have to set your variables via $GLOBALS which is NO GOOD if you have tons of files setting global variables everywhere. OK some may say that the problem is in the legacy system but that doesn't mean we cannot do tests on such system. With SimpleTest such thing is simple. I suppose if PHPUnit allows us to include a file globally, not within any class/function scope then it wouldn't be too much of an issue as well.
Another promising solution is http://www.enhance-php.com, looks nice :)
when there are thousands functions to test at one go, phpunit is way to go, simple test is falling short as it web based.
I am still using simple web to for small scale test .
But both are good
Related
I'm trying to build a full test suite for Joomla and others. There are some docs around but they're quite limited and I'm wondering somebody did wrote already some scripts. By 'full' I do mean really a complete testing :
1. installing an extension on Windows, Max, Linux (Vagrant, VM)
2. configuring the extensions with the CMS option panels
3. doing things in a full Ajax application
I guess I'll end up with lots of bash-scripts, triggering other scripts within a virtual setup, right ?
I must admit, I am not really familiar with all this testing frameworks and products and I'd be already very happy to get pointed to anything. Doing pure unit tests doesn't seem enough given the nature of such systems (namespace collision, interfering plugins,...)
Thank you for any hints
I am not familiar with wordpress and joomla but those are just PHP code, so using PHPUnit can be suitable. With PHPUnit you are not only doing unit testing but you can also do other kinds of tests (it depends on how much time you are willing to spend on testing but I would say that it can cover pretty much any aspects)
As for front-end testing, there are several choices
Using selenium to write tests
Using one of the many headless testing frameworks out there (for instance, casperjs in Javascript, Watir for Ruby etc...)
For setting up virtual environment, apart from vagrant you can check docker
There are many programs you can use to do the normal interaction tests; if you want to handle logins, some scripting will be necessary to grab the token.
But for a full integration test including ajax I guess you'd be better off with a headless browser, take a look at this Real headless browser where the same requirement is discussed in detail
I really appreciate Python's nose utility, which is a testing framework extension that watches your source code for changes and runs the unittests anytime something is modified.
I was wondering whether anything such as this existed for PHP / as a PHPUnit extension? I'm fully aware that this sort of thing wouldn't be difficult to conjure up in bash using watch, but I work across platforms and would ideally like to avoid writing separate implementations.
You can use watchr for this.
PHPUnit Watchr config
Another PHPUnit Watchr config
Guide on how to combine Watchr, Growl and PHPUnit
This question already has answers here:
Closed 11 years ago.
Possible Duplicates:
Setting up a deployment / build / CI cycle for PHP projects
Recommended server for Continuous Integration for PHP Project
Hello, guys!
Recently I faced the need for continuous integration for some of my projects. And, to my surprise, there exist many tools for this purpose like CruiseControl with its plugin PhpUnderControl, Xinc (written in PHP), Hudson with lots of functionality etc. Also I studied the matter and, as far as I understood, installing and configuring such a tool takes quite a long time. Thus, it would be very dissapointing to spend much time for setting everything up and get to know that the tool lacks some crucial functionality. I address to those who have some experience with this matter and can give a piece of reasonable advice. Thank you!
Updated 26.04.2017: Years passed, I still use CI, but abandoned Jenkins in favor of Gitlab CI: it seems simpler, CI jobs can be configured directly in the project repository. And along with Docker, I believe, it's a very powerful solution that can be tailored for almost any kind of project.
Jenkins (formerly Hudson) is by far the best solution. It has a nice PHP integration (see http://jenkins-php.org/)
I can't give you a definitive answer as to what tools are the best (it's a very subjective question, and even if there was a clear leader, it may not remain the best in the future), but I'll do the next best thing and tell you what we're running:
We have a Hudson installation which runs a set of Phing scripts. The Phing scripts run PHPUnit (and a few other tools). Some of the PHPUnit tests are stand-alone unit tests; others run Selenium and test the software in a variety of browsers.
From what I gather, this seems to be a fairly common setup, and it works well for us.
That said, we use more than just PHP in-house - we have Java and .Net teams as well. Some of the tools may have been chosen in order to help us maintain a consistent interface for our continuous integration across the various teams.
Hope that helps.
PHPUnderControl (Plugin for CruiseControl) and Jenkins-PHP (formerly known as Hudson-PHP) are the two big players. I recently switched from phpuc to jenkins. Its more flexible.
Cruise Control with PHP Under Control(phpuc) is one of the best solution for this.
http://cruisecontrol.sourceforge.net/
I'm in charge of at least one large body of existing PHP code, that desperately needs tests, and as well I need some method of checking the production site for errors.
I've been working with PHP for many years, but am unfortunately new to testing. (Sorry!).
While writing tests for code that has predictable outcomes seems easy enough, I'm having trouble wrapping my head around just how I can test the live site, to ensure proper output.
I know that in a test environment, I could set up the database in a known state... but are there proper methods or techniques for testing a live site? Where should I begin?
[I am aware of PHPUnit and SimpleTest, but haven't chosen one over the other yet]
Unit testing frameworks like PHPUnit are built more for testing the functionality of separate, logical units (i.e. classes), not so much the behaviour of entire live sites. One household name for that is Selenium, a web application testing system. It is my understanding that Selenium tests can in turn be integrated with PHPUnit.
As for the choice between Simpletest and PHPUnit, check out my recent question about PHPUnit vs. SimpleTest - the answers to that question, notably #Gordon's, managed to convince me of PHPUnit.
I second Pekka's suggestion. Also, I strongly suggest using PHPUnit, as it is the de-facto Standard in UnitTesting frameworks in the PHP World.
Another thing you can do is head over to phpqatools.org (edit: this website is no longer active) and use the given tools to analyze your codebase, find dead code, copy and paste, code violations, etc.
Also profile your code with XDebug or Zend Debugger to find out what it is actually running how often. This way you will not only get an idea of which code you should test first (those that runs most often), but also how it performs, which is a good starting point when you wil optimize it after you have written the Unit-Tests.
In addition, check out:
Legacy Code Nightmare and
Theory and practice – migrating your legacy code
PHPUnit Manual
We currently use a hand-rolled setup and configuration script and a hand-rolled continuous integration script to build and deploy our application. I am looking at formalizing this somewhat with a third party system designed for these purposes.
I have looked into Phing before, and I get that it's basically like Ant. But, my Ant experience is somewhat limited so that doesn't help me much. (Most of the Java work I have done was just deployed as a jar file).
I have looked into Cruise Control before, and I understand that phpUnderControl is a plug-in for CC. But, Phing says it also works with CC. So I am not clear on the overlap here. Do I need both Phing and phpUnderControl to work with CruiseControl, or are they mutually exlclusive?
What I need exactly is something that can:
Check out source from SVN
Install the database from SQL file
Generate some local configuration files from a series of templates and an ini file
Run all of our unit tests (currently ST, but easy to convert to PHPUnit) and send an email to the dev team if any tests break (with a stack trace of course)
Generate API documentation for the application and put it somewhere
Run a test coverage report
Now, we have just about all of this in one form or another. But, it'd be nice to have it all automated and bundled together in one process.
phing is pretty much ant written in PHP where phpUnderControl adds support for PHP projects to CruiseControl and uses phing or ant on the backend to parse the build.xml file and run commands.
I just set up CruiseControl and phpUnderControl and it's been working great. It checks out my SVN, runs it through phpDocumentor, PHP_CodeSniffer, and PHPUnit whenever we do a check in. Since it's all based off of the build.xml file you can run just about any software you want through it.
I'm sure lots of people will say this by the time I've typed this but...
I know it's not PHP but we're finding Capistrano just the job for this kind of thing. It really is an excellent piece of software.
We've been using Phing, and the cost to set it up has been very low; it's really easy to learn even if you don't know ANT. I've had very bad experiences with CruiseControl (instability - going down randomly) - so I like the simplicity of Phing. Plus, it's easily extensible using PHP (in case you have a custom task that they don't support out of the box).