PhpStorm has some sort of weird behaviour that is driving me crazy. I've got my project setup to have a source and a test directory. The source directory should be for productive code, while the test directory should contain all the phpunit tests.
In those directories, the folder structure is mirrored - if I have a class \foo\Bar,
then there is
src/foo/Bar.php and test/foo/BarTest.php.
I've marked src/ and test/ as source and test directory in PhpStorm, however, every time I want to create a new test suite for a class, PhpStorm defaults to put the test class into the src/ directory next to the class under test.
It's possible to change that, however it get's more annoying when that same directory in test/ does not exist yet. Instead of creating it, PhpStorm will just reject to create that test suite for me.
Maybe I am misunderstanding the concept behind test management in PhpStorm quite a bit - because it just can't be that bad user experience.
Is there something I am doing wrong, or something that I can configure to make the situation less painful?
You are doing nothing wrong. It just does not work this way -- devs coded it to behave in a way it is right now.
Originally IDE used PHPUnit's ability to generate test classes (back then such functionality was part of actual PHPUnit). Then PHPUnit creator(s) have moved this functionality into separate package (phpunit-skelgen) .. and you had to install it manually/separately. Then devs decided to drop phpunit-skelgen support completely and implemented current implementation (which is also more in line with similar routines in other IDEs built on IDEA platform/technologies supported by those IDEs).
There are quite few tickets about changing such behaviour to the same as you described/desired .. but so far it does not look like it's in their priority list...
https://youtrack.jetbrains.com/issue/WI-2850
https://youtrack.jetbrains.com/issue/WI-24358
https://youtrack.jetbrains.com/issue/WI-21890
Subscribe to those tickets (star/vote/comment) to get notified on progress.
As of release 2016.3 PHPStorm now takes places tests in the specified tests directory, providing exactly what you ask for in your question.
Related
I am trying to work what the point of the environments folder is.
Originally I had the idea that you could point the webserver to the different dev and prod folders in the environment folder but after reading up a bit I realise this is not the case.
In Yii 1 you would solve this by just having multiple index.php's i.e.:
index.php
index-local.php
So the question is what benefit does this new environment structure actually give me over the old way?
I've found environments very useful in allowing me to keep a common code base for multiple client projects (based on Yii App Advanced) and setting up a different environment for each specific client, keeping their custom code private and separate.
To do this I store the environments folder in a separate git repo from the rest of the code and pull down the relevant folder on a client/project basis.
This lets me use a base common code for all projects and add/override any file for a specific client or project whilst still allowing separate dev/prod config settings. If the client uses other developers too, they are also catered for. In this way, only common code I choose will be shared amongst clients and custom code will be kept private.
I've also moved the composer.json file into the environments folder so I can pull in different extensions per client/project keeping those private too.
That init command can be a very powerful tool and you don't have to limit yourself to the template provided by the core developers.
If you don't need environments, then don't use them, but I assure you some people will find it very useful.
Yii2 documentation in WIP, but you should read this :
https://github.com/yiisoft/yii2/blob/master/docs/guide/apps-advanced.md#configuration-and-environments
You need to use yii init command to switch between these environments.
EDIT :
This new environment feature is more than just use different config file. You can use different folder structure, different entry script...etc
Personnaly I won't use this feature, I don't need it (I will use a different entry script as with Yii 1), but I think this is not useless.
I think you didn't get the real purpose of environments introduced in Yii2.
I'll try to explain what was the main purpose of adding environments to yii from the developers point of view on an example and hope you will really appreciate its usefulness.
Let's suppose for a moment that you are a team of developers (e.g. 5-7 person) working on mid-to-large project implemented in Yii. To effectively work on that project your team decides to use some CVS or SVN (e.g. GIT) and keep all the files of the project in repository in cloud for the whole team. That's de facto standard while working on mid-to-large projects in teams and nobody will resist that it's the only comfortable and easy way.
Ok, now let's suppose you use Yii 1.x or Yii2 with the approach of different entry scripts to differentiate between local (development) and production environments to connect to db or set some other environment specific configs. Everything is ok and working. But suppose your team members implemented something new on the project and you check out repository to work on updated version and you suddenly find out that your local config file (in this case entry script with config) is overwritten with other team member's file who pulled the changes to repository (because each of you is using your local machine db with other database name or OS, or config, or simply because your team uses one local development server db, but you are on vacation and can't use anything except your local machine).
So generally Yii2 environment adds more flexibility for using different environments each with it's own specific configurations while using also general (common) configs when working in teams on mid-to-large projects hence why the example in guide is given on advanced app project.
Surely you can overcome everything stated above with some solutions or .gitignore which is used by default to overcome the problem stated in Yii2 with environments. But:
Why bother if everything is already done?
and
It was just one little example of usefulness of Yii2 environments. More depends on the project and your imagination.
Overall Yii2 is great product. Not only it adds many new features to already great framework, but it also is more robust and flexible than Yii 1.x (despite the fact that Yii 1.x was already very robust).
As for Laravel or any other PHP framework, it really depends... Everyone will find his/her own favorite.
For those who are tired of copying files around, I created a useful script that you can run in background to keep the files in sync on your dev environment:
File sync-env-files.sh
#!/bin/bash
ENVIRONMENT_DIR="/var/www/example.com/environments/dev/"
DIR="/var/www/example.com/"
while [ true ]; do
for envFile in `find $ENVIRONMENT_DIR -type f`
do
file=${envFile/$ENVIRONMENT_DIR/$DIR}
if [ `stat -c "%Y" $file` -gt `stat -c "%Y" $envFile` ]; then
#echo "copying modified file $file to $envFile"
/bin/cp -f $file $envFile
fi
done
sleep 2
done
Then run the script in background or add to cron with flock
nohup server/sync-env-files.sh >/dev/null 2>&1 &
I would like to mention in addition to #AngelCoding, since this question still gets seen, that I use the environments folder lots now and definitely see the point of it.
The very first things I do in any open source project is create one project for the code base on GitHub and then another, private, one on Bitbucket for the configuration, in other words the environments folder.
Having this folder has made it a lot easier for me to separate my configuration into a private repository.
So the environments folder has a lot of uses and really helps to separate configuration for easier usage even if it does not seem like it initially.
I am writing a ZF application. I had some abstract parent classes in a library directory. Netbeans could generate test skeletons for the child classes no problems, and I could run the tests no problem.
Later on I decided to move the abstract parents out of the library and in to the application directory (to improve readability). I updated my application code accordingly, and it runs no problem.
However, now when I use netbeans to generate test skeletons for the child classes, it gives a fatal error saying it can not find the parent. I then constructed a test class manually and ran it from the command line, and PHPUnit gave the same error.
What do I need to do to get this pathing working correctly? In the PHPUnit bootstrap I tried adding the Application directory to the include path, and registering Application as a namespace with the Zend Autoloader. I don't think this is a recommended practice, and it failed anyhow...
I really do struggle with path issues, finding files, etc...
Any assistance is much appreciated.
OK, so I can move the classes from the library to the core code directories, and the core code finds them no problems because they conform to the standard naming conventions autoladed by ZF.
Other applications (e.g. PHPUnit) won't be able to find them though. I could get around this by setting their autoloading the same ZF. Alternatively, I could incorporate required files with require_once (which would negate the use of autoloading in the first place).
So, it seems that my best shot is to put the shared classes back in the library. My original PHPUnit bootstrap adds the library prefix to the ZF autoloader, so I'm good to go.
All-in-all, I've just learned another good use for libraries.
[EDIT:] Another Alternative...
My test classes are boostrapping Zend Application which sets up the autoloading for PHPUnit. The only drawback doing it this way is that PHPUnit (via my netbeans IDE) can't create the skeleton tests for me. I guess I'll have to live with that.
How do you manage your PHPUnit files in your projects?
Do you add it to your git repository or do you ignore them?
Do you use #assert tag in your PHPdocs codes?
Setup
I'm not using php currently, but I'm working with python unit testing and sphinx documentation in git. We add our tests to git and even have certain requirements on test passing for pushing to the remote devel and master branches (master harder than devel). This assures a bit of code quality (test coverage should also be evaluated, but thats not implemented yet :)).
We have the test files in a separate directory next to the top-level source directory in the directories where they belong to, prefixed with test_, so that the unit testing framework finds them automagically.
For documentation its similar, we just put the sphinx docs files into their own subdirectory (docs), which is in our case an independent git submodule, which might be changed in the future.
Rationale
We want to be able to track changes in the tests, as they should be rare. Frequent changes indicate immature code.
Other team members need access to the tests, otherwise they're useless. If they change code in some places, they must be able to verify it doesn't break anything.
Documentation belongs to the code. In case of python, the code directly contains the documentation. So we have to keep it both together, as the docs are generated from the code.
Having the tests and the docs in the repository allows for automated testing and doc building on the remote server, which gives us instantaneous updated documentation and testing feedback. Also the implementation of “code quality” restrictions based on test results works that way (its actually more a reminder for people to run tests, as code quality cannot be checked with tests without looking at test coverage too). Refs are rejected by the git server if tests do not pass.
We for example require that on master, all tests have to pass or be skipped (sadly, we need skipped, as some tests require OpenGL, which is not available on headless), while on devel its okay if tests just “behave like expected” (i.e. pass, skip or expected failure, no unexpected success, error or failure).
Yes, to keeping them in git. Other conventions I picked up by looking at projects, including phpunit itself. (A look at the doctrine2 example shows it seems to follow the same convention.)
I keep tests in a top-level tests directory. Under that I have meaningfully named subdirectories, usually following the main project directory structure. I have a functional subdirectory for tests that test multiple components together (where applicable).
I create phpunit.xml.dist telling it where to find the tests (and also immediately telling anyone looking at the source code that we use phpunit, and by looking at the xml file they can understand the convention too).
I don't use #assert or the skeleton generator. It feels like a toy feature; you do some typing in one place (your source file) to save some typing in another place (your unit test file). But then you'll expand on the tests in the unit test files (see my next paragraph), maybe even deleting some of the original asserts, and now the #assert entries in the original source file are out of date and misleading to anyone looking at just that code.
You have also lost a lot of power that you end up needing for real-world testing of real-world classes (simplistic BankAccount example, I'm looking at you). No setUp()/tearDown(). No instance variables. No support for all the other built-in assert functions, let alone custom ones. No #depends and #dataProvider.
One more reason against #assert, and for maintaining a separate tests directory tree: I like different people to write the tests and the actual code, where possible. When tests fail it sometimes points to a misunderstanding in the original project specs, by either your coder or your tester. When code and tests live close together it is tempting to change them at the same time. Especially late on a Friday afternoon when you have a date.
We store our tests right with the code files, so developers see the tests to execute, and ensure they change the tests as required. We simply add an extension of .test to the file. This way, we can simply include the original file automatically in each test file, which may then be created with a template. When we release the code, the build process deletes the .test files from all directories.
/application/src/
Foo.php
Foo.php.test
/application/src/CLASS/
FOO_BAR.class
FOO_BAR.class.test
require_once(substr(__FILE__, 0, -5)); // strip '.test' extension
I am currently going to start from scratch with the phpunit tests for a project. So I was looking into some projects (like Zend) to see how they are doing things and how they organizing their tests.
Most things are pretty clear, only thing I have some problems with is how to organize the test suites properly.
Zend has an AllTests.php from which loads others test suites.
Tough looking at the class it is useing PHPUnit_Framework_TestSuite to create a suite object and then add the other suites to it, but if I look in the PHPUnit docs for organizing tests in PHPUnit versions after 3.4 there is only a description for XML or FileHierarchy. The one using classes to organize the tests was removed.
I haven't found anything that this method is deprecated and projects like Zend are still using it.
But if it is deprecated, how would I be able to organize tests in the same structure with the xml configuration? Executing all tests is no problem, but how would I organize the tests (in the xml) if I only wanted to execute a few tests. Maybe creating several xmls where I only specify a few tests/test suites to be run?
So if I would want to only test module1 and module2 of the application, would I have an extra xml for each and defining test suites only for those modules (classes used by the module) in it. And also one that defines a test suite for all tests?
Or would it be better to use the #group annotation on the specific tests to mark them to be for module1 or module2?
Thanks in advance for pointing me to some best practices.
I'll start of by linking to the manual and then going into what I've seen and heard in the field.
Organizing phpunit test suites
Module / Test folder organization in the file system
My recommended approach is combining the file system with an xml config.
tests/
\ unit/
| - module1
| - module2
- integration/
- functional/
with a phpunit.xml with a simple:
<testsuites>
<testsuite name="My whole project">
<directory>tests</directory>
</testsuite>
</testsuites>
you can split the testsuites if you want to but thats a project to project choice.
Running phpunit will then execute ALL tests and running phpunit tests/unit/module1 will run all tests of module1.
Organization of the "unit" folder
The most common approach here is to mirror your source/ directory structure in your tests/unit/ folder structure.
You have one TestClass per ProductionClass anyways so it's a good approach in my book.
In file organization
One class per file.
It's not going to work anyways if you have more than one test class in one file so avoid that pitfall.
Don't have a test namespace
It just makes writing the test more verbose as you need an additional use statement so I'd say the testClass should go in the same namespace as the production class but that is nothing PHPUnit forces you to do. I've just found it to be easier with no drawbacks.
Executing only a few tests
For example phpunit --filter Factory executes all FactoryTests while phpunit tests/unit/logger/ executes everything logging related.
You can use #group tags for something like issue numbers, stories or something but for "modules" I'd use the folder layout.
Multiple xml files
It can be useful to create multiple xml files if you want to have:
one without code coverage
one just for the unit tests (but not for the functional or integration or long running tests)
other common "filter" cases
PHPBB3 for example does that for their phpunit.xmls
Code coverage for your tests
As it is related to starting a new project with tests:
My suggestion is to use #covers tags like described in my blog (Only for unit tests, always cover all non public functions, always use covers tags.
Don't generate coverage for your integration tests. It gives you a false sense of security.
Always use whitelisting to include all of your production code so the numbers don't lie to you!
Autoloading and bootstrapping your tests
You don't need any sort of auto loading for your tests. PHPUnit will take care of that.
Use the <phpunit bootstrap="file"> attribute to specify your test bootstrap. tests/bootstrap.php is a nice place to put it. There you can set up your applications autoloader and so on (or call your applications bootstrap for that matter).
Summary
Use the xml configuration for pretty much everything
Seperate unit and integration tests
Your unit test folders should mirror your applications folder structure
To only execute specif tests use phpunit --filter or phpunit tests/unit/module1
Use the strict mode from the get go and never turn it off.
Sample projects to look at
Sebastian Bergmanns "Bank Account" example project
phpBB3 Even so they have to fight some with their legacy ;)
Symfony2
Doctrine2
Basic Directory Structure:
I have been experimenting with keeping the test code right next to the code being tested, literally in the same directory with a slightly different file name from the file with the code it is testing. So far I am liking this approach. The idea is you don't have to spend time and energy keeping the directory structure in sync between your code and your test code. So if you change the name of the directory the code is in, you don't then also need to go and find and change the directory name for the test code. This also causes you to spend less time looking for the test code that goes with some code as it is right there next to it. This even makes it less of a hassle to create the file with the test code to begin with because you don't have to first find the directory with the tests, possibly create a new directory to match the one you are creating tests for, and then create the test file. You just create the test file right there.
One huge advantage of this is it means the other employees (not you because you would never do this) will be less likely to avoid writing test code to begin with because it is just too much work. Even as they add methods to existing classes they will be less likely to not feel like adding tests to the existing test code, because of the low friction of finding the test code.
One disadvantage is this makes it harder to release your production code without the tests accompanying it. Although if you use strict naming conventions it still might be possible. For example, I have been using ClassName.php, ClassNameUnitTest.php, and ClassNameIntegrationTest.php. When I want to run all the unit tests, there is a suite that looks for files ending in UnitTest.php. The integration test suite works similarly. If I wanted to, I could use a similar technique to prevent the tests from getting released to production.
Another disadvantage of this approach is when you are just looking for actual code, not test code, it takes a little more effort to differentiate between the two. But I feel this is actually a good thing as it forces us to feel the pain of the reality that test code is code too, it adds its' own maintenance costs, and is just as vitally a part of the code as anything else, not just something off to the side somewhere.
One test class per class:
This is far from experimental for most programmers, but it is for me. I am experimenting with only having one test class per class being tested. In the past I had an entire directory for each class being tested and then I had several classes inside that directory. Each test class setup the class being tested in a certain way, and then had a bunch of methods each one with a different assertion made. But then I started noticing certain conditions I would get these objects into had stuff in common with other conditions it got into from other test classes. The duplication become too much to handle, so I started creating abstractions to remove it. The test code became very difficult to understand and maintain. I realized this, but I couldn't see an alternative that made sense to me. Just having one test class per class seemed like it would not be able to test nearly enough situations without becoming overwhelming to have all that test code inside one test class. Now I have a different perspective on it. Even if I was right, this is a huge dampener on other programmers, and myself, wanting to write and maintain the tests. Now I am experimenting with forcing myself to have one test class per class being tested. If I run into too many things to test in that one test class, I am experimenting with seeing this as an indication that the class being tested is doing too much, and should be broken up into multiple classes. For removing duplication I am trying to stick to simpler abstractions as much as possible that allows everything to exist in one readable test class.
UPDATE
I am still using and liking this approach, but I have found a very good technique for reducing the amount of test code and the amount of duplication. It is important to write reusable assertion methods inside the test class itself that gets heavily used by the test methods in that class. It helps me to come up with the right types of assertion methods if I think of them as internal DSLs (something Uncle Bob promotes, well actually he promotes actually making internal DSLs). Sometimes you can take this DSL concept even further (actually make a DSL) by accepting a string parameter that has a simple value that refers to what kind of test you are trying to perform. For example, one time I made a reusable asssertion method that accepted a $left, $comparesAs, and a $right parameter. This made the tests very short and readable as the code read something like $this->assertCmp('a', '<', 'b').
Honestly, I can't emphasize that point enough, it is the entire foundation of making writing tests something that is sustainable (that you and the other programmers want to keep doing). It makes it possible for the value that tests add to outweigh what they take away. The point is not that you need to use that exact technique, the point is you need to use some kind of reusable abstractions that allow you to write short and readable tests. It might seem like I'm getting off topic from the question, but I'm really not. If you don't do this, you will eventually fall into the trap of needing to create multiple test classes per class being tested, and things really break down from there.
I would like to package PHPUnit and various other test dependencies into a phar and put that into svn. This way I can run phpunit on any client machine without needing pear. Can this be done?
Current status:
Work on a phpunit.phar has started in the phpunit repo but the generated phar is not stable and not feature complete.
If it gets there there will be official releases
Original answer:
If you can I'll give you 500 rep, a 100 Bucks and my first born.. well no.. just the first two.
To be serious:
I've nagged the creator of PHPUnit about this topic on at least 3 conferences now and well.. it doesn't seem like it's possible.
There are a couple of issues with that. First off PHPUnit spawns new php processes for test isolation. The problem with this is that a phar can't tell which php executable called it. So if you start phpunit with a custom compiled version it would use the "default" php installed to spawn the isolated tests.
Secondly as far as i know and have been told it's not possible to put static files like images and css in a phar. Which makes generating code coverage a lot harder. That would require some work on that part.
There are other issues i can't recall exactly recall right having to do with xDebug being able to provide code coverage for phars (and phpunit relying on not generating overage for it's own code and so) and others things.
There once was a phar but from my understanding that just doesn't work out with the current state of phpunit and never really worked completly.
I'm not saying it can't be done just that various people already have given up on creating a phpunit.phar including the guy how develops it. (That is just my impression, i of course can't speak for Sebastian here and might be completely wrong, take this as a little disclaimer)
Putting PHPUnit into SVN
You don't have to build a .phar to do so!
For my company I maintain a svnd version of PHPUnit. It's not the recommended way of using it but it works without much issues!
Follow the "using from a git checkout" instructions on the phpunit github site. You then need to put those files into your php include path and it works.
My suggestion would be to create a custom phpunit.sh that modifies the include path and then calls the original phpunit.sh passing along all arguments. It's a little bit of work but it works quite well and it is a hell of a lot easier than creating a phar archive :)
From the new PHPUnit page:
We distribute a PHP Archive (PHAR) that contains everything you need in order to use PHPUnit. Simply download it from here, make it executable, and put it into your $PATH, for instance......