I am doing some fairly complex unit tests using PHPUnit. In these tests some files are being generated in temp dirs. After test is finished all this get's wiped. Is there a way to say to framework to leave generated content untouched?
There are 2 ways you could achieve this. Without knowing what exactly clears those files, my best bet is to subclass PHPUnit\Framework\TestCase and implement tearDown or tearDownAfterClass there (and have the relevant test cases subclass that instead), or alternatively by using register_shutdown_function in your bootstrap script.
The tearDown/shutdown method could simply rename the temp dir and mkdir a new one so there'll be nothing to clear, but it's still best to not have those files cleared in the first place. If that code sits inside your vendor/ directory, it's still possible to modify those files.
Related
I'm writing some unit tests (PHPUnit) for a project that will be used by another project. The project that I am testing has a class that will include files that will be present only in the other project.
Now how do I write a unit test for this class without having the files that it needs to include available? Would you recommend setting up a complete testing project with stub files (the file in question is a file that contains some settings) and running all the unit tests there? Or should I create directories and files using for example the setUp() method?
Edit:
To be more specific, I have a base project A, which is a website. I have a project B, which contains a class that generates a form. The form class will be installed in project A using Composer. The form class will in project A check for the existence of a dir with a settings file. If it exists, it will include it and load the settings in it. To test the form class, do you think I should create a project C (just for testing) which installs project B and in which I set up the directory with the settings file for testing? Or do you think it's a better way to go to create the directory with the settings file in project B itself? The latter seems a bit odd to me, as I don't want to have all this unit testing material available in project A when I composer-install project B in it.
Yes!! To everything:
Now how do I write a unit test for this class without having the files that it needs to include available?
You can create test fixtures. What is the specification your programming your code to? As you develop your code are you using a test file? reading documentation? Given a specification by the client? You could create an input file that fulfills the spec and provide it to your function.
Would you recommend setting up a complete testing project with stub files (the file in question is a file that contains some settings) and running all the unit tests there?
Yes, but only the absolute smallest amount of tests necessary to programmatically ensure that the functionality that you are saying you're providing your client is being delivered! If the function is provided a file path and parses it and then loads the settings I feel like there needs to be at least a couple test cases that ensure that a file can be read from the operating system. Having a fixture file that your test loads to verify that the file opening logic is correct should be a pretty reliable test. I think the tricky part is minimizing the number of these tests.
For example, if you need to test your settings parsing logic, it may seem easy to create a settings file and have your test load and parse that settings file. For a couple tests this will be plenty fast and reliable. But as your test suite grows it becomes orders of magnituded slower than in memory tests. Instead of going through the file system to test the settings parsing logic, you could excercise a settings parsing function directly by providing it with the string contents of the file. This way you could build a settings string in memory, in your test function and pass it into the parsing function, avoiding any file system reads. If the file is too large and expects a file like object so that it can incrementally read data from the file system you could create an in memory stub object which you could use.
I'm not sure of the php api for that but if there is like a readline method you could create a fake file object provide it with the PHP file api and create your fake settings file in memory during the test, also avoiding the file system.
Or should I create directories and files using for example the setUp() method?
What is the benefit of this over having a static file? In my experiences minimizing test complexity and test logic is huge for test suite maintenance and performance.
You could create/delete the files and directories during setup and tear down, but be aware that this can make your tests flakey as sometimes files can not be deleted, e.g. due to lock issues or when a test fails and tear down is not called.
A safer way to interact with "your local filesystem" is vfsStream which will never actually write to disk. The documentation contains a nice basic example both for using the filesystem directly or using vfsStream to mock the filesystem access: https://github.com/mikey179/vfsStream/wiki/Example
I'm new to php and inherited a website project with hundreds of pages, all procedural (when I do a text search of the files, there isn't even a function definition anywhere). I'm coming from the c# and Java worlds. I'm looking for a way to incrementally add OOP. (They want me to update the front end and I am trying to convince them of fixing the backend at the same time and they don't want to use a framework (dammit)).
While looking into autoloader... Well, here's my understanding. It's a method of registering folders where classes are stored and when you instantiate a class, trait, etc. it searches the folder based on the class/filename/namespace and loads the appropriate definitions.
I have a few questions:
Does autoloader search the folder and load the appropriate definitions on every page lifecycle (or does it cache them)?
Pre-loading:
Is there a way to use autoloader, or some alternative, to pre-load ALL class definitions into memory and make them available across all sessions?
If so, when updating class files, how would I tell this mechanism to reload everything to memory when I make changes to class files?
UPDATE TO QUESTIONS:
Thank you both for your answers and it helps a little, but... I do have a bad habit of posing the wrong question(s) on StackOverflow.
The thing I want to avoid is slowing down pages by adding classes. So let's say I add a library and register the paths with autoloader. A page instanciates a class with multiple dependencies. Let's say that the dependency graph includes 15 files. For each request lifecycle, the server loads the page and 15 other files just on that one page.
Since I am coming from compiled languages, I feel a little strange not loading these classes into memory. All the classes together should not be over say 5MB.
(Or maybe I should just create a RAM Disk and copy all the files in there on boot and just have a symlink?)
Auto loaders in PHP are lazy. When PHP encounters a the use of a class it doesn't know about, it will ask the registered autoloader (or chain of autoloaders) to go find it. It's the autoloader's job to figure out where to get the file the class is defined in and include it. Having some sort of convention for naming your classes and organizing your class files is key to having a useful autoloader, and several conventions have arisen in the PHP community, such as PSR-4.
Does autoloader search the folder and load the appropriate definitions on every page lifecycle (or does it cache them)?
The autoloader(s) is(are) called on every request, but only when the need to autoload a class arises.
Pre-loading: Is there a way to use autoloader, or some alternative, to pre-load ALL class definitions into memory and make them available across all sessions?
I don't believe so, but as the number of classes grow, this becomes more and more wasteful.
Welcome to the wonderful[citation needed] world of legacy PHP, I highly recommend you check out Modernizing Legacy Applications In PHP. It's like a strategy guide for getting from Mordor back to the Shire.
I think you may misunderstand the purpose of autoloading. It is simply instructions on what to do when your code calls for a class that PHP doesn't recognize. That's it. The autoloader just calls requires /path/to/classfile so that PHP will see the class.
Does autoloader search the folder and load the appropriate definitions
on every page lifecycle (or does it cache them)?
There is no caching across requests, so if you make a change to file, the next http request will incorporate those changes. It's just as if you changed any other instruction in your script, for example change echo 1 to echo 2
Pre-loading: Is there a way to use autoloader, or some alternative, to
pre-load ALL class definitions into memory and make them available
across all sessions?
There is no need for this. A well written autoloader has instructions for where to find any class, so loading all possible classes ahead of time is wasteful. If you're still running into undefined classes errors, you need to either improve the autoloader or place the class files in accordance with the current autoloader instructions.
If you really want to preload all your classes, use the auto_prepend_file setting in php.ini. The docs say
Specifies the name of a file that is automatically parsed before the
main file
Set it to an initialization script. In that script have something like:
//put all your class files in this folder
$dir = '/path/to/classes/folder';
$handle = opendir($dir);
//require all PHP files from classes folder
while (false !== ($item = readdir($handle))){
$path = $dir.'/'.$item;
if(is_file($path) && pathinfo($path,PATHINFO_EXTENSION)==='php')
require_once $path;
}
This is simplified. There is significant risk in just including all files in any directory into your script so I would not do this. You would also need to adjust this if you want to include files in subdirectories.
Basically, don't do this. Just have a good autoloader.
No one posted what I was looking for but it seems the best route is the OptCache that's prebuilt into php 5.5 and above (my client is using 5.3 so I didn't know about it).
https://github.com/zendtech/ZendOptimizerPlus
The Zend OPcache
The Zend OPcache provides faster PHP execution through opcode caching
and optimization. It improves PHP performance by storing precompiled
script bytecode in the shared memory. This eliminates the stages of
reading code from the disk and compiling it on future access. In
addition, it applies a few bytecode optimization patterns that make
code execution faster.
I am trying to use PHPunit to do testing.
One of the tests I am writing requires scanning a directories sub directories for a certain folder.
I want to write a test that checks to make sure it does not fail if the folder does not exist.
The issue is that the directory it scans may or may not have the folder in it. so I am confused as to how I am meant to do this.
One option I could see is to move everything out of the directory, run the assertion, and then move everything back in, but that seems messy to me. What is the best way to do this?
phpunit actually provides a means to mock the filesystem using vfsStream. This is described in the PHPUnit documentation (specifically look at example 10.19 to see how this is used). vfsStream acts as a wrapper for the filesystem.
How do you manage your PHPUnit files in your projects?
Do you add it to your git repository or do you ignore them?
Do you use #assert tag in your PHPdocs codes?
Setup
I'm not using php currently, but I'm working with python unit testing and sphinx documentation in git. We add our tests to git and even have certain requirements on test passing for pushing to the remote devel and master branches (master harder than devel). This assures a bit of code quality (test coverage should also be evaluated, but thats not implemented yet :)).
We have the test files in a separate directory next to the top-level source directory in the directories where they belong to, prefixed with test_, so that the unit testing framework finds them automagically.
For documentation its similar, we just put the sphinx docs files into their own subdirectory (docs), which is in our case an independent git submodule, which might be changed in the future.
Rationale
We want to be able to track changes in the tests, as they should be rare. Frequent changes indicate immature code.
Other team members need access to the tests, otherwise they're useless. If they change code in some places, they must be able to verify it doesn't break anything.
Documentation belongs to the code. In case of python, the code directly contains the documentation. So we have to keep it both together, as the docs are generated from the code.
Having the tests and the docs in the repository allows for automated testing and doc building on the remote server, which gives us instantaneous updated documentation and testing feedback. Also the implementation of “code quality” restrictions based on test results works that way (its actually more a reminder for people to run tests, as code quality cannot be checked with tests without looking at test coverage too). Refs are rejected by the git server if tests do not pass.
We for example require that on master, all tests have to pass or be skipped (sadly, we need skipped, as some tests require OpenGL, which is not available on headless), while on devel its okay if tests just “behave like expected” (i.e. pass, skip or expected failure, no unexpected success, error or failure).
Yes, to keeping them in git. Other conventions I picked up by looking at projects, including phpunit itself. (A look at the doctrine2 example shows it seems to follow the same convention.)
I keep tests in a top-level tests directory. Under that I have meaningfully named subdirectories, usually following the main project directory structure. I have a functional subdirectory for tests that test multiple components together (where applicable).
I create phpunit.xml.dist telling it where to find the tests (and also immediately telling anyone looking at the source code that we use phpunit, and by looking at the xml file they can understand the convention too).
I don't use #assert or the skeleton generator. It feels like a toy feature; you do some typing in one place (your source file) to save some typing in another place (your unit test file). But then you'll expand on the tests in the unit test files (see my next paragraph), maybe even deleting some of the original asserts, and now the #assert entries in the original source file are out of date and misleading to anyone looking at just that code.
You have also lost a lot of power that you end up needing for real-world testing of real-world classes (simplistic BankAccount example, I'm looking at you). No setUp()/tearDown(). No instance variables. No support for all the other built-in assert functions, let alone custom ones. No #depends and #dataProvider.
One more reason against #assert, and for maintaining a separate tests directory tree: I like different people to write the tests and the actual code, where possible. When tests fail it sometimes points to a misunderstanding in the original project specs, by either your coder or your tester. When code and tests live close together it is tempting to change them at the same time. Especially late on a Friday afternoon when you have a date.
We store our tests right with the code files, so developers see the tests to execute, and ensure they change the tests as required. We simply add an extension of .test to the file. This way, we can simply include the original file automatically in each test file, which may then be created with a template. When we release the code, the build process deletes the .test files from all directories.
/application/src/
Foo.php
Foo.php.test
/application/src/CLASS/
FOO_BAR.class
FOO_BAR.class.test
require_once(substr(__FILE__, 0, -5)); // strip '.test' extension
I am currently going to start from scratch with the phpunit tests for a project. So I was looking into some projects (like Zend) to see how they are doing things and how they organizing their tests.
Most things are pretty clear, only thing I have some problems with is how to organize the test suites properly.
Zend has an AllTests.php from which loads others test suites.
Tough looking at the class it is useing PHPUnit_Framework_TestSuite to create a suite object and then add the other suites to it, but if I look in the PHPUnit docs for organizing tests in PHPUnit versions after 3.4 there is only a description for XML or FileHierarchy. The one using classes to organize the tests was removed.
I haven't found anything that this method is deprecated and projects like Zend are still using it.
But if it is deprecated, how would I be able to organize tests in the same structure with the xml configuration? Executing all tests is no problem, but how would I organize the tests (in the xml) if I only wanted to execute a few tests. Maybe creating several xmls where I only specify a few tests/test suites to be run?
So if I would want to only test module1 and module2 of the application, would I have an extra xml for each and defining test suites only for those modules (classes used by the module) in it. And also one that defines a test suite for all tests?
Or would it be better to use the #group annotation on the specific tests to mark them to be for module1 or module2?
Thanks in advance for pointing me to some best practices.
I'll start of by linking to the manual and then going into what I've seen and heard in the field.
Organizing phpunit test suites
Module / Test folder organization in the file system
My recommended approach is combining the file system with an xml config.
tests/
\ unit/
| - module1
| - module2
- integration/
- functional/
with a phpunit.xml with a simple:
<testsuites>
<testsuite name="My whole project">
<directory>tests</directory>
</testsuite>
</testsuites>
you can split the testsuites if you want to but thats a project to project choice.
Running phpunit will then execute ALL tests and running phpunit tests/unit/module1 will run all tests of module1.
Organization of the "unit" folder
The most common approach here is to mirror your source/ directory structure in your tests/unit/ folder structure.
You have one TestClass per ProductionClass anyways so it's a good approach in my book.
In file organization
One class per file.
It's not going to work anyways if you have more than one test class in one file so avoid that pitfall.
Don't have a test namespace
It just makes writing the test more verbose as you need an additional use statement so I'd say the testClass should go in the same namespace as the production class but that is nothing PHPUnit forces you to do. I've just found it to be easier with no drawbacks.
Executing only a few tests
For example phpunit --filter Factory executes all FactoryTests while phpunit tests/unit/logger/ executes everything logging related.
You can use #group tags for something like issue numbers, stories or something but for "modules" I'd use the folder layout.
Multiple xml files
It can be useful to create multiple xml files if you want to have:
one without code coverage
one just for the unit tests (but not for the functional or integration or long running tests)
other common "filter" cases
PHPBB3 for example does that for their phpunit.xmls
Code coverage for your tests
As it is related to starting a new project with tests:
My suggestion is to use #covers tags like described in my blog (Only for unit tests, always cover all non public functions, always use covers tags.
Don't generate coverage for your integration tests. It gives you a false sense of security.
Always use whitelisting to include all of your production code so the numbers don't lie to you!
Autoloading and bootstrapping your tests
You don't need any sort of auto loading for your tests. PHPUnit will take care of that.
Use the <phpunit bootstrap="file"> attribute to specify your test bootstrap. tests/bootstrap.php is a nice place to put it. There you can set up your applications autoloader and so on (or call your applications bootstrap for that matter).
Summary
Use the xml configuration for pretty much everything
Seperate unit and integration tests
Your unit test folders should mirror your applications folder structure
To only execute specif tests use phpunit --filter or phpunit tests/unit/module1
Use the strict mode from the get go and never turn it off.
Sample projects to look at
Sebastian Bergmanns "Bank Account" example project
phpBB3 Even so they have to fight some with their legacy ;)
Symfony2
Doctrine2
Basic Directory Structure:
I have been experimenting with keeping the test code right next to the code being tested, literally in the same directory with a slightly different file name from the file with the code it is testing. So far I am liking this approach. The idea is you don't have to spend time and energy keeping the directory structure in sync between your code and your test code. So if you change the name of the directory the code is in, you don't then also need to go and find and change the directory name for the test code. This also causes you to spend less time looking for the test code that goes with some code as it is right there next to it. This even makes it less of a hassle to create the file with the test code to begin with because you don't have to first find the directory with the tests, possibly create a new directory to match the one you are creating tests for, and then create the test file. You just create the test file right there.
One huge advantage of this is it means the other employees (not you because you would never do this) will be less likely to avoid writing test code to begin with because it is just too much work. Even as they add methods to existing classes they will be less likely to not feel like adding tests to the existing test code, because of the low friction of finding the test code.
One disadvantage is this makes it harder to release your production code without the tests accompanying it. Although if you use strict naming conventions it still might be possible. For example, I have been using ClassName.php, ClassNameUnitTest.php, and ClassNameIntegrationTest.php. When I want to run all the unit tests, there is a suite that looks for files ending in UnitTest.php. The integration test suite works similarly. If I wanted to, I could use a similar technique to prevent the tests from getting released to production.
Another disadvantage of this approach is when you are just looking for actual code, not test code, it takes a little more effort to differentiate between the two. But I feel this is actually a good thing as it forces us to feel the pain of the reality that test code is code too, it adds its' own maintenance costs, and is just as vitally a part of the code as anything else, not just something off to the side somewhere.
One test class per class:
This is far from experimental for most programmers, but it is for me. I am experimenting with only having one test class per class being tested. In the past I had an entire directory for each class being tested and then I had several classes inside that directory. Each test class setup the class being tested in a certain way, and then had a bunch of methods each one with a different assertion made. But then I started noticing certain conditions I would get these objects into had stuff in common with other conditions it got into from other test classes. The duplication become too much to handle, so I started creating abstractions to remove it. The test code became very difficult to understand and maintain. I realized this, but I couldn't see an alternative that made sense to me. Just having one test class per class seemed like it would not be able to test nearly enough situations without becoming overwhelming to have all that test code inside one test class. Now I have a different perspective on it. Even if I was right, this is a huge dampener on other programmers, and myself, wanting to write and maintain the tests. Now I am experimenting with forcing myself to have one test class per class being tested. If I run into too many things to test in that one test class, I am experimenting with seeing this as an indication that the class being tested is doing too much, and should be broken up into multiple classes. For removing duplication I am trying to stick to simpler abstractions as much as possible that allows everything to exist in one readable test class.
UPDATE
I am still using and liking this approach, but I have found a very good technique for reducing the amount of test code and the amount of duplication. It is important to write reusable assertion methods inside the test class itself that gets heavily used by the test methods in that class. It helps me to come up with the right types of assertion methods if I think of them as internal DSLs (something Uncle Bob promotes, well actually he promotes actually making internal DSLs). Sometimes you can take this DSL concept even further (actually make a DSL) by accepting a string parameter that has a simple value that refers to what kind of test you are trying to perform. For example, one time I made a reusable asssertion method that accepted a $left, $comparesAs, and a $right parameter. This made the tests very short and readable as the code read something like $this->assertCmp('a', '<', 'b').
Honestly, I can't emphasize that point enough, it is the entire foundation of making writing tests something that is sustainable (that you and the other programmers want to keep doing). It makes it possible for the value that tests add to outweigh what they take away. The point is not that you need to use that exact technique, the point is you need to use some kind of reusable abstractions that allow you to write short and readable tests. It might seem like I'm getting off topic from the question, but I'm really not. If you don't do this, you will eventually fall into the trap of needing to create multiple test classes per class being tested, and things really break down from there.