I'm adding unit tests to a legacy PHP application that uses a MySQL compatible database. I want to write genuine unit tests that don't touch the database.
How should I avoid accidentally using a database connection?
Lots of parts of the application use static method calls to get a reference to a database connection wrapper object. If I'm looking at one of these calls I know how to use dependency injection and test doubles to avoid hitting the database from a test, but I'm not sure what to do about all the database queries that I'm not looking at at any one time, which could be some way down the call stack from the method I'm trying to test.
I've considered adding a public method to the database access class that would be called from the PHPUnit bootstrap file and set a static variable to make any further database access impossible, but I'm not keen on adding a function to the application code purely for the sake of the tests that would be harmful if called in production.
Adding tests to a legacy application can be delicate, especially unit tests. The main problem you will likely have is that most tests will be hard to write and easily become unintelligible, because they involve massive amount of setting up and mocking. At the same time you will likely not have much freedom to refactor them, so they become easier to test, because that will lead to ripple effects in the code base.
That's why I usually prefer end to end-tests. You can cover lots of ground without having to test close to the implementation and those tests are usually more useful when you want to do large scale refactoring or migrate the legacy code base later, because you ensure that the most important features you were using still work as expected.
For this approach you will need to test through the database, just not the live database. In the beginning it's probably easiest to just make a copy, but it's absolutely worthwhile to create a trimmed down database with some test fixtures from scratch. You can then use something like selenium to test your application through the web interface by describing the actions you take on the site, like go to url x, fill out a form and submit it and describe the expected outcome, like I should be on url y now and there should be a new entry in the database. As you can see these kinds of tests are written very close to what you see on the website and not so much around the implementation or the single units. This is actually intended because in a migration you might want to rip out large chunks and rewrite them. The unit tests will become completely useless then, because the implementation might change drastically, but those end2end-tests describing the functionality of the site will still remain valid.
There are multiple ways you can go about this. If you are familiar with PHPUnit you might want to try the selenium-extension. You should find tutorials for this online, for example this one: https://www.sitepoint.com/using-selenium-with-phpunit/
Another popular option for these kind of tests is Behat with the MinkExtension. In both cases the hardest part is setting up selenium, but once you are able to write a simple test, that for example goes to your frontpage and checks for some text snippet and get that running. You can write tests really fast.
One big downside of these tests is that they are very slow, because they do full web requests and in some cases have to do some waiting for JavaScript. So you should probably not test everything. Instead try to focus on the most important features. If you have some e-commerce project, maybe go through a very generic checkout procedure. Then expand on different variations that are important to you, e.g. logged in user vs. new user or adding vouchers to the basket. Another good way to start is write very stupid tests, that just check whether your urls are actually accessible, so go to url and check for status code and some expected text snippet. Those are not really that useful in terms of making sure your application behaves correctly, but they still give you some safety as to whether some random 500 errors appear out of the blue.
This is probably the best approach for making safe your app works well and make it easier to upgrade, refactor or migrate your application or parts of it. Additionally whenever you add new features, try to write some actual unit tests for them. It's probably easiest if they are not too connected with the old parts of the code. In the best case scenario you won't have to worry too much about the database, because you can replace the data you get from the database with some instances you prepare yourself in the test and then just test whatever feature. Obviously if it's something like a simple we want to have a form that adds this data to the database, you will probably still not want to write a unit test, but instead write one of those bigger end to end-tests instead.
Related
When i started making a mobile app (that uses laravel on the server) i decided to not dig into testing or coding standards as i thought it was better to actually get a working app first.
A few months later i have a working app but my code doesn't follow any standards and nor have i written any tests.
Current directory structure i'm using is:
app/controllers : Contains all the controllers the app uses. The controllers aren't exactly thin, they contain most of the logic and some of them even have multiple conditional statements (if/else). All the database interactions occur in the controllers.
app/models : Define relationships with other models and contain certain functions relative to the particular model eg. A validate function.
app/libraries : contain custom helper functions.
app/database : contains the migrations and seeds.
My app is currently working and the reason for the slack is probably because i work alone on the app.
My concerns:
Should i go ahead and release the app and then see if its even worth
making the effort to refactor or should i first refactor.
I do wish to refactor the code but i'm unsure on as to what approach
i should take. Should i first get the standards right and then make
my code testable? Or should i not worry about standards( and
continue using classmap to autoload) and just try and make my code
testable?
How should i structure my files?
Where should i place interfaces,abstract classes etc?
Note: I am digging into testing and coding standards from whatever resources i can find, but if you guys could point me to some resource i would appreciate it.
Oh dear. Classic definition of legacy code, is code with no unit tests. So now you are in that unfortunate position where, if you are ever going to have to amend/enhance/resuse this code base, you are going to have to absorb a significant cost. The further away from your current implementation you get, the higher that cost is going to get. That's called technical debt. Having testable code doesn't get rid of it though, it simply reduces the interest rate.
Should you do it? Well if you release now, and it achieves some popularity and therefore need to do more to it...
If it's going to be received with total indifference or screams of dismay, then there's no point in releasing it at all, in fact that would be counter-productive.
Should you choose to carry on with the code base, I can't see any efficient way to re-factor for a coding standard without a unit test to prove you haven't broken the app in doing so. If you find a coding standard where testable and tested aren't a key part of it, ignore it, it's drivel.
Of course you still have the problem of how do you change code to make it testable without breaking it. I suspect you can iterate towards that with say delegation in your fat controllers without too much of a difficulty.
Two key points.
Have at least some test first, even if it's a wholly manual one, say a printout of the web page before you do anything. A set of integration tests, perhaps using an automation suite would be very useful, especially if you already have a coupling problem.
The other is, don't make too many changes at once and by too many I mean how many operations in the app will be affected by this code.
Robert C Martin's (no affiliation whatsoever) Working With Legacy Code would be a good purchase if you want to learn more.
Hopefully this lesson has already been learnt. Don't do this again.
Last but not least doing this sort of exercise is a great learning experience, rest assured you will run into situations (lots of them) where this perennial mistake has been made.
With tests, using the #depends you can make a test method depend on another one, that way, should the first test fail, the test that depends on it will be ignored, is there a way to do that with Test Classes
For example, say i have a page which i test the layout of, checking that the images are showing up, if links are correct, this will be one Test Class, on this page is a link to a form, for that page i would create a new test class checking its layout, validation ect.
what i want to try and pull of is if any test of the first Test Class fails then the second one should be skipped since the first page should be correct as it will be the page a user will see before entering the first (unless they type the second page's url in but have to assume users are stupid and as such don't know how the address bar work)
i should also note that all the tests are being stored using TFS so while the team will have the same tests we may have different phpunit.xml files (one person apparently uses php.xml.dist and wont change from it because it is mentioned once in Magento TAF even though i use .xml and have had no problems), as such, trying to enforce an order in phpunit.xml wont be all that useful (also enforcing the order in the .xml wont do this dependency thing)
Just because we can do a thing, does not mean we should. Having tests that depend on other tests' successful execution is a bad idea. Every decent text on testing tells you this. Listen to them.
I don't think phpUnit has a way to make one test class depend on another. (I started writing down some hacks that spring to mind, but they are so ugly and fragile and I deleted them again.)
I think the best approach is one big class for all your functional tests. Then you can use #depends as much as you need to. <-- That is where my answer to your actual question ends :-)
In your comment on Ross's answer you say: "I was taught that if you have a large number of (test) methods in one class then you should break it into separate classes" To see why we are allowed to break this rule you have to go below the surface to why that is usually a good idea: a lot of code in one class suggests the class is doing too much, making it harder to change, and harder to test. So you use Extract Class refactoring to split classes up into finer functionality. But never mechanically: each class should still be a nice, clean abstraction of something.
In unit testing the class is better thought of as a way to collect related tests together. When one test depends on another then obviously they are related, so they should be in the same class.
If a 2000-line file make you unhappy, one thing you can do, and should do, is Extract Parent Class. Into your parent class goes all the helper functions and custom asserts. You will leave all the actual tests in the derived class, but go through each of them and see what common functionality could be moved to a shared function, and then put that shared function in the parent class.
Responding to Ross's suggestion that #depends is evil, I prefer to think of it as helping you find the balance between idealism and real-world constraints. In the ideal world you want all your tests to be completely independent. That means each test needs to do its own fixture creation and tear down. If using a database, it should be creating its own database (a unique name, so in the future they can be run in parallel), then creating the tables, filling them with data, etc. (Use the helper functions in your parent class to share this common fixture code.)
On the other hand, we want our tests to finish in under 100 milliseconds, so that they don't interrupt our creative flow. Fixture sharing helps speed up tests at the cost of removing the independence.
For functional tests of a website I would recommend using #depends for obvious things like login. If most of your tests will first log into the site, then it makes a lot of sense to make a loginTest(), and have all other tests #depend on that. If login is not working, you know for sure that all your other tests are going to fail... and are wasting huge amounts of the most valuable programmer resource in the process.
When it is not so clear-cut, I'd err on the side of idealism, and come back and optimize later, if you need to.
I want to make some unit tests in my project (i am new to testing), but tutorials online seem to show examples testing only simpliest stuff.
What I want to test is case when after sending POST to addAction in my SurveyController will result in adding corresponding rows to my survey and question tables (one-to-many).
What are the best practices to test database related stuff? Do I create separate db for my test environment and run tests on it? That is the only and right option?
It depends on your circumstances.
Here is my take on this:
The idea is to test but also be DRY (Don't repeat yourself). Your tests must cover all or as many different cases as possible to ensure that your component is thoroughly tested and ready to be released.
If you use an already developed framework to access your database like Doctrine, Zend Framework, PhalconPHP etc. then you can assume that the framework has been tested and not test the actual CRUD operations. You can concentrate on what your own code does.
Some people might want to test even that but in my view it is an overkill and waste of resources. One can actually include the tests of the particular framework with their own if they want to just have more tests :)
If however you were responsible for the database layer classes and its interaction with your application, yeah tests are a must. You might not run them all the time you need to prove a database operation works or not through some piece of code, but you need to have them.
Finally you can use Mock tests as Mark Baker suggested in your code and assume that the database will respond as you expect it to (since it has already been tested). You can then see how your application will react with different responses or results.
Mocking database operations will actually make your tests run faster (along with the other benefits that come with this strategy) since there won't be any database interactions in the tests themselves. This can become really handy in a project with hundreds if not thousands of tests and continuous integration.
HTH
I inherited a fairly large, homemade, php4+MySQL, ecommerce project from developers that literally taught themselves programming and html as they wrote it. (I would shudder except that it's really impressive that they were able to do so much by starting from scratch.) My job is to maintain it and take it forward with new functionality.
Functionality of the code depends on $_SESSION data and other global state structures, which then affect the flow of the code and which parts of the site are displayed through require statements. When I took it over last year, my first task was abstracting all of the repetition into separate files which get included via require statements and also removing most of the 'logic' code from the 'display' or output code, but I couldn't remove it all. I have moved code into functions where I can, but that's still quite limited. Classes and methods are definitely out of the question right now.
All testing is being done manually/visually. I would like to start automating some testing, but I simply don't know where to start. Unit testing of functions is pretty straightforward, but very little of the code is in functions and most of that is pretty simple. I've looked at phpUnit and DbUnit, but all of the examples and discussion about them focus on classes and methods.
So, what options do I have to begin implementing unit testing on anything more than the most trivial parts of my project?
First off PHPUnit can be used to test procedural code just fine. Don't let the fact that PHPUnit examples only shows classes deter you. It's just how PHPUnit tests are organized.
You can just write test classes and test your function from them without any problems and that should be your smallest problem :)
If the code doesn't run on PHP 5.2+ then you can't use a current PHPUnit Version which is definitely more of a concern and my first general recommendation is to find any issues an PHP 5 upgrade might bring.
To start off one book recommendation to save you some troubles:
Working Effectively with Legacy Code
The book will help you avoid a lot of small mistakes you'd have to make yourself once instead and will get you in the right mindset. It's Java based but that is not really an issue as most of the stuff is easily adaptable.
Testing is hard because you don't even know what the application is supposed to in the first place
Getting unit tests up an running takes quite some time and doesn't give you a status "is it still working" so my first point would be to get some integration and front-end tests set up.
Tools like Selenium and the web testing parts of Behat can help you A LOT with this.
The advantage of using Behat would be that you can write nice documentation for what the product actually is supposed to do. No matter how to project goes along these docs will always have value for you.
The rests read something like: "When I go to this url and enter that data a user should be created and when I click there I get an email containing my data export". Check it out. It might be useful.
The most important thing is to get a quick green/red indicator if the thing is still working!
If you then find out that it was broken despite your "light" being green you can expand the tests from there on out.
When you don't know when it is broken you will never be confident enough to change enough stuff around so that you can incrementally improve what needs fixing or changing.
After you have a general sense of how everything work and you trust your small tests to show you when you break "the whole thing" I'd say it's time to set up a small Continuous integration server like Jenkins for PHP that allows you to track the status of your project over time. You don't need all the QA stuff at the start (maybe to get an overview over the project) but just seeing that all the "does it still work" stuff returns "yes" is very important and saves you lots of timing making sure of that manually.
2% Code Coverage is boring
When you are at a point where unit testing and code coverage come into play you will be faced with a mean read 0%. That can be quite annoying to never see this number rise a lot.
But you want to make sure you test NEW code so I'd suggest using PHP_Change_Coverage as described in this blog posting so make sure everything you touch has tests afterwards.
PHP Black Magic
function stuff() {
if(SOME_OLD_UGLY_CONST == "SOME SETTING") {
die("For whatever reasons");
}
return "useful stuff";
}
When testing it is really annoying when your scripts die() but what to do?
Reworking all scripts without tests can be more hurtful than not doing anything at all so maybe you want to hack to get tests first.
For this an other solutions to scary things there is the php test helpers extension.
<?php
set_exit_overload(function() { return FALSE; }
exit;
print 'We did not exit.';
unset_exit_overload();
exit;
print 'We exited and this will not be printed.';
?>
I would probably start testing with a tool like Watir or Selenium. This will allow you to do black box testing of the the whole page automatically. Once you have set up these tests, you can start to refactor the PHP pages, and build up unit tests as you are doing the refactoring.
Legacy code without unit tests is always a pain. There's no real solution because most of the time the code isn't written in a way it's unit testable at all. At work we had to handle lots of legacy code too. We wrote unit tests for new written code (which is a pain too, because you need to be able to set up some test data and stuff like that). This way it doesn't get worse and you will cover more and more old legacy code that is called within your new code. Doing this you will get closer each time to code that is covered by unit tests.
You face a big task. You likely can't code all the tests that are needed in any reasonable length of time, and your managers won't care if you do, if they believe the application mostly works.
What you should do is focus on building tests for new or modified application code. There will be plenty of that, if your application has any life, to keep you busy.
You should
also verify that you are actually testing that new code. By diffing new check-ins (you are using source control, right? if not, FIX THAT FIRST!) against old, you can see what has changed and get precise location information about the location of the changes. You (well, your manager should) want proof that those changes have been tested. You can use a code coverage tool to tell what has been tested, and take the intersection of that with the location of your new changes; the intersection better include all the modified code.
Our PHP Test Coverage can provide coverage information, and can compute such intersection data.
I'm writing unit tests for a project (written in PHP, using PHPUnit) that have its entire environment (loaded components, events, configuration, cache, per-environment singletons, etc) held in an object which all the components use to interact with each other (using a mediator pattern).
In order to make the unit tests run faster, I'm sharing the environment object and some other objects (for example, in my test case for view object [as in the V of MVC], the view manager object [which acts as a factory for view objects and responsible for the actual rendering]) among tests in the same test case (using PHPUnit's setUpBeforeClass() and static properties).
Even though, to the best of my knowledge, the objects I share shouldn't effect the integrity of the tests (in the views case, for example, the environment and view manager object are shared, but a separate view object is created for every test - which is the object that's actually being tested by the test case), it just feels increasingly wrong to me.
I would prefer it if each test used a completely isolated environment and couldn't effect other tests in the same test case in any way. However, that would make the tests run much slower and it feels like a big price for something that I can't really pinpoint the downside of and mainly just "feels wrong".
What do you think? Can you pinpoint any downsides so I can convince myself its worth the longer execution time? Or am I just over reacting and its completely fine?
I share your feelings so maybe i just state my goals and my solution when i faced that issue:
Devs should have a test suite that runs very very fast
At least single test cases should execute in less than a second
I really want to be sure i don't have interdependencies in my test cases
I'm going to assume you have a Continuous Integration Server running. If not a cronjob might do but consider setting up jenkins, it's really really easy.
For normal usage:
Just share as much fixtures as you need to get the speed you need. It might not be pretty and there might be better solutions along the way but if you have something that is expensive to create just do it once.
I'd suggest helper methods getFoo() { if(!self::$foo) .... create ... return $foo;} over setUpBeforeClass because it can make sharing easier but mainly because of the following point.
Once a night:
Run your test suite with --process-isolation and in that bootstrap recreate your complete database and everything.
It might run 6 hours (disable code coverage for that!) but how cares. Your fixtures will be recreated for every single test case since it's a new php process and the static vars don't exist.
Using this way you can be sure that you don't have created dependent once a day. Thats good enough to remember what you did (and you can run with --filter and --process-isolation if you need to fix something).
Like writing "normal" code, when you write test cases, it's fine to rely on knowledge of how fixture objects work.
If a given factory method is documented as generating new instances every time, then I see only downside in creating the factory method anew each time, especially if the factory creation is itself expensive.
It helps to keep in mind a key goal around writing unit tests. You want to know within 5-10 minutes whether you broke the build. That way you can go out to lunch, go to a meeting, go home, etc. after you get the "all clear". If you know that some part of the fixture is reusable without creating interactions, then you should use that knowledge to make your tests even more comprehensive within that 5-10 minute window. I understand the purist impulse here, but it buys you nothing in terms of test independence, and it unnecessarily limits what your test suite will accomplish for you.
Perhaps this is blasphemous, but if you can still manage a significant threshold of code coverage and you can also guarantee that no state pollution can exist (and that is your real issue -- are you making sure that each test will not be effected by the data left over from the test before), I see no problem with leaving things the way they are.
Let the tests run quickly, and when a bug is found (which is inevitable in integration testing) then you have reason to invest the time in localizing one particular test or set of tests. If you have a toolkit which works for the general case, however, I would leave it as is.