I'm creating a php script, which should process the POST data it receives (from AJAX or else) and send it further (to another script).
I'm wondering how to develop it in a "BDD way".
So far I've done the "processing part" by writting features with Behat and created the required blocks (classes) using phpspec.
But then I'm blocked when it comes to testing those following features:
the script only processes / accepts POST data,
the script sends only valid data further after processing,
the script sends back errors in case of invalid data.
It seems to me that I could write the tests against the script itself, but then I'm wondering:
if it's a good idea (it does seem simple enough but a bit messy though because there is not much isolation)
how to do this elegantly in behat (it seems messy for me to have to manually run my local server and have its url hardcoded in my tests / contexts, but maybe it's just the way to do it)
Any ideas or suggestions?
BDD:
Let's try to change your mindset a bit. The BDD is about a collaboration and the automation part (tests) are only the last part of it.
Your acceptance tests, this is what you do via Behat, should only cover a specification of your feature via examples. It means, don't focus on testing of all possible scenarios like you would do via unit/integration tests but specify only the minimum which describes your feature enough to be revealing the intention of that feature.
In most cases, the examples cover only positive scenarios and 1-5 are enough.
Little help here. Ask yourself, what would you mention if these examples would be a part of a documentation for customers? The specification by examples is no more no less than the documentation of an application which has an ability to be automatically tested.
Testing level:
Unfortunately, I don't know the technical background of your script so the answer will be more a theory.
There are more levels of acceptance tests and the higher you test the more you cover but the more expensive is their creation and maintenance:
UI
HTTP request via infrastructure
Initiate the application and inject a fake request
Call a controller directly
Call an application service which is processing the domain logic
There is my personal practice. Because BDD is the best with TDD, I always start with the point 5), and sometimes, I also add the higher level 3) to be sure that the application works correctly as the whole. I use levels 2) and 1) very rarely as I don't need to test my infrastructure via acceptance tests, it's not their purpose.
This is more of a comment rather than a proper answer, but it would not fit well in a comment either...
So first thing, good that you're trying to test all the things. But in order to do BDD (hence using Behat), there has to be some sort of benefit for a stakeholder or part of the business with who you should have a conversation about the feature at hand.
The fact that you're just describing a script that receives a particular input and transforms it into a particular output, sounds pretty much what you'd expect from a Unit test, right?
On the other hand, if this particular script is solving a stakeholder's need and there's a story or scenario that this script is helping to satisfy, I assume that there would be some sort of change in the state of the system which you can test for. If so, then go ahead and describe it using Behat, having a conversation with the appropriate stakeholders. I guess that you'd need to set up an environment for your system in which you'd then run your script and then check that the state of it has been changed appropriately.
Related
I have some confused question about testing.
We're using Slim as framework of the system. As I know, Unit Test is the minimum of testing. For example, to test if the class or method works as expected.
To test another functions of the system, for example, the system provide an API to search products information, this search function designed by the following thoughts:
Clients post keyword to the API entry. eg. /search
Handle the request and inject SearchService into the controller.
Set the keyword to the SearhService.
Push the keyword into the database. eg. search_history.
Fetch the search result back to controller, and response to clients.
This is how search API designs.
And we created a functional test for the Search API called SearchTest. This is how we done:
Use $this->runApp() method provided by Slim to sending requests to the API entry.
Set several test cases to test different kind of scenario, such as "No keyword provided", "No search result" ... etc.
Assert the responses that return value or HTTP status as expected.
Some questions are confusing:
Sending requests and asserting the responses, is that all? Do we misunderstand so called "functional testing"?
We shouldn't care about how it works, we just need to send request and assert the responses, right?
If not, should we check dependencies or used components work as expected? For example, should we connect to the database, and check if the keyword pushing success or not?
Automated Testing (and testing in general) is considered a good practice in Software Engineering. However, there exist a lot of hot discussion on what are the boundaries for test methodology and types classification.
In this sense, a practical approach to correctly implement a testing methodology is learning from those who you may consider respectable in your context of software development AND making sure that the practice you adopt plays nice for the goal you are willing to achieve. Try to be consistent on this.
Just as a reference, lets take this definition.
Functional testing refers to activities that verify a specific action or function of the code. These are usually found in the code requirements documentation, although some development methodologies work from use cases or user stories. Functional tests tend to answer the question of "can the user do this" or "does this particular feature work."
Considering this approach, you want your tests to give you confidence that the user is able to execute some function (or feature) provided by your application. It does not matter (here) how that feature is provided, just get into your user's shoes and think "Am I getting what I'm expecting from this action?", the answer to that question should hint you the assertions for your test.
Non-functional testing refers to aspects of the software that may not be related to a specific function or user action, such as scalability or other performance, behavior under certain constraints, or security. Testing will determine the breaking point, the point at which extremes of scalability or performance leads to unstable execution. Non-functional requirements tend to be those that reflect the quality of the product, particularly in the context of the suitability perspective of its users.
If you want to test how a certain function is executed, it's probably a Unit Test what may help you to do those assertions.
But again, remember it's not that of a sharp limit on what your tests should perform out from their name, but just to know what classification would fit best for the type of assertions you are performing (that helps you better structure your tests and give you a clear idea of what are you testing). Try to be consistent and focus on testing what you really need to be confident on.
I'm adding unit tests to a legacy PHP application that uses a MySQL compatible database. I want to write genuine unit tests that don't touch the database.
How should I avoid accidentally using a database connection?
Lots of parts of the application use static method calls to get a reference to a database connection wrapper object. If I'm looking at one of these calls I know how to use dependency injection and test doubles to avoid hitting the database from a test, but I'm not sure what to do about all the database queries that I'm not looking at at any one time, which could be some way down the call stack from the method I'm trying to test.
I've considered adding a public method to the database access class that would be called from the PHPUnit bootstrap file and set a static variable to make any further database access impossible, but I'm not keen on adding a function to the application code purely for the sake of the tests that would be harmful if called in production.
Adding tests to a legacy application can be delicate, especially unit tests. The main problem you will likely have is that most tests will be hard to write and easily become unintelligible, because they involve massive amount of setting up and mocking. At the same time you will likely not have much freedom to refactor them, so they become easier to test, because that will lead to ripple effects in the code base.
That's why I usually prefer end to end-tests. You can cover lots of ground without having to test close to the implementation and those tests are usually more useful when you want to do large scale refactoring or migrate the legacy code base later, because you ensure that the most important features you were using still work as expected.
For this approach you will need to test through the database, just not the live database. In the beginning it's probably easiest to just make a copy, but it's absolutely worthwhile to create a trimmed down database with some test fixtures from scratch. You can then use something like selenium to test your application through the web interface by describing the actions you take on the site, like go to url x, fill out a form and submit it and describe the expected outcome, like I should be on url y now and there should be a new entry in the database. As you can see these kinds of tests are written very close to what you see on the website and not so much around the implementation or the single units. This is actually intended because in a migration you might want to rip out large chunks and rewrite them. The unit tests will become completely useless then, because the implementation might change drastically, but those end2end-tests describing the functionality of the site will still remain valid.
There are multiple ways you can go about this. If you are familiar with PHPUnit you might want to try the selenium-extension. You should find tutorials for this online, for example this one: https://www.sitepoint.com/using-selenium-with-phpunit/
Another popular option for these kind of tests is Behat with the MinkExtension. In both cases the hardest part is setting up selenium, but once you are able to write a simple test, that for example goes to your frontpage and checks for some text snippet and get that running. You can write tests really fast.
One big downside of these tests is that they are very slow, because they do full web requests and in some cases have to do some waiting for JavaScript. So you should probably not test everything. Instead try to focus on the most important features. If you have some e-commerce project, maybe go through a very generic checkout procedure. Then expand on different variations that are important to you, e.g. logged in user vs. new user or adding vouchers to the basket. Another good way to start is write very stupid tests, that just check whether your urls are actually accessible, so go to url and check for status code and some expected text snippet. Those are not really that useful in terms of making sure your application behaves correctly, but they still give you some safety as to whether some random 500 errors appear out of the blue.
This is probably the best approach for making safe your app works well and make it easier to upgrade, refactor or migrate your application or parts of it. Additionally whenever you add new features, try to write some actual unit tests for them. It's probably easiest if they are not too connected with the old parts of the code. In the best case scenario you won't have to worry too much about the database, because you can replace the data you get from the database with some instances you prepare yourself in the test and then just test whatever feature. Obviously if it's something like a simple we want to have a form that adds this data to the database, you will probably still not want to write a unit test, but instead write one of those bigger end to end-tests instead.
I have a question on how to implement behat/mink functional tests.
In my web app, I have users that can access some data sheets if they have the required credentials (i.e. no access/ read only / write).
I want to be able to test all the possible contexts via behat/mink.
The question is what is the best practice for such testing ?
Some dev told me that I have to create a scenario for each type of user I would like to use. Then, I will have to use the user I created in others tests.
But I am not very confortable with this idea : I believe that it introduces coupling between my tests. If the test that creates the user fails, then the test that checks the access over my datasheet for that specific user will also fail.
So, I believed that I could use some fixtures : before testing my app, I run a script that will insert me all the profiles I need. I will have some tests dedicated for creating users, and I will use the fixtures to check if a specific user is allowed to access a specific datasheet.
The counterpart with this solution is that I will have to maintain the fixtures set.
Do you have any suggestion / idea ?
Hi user3333860 (what a username XD),
I'm not an expert in testing and these days, I'm more on ruby/rspec but I personally think that both solutions are good and are used.
Use a feature to create your User:
If your test for creating a user fails, It may means that your User creation refactoring is also messed up.
So the fact that other tests fails doesn't seems to be a drawback to me. But I do understand the fact that you don't want to have coupling between your tests.
The main point is, are your test ran in a static order or are they run randomly (ie: rspec don't always launch tests in the same order) and are you ready to have the same test run multiple time in different features so that your other tests can successfully complete.
Use a fixtures:
Well it is also a good and popular solution (in fact, the one I use) and you already pinpoint the fact that you will have to maintain.
In the end, I'd take the fixtures path ONLY with a tools helping like FactoryGirl that helps you maintain your objects templates (here is the PHP version of it)
https://github.com/breerly/factory-girl-php
Hope I helped with your dilemma.
What I would propose would probably sound strange, but I have my reasons for it.
For a long time, we've had this Spring based API, which originated as an abstract set of REST services for CRUD functionality. With time however, we started building business and presentation layers on top until we reached a state of a dead end. Do not get me wrong, Spring/Hibernate are great frameworks, and being built on top of the JVM has its definite advantages, including performance over other web technologies, such as PHP. Compared to PHP, it gives us a much deeper way of controlling aspects such as transactioning, multithreading, dealing with byte data, control over native C++ applications, using JNI, etc.
Where the stack clearly hits a hard wall, is where the requirements get changed most often, namely the business and presentation layers. Turning the applciation into a modern, user-centered social applciation, we've experienced the toughest challenge in out careers. Java EE presentation technologies are tough to work with. Also, changing the business requirements became a tremendously long cycling because of the traditional hurdles in building testing and deploying massive Java applications.
It also felt like for a large part, we are trying to reinvent the wheel. In the PHP world, so many projects exist already, which give you a complete management system, agnostic to any kind of backend system (mapping hooks to REST/SOAP endpoints) . Many of them have all these functionalities ready, allow for admin-friendly change of scenarious, and rules, have templating, etc. Plus, being PHP based means absolutely no time wasted in building and deploying. Write the change, test, asert that it works, and switch.
Our idea now is to find a way to move the business and presentation layers in such a kind of front-server PHP-based application, and leave the pure backend stuff to Spring/Hibernate. We have a few concerns though, coming from our relative inexperience with Spring.
If we implement the business methods using PHP methods, how do we keep transactional security? I mean if a business method has to make three separate HTTP requests to the JAVA, how can we guarantee that they will all be executed in the same transaction, DB-wise?
Is there a way to use proxies or promise objects between both systems? For instance, if the PHP business method needs to call a Spring search method get a collection of objects from the database, and then pass it on to another spring method, this will mean that teh whole collection will have to be sent back and forth. Perhaps, one could store it in a session object on the JAVA side, and simply sent an empty proxy back to the frontend, which the frontend can bass back to another jav method.
A lot of our Spring based functionailities rely on a structure of plugins, using Spring events. How can we make so that our frontend server also gets notified on every ackend event that happens.
I have two ideas here: a post-processing-filter that simply makes a POST request to a controller on the frontend server, using some naming convention.
or ... using some kind of a message queue, such as JMS or RabbitMQ, or why not even something like Reddis, where one can watch data for changes
Anyone who has done that before? Is this a good idea in general? Any suggestions how to resolve the aforementioned issues?
This is not what you asked, but I think it's worth saying that some of the problems you are experiencing with Java are problems that most experienced Java developers faced some day, and most have found some solution. For instance, you can use Arquillian to do a "faster" TDD in Java. And you can use JRebel to do a "code-and-refresh" approach, like in PHP. Modern IDEs also helps in refactoring, something that helps in the "requirements change" scenario.
Granted, the presentation part is still a big problem with Java. I personally don't like JSF, and (to me), most of the other presentation technologies are either non-intuitive or they are cumbersome.
I would say that this is the reason why a lot of Java developers are becoming adopters of HTML5 and Javascript (backbone.js, underscore.js, jquery, ...) for the frontend, with REST in the backend. There's no need to have PHP in the middle.
I'm afraid I cannot answer your other questions, but perhaps a good start would be to see if PHP could be run from inside a Java EE container? I know this works for Ruby and Python apps, as JRuby and Jython would take care of the bridge between the two world.
Largely what you are trying to do is what we call: SOFEA
In this case PHP is your frontend and Java is your service layer.
Concern 1
Transactions over traditional REST or WS are a PITA.
Either consider aggregating your service calls to do more work so that the whole entire transaction is in one service calls or have Java handle your frontend. Otherwise see Concern 3 with Finagle.
Concern 2
I recommend you stay stateless and do REST or RCP.
It sounds like your concerned with speed which I think you shouldn't worry about till its a problem. However if you do want to do user based caching of objects I would use Redis and its pub/sub feature (or RabbitMQ + memcache).
Concern 3
Use RabbitMQ, or Redis, or/and Finagle.
MY final Opinion
I don't think you need PHP particularly if you know Java.
So much of the web app world today is moving towards Javascript (in the client) I don't see any benefit to using something like PHP serving pages when there are better options like Node.js or just sticking with Java.
And #jpkrohling is right... templating and frotend used to suck in Java but its gotten much better particularly since most of your UI code will be in Javascript anyway.
If you need an example of more modern web stuff in Java have a look at this project: MWA and also by the same author: Handlebars in Java.
To sum it up my opinion and go back to SOFEA: backend = Java and frontend = Javascript
Haven't done this before, not sure if it's a good idea, but here are some thoughts on your questions:
First of all, in the PHP world, each request (whether it uses keep-alive or not) is run as an isolated process, so this scenario is typically unheard of; it's possible to use persistent database connections, but any non-committed transactions are automatically rolled back when the request is done. Now, this is where my knowledge about Spring is lacking, because I'm not sure whether Java can keep a database connection in the same state between requests as long as the same HTTP connection is used; if so, you can use PHP + cURL to perform three HTTP requests while reusing the network connection.
Object (un)marshaling between PHP and Java will probably not work very well; a representation of the collection would have to be made in the form of perhaps a result identifier that can be fetched via REST by PHP and passed back to Java as an opaque string.
It should be noted that PHP requests are typically short-lived, which makes it a good candidate for horizontal scaling; long running processes, such as those for Pub/Sub systems, are therefore seen less often. Personally I prefer using the alternative of sending HTTP requests so that the frontend caching can be abstracted away from the Java service.
I'll add more when I think of anything else, do let me know if my answer gives you more questions :)
There is definitely no straightforward way out. You have to re-write your backend controller layer which exposes REST calls to PHP. You need to group some fine grained services and make them coarse grained.
A word of advise - you can make things work at the end day with tweaking here and there, introducing a queue or another framework or library etc - but you would have also introduced lot of Accidental complexity..
Things can get real nasty in future during maintenance, bug fixes, upgrades etc
If speed of development is a concern, explore other alternative like Play Framework 1.2.x framework, so that you can re-use your server side code as it is.
I have a legacy codebase and I need to test that code with PHPUnit. So I am asking for suggestions based on your experiences. Which classes I should test first? Or give the priority?
Should I start with the easy/small classes or with the base/super class?
My general suggestion for introducing unit testing to an existing codebase would be the following:
Start of testing some very simple classes to get a feel for writing tests
Maybe even rewrite those tests again and make proper "this is how we should do it" examples out of them
Take the one of the biggest and meanest classes in the whole system and get that class tested as best as you can. This is an important step to show everyone in your team (and maybe management) that unit testing your codebase CAN WORK OUT and is doable.
After that I'd suggest that you focus on three things:
Make sure new code gets tests
If you fix a bug create a test before fixing it to "prove" the bug is actually fixed
Use tests as a tool when you touch/change old code so you get better an better test coverage.
PHPUnit will provide you with a CodeCoverage Report showing you how well your codebase is tested. It can be pretty cool to see the number rise from 0.3% to 5% to 20% over the course of month but it not a really strong motivator.
To make sure you test NEW code I'd suggest using PHP_Change_Coverage as described in this blog posting
This tool will help you a lot generating meaningful coverage reports as it only shows NEWLY CREATED CODE as UNTESTED and not all the old stuff you have lying around.
With that you have something at hand that makes it really easy to "get a high % very early and keep testing the new stuff" while you create tests for everything old.
Before PHP Change Coverage:
and After:
There's often too much code in a system to test it all as a first step. But most of that code already works.
I'd start with methods that were modified recently. Presumably most of the rest of the software works to some degree, and testing that won't likely find as many errors as will be found in new or newly revised code.
Should you run out of work (I doubt it in the near future if you have 1 or more developers actively working near you), you can move up to methods that use the methods that were modified, to methods that have high complexity according to software metrics, and to methods that are critical for safe system operation (login with password, storage of customer charge data, etc.)
One way to help decide what to consider testing next is to use a test coverage tool. Normally one uses this to determine how well tested the software is, but if you don't have many tests you already know with it will tell you: your code isn't very well tested :-{ So there's no point in running it early in your test construction process. (As you get more tests you and your managers will eventually want to know this). However, test coverage tools also tend to provide complete lists of code that has been exercised or not as part of your tests, and that provides a clue as to what you should test node: code that has not been exercised.
Our SD PHP Test Coverage tool works with PHP and will provide this information, both through and interactive viewer and as a generated report. It will tell you which methods, classes, files, and subystems (by directory hierarchy) have been tested and to what degree. If the file named "login.php" hasn't been tested, you'll be able to easily see that. And that explicit view makes it much easier to intelligently decide what to test next, than simply guessing based on what you might know about the code.
Have a look at PHPure,
Their software records the inputs and outputs from a working php website, and then automatically writes phpunit tests for functions that do not access external sources (db, files etc..).
It's not a 100% solution, but it's a great start