I am working on an extension (app) of nextcloud (which is based on Symfony). I have a helper class to extract data from the request that is passed by the HTTP server to PHP. A much-reduced one could be something like this (to get the point here):
<?php
namespace OCA\Cookbook\Helpers;
class RequestHelper {
public function getJson(){
if($_SERVER['Request_Method' === 'PUT'){ // Notice the typos, should be REQUEST_METHOD
$raw = file_get_content('php://input');
return json_decode($raw, true);
} else { /* ... */ }
}
}
Now I want to test this code. Of course, I can do some unit testing and mock the $_SERVER variable. Potentially I would have to extarct the file_get_content into its own method and do a partial mock of that class. I get that. The question is: How much is this test worth?
If I just mimick the behavior of that class (white box testing) in my test cases I might even copy and paste the typo I intentionally included here. As this code is an MWE, real code might get more complex and should be compatible with different HTTP servers (like apache, nginx, lighttpd etc).
So, ideally, I would like to do some automated testing in my CI process that uses a real HTTP server with different versions/programs to see if the integration is working correctly. Welcome to integration testing.
I could now run the nextcloud server with my extension included in a test environment and test some real API endpoints. This is more like functional testing as everything is tested (server, NC core, my code and the DB):
phpunit <---> HTTP server <---> nextcloud core <---> extension code <---> DB
^
|
+--> RequestHelper
Apart from speed, I have to carefully take into account to test all possible paths through the class RequestHelper (device under test, DUT). This seems a bit brittle to me in the long run.
All I could think of is adding a simple endpoint only for testing the functionality of the DUT, something like a pure echo endpoint or so. For the production use, I do not feel comfortable having something like this laying around.
I am therefore looking for an integration test with a partial mock of the app (mocking the business logic + DB) to test the route between the HTTP server and my DUT. In other words, I want to test the integration of the HTTP server, nextcloud core, my controller, and the DUT above without any business logic of my app.
How can I realize such test cases?
Edit 1
As I found from the comments the problem statement was not so obviously clear, I try to explain a bit at the cost of the simplicity of the use-case.
There is the nextcloud core that can be seen as a framework from the perspective of the app. So, there can be controller classes that can be used as targets for URL/API endpoints. So for example /apps/cookbook/recipe/15 with a GET method will fetch the recipe with id 15. Similarly, with PUT there can be a JSON uploaded to update that recipe.
So, inside the corresponding controller the structure is like
class RecipeController extends Controller {
/* Here the PUT /apps/cookbook/recipe/{id} endpoint will be routed */
public function update($id){
$json = $this->requestHelper->getJson(); // Call to helper
// Here comes the business logic
// aka calls to other classes that will save and update the state
// and perform the DB operation
$this->service->doSomething($json);
// Return an answer if the operation terminated successfully
return JsonResponse(['state'=>'ok'], 200);
}
}
I want to test the getJson() method against different servers. Here I want to mock at least the $this->service->doSomething($json) to be a no-op. Ideally, I would like to spy into the resulting $json variable to test that exactly.
No doubt, in my test class it would be something like
class TestResponseHandler extends TestCase {
public function setUp() { /* Set up the http deamon as system service */}
public testGetJson() {
// Creat Guzzle client
$client = new Client([
'base_uri' => 'http://localhost:8080/apps/cookbook',
]);
// Run the API call
$headers = ...;
$body = ...;
$response = $client->put('recipe/15', 'PUT', $headers, $body);
// Check the response body
// ....
}
}
Now, I have two code interpreters running: Once, there is the one (A) that runs phpunit (and makes the HTTP request). Second, there is the one (B) associated with the HTTP server listening on localhost:8080.
As the code above with the call to getJson() is running inside a PHP interpreter (B) outside the phpunit instance I cannot mock directly as far as I understand. I would have to change the main app's code if I am not mistaken.
Of course, I could provide (more or less) useful data in the test function and let the service->doSomething() method do its job but then I am no longer testing only a subset of functions but I am doing functional or system testing. Also, this makes it harder to generate well-aimed test cases if all these side-effects need to be taken into account.
Related
Given the following pest test:
it('allows admins to create courses', function () {
$admin = User::factory()->admin()->create();
actingAs($admin);
$this->get('/courses')->assertDontSee('WebTechnologies');
$this->followingRedirects()->post('/courses', [
'course-name' => 'WebTechnologies',
])->assertStatus(200)->assertSee('WebTechnologies');
});
The above should fully work; however, the second request post('/courses')...
fails saying that:
Failed asserting that <...> contains "WebTechnologies".
If I remove the first request:
it('allows admins to create courses', function () {
$admin = User::factory()->admin()->create();
actingAs($admin);
$this->followingRedirects()->post('/courses', [
'course-name' => 'WebTechnologies',
])->assertStatus(200)->assertSee('WebTechnologies');
});
The test passes.
If I remove the second request instead:
it('allows admins to create courses', function () {
$admin = User::factory()->admin()->create();
actingAs($admin);
$this->get('/courses')->assertDontSee('WebTechnologies');
});
It also passes.
So why should the combination of the two cause them to fail? I feel Laravel is caching the original response, but I can't find anything within the documentation supporting this claim.
I have created an issue about this on Laravel/Sanctum as my problem was about authentication an stuff...
https://github.com/laravel/sanctum/issues/377
One of the maintainers of Laravel Said:
You can't perform two HTTP requests in the same test method. That's not supported.
I would have wanted a much clearer explanation on why it's not supported.
but I guess, we would never know. (Unless we dive deep into the Laravel framework and trace the request)
UPDATE:
My guess is that, knowing how Laravel works, for each REAL request Laravel initializes a new instance of the APP...
but when it comes to Test, Laravel Initializes the APP for each Test case NOT for each request, There for making the second request not valid.
here is the file that creates the request when doing a test...
vendor/laravel/framework/src/Illuminate/Foundation/Testing/Concerns/MakesHttpRequests.php
it's on the call method line: 526 (Laravel v9.26.1)
as you can see...
Laravel only uses 1 app instance... not rebuilding the app...
Line 528: $kernel = $this->app->make(HttpKernel::class);
https://laravel.com/docs/9.x/container#the-make-method
the $kernel Variable is an instance of vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php
My guess here is that the HttpKernel::class is a singleton.
P.S. I can do a little more deep dive, but I've procrastinated too much already by answering this question, it was fun thou.
TL;DR.
You can't perform two HTTP requests in the same test method. That's not supported.
UPDATE:
I was not able to stop myself...
I found Laravel initializing Kernel as a singleton
/{probject_dir}/bootstrap/app.php:29-32
Please make sure to not use any classic singleton pattern which isn't invoked with singleton binding or facades.
https://laravel.com/docs/9.x/container#binding-a-singleton
$this->app->singleton(Transistor::class, function ($app) {
return new Transistor($app->make(PodcastParser::class));
});
The Laravel app won't be completely restarted during tests unlike different incoming HTTP requests - even if you call different API endpoints in your tests
I am currently busy with a PSR-7 project with responses and requests.
Currently we are setting up an application in our index.php by doing something like:
$app = new Application();
$app->loadConfiguration(
'../config/global.yml',
);
// Should return the response?
$app->run((new ServerRequestFactory())->createServerRequestFromGlobals());
Here the run method also calls an emit method that is responsible for sending the headers and printing the body of the response.
The request and respons are now linked together in one call which makes it hard to test since you don't want to send the response with the headers straight to PHPUnit.
I have removed the emit call in the chain of the run method and added this to the index after the run method call:
// Send the response.
$app->send();
This way they are decoupled but the downside is I now have to hold a instance of my response in a response property inside my Application.php($app) class.
I want to move the response instance to the response class itself but my co-workers thinks a class should never hold an instance of itself. Yet when I look at frameworks this happens quite a lot. Is he right about this?
What arguments can I make to decouple my request and response besides easier testing?
I am pretty new to unit testing, one of the arguments I have already heard is that I should not test the full application anyways but rather separate components and therefore should not be worried about de-coupling the request and response.
I'm (we're) creating a package that acts as a core component for our future CMS and of course that package needs some unit tests.
When the package registeres, the first thing it does is set the back/frontend context like this:
class FoundationServiceProvider extends ServiceProvider
{
// ... stuff ...
public function register()
{
// Switch the context.
// Url's containing '/admin' will get the backend context
// all other urls will get the frontend context.
$this->app['build.context'] = request()->segment(1) === 'admin'
? Context::BACKEND
: Context::FRONTEND;
}
}
So when I visit the /admin url, the app('build.context') variable will be set to backend otherwise it will be set to `frontend.
To test this I've created the following test:
class ServiceProviderTest extends \TestCase
{
public function test_that_we_get_the_backend_context()
{
$this->visit('admin');
$this->assertEquals(Context::BACKEND, app('build.context'));
}
}
When I'm running the code in the browser (navigating to /admin) the context will get picked up and calling app('build.context') will return backend, but when running this test, I always get 'frontend'.
Is there something I did not notice or some incorrect code while using phpunit?
Thanks in advance
Well, this is a tricky situation. As I understand it, laravel initiates two instances of the framework when running tests - one that is running the tests and another that is being manipulated through instructions. You can see it in tests/TestCase.php file.
So in your case you are manipulating one instance, but checking the context of another (the one that did not visit /admin and is just running the tests). I don't know if there's a way to access the manipulated instance directly - there's nothing helpful in documentation on this issue.
One workaround would be to create a route just for testing purposes, something like /admin/test_context, which would output the current context, and the check it with
$this->visit('admin/test_context')->see(Context::BACKEND);
Not too elegant, but that should work. Otherwise, look around in laravel, maybe you will find some undocumented feature.
I am testing the endpoints/API of a web application. I have multiple small tests that depend on the return values of the preceding tests. Some of the tests even depend on side effects that are made by the preceding tests. Here's an example of how it goes (the numbered list items represent individual test cases):
Make a request to an endpoint and assert the http code is 200, return the response
Parse the response body and do some assertions on it, return the parsed response body
Do some assertions on the debug values of the parsed response body
Make a new request to another endpoint and assert the http code is 200, return the response
Parse the response body and assert that the side effects from test 1 actually took place
As you can see the tests sort of propagate from test 1, and all the other tests depend on its return value or side effects.
Now I want to execute these tests with data from a data provider to test the behavior with multiple users from our application. According to the phpunit documentation, this is not possible. From the docs:
When a test depends on a test that uses data providers, the depending test will be executed when the test it depends upon is successful for at least one data set. The result of a test that uses data providers cannot be injected into a depending test.
Just to be clear, what I want is for test 1 to execute x number of times with y values, and have all the other tests propagate its return value or check its side effects each time.
After some googling, the only solution that comes to mind is to put all the tests into one single test to remove all dependencies. However I have multiple test suites with this behavior, and some of the tests would get really big and unwieldy.
So, how can I keep the dependencies between small test cases while using data providers? I'm using php 5.5 along with Silex 1.3 and phpunit 4.8
EDIT: I should mention that my tests are extending Silex' WebTestCase, though I'm not sure if it makes a difference.
Here's an example in case I was unclear:
public function testValidResponse()
{
$client = $this->createClient();
$client->request('POST', '/foo', $this->params);
$this->assertEquals(200, $client->getResponse()->getStatusCode());
return $client->getResponse();
}
/**
* #depends testValidResponse
*/
public function testStatusIsOk(Response $response)
{
$json = json_decode($response->getContent(), true);
$this->assertTrue($json['status']);
return $json;
}
/**
* #depends testStatusIsOk
*/
public function testExecutionTime($json)
{
$this->assertLessThan($this->maxExecutionTime, $json['debug']['executionTimeSec']);
}
/**
* #depends testValidResponse
*/
public function testAnotherEndpointValidResponse()
{
$client = $this->createClient();
$client->request('GET', '/bar');
$this->assertEquals(200, $client->getResponse()->getStatusCode());
return $client->getResponse();
}
/**
* #depends testAnotherEndpointValidResponse
*/
public function testSideEffectsFromFirstTest(Response $response)
{
// ...
}
I think the main problem is that the tests are too complex, and should not depend that much on each other. The solution was to reduce some of the complexity by refactoring the tests. Here's a rough outline of what I did:
Moved some of the more complex test cases from Integration testing
to Unit testing. So I'm not testing the endpoint any more, but the methods that are executed when you go to the endpoint.
Added a generic test case that operates on data from a data provider which includes, not exclusively, the urls to the different endpoints. This test case simply tests that the endpoints are returning the expected http code and some other small things. This allowed me to put all of my Integration testing into one simple test case.
I'm pretty happy with the solution, I deleted 7-8 test classes, and the complex testing that got moved to Unit testing is also simplified.
The more I read about dependency injection the more I get confused. I know what it is for, that is not the problem. Trying to do some design on paper this is what I came up with and somehow it seems to me I am overlooking something.
First I imagined building an actual server that would accept incoming requests and returns responses to the user.
class Server {
private $responseBuilder;
public function __construct($responseBuilder) {
$this->responseBuilder = $responseBuilder;
}
public function run() {
// create socket, receive request
$response = $this->responsebuilder->build($request);
// send response
}
}
class Response {
private $method;
private $message;
private $url;
// getters & setters
}
class ServerBuilder {
public build() {
// construction logic
return new Server(new ResponseBuilder());
}
}
Since Apache is used to handle server requests we could replace the server with something that just send the response.
$bldr = new ResponseBuilder();
$response = $bldr->build();
// send response some way
Note that ResponseBuilder has direct access to the request ($_SERVER['..'])
and so it has everything it needs to choose the right response.
PHP however allows us to build and send responses inline. So we could have a Controller object for each page or something else that send the response and have a builder for that.
$bldr = new ControllerBuilder();
$controller = $bldr->build();
$controller->run();
class ExampleController implements Controller {
public function run() {
header("HTTP/1.1 404 Not Found");
echo 'sorry, page not found';
}
}
This all makes sense to me. But let's look at the server example again.
It calls $responseBuilder->build() and gets a response back. But this would mean that the builder (or other builders if we split it) is also responsible for anything else that might occur like authenticating a user, writing to the database,... and I can't get my head around the fact that writing to a database would be part of the object graph construction.
It would be like: Send me your request. Oh you want the homepage? I will build you your response and while I'm at it I will also do some things that have nothing to do with building it like logging what I just did and saving some of your data in a cookie and sending a mail to the administrator that you are the first visitor on this page ever, ...
You should decouple them. You have a few assumptions that I think are a bit strange. Let's start with them.
The main purpose of an incoming http request is to give back some html
I have built PHP backends that only return JSON, instead of HTML. I had a really strong border between back and front end. I only used the backend to give me data from the database, or add/edit data in the databse. The front end was just a PHP script that would build the pages any way i wanted.
Since it is the web there is in theory no use for setters since
everything can be injected in the constructor
You could use the constructor, but you don't have too. You can use setters. Dependency injection is actually just turning the flow around.
You are on the right track though. You want some class that is responsible for building your pages. So, make it only responsible for your building your pages, and take out the other responsibilities. Things like logging, authentication etc should be outside of that.
For instance if you want logging, you could have your builder create your page, and your logger could then listen to all the things your builder is doing (with the observer pattern for instance). So if your builder says: "i created the home page", you can log it with your logger, who is actually listening to your builder.
Authentication for instance should happen even before your builder starts. You don't want your builder to go to work if you can already figure out that a user is not supposed to be on a page. You could use a database for that, and whitelist any usertype/pagerequest combination.
Then for data handling, i would create a backend, that only handles requests that are supposed to give back data, or save it. The front end could then communicate to get it's content by pulling it.
I hope this clears up a few things, but I'll be happy to answer more indept questions.