I have an API, written in PHP with Slim (3, will be upgraded to 4 soon), I also have an extensive specification written in openapi3 YML format (I could convert to another format if necessary, but I think it would be the best to keep oas3). Currently I am testing all endpoint against this specification with Dredd. This is a Tool which goes through a API specification an, sends example data to the real API and checks if the result matches the spec. That works, but not very good. For a test run I have to wipe out and re-initialize the db with PHP and then run Dredd with npm. Plus since Dredd does not support all features of oas3 I have to do some conversion first.
Since I have some Unit-tests with PHPUnit anyway, I would love to run the other tests with PHPUnit as well (and get rid of the Node-stuff at all). I found http://opensource.byjg.com/php-swagger-test/ and https://github.com/Maks3w/SwaggerAssertions. Both of them provide this very feature, but I would have to write a separate function for every endpoint, containing sample data and so on - stuff which is already in the API spec. Any Idea how I could avoid this afford and just use my API spec as test defintion with PHPUnit or at least any PHP library?
Example from the specs:
/users:
get:
summary: get a list of users
description: get info about all registered users - in whole application or in a workspace.
parameters:
- in: header
name: AuthToken
schema:
$ref: '#/components/schemas/auth'
example:
at: 132token
- in: query
name: ws
description: id of a workspace to get users from. can be omitted for all users in system
required: false
examples:
a:
value: 0
b:
value: 1
With byjg php-swagger-test I would have to write something like this
public function usersA()
{
$request = new \ByJG\Swagger\SwaggerRequester();
$request
->withMethod('GET')
->withPath("/users")
->withHEader(blabla)
->withRequestBody(['ws'=>0]);
$this->assertRequest($request);
}
public function usersB()
{
$request = new \ByJG\Swagger\SwaggerRequester();
$request
->withMethod('GET')
->withPath("/users")
->withHEader(blabla)
->withRequestBody(['ws'=>1]);
$this->assertRequest($request);
}
Two tests (functions) containing only information which is already in the spec. Is there a better tool/way to run tests over all endpoints against the spec without writing all those?
Related
I am working on an extension (app) of nextcloud (which is based on Symfony). I have a helper class to extract data from the request that is passed by the HTTP server to PHP. A much-reduced one could be something like this (to get the point here):
<?php
namespace OCA\Cookbook\Helpers;
class RequestHelper {
public function getJson(){
if($_SERVER['Request_Method' === 'PUT'){ // Notice the typos, should be REQUEST_METHOD
$raw = file_get_content('php://input');
return json_decode($raw, true);
} else { /* ... */ }
}
}
Now I want to test this code. Of course, I can do some unit testing and mock the $_SERVER variable. Potentially I would have to extarct the file_get_content into its own method and do a partial mock of that class. I get that. The question is: How much is this test worth?
If I just mimick the behavior of that class (white box testing) in my test cases I might even copy and paste the typo I intentionally included here. As this code is an MWE, real code might get more complex and should be compatible with different HTTP servers (like apache, nginx, lighttpd etc).
So, ideally, I would like to do some automated testing in my CI process that uses a real HTTP server with different versions/programs to see if the integration is working correctly. Welcome to integration testing.
I could now run the nextcloud server with my extension included in a test environment and test some real API endpoints. This is more like functional testing as everything is tested (server, NC core, my code and the DB):
phpunit <---> HTTP server <---> nextcloud core <---> extension code <---> DB
^
|
+--> RequestHelper
Apart from speed, I have to carefully take into account to test all possible paths through the class RequestHelper (device under test, DUT). This seems a bit brittle to me in the long run.
All I could think of is adding a simple endpoint only for testing the functionality of the DUT, something like a pure echo endpoint or so. For the production use, I do not feel comfortable having something like this laying around.
I am therefore looking for an integration test with a partial mock of the app (mocking the business logic + DB) to test the route between the HTTP server and my DUT. In other words, I want to test the integration of the HTTP server, nextcloud core, my controller, and the DUT above without any business logic of my app.
How can I realize such test cases?
Edit 1
As I found from the comments the problem statement was not so obviously clear, I try to explain a bit at the cost of the simplicity of the use-case.
There is the nextcloud core that can be seen as a framework from the perspective of the app. So, there can be controller classes that can be used as targets for URL/API endpoints. So for example /apps/cookbook/recipe/15 with a GET method will fetch the recipe with id 15. Similarly, with PUT there can be a JSON uploaded to update that recipe.
So, inside the corresponding controller the structure is like
class RecipeController extends Controller {
/* Here the PUT /apps/cookbook/recipe/{id} endpoint will be routed */
public function update($id){
$json = $this->requestHelper->getJson(); // Call to helper
// Here comes the business logic
// aka calls to other classes that will save and update the state
// and perform the DB operation
$this->service->doSomething($json);
// Return an answer if the operation terminated successfully
return JsonResponse(['state'=>'ok'], 200);
}
}
I want to test the getJson() method against different servers. Here I want to mock at least the $this->service->doSomething($json) to be a no-op. Ideally, I would like to spy into the resulting $json variable to test that exactly.
No doubt, in my test class it would be something like
class TestResponseHandler extends TestCase {
public function setUp() { /* Set up the http deamon as system service */}
public testGetJson() {
// Creat Guzzle client
$client = new Client([
'base_uri' => 'http://localhost:8080/apps/cookbook',
]);
// Run the API call
$headers = ...;
$body = ...;
$response = $client->put('recipe/15', 'PUT', $headers, $body);
// Check the response body
// ....
}
}
Now, I have two code interpreters running: Once, there is the one (A) that runs phpunit (and makes the HTTP request). Second, there is the one (B) associated with the HTTP server listening on localhost:8080.
As the code above with the call to getJson() is running inside a PHP interpreter (B) outside the phpunit instance I cannot mock directly as far as I understand. I would have to change the main app's code if I am not mistaken.
Of course, I could provide (more or less) useful data in the test function and let the service->doSomething() method do its job but then I am no longer testing only a subset of functions but I am doing functional or system testing. Also, this makes it harder to generate well-aimed test cases if all these side-effects need to be taken into account.
I have the following simple PHP method like the following
/**
*
* (swagger annotation to be called from a different class)
*
*/
public function getApiCall()
{
//Do something
}
and I need to include long Swagger documentation into the annotation above the method, so
is it possible to write the annotation in a different class ? and call it here with something like
/**
*
*call('App\Http\Controllers\testAnnotation');
*/
The main purpose is to have a clean class without so many lines of documentation and annotations in it.
Loading "annotations from a different class" is not something that makes a lot of sense. Annotations are read in the annotated code, that's their whole purpose.
But if you want to keep configuration and code separated, you do not have to use Swagger-Php to generate your swagger configuration file.
The package is simply a convenience way to generate the swagger.json file from code annotations.
But if you do not want to use annotations in the first place, and keep your classes clean from extraneous configuration (something that I personally applaud), just... do not use Swagger-Php and build your own configuration files outside of your classes.
You could even write it in YAML, if you feel more comfortable than writing JSON by hand. For example::
openapi: 3.0.0
info:
title: 'Search API'
version: 1.0.0
servers:
- url:
description: Current host server
- url: https:your-server.com
description: Prod server
paths:
/foo:
post:
summary: 'Creates a new foo'
description: 'Builds a new Foo and makes it available to Bar'
requestBody:
description: 'Foo '
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/Foo'
responses:
'201':
description: Foo created
'202':
description: Foo queued, it will be eventually created.
components:
schemas:
Foo:
type: object
required:
- name
- size
properties:
name:
type: string
size:
type: integer
This, once converted to JSON (there are many libraries to do this, or you could even use a free service like this one), you can feed the resulting JSON to swagger directly.
E.g. the YAML above parses to this JSON file. You can easily test it out by heading to the Swagger demo instance, and past the JSON URL in the "explore" location bar, and you'll get something like this:
In the end, it's not much more work than using annotations (if any more work at all), and you can keep your entity classes clean from configuration concerns.
Codeception has documentation for functional tests at: https://codeception.com/docs/05-UnitTests
So following that on my laravel/homestead project I do as follows:
in functional.sute.yml:
class_name: FunctionalTester
modules:
enabled:
- Laravel5
- \Helper\Functional
My test:
<?php
class LoginCest
{
public function _before(FunctionalTester $I)
{
}
public function _after(FunctionalTester $I)
{
}
// tests
public function tryLogin (FunctionalTester $I)
{
$I->amOnPage('/login');
$I->fillField('email', 'someemail');
$I->fillField('password', 'somepw');
$I->click('Login');
$
I->see('some text');
}
}
So when I run the test, it fails:
There was 1 error:
---------
1) LoginCest: Try login
Test tests/functional/LoginCest.php:tryLogin
[ExternalUrlException] Codeception\Module\Laravel5 can't open external URL: http://myapp.test/login
Scenario Steps:
4. $I->click("Login") at tests/functional/LoginCest.php:20
3. $I->fillField("password","somepw") at tests/functional/LoginCest.php:19
2. $I->fillField("email","someemail") at tests/functional/LoginCest.php:18
1. $I->amOnPage("/login") at tests/functional/LoginCest.php:17
#1 Codeception\Lib\InnerBrowser->click
#2 /home/vagrant/Code/my-app/tests/_support/_generated/FunctionalTesterActions.php:1114
#3 /home/vagrant/Code/my-app/tests/functional/LoginCest.php:20
#4 LoginCest->tryLogin
ERRORS!
Tests: 1, Assertions: 0, Errors: 1.
My app url is someapp.test which is running on homestead.
In looking at my LoginController I see:
$this->redirectTo();
at the very end.
Now I understand that functional tests do not require a webserver and I could probably make it work using the acceptance test. But really having a hard time understanding on why anyone would use codeception to do functional tests if you cant even specify a url. Also why would codeception use a login example for functional test when others may face similar issues?
Some background first.
URL consists of these main parts: PROTOCOL://DOMAIN:PORT/URI?QUERY_STRING#HASH
HASH is only used in client side and not passed to webserver, so it can't be used for routing.
PROTOCOL and PORT can be used for routing, but that is very unusual.
Some websites display different contents depending on DOMAIN which is used to access them,
but most only use URI and/or QUERY_STRING parts for routing and displaying the right page.
The main distinction of functional testing using Codeception is that it doesn't require webserver and because of that mostly doesn't care about domain names.
Website code usually doesn't care if it is accessed using http://myapp.test/ or http://google.com/ and happily returns your front page for either.
Though if a click on http://google.com/ link rendered your front page it almost certainly would be wrong.
To prevent that, external domain check was implemented years ago.
All internal links in your website must have no domain component or it must match the one passed using Host header.
Exception is made for domains that are using for domain based routing, such domains can be used in tests.
I've been using Couchbase for a large project where we only consume data, but have no requrirement to actually write to the Couchbase cluster. I want to write some management tools and be able to create Design Documents and Views through the API and not the web console.
Creating Documents is not an issue at all, but whenever I use setDesignDoc() I get the following error message:
Fatal error: Uncaught exception 'CouchbaseLibcouchbaseException' with message 'Failed to store design doc: Invalid input/arguments' in ....
I can get the actual view data from the source with getDesignDoc() and use it for the creating that design document on the destination server like so:
$connandleDestination->setDesignDoc( "myDesignDoc", $connSource->getDesignDoc("myDesignDoc") );
And that's where I get the above error message.
I'm using the PHP SDK by the way with the latest version against Couchbase 2.5.1
Can you confirm if what you've given as your example is the exact code you're using?
$connandleDestination->setDesignDoc( "myDesignDoc", $connSource->getDesignDoc("myDesignDoc") );
What you've given would not work as, from the API docs, the syntax is as follows:
setDesignDoc(string $name, string $document) : bool
Hence, it would appear you are trying to set a new doc, myDesignDoc to be equal to the design doc myDesignDoc, which would either not exist, or have no effect (as setting something to equal itself causes no change - and creating a new design doc with the name of an existing one will simply overwrite it).
Did you instead mean to just use get()? get() would return a document (which would be valid in the setDesignDoc input, and that document could have the same name as the new design doc to be created.
I've been working on a project (which I'll keep specific details out of this post with randomized data) that involves integrating our system (PHP 5.3.x+) with an API (they provided a SDK) of a major company. They provided a WSDL and claimed ours needed to match their methods and they provided examples of how output (XML generated by the Soap Server) should look.
Right now, everything has been working as expected. When I send a XML request from SoapUI (an app I'm using to test) it all processes properly and such, but the XML output isn't matching closely with their examples and we believe they said we must be close to their examples.
Basically, we created an agnostic class we initialize with a service name and it initializes into a non-agnostic class which is used via the following:
/**
* The following is used to process Soap Server based on config and any optional settings.
*
* #param string $className
* #param array $options
* #param object $config
* #return Zend_Soap_Server
*/
public static function init($className, Array $options = null, $config = null)
{
// Used to define the class and return object.
$soap_server = new Zend_Soap_Server(null, $options);
$soap_server->setClass($className, null, (isset($config) ? $config : null));
$soap_server->handle();
exit;
}
The problem itself lies within the outputted response. How would you guys suggest we build the XML output if they're very specific about everything?
1.) One of our methods is moneyTransferRequest. When I send the XML over for this, it does find the method and processes it. However, they want it to show the method name, in the response, as moneyTransferResponse but it outputs moneyTransferRequestResponse.
2.) Our output (for variables and such sent back as an object) has multiple variables, we'll say $money for example. The field for this would return as:
<money xsi:type="xsd:string">10.0</money>
They would like it to be:
<ns1:money xsi:type="xsd:string">10.0</money>
in the return.
I appreciate any help and input on the subject.
The key feature of SOAP is that it uses XML, and XML can come in a bunch of different styles but still mean the same.
I think (but I can only guess because you didn't provide details) that your two issues might be non-existing.
1.) The name of the response XML structure should align with the name mentioned in the WSDL. YOU are publishing the WSDL, so you should check if these two match. Note that the important entry point is the SOAP method - everything thereafter is defined in the WSDL itself, any consuming client should be able to figure it out as long as the names mentioned are correctly used.
2.) This is basically the same, but even easier: XML allows to use namespaces, and these can be defined in several locations, with the result being not literally the same, but every XML parser will understand that they are the same. So you should check whether the namespace that is required as "ns1" is mentioned in the XML header of your response. Every XML document has a base namespace, which does not need to be repeated on every element that belongs to it.
This is the case with the <money> element. Your style of writing uses that base namespace, their style of writing uses a namespace shortcut ns1 also introduced in the XML header, but not declared as the base namespace. So as long as there are traces of the correct XML namespace in both responses, I'd assume they are equivalent.
And the bad news would be that you cannot change how the PHP SoapServer generates the XML. You'd need to create your own implementation of a SOAP server, which I'd say is a complete waste of resources.