I'm a bit lost between definitions, generated code and many things that are a bit of a black box, even after debugging.
But let's start at the beginning. I have an API written with node using NestJS as framework. NestJS automatically creates the swagger/openapi json file. Using swagger-codegen I create a PHP class to access the API from another server. Works like a charm for simple API request.
Now the problem are API request returning a bigger response, i.e. >1000 rows from one or more DBs. Do make the client not wait a long time and create a big JSON response on the server I've switched to NDJSON, which splits the reponse in chunks of smaller JSON parts, each on it's own line. This also works when I create my request by hand with curl or a HTTP wrapper in PHP using fopen and fread. The response type is application/x-ndjson.
But the code generated by swagger-codegen is always waiting until the whole response has been received. It's even worse, because it fails to decode NDJSON with json_decode() and just returns null. Underneath Guzzle is used, which uses PSR7 streams for the response.
Now I could just skip the autogenerated code for the NDJSON endpoints. But I'd prefer not to add special handling and lose all the useful generated checks.
So is it somehow possible to make swagger-codegen give access to the stream of the response? Am I missing a parameter to codegen or something in the swagger JSON? It does have a produce with application/x-ndjson.
Do answer my own question it is possible, but not easily - meaning there is no option or parameter.
First create a class that extends the autogenerated API class. There you have access to all of the protected methods. We assume the endpoint or api method is called testMethodGet. The only thing that can be reused is the request method, i.e. testMethodGetRequest, but it does all of the client side validation and transforming of input data, so that's already a big win. Do also get some of the boilerplate result validation you can copy the "http info method", i.e. testMethodGetWithHttpInfo (use async version if you wish). Remove the if/else block after $responseBody = $response->getBody(); and in the return replace the ObjectSearializer line with just $responseBody.
What you are now getting back is the body as PSR7 stream, but there is one last catch. By default the whole response is drained into a temporary file. To really stream the result body from the server in the created method add an option to $options: $options['stream'] = true; Now the method returns as soon as the body is started. No need to wait for the whole body content.
One more thing. As the result is a PSR7 stream you can use $stream->detach() to get the PHP stream resource if you prefer to use the normal file and stream methods.
I'm building an api at my company using laravel.
The problem I'm encountering is that if you send an api request without defining the correct header with the request you will get html back if there is a failure e.g. authorization failure or findOrFail() failure.
My thinking is that you never want to return html (even if the user has the wrong header).
I have a couple of solutions. In BeforeMiddleware.php I can manually insert a header into the request such as:
// Check if we are on an api route
$apiRoute = strncmp($uri, '/api/', 5) == 0;
// Insert the request header to force json response
if ($apiRoute){
$language = $request->header->add('Accept', 'application/json');
}
The 2nd solutions would be to throw an error if they don't have the correct header.
What would be the best way to enforce a json response, what is a good practice for handling api responses in laravel?
Once you detected that you are on your api path you are out of the woods and can indeed tackle your problem in the app\Exceptions\Handler.php file like suggested on How do you force a JSON response on every response in Laravel?.
For an open source project I created JSON exception objects by Microsoft format as output, but you can choose the jsonapi format (http://jsonapi.org/examples/#error-objects-basics) as you like:
https://github.com/StadGent/laravel_site_opening-hours/blob/develop/app/Exceptions/Handler.php
(note that on this implementation it is indeed depending from the headers, but you can use your path detection I think)
In the book of laravel I read, and also my co-worker who has experience with laravel said that generating JSON I should in laravel way.
Why do I need to do this:
Route::get('markdown/response', function()
{
$data = array('iron', 'man', 'rocks');
return Response::json($data);
});
As I read it sends also content-type header when using this.
When I was using codeigniter I used to do simply this:
echo json_endode($data);
and never ever ever had any problems. Even if it is not set content type. Actually I dont know maybe php sets it automatically, but since I did not have problems, I did not care.
And when using 'new' technology I really want to know why it is better than good old one.
With respect, by not providing a content-type header, you were doing it "wrong" when coding in CodeIgniter.
Most clients (browsers, ajax requests, especially jQuery) can still can guess how to handle the response correctly and so probably "just worked" for you. You were likely always implicitly returning a Content-Type: text/html with your response, which is a default header in CodeIgniter.
You should always return a proper content type with your HTTP responses so the consuming client knows how to treat this content. Note that this is a mechanism of HTTP as defined in specification, not specific to any framework or even a language.
Response::json()
The above code is just a convenience function, where Laravel will automatically set the application/json header for you, as well as convert an array of data into JSON format. The only effective difference from your CodeIgniter code is the setting of the header, as you've pointed out.
It's worth noting that the Response object extends Symfony's response object, which is very "powerful" - in other words, it's a very good implementation of the HTTP protocol.
The response object returned from Response::json (and other Response static methods) are highly modifiable.
$response = Response::json($data);
$response->header('Content-Type', 'application/json');
return $response;
You can check for more available methods in the Laravel and Symfony code API.
http://laravel.com/api/class-Illuminate.Http.Response.html
http://api.symfony.com/2.1/Symfony/Component/HttpFoundation/Response.html
Just because it worked doesn't mean it wasn't wrong. JSON isn't HTML, so text/html is an inaccurate Content-Type for it.
Sending the correct header means libraries like jQuery understand what sort of data they're getting back, and thus are able to handle it on their own without guidance. Browsers may also do things like pretty-printing the JSON data or making it otherwise easier to read.
Depends what you are trying to do with the route. if you only want to return json data you can just return json_encode($data) and that will work, To actually return a json response for use with something like an ajax request you need the headers set properly or the accepting route just thinks its getting a string. Response::json is for setting the response which sets the headers appropriately.
This is something that has been bugging me for a while.. I'm building of a RESTful API that has to receive files on some occasions.
When using HTTP POST, we can read data from $_POST and files from $_FILES.
When using HTTP GET, we can read data from $_GET and files from $_FILES.
However, when using HTTP PUT, AFAIK the only way to read data is to use the php://input stream.
All good and well, untill I want to send a file over HTTP PUT. Now the php://input stream doesn't work as expected anymore, since it has a file in there as well.
Here's how I currently read data on a PUT request:
(which works great as long as there are no files posted)
$handle = fopen('php://input', 'r');
$rawData = '';
while ($chunk = fread($handle, 1024)) {
$rawData .= $chunk;
}
parse_str($rawData, $data);
When I then output rawData, it shows
-----ZENDHTTPCLIENT-44cf242ea3173cfa0b97f80c68608c4c
Content-Disposition: form-data; name="image_01"; filename="lorem-ipsum.png"
Content-Type: image/png; charset=binary
�PNG
���...etc etc...
���,
-----ZENDHTTPCLIENT-8e4c65a6678d3ef287a07eb1da6a5380
Content-Disposition: form-data; name="testkey"
testvalue
-----ZENDHTTPCLIENT-8e4c65a6678d3ef287a07eb1da6a5380
Content-Disposition: form-data; name="otherkey"
othervalue
Does anyone know how to properly receive files over HTTP PUT, or how to parse files out of the php://input stream?
===== UPDATE #1 =====
I have tried only the above method, don't really have a clue as to what I can do else.
I have gotten no errors using this method, besides that I don't get the desired result of the posted data and files.
===== UPDATE #2 =====
I'm sending this test request using Zend_Http_Client, as follows:
(haven't had any problems with Zend_Http_Client so far)
$client = new Zend_Http_Client();
$client->setConfig(array(
'strict' => false,
'maxredirects' => 0,
'timeout' => 30)
);
$client->setUri( 'http://...' );
$client->setMethod(Zend_Http_Client::PUT);
$client->setFileUpload( dirname(__FILE__) . '/files/lorem-ipsum.png', 'image_01');
$client->setParameterPost(array('testkey' => 'testvalue', 'otherkey' => 'othervalue');
$client->setHeaders(array(
'api_key' => '...',
'identity' => '...',
'credential' => '...'
));
===== SOLUTION =====
Turns out I made some wrong assumptions, mainly that HTTP PUT would be similar to HTTP POST. As you can read below, DaveRandom explained to me that HTTP PUT is not meant for transferring multiple files on the same request.
I have now moved the transferring of formdata from the body to url querystring. The body now holds the contents of a single file.
For more information, read DaveRandom's answer. It's epic.
The data you show does not depict a valid PUT request body (well, it could, but I highly doubt it). What it shows is a multipart/form-data request body - the MIME type used when uploading files via HTTP POST through an HTML form.
PUT requests should exactly compliment the response to a GET request - they send you the file contents in the message body, and nothing else.
Essentially what I'm saying is that it is not your code to receive the file that is wrong, it is the code that is making the request - the client code is incorrect, not the code you show here (although the parse_str() call is a pointless exercise).
If you explain what the client is (a browser, script on other server, etc) then I can help you take this further. As it is, the appropriate request method for the request body that you depict is POST, not PUT.
Let's take a step back from the problem, and look at the HTTP protocol in general - specifically the client request side - hopefully this will help you understand how all of this is supposed to work. First, a little history (if you're not interested in this, feel free to skip this section).
History
HTTP was originally designed as a mechanism for retrieving HTML documents from remote servers. At first it effectively supported only the GET method, whereby the client would request a document by name and the server would return it to the client. The first public specification for HTTP, labelled as HTTP 0.9, appeared in 1991 - and if you're interested, you can read it here.
The HTTP 1.0 specification (formalised in 1996 with RFC 1945) expanded the capabilities of the protocol considerably, adding the HEAD and POST methods. It was not backwards compatible with HTTP 0.9, due to a change in the format of the response - a response code was added, as well as the ability to include metadata for the returned document in the form of MIME format headers - key/value data pairs. HTTP 1.0 also abstracted the protocol from HTML, allowing for the transfer of files and data in other formats.
HTTP 1.1, the form of the protocol that is almost exclusively in use today is built on top of HTTP 1.0 and was designed to be backwards compatible with HTTP 1.0 implementations. It was standardised in 1999 with RFC 2616. If you are a developer working with HTTP, get to know this document - it is your bible. Understanding it fully will give you a considerable advantage over your peers who do not.
Get to the point already
HTTP works on a request-response architecture - the client sends a request message to the server, the server returns a response message to the client.
A request message includes a METHOD, a URI and optionally, a number of HEADERS. The request METHOD is what this question relates to, so it is what I will cover in the most depth here - but first it is important to understand exactly what we mean when we talk about the request URI.
The URI is the location on the server of the resource we are requesting. In general, this consists of a path component, and optionally a query string. There are circumstances where other components may be present as well, but for the purposes of simplicity we shall ignore them for now.
Let's imagine you type http://server.domain.tld/path/to/document.ext?key=value into the address bar of your browser. The browser dismantles this string, and determines that it needs to connect to an HTTP server at server.domain.tld, and ask for the document at /path/to/document.ext?key=value.
The generated HTTP 1.1 request will look (at a minimum) like this:
GET /path/to/document.ext?key=value HTTP/1.1
Host: server.domain.tld
The first part of the request is the word GET - this is the request METHOD. The next part is the path to the file we are requesting - this is the request URI. At the end of this first line is an identifier indicating the protocol version in use. On the following line you can see a header in MIME format, called Host. HTTP 1.1 mandates that the Host: header be included with every request. This is the only header of which this is true.
The request URI is broken into two parts - everything to the left of the question mark ? is the path, everything to the right of it is the query string.
Request Methods
RFC 2616 (HTTP/1.1) defines 8 request methods.
OPTIONS
The OPTIONS method is rarely used. It is intended as a mechanism for determining what kind of functionality the server supports before attempting to consume a service the server may provide.
Off the top of my head, the only place in fairly common usage that I can think of where this is used is when opening documents in Microsoft office directly over HTTP from Internet Explorer - Office will send an OPTIONS request to the server to determine if it supports the PUT method for the specific URI, and if it does it will open the document in a way that allows the user to save their changes to the document directly back to the remote server. This functionality is tightly integrated within these specific Microsoft applications.
GET
This is by far and away the most common method in every day usage. Every time you load a regular document in your web browser it will be a GET request.
The GET method requests that the server return a specific document. The only data that should be transmitted to the server is information that the server requires to determine which document should be returned. This can include information that the server can use to dynamically generate the document, which is sent in the form of headers and/or query string in the request URI. While we're on the subject - Cookies are sent in the request headers.
HEAD
This method is identical to the GET method, with one difference - the server will not return the requested document, if will only return the headers that would be included in the response. This is useful for determining, for example, if a particular document exists without having to transfer and process the entire document.
POST
This is the second most commonly used method, and arguably the most complex. POST method requests are almost exclusively used to invoke some actions on the server that may change its state.
A POST request, unlike GET and HEAD, can (and usually does) include some data in the body of the request message. This data can be in any format, but most commonly it is a query string (in the same format as it would appear in the request URI) or a multipart message that can communicate key/value pairs along with file attachments.
Many HTML forms use the POST method. In order to upload files from a browser, you would need to use the POST method for your form.
The POST method is semantically incompatible with RESTful APIs because it is not idempotent. That is to say, a second identical POST request may result in a further change to the state of the server. This contradicts the "stateless" constraint of REST.
PUT
This directly complements GET. Where a GET requests indicates that the server should return the document at the location specified by the request URI in the response body, the PUT method indicates that the server should store the data in the request body at the location specified by the request URI.
DELETE
This indicates that the server should destroy the document at the location indicated by the request URI. Very few internet facing HTTP server implementations will perform any action when they receive a DELETE request, for fairly obvious reasons.
TRACE
This provides an application-layer level mechanism to allow clients to inspect the request it has sent as it looks by the time it reaches the destination server. This is mostly useful for determining the effect that any proxy servers between the client and the destination server may be having on the request message.
CONNECT
HTTP 1.1 reserves the name for a CONNECT method, but does not define its usage, or even its purpose. Some proxy server implementations have since used the CONNECT method to facilitate HTTP tunnelling.
I've never tried using PUT (GET POST and FILES were sufficient for my needs) but this example is from the php docs so it might help you (http://php.net/manual/en/features.file-upload.put-method.php):
<?php
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
$fp = fopen("myputfile.ext", "w");
/* Read the data 1 KB at a time
and write to the file */
while ($data = fread($putdata, 1024))
fwrite($fp, $data);
/* Close the streams */
fclose($fp);
fclose($putdata);
?>
Here is the solution that I found to be the most useful.
$put = array();
parse_str(file_get_contents('php://input'), $put);
$put will be an array, just like you are used to seeing in $_POST, except now you can follow true REST HTTP protocol.
Use POST and include an X- header to indicate the actual method (PUT in this case). Usually this is how one works around a firewall which does not allow methods other than GET and POST. Simply declare PHP buggy (since it refuses to handle multipart PUT payloads, it IS buggy), and treat it as you would an outdated/draconian firewall.
The opinions as to what PUT means in relation to GET are just that, opinions. The HTTP makes no such requirement. It simply states 'equivalent' .. it is up to the designer to determine what 'equivalent' means. If your design can accept a multi-file upload PUT and produce an 'equivalent' representation for a subsequent GET for the same resource, that's just fine and dandy, both technically and philosophically, with the HTTP specifications.
Just follow what it says in the DOC:
<?php
/* PUT data comes in on the stdin stream */
$putdata = fopen("php://input", "r");
/* Open a file for writing */
$fp = fopen("myputfile.ext", "w");
/* Read the data 1 KB at a time
and write to the file */
while ($data = fread($putdata, 1024))
fwrite($fp, $data);
/* Close the streams */
fclose($fp);
fclose($putdata);
?>
This should read the whole file that is on the PUT stream and save it locally, then you could do what you want with it.
I'm developing a REST CodeIgniter Controller and need to POST and PUT to a URL in multiple formats.
The main formats I can think of would be:
XML
JSON
HTML Form data
There will be a response format defined in the URI e.g. /format/xml or /format/csv.
I don't want to have to define the request format in the URI as well.
I was wondering if anyone has any suggestions on finding out the request format so my script can complete the request.
I'm thinking it may be possible to get this data from the request headers "content-type"?
"content-type: text/xml" = XML
"content-type: application/json" = JSON
"content-type: text/plain" = HTLM Form data **i think!*
Would this method be robust or are there better approaches out there?
Thanks.
The content-type is the correct way to get the information you want.
Just make sure to throw an exception with some feedback and the correct http error code if the client calls it on the wrong format or do not pass the content-type header (you could also assume one content-type as default)
Also, you don't really have to use format/format_of_the_response . A better way would be to use the header Accept on the same way you use the header content-type
That method is robust as long as you know the your REST client will follow your content-type rule. If you control the client then this is fine. If you don't control the client then how acceptable is failure of the call when an unusual (or no) content-type is passed in?
Oh and for reference HTML form data has a content type of 'application/x-www-form-urlencoded'
I know this has been answered, but Phil Sturgeon wrote a REST Client and REST Server Library for codeigniter, might help you out a little bit.
http://github.com/philsturgeon/codeigniter-restclient