How to stream an HTTP file upload without the Content-Length header? - php

Is it possible to upload a file to an apache php server without including the content-length header ?
I am trying to stream a file that I am creating on the fly as a file upload. When I don't use the content-length header I got the apache "501 Method Not Implemented".
$sock = fsockopen($host,80,$errno, $error);
fwrite($sock, "POST $resource HTTP/1.1\r\n" .
"Host: $host\r\n\r\n");
fwrite($sock,fread($readHandle,filesize($file)));
If I include the content-length it works fine.
The server is reading from php://input

According to the HTTP spec you aren't technically required to specify the Content-Length header. From RFC 2616 14.13:
Applications SHOULD use this field to indicate the transfer-length of
the message-body, unless this is prohibited by the rules in section
4.4.
However, this is a pretty standard requirement for most servers, and they'll generally send back an error response if the Content-Length header is missing or incorrectly specified. For all intents and purposes, SHOULD in this case equates to MUST.
The problem is that (especially with keep-alive connections), the server doesn't know when your request message actually ends without the Content-Length header. The other option if you're streaming a request entity body is to send a Transfer-Encoding: chunked header and manually send one chunk of the entity body at a time.
So in summary, if you want to send an entity body with your message but don't want to send a Content-Length header, your only real option is to send a chunked HTTP message. This is basically required if you want to stream that entity body and don't know its length ahead of time.
How to chunk-encode an HTTP entity body for streaming ...
Transfer-Encoding: chunked means that you're encoding the entity body of the HTTP message according to the constraints laid out in RFC2616 Sec3.6.1. This encoding format can be applied to either requests or responses (duh, they're both HTTP messages). This format is extremely useful because it allows you to start sending an HTTP message right away before you know the size of the entity body or even exactly what that entity body is going to be. In fact, this is exactly what PHP does transparently for you when you echo any output without having sent a length header like header('Content-Length: 42').
I'm not going to get into details of chunked encoding -- that's what the HTTP spec is for -- but if you want to stream a request entity body you need to do something like this:
<?php
$sock = fsockopen($host,80,$errno, $error);
$readStream = fopen('/some/file/path.txt', 'r+');
fwrite($sock, "POST /somePath HTTP/1.1\r\n" .
"Host: www.somehost.com\r\n" .
"Transfer-Encoding: chunked\r\n\r\n");
while (!feof($readStream)) {
$chunkData = fread($readStream, $chunkSize);
$chunkLength = strlen($chunkData);
$chunkData = dechex($chunkLength) . "\r\n$chunkData\r\n";
fwrite($sock, $chunkData);
}
fwrite($sock, "0\r\n\r\n");

Related

What are this Weird Character on my Ajax result

What are this Weird Characters on my Ajax result 138d on the start and 0 on the end of my ajax result. how to disable this?
138d
{"feeds":[{"pubdate":"Sun, 28 Nov 2021 23:00:00 EST"]}
0
Your HTTP client is buggy. It improperly extracted the body from the HTTP response.
There are three ways to signaling the end of an HTTP response:
Using a Content-Length header.
Using the chunked transfer coding.
Closing the socket.
The first two methods allow more than one request-response exchange to occur over the same connection, which is far more efficient than creating a new connection for each request. (Especially for HTTPS.)
The Content-Length header is quite simple to use, but it requires knowing the size of the message body before starting to send it. The chunked transfer coding doesn't have that limitation. It allows servers to start sending a response before its size becomes known.
An HTTP response that uses the chunked transfer coding might look like this:
HTTP/1.1 200 OK␍␊
Transfer-Encoding: chunked␍␊
Content-Type: application/json␍␊
␍␊
28␍␊
..............................␍␊
28␍␊
..............................␊␍␊
0␍␊
The message body of that response is
............................................................␊
The response that returned your JSON apparently used the chunked transfer coding. And whatever you used to parse the HTTP response incorrectly returned the still-chunked message body. This is a bug. Quote RFC2616,
All HTTP/1.1 applications MUST be able to receive and decode the
"chunked" transfer-coding

PHP response to HEAD request

I have a PHP script that serves portions of a PDF file by byte ranges.
If an HTTP HEAD request is received, it should send back headers (including the PDF file size) but not the actual file contents. I have tried this:
header('HTTP/1.1 200 OK');
header('Content-Type: application/pdf');
header('Accept-Ranges: bytes');
header('Content-Length: '.filesize($Pathname));
die;
The problem is that something (I assume the web server == LiteSpeed) replaces the Content-Length header with Content-Length: 0 - which defeats the whole purpose.
Can anyone suggest what I should be doing? Thanks
From w3c Hypertext Transfer Protocol -- HTTP/1.1:
When a Content-Length is given in a message where a message-body is
allowed, its field value MUST exactly match the number of OCTETs in
the message-body. HTTP/1.1 user agents MUST notify the user when an
invalid length is received and detected.
And:
The Content-Length entity-header field indicates the size of the
entity-body, in decimal number of OCTETs, sent to the recipient or, in
the case of the HEAD method, the size of the entity-body that would
have been sent had the request been a GET.
So, I suppose, your code will properly work if you send real HEAD request to your server.
It's the webserver job, not yours.
In my case I left everything to the Apache webserver and nothing changed in my php code except of how the requests is being parsed
For example things like
if($_SERVER['REQUEST_METHOD'] === "GET"){
//ok
}else{
//send 400 Bad Request
}
are changed to
if($_SERVER['REQUEST_METHOD'] === "GET" || $_SERVER['REQUEST_METHOD'] === "HEAD"){
//ok
}else{
//send 400 Bad Request
}
and Apache did all the heavy lifting (striped the response body).
(don't try to ob_clean() or die("") or things like this).
related resources:
http://hc.apache.org/httpclient-3.x/methods/head.html
https://security.stackexchange.com/questions/62811/should-i-disable-http-head-requests
Apache 2.2.2 response on HEAD requests
As Lurii mentioned, the content length is affected by your request type.
With GET requests, a non-matching content length may result in a hanging client, so LiteSpeed will verify the content length before sending the header to the client.
Using a HEAD request should return the content length as expected.

Guzzle response with content-encoding: gzip comes back with incorrect content-length header

I have a symfony2 application where I am using the Guzzle http client to send a GET request to a server in order to retrieve the contents of a json file. The Guzzle response gets transformed into a Symfony2 response to the browser.
The Guzzle response comes back with the following headers:
Content-Encoding: gzip
Content-Length: 2255
Content-Type: application/json
When outputting the data to the UI/browser I notice that it gets cut off because the Content-Length is incorrect. The size of the file is closer to 4905 bytes, not 2255. 2255 is the exact length of the data up to the cut-off point. I suspect that the 2255 is the size of the gzipped data and it gets uncompressed at some point without updating the content-length. Now I did verify that I get all of the data back, however the content-length header is honored which is why the data gets cut off when I forward it to the browser. Interestingly, hitting the url to the json file directly yields the full contents even though the content-length is 2255 which means it gets ignored by Chrome when hitting the file directly. Same if I use the POSTman REST client to make the GET request - full contents get displayed.
By default, Guzzle has a request option decode_content = true for how the responses should be handled. I set it to false when submitting the request but that didn't seem to resolve the issue.
Before converting the Guzzle response to a Symfony response I removed the content-length header and that seems to solve the problem however I am not sure that's the best approach since RFC protocol states that a content-length header should be present unless a transfer-encoding header is present, which it isn't. https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html
Another alternative is, since this is a streamed response, to get the size of the stream and correct the content-length, however the Guzzle implementation uses strlen() for this which has the undesirable affect of reading the whole stream.
What possible issues might I run into if I choose to omit the content-length header? And alternatively, is there a way to get the TRUE length of the contents without reading the whole stream and simply update the content-length header with the correct amount?

How to generate Chunked response with a Trailer in Apache/PHP?

I know I can generate chunked response in PHP simply by introducing a sleep() in the output.
But is it possible to also generate a Trailer HTTP section in PHP? If not, is it possible in general in Apache 2.2?
I need it for testing purposes.
PHP will send a chunked response by default if headers are sent and no Content-Length header was specified. If you're familiar with the HTTP spec, this is the only logical thing to do since the client on the other end needs to know when the HTTP message you're sending ends so it can stop reading.
If you want to do this manually, you need to ...
Send the appropriate headers yourself and call flush()
Manually output the chunked HTTP message
So you might do something like the following. The idea is that you need to manually send your own headers and manually chunk your own message. If you simply don't send a Content-Length header, however, PHP will send a chunked message for you by default.
header("Transfer-encoding: chunked");
header("Trailer: X-My-Trailer-Header");
flush();
echo dechex(strlen($myChunk)) . "\r\n";
echo $myChunk;
echo "\r\n";
flush();
echo "0\r\n";
flush();
echo "X-My-Trailer-Header: some-value\r\n";
flush();

Detect the end of a HTTP packet

I have to write a PHP script that works as a client against another HTTP Server. This Server ignores the HTTP Connection:Close header and keeps the TCP connection open unless it is closed by the client. And here is my dilemma. I (the client) have to deciede when a HTTP request/response has finished and then close the connection. Simply use:
$data = file_get_contents($url);
.. won't work, as file_get_contents returns only if the connection timeout (default 30 seconds) has reached.
So I have to write my own read - loop like this (pseudo code):
$sock = fsockopen(...);
$data = '';
while($line = fgets($sock)) {
$data .= $line;
if(http_package_recieved()) {
break;
}
}
Unfortunately there is no Content-Length header in the response. My question is, how the function
http_package_recieved()
... should look like.
Greets
Thorsten
You can check if $line is empty to see if the server isn't sending anything. You can also set a small read timeout on the socket with stream_set_timeout() , and then inside the loop check stream_get_meta_data() to see if it has been reached in order to break out.
If it doesn't close the connection and it doesn't tell you the total length of the response, you have no way to know whether all the data has been received.
You could specify a maximum time interval between packets, but that won't be reliable.
You'd be better of using a library, such as cURL (http://uk.php.net/manual/en/intro.curl.php), to handle this. The HTTP spec isn't simple: https://www.rfc-editor.org/rfc/rfc2616 (see Section 4.4) and you'd likely miss something crucial.
When the entity ends is either guided by:
Content-Length header (which you don't have)
HTTP Chunked Transfer Encoding (see Transfer-Encoding: chunked header: do you have one of these?).
It's possible you may have to process this chunked transfer encoding if you get this header. There are libraries to do so.
feof($sock) would be OK

Categories