Here's a strange one:
I've got nginx reverse proxying requests to apache 2 with mod_php.
A user (using firefox 3.1b3) reported that recently, he's started getting sporadic "What should firefox do with this file?" popups during normal navigation. We haven't had any other reports of this issue, and haven't been able to reproduce it ourselves.
I checked Nginx and apache's logs. Nothing in the error logs, and they both show a normal HTTP 200 for the request.
I had him send me the downloaded file, and it's generated HTML, as it should be -- except it has some trailing and leading bytes tacked on.
The opening byte sequence is the magic gzip header: 1F8B08
Here are the opening characters, C-escaped for convenience:
\x1F\x8B\x089608\r\n<!DOCTYPE HTML ...
and the file ends with:
...</html>\n\r\n0\r\n\r\n
When I fetch the same URL via wget, it starts with as expected; the mysterious opening and closing bytes are nowhere to be seen.
Has anyone ever seen anything similar to this? Could this be a FF 3.1b3 bug?
Never seen an issue exactly like that, but I did have an issue once with a transparent proxy that would claim to the web server that it could handle gzip compressed content when, in fact, it received the gzipped content from the server, stripped the gzip headers without decompressing it, and sent the result to the browser. The behavior we saw was what you describe: a save/open file dialog for what should have been a normal web page. In this case the browser in question was IE.
I'm not sure if you're problem is related, but as an experiment, you could look at the requests between the proxy and Apache and see if they are gzipped, or else turn off the gzip compression for requests in Apache and see if that fixes the issue. If so, then you probably have a problem with gzip handling in your proxy.
wget doesn't request a compressed response. Try:
curl --compressed <URL>
You could also try adding a -v to print the response headers, and check that a sensible Content-Type is being returned.
Related
We have problem in our PHP application that runs in GCP Cloud run.
Environment: PHP 8 (nginx php-fpm) in docker - official image "php:8.1-fpm".
There are certain files that are failed to be sent to the client when client opens a page in our system. I am not able to reproduce the same behavior locally in docker (exactly the same image), so I guess this is Cloud run specific.
The file is served by this PHP code:
...
header('Last-Modified: ' . $lmstring);
header('Content-Lenght: ' . filesize($filename));
readfile($filename);
die;
}
Getting this warning from cloud run:
"Truncated response body. Usually implies that the request timed out or the application exited before the response was finished."
Info log from Google HTTPS load balancer (error code explained here):
statusDetails: "client_disconnected_after_partial_response"
Real example and some facts I already found out:
Opening page X in the internal system. This page downloads Y other files - most of these Y files (css+js) are sent successfully, but some are truncated to ~3641 characters. All files that are sent incompletely are usually truncated to the same length (rarely a bit less). It's always the same files trucated and I didn't spot anything they have in common. Sometimes it is js file, sometimes it is css file. The response code is HTTP 200.
I didn't find any error in PHP error output - trying to turn errors on on all levels. We get errors from PHP normally in other cases, so I don't expect anything is silenced.
Anybody has any idea why might this be happening and how to deal with that?
Seems like I found the solution. Answer in following thread has helped:
Google Cloud Run website timeouts when content length is between 4013-8092 characters. What is going on?
Try to explain the behavior is in the above thread too.
Turning the fastcgi bufferring off in nginx helped to get rid of the problem:
fastcgi_buffering off; (nginx.conf)
I have the following setup:
Some files are dynamically generated dependent on some (only a few) session parameters. Since they have not a great diversity, i allow caching in proxys/browsers. The files get an etag on their way, and the reaction of the whole web application at first glance seems correct: Files served in correct dependence from session situations, traffic saved.
And then this erroneous behavior:
But at closer inspection, i found that in his answer in case of a 304 for those dynamically generated files, apache wrongly sends a "Connection: close" Header instead of the normally sent "Connection: KeepAlive". What he should do is: Simply do not manipulate anything concerning "connection".
I cannot find any point where to pinpoint the cause of this behavior: Nowhere in the apache config files is anything written except one single line in one single file where it is instructed to send a keepalive - which it does - as long as it does not send a 304 response for a dynamically generated file. Nowhere in PHP do i instruct that guy to send anything other than keepalives (and the latter only to try to counter the connection:close).
The apache does not do this when it serves "normal" (non-dynamic) files (with 304 answers). So in some way i assume that maybe the PHP kernel is the one who interferes here without permission or being asked. But then, an added "Header set Connection 'Keep-Alive'" in the apache config, which i too added to counter the closing of the connection, does not work, too. Normally, when you put such a header set rule (not of "early" type) in the apache config, this rules takes action AFTER finalization of any subordered work on the requested document (thus AFTER finalization of the PHP output). But in my case, nothing happens - well: in case of a 304 response. In all other cases, everything works normal and correct.
Since there do some other files go over the line at a page request, i would appreciate to get the apache rid of those connection-closures.
Is there anybody who has an idea what to do with this behavior?
P.S.: One day (and a good sleep) later, things are clearing:
The culprit in this case was a shortsightedly (on my behalf) copied example snippet, which had "HTTP/1.>>>0<<< 304" (the Null!) in it.
This protocol version number gets (correctly) post-processed by apache (after everything otherwise - including any apache modules work - got finalized), in that it decides not to send a "Connection: Keep-Alive" over the wire, since that feature didn't exist in version HTTP/1.0.
The problem in this case was to get the focus on the fact that everything inside php and apache modules worked correctly and something in the outer environment of them must have been wrong, and thereafter to shift the view to anything in the code that could possibly influence that outer environment (e.g. the protocol version).
I ran into an issue where regardless of the preload attribute setting, when IE9 makes a request for a video, and the video is served by x-sendfile, the request is listed as pending and keeps the connection open.
Consequently, if you have 10 videos trying to load, IE9 will quickly eat up all of its available connections and the browser will not be able to make further requests.
When telling IE9 to request the same video from Apache, without X-Sendfile, Apache serves a small portion of the file as a 200 request. Then the browser makes a request later when the play button is pressed to serve a range of the file.
It looks like X-Sendfile is causing Apache to serve the entire file initially, instead of serving just a part of it.
How can I make X-Sendfile requests via Apache function the same as a regular request to Apache?
Setting the "Accept-Ranges" header like header("Accept-Ranges: bytes"); tells IE9 to attempt to stream the file by default, instead of serve it in one chunk.
It's recommended to check that the HTTP request is version 1.1 though before setting, since 1.0 doesn't support the header.
if (isset($_SERVER['SERVER_PROTOCOL']) && $_SERVER['SERVER_PROTOCOL'] === 'HTTP/1.1') {
header("Accept-Ranges: bytes");
}
I wasn't able to find any documentation on this anywhere, so I'm posting my solution here.
I'm moving a website to a new server. (Old server had php 5.3.2, the new one has php 5.5.9)
Centos, httpd Apache/2.2.26.
I've copied files and it works fine except the only weird thing:
some strange HEX number is inserted before the content of pages:
Also, in the bottom of the page, 0 is inserted after the </html> tag.
I've noticed two things:
1) In my case only two headers are sent from php script:
header("HTTP/1.1 200 OK");
header("Status: 200");
If I comment the first header than It will be ok - no strange numbers.
2) It looks like that number is the number of characters on the page (I've checked it).
And if the page is less then 8000 characters, than the number doesn't appear, but if page has 8001 characters than 1F41 appears
P.S. I was advised to remove all BOM from files. Files were OK - already without BOM. So it's not about BOM.
UPD:
I made a very simple test (index.php):
<?php header("HTTP/1.1 200 OK"); ?>
Lorem Ipsum ... 8000 characters
Everything is OK.
<?php header("HTTP/1.1 200 OK"); ?>
Lorem Ipsum ... 8001 characters
Bug happens 1f41 before Lorem Ipsum.
This is not PHP nor a BOM. You have a problem with Content-Transfer-Encoding.
The server is sending along a Chunked encoding (this is usually done when the Content-Length is unavailable) which the client apparently doesn't know how to handle, or the header combination makes the client believe that it can bypass dechunking.
Therefore, what is actually the "length of next chunk" gets interpreted by the client, and you see it as a hex bit before the content:
05
These
05
Are
03
the
1F
first characters of the senten
03
ce.
instead of
Content-Length: 48
These are the first characters of the sentence.
(I calculated the lengths by eye, they're probably wrong)
The likely cause is that you have some kind of buffering which interferes with Content Encoding. If everything stays in the buffer, all well and good, a Content-Length is available, gets sent and Bob's your uncle. But if you send more than the 8000 bytes of the buffer, the buffer is flushed and something untoward happens. Try looking in the documentation for zlib and output buffering, you might have some conflict in php.ini between what Apache does and what PHP does.
Interesting links that show how exactly 8000 is (was?) a buffer size used in some portions of apache:
https://bugs.php.net/bug.php?id=45945
https://serverfault.com/questions/366996/how-can-the-apache-2-2-deflate-module-length-limit-be-increased
This last link suggests me to suggest you to try checking whether zlib, mod_gzip, ob_gzhandler and/or mod_deflate are enabled, and possibly conflicting, or to try exchanging the one for the other.
Update (as per comment)
Adding the appropriate Transfer-Encoding header fixes the problem.
So what really happened? We know that the output was correctly chunked-encoded (i.e. the chunking itself was correct), and the client was able to interpret it after being told to. So what was missing was simply the knowledge of the content being chunked, which looks absurd and a good bit buggy: for whatever chunked the content had to know it would get chunked (d'oh!), so it had responsibility of adding the appropriate header, and yet it did not (or it did, but something else stripped it), until the OP rectified the situation adding a header himself by hand.
The problem is now solved, but I think there must be still a bug lingering somewhere in the workflow, either in the modules, the application, or in how the headers are processed.
As discussed in the PHP chat room yesterday 9F 1A is the BOM for UTF-16BE Encoding
In my case only two headers are sent from php script:
header("HTTP/1.1 200 OK"); header("Status: 200");
What's going here? Why are you setting both? One is a proper http response, the other is a CGI response. You shouldn't need to set either, as PHP will default to a 200 OK if the script completes. And setting a CGI style header is probably wrong, unless you're definitely running PHP in CGI mode through Apache.
Had this problem for serving HTML as well as plain text.
EXPLICITLY setting header Transfer-Encoding to "chunked"
header("Transfer-Encoding: chunked");
indeed solved the problem (not sure how reliably though).
That Transfer-Encoding header was being added automatically even if i didn't set it - but the problem was still there. Setting the header explicitly in PHP code solved it, but not exactly - see below. Tested using websniffer.cc, PHP 7.4.9, CentOS 7.8
UPDATE: adding the header might confuse some clients. https://securityheaders.com/ failed to connect to the site with this header explicitly set on (regardless of content length)
When I set a custom header in Apache + mod_php5, this works fine:
header('Foo: Bar');
But when I try this while also sending a 304 Not Modified response, the header appears to be removed by apache (along with X-Powered-By and other standard headers).
header('HTTP/1.1 304 No Content');
header('Foo: Bar');
Does anyone know how to solve this issue?
Does this not answer the question?
If the conditional GET used a strong cache validator (see section 13.3.3), the response SHOULD NOT include other entity-headers. Otherwise (i.e., the conditional GET used a weak validator), the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers.
from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5
As of Apache 2.4.23 (the latest release as of today, as far as I know), you're not going to be able to get around that problem when you send a 304 "Not Modified" response because, indeed, Apache does explicitly remove all non-whitelisted headers:
http://svn.apache.org/viewvc/httpd/httpd/tags/2.4.23/modules/http/http_filters.c?view=markup#l1331
So, whether we like it or not (because I'm on the same boat of having my CORS headers removed by Apache from the response when I send a 304), it does seem like Apache is following the RFC recommendation and it's indeed treating everything that falls outside of that list as entity headers.
One solution is to patch-up the Apache source to extend that list and turn to deploying your home-grown package to your server(s), but that's definitely not without a long list of implications of its own. On the flip side, I hear that nginx doesn't suffer from this problem.
The content that I'm delivering will be consumed, among others, by WebGL runtimes in standard browsers, so if they do complain about the lack of CORS in my 304 responses I'm going to have to turn everything to 200 OK and forego the bandwidth savings.
I do a trick to solve this issue by :
1. put all header before 304 header
2. flush these header before send 304
header('Foo: Bar');
flush();
header('HTTP/1.1 304 No Content');
Apache will not remove any header until it found 304.
We force other header send out by flush() before send 304.
That apache can not hurt me.
Try:
header('Foo: bar', true, 304);