When I set a custom header in Apache + mod_php5, this works fine:
header('Foo: Bar');
But when I try this while also sending a 304 Not Modified response, the header appears to be removed by apache (along with X-Powered-By and other standard headers).
header('HTTP/1.1 304 No Content');
header('Foo: Bar');
Does anyone know how to solve this issue?
Does this not answer the question?
If the conditional GET used a strong cache validator (see section 13.3.3), the response SHOULD NOT include other entity-headers. Otherwise (i.e., the conditional GET used a weak validator), the response MUST NOT include other entity-headers; this prevents inconsistencies between cached entity-bodies and updated headers.
from http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.5
As of Apache 2.4.23 (the latest release as of today, as far as I know), you're not going to be able to get around that problem when you send a 304 "Not Modified" response because, indeed, Apache does explicitly remove all non-whitelisted headers:
http://svn.apache.org/viewvc/httpd/httpd/tags/2.4.23/modules/http/http_filters.c?view=markup#l1331
So, whether we like it or not (because I'm on the same boat of having my CORS headers removed by Apache from the response when I send a 304), it does seem like Apache is following the RFC recommendation and it's indeed treating everything that falls outside of that list as entity headers.
One solution is to patch-up the Apache source to extend that list and turn to deploying your home-grown package to your server(s), but that's definitely not without a long list of implications of its own. On the flip side, I hear that nginx doesn't suffer from this problem.
The content that I'm delivering will be consumed, among others, by WebGL runtimes in standard browsers, so if they do complain about the lack of CORS in my 304 responses I'm going to have to turn everything to 200 OK and forego the bandwidth savings.
I do a trick to solve this issue by :
1. put all header before 304 header
2. flush these header before send 304
header('Foo: Bar');
flush();
header('HTTP/1.1 304 No Content');
Apache will not remove any header until it found 304.
We force other header send out by flush() before send 304.
That apache can not hurt me.
Try:
header('Foo: bar', true, 304);
Related
I have the following setup:
Some files are dynamically generated dependent on some (only a few) session parameters. Since they have not a great diversity, i allow caching in proxys/browsers. The files get an etag on their way, and the reaction of the whole web application at first glance seems correct: Files served in correct dependence from session situations, traffic saved.
And then this erroneous behavior:
But at closer inspection, i found that in his answer in case of a 304 for those dynamically generated files, apache wrongly sends a "Connection: close" Header instead of the normally sent "Connection: KeepAlive". What he should do is: Simply do not manipulate anything concerning "connection".
I cannot find any point where to pinpoint the cause of this behavior: Nowhere in the apache config files is anything written except one single line in one single file where it is instructed to send a keepalive - which it does - as long as it does not send a 304 response for a dynamically generated file. Nowhere in PHP do i instruct that guy to send anything other than keepalives (and the latter only to try to counter the connection:close).
The apache does not do this when it serves "normal" (non-dynamic) files (with 304 answers). So in some way i assume that maybe the PHP kernel is the one who interferes here without permission or being asked. But then, an added "Header set Connection 'Keep-Alive'" in the apache config, which i too added to counter the closing of the connection, does not work, too. Normally, when you put such a header set rule (not of "early" type) in the apache config, this rules takes action AFTER finalization of any subordered work on the requested document (thus AFTER finalization of the PHP output). But in my case, nothing happens - well: in case of a 304 response. In all other cases, everything works normal and correct.
Since there do some other files go over the line at a page request, i would appreciate to get the apache rid of those connection-closures.
Is there anybody who has an idea what to do with this behavior?
P.S.: One day (and a good sleep) later, things are clearing:
The culprit in this case was a shortsightedly (on my behalf) copied example snippet, which had "HTTP/1.>>>0<<< 304" (the Null!) in it.
This protocol version number gets (correctly) post-processed by apache (after everything otherwise - including any apache modules work - got finalized), in that it decides not to send a "Connection: Keep-Alive" over the wire, since that feature didn't exist in version HTTP/1.0.
The problem in this case was to get the focus on the fact that everything inside php and apache modules worked correctly and something in the outer environment of them must have been wrong, and thereafter to shift the view to anything in the code that could possibly influence that outer environment (e.g. the protocol version).
I'm moving a website to a new server. (Old server had php 5.3.2, the new one has php 5.5.9)
Centos, httpd Apache/2.2.26.
I've copied files and it works fine except the only weird thing:
some strange HEX number is inserted before the content of pages:
Also, in the bottom of the page, 0 is inserted after the </html> tag.
I've noticed two things:
1) In my case only two headers are sent from php script:
header("HTTP/1.1 200 OK");
header("Status: 200");
If I comment the first header than It will be ok - no strange numbers.
2) It looks like that number is the number of characters on the page (I've checked it).
And if the page is less then 8000 characters, than the number doesn't appear, but if page has 8001 characters than 1F41 appears
P.S. I was advised to remove all BOM from files. Files were OK - already without BOM. So it's not about BOM.
UPD:
I made a very simple test (index.php):
<?php header("HTTP/1.1 200 OK"); ?>
Lorem Ipsum ... 8000 characters
Everything is OK.
<?php header("HTTP/1.1 200 OK"); ?>
Lorem Ipsum ... 8001 characters
Bug happens 1f41 before Lorem Ipsum.
This is not PHP nor a BOM. You have a problem with Content-Transfer-Encoding.
The server is sending along a Chunked encoding (this is usually done when the Content-Length is unavailable) which the client apparently doesn't know how to handle, or the header combination makes the client believe that it can bypass dechunking.
Therefore, what is actually the "length of next chunk" gets interpreted by the client, and you see it as a hex bit before the content:
05
These
05
Are
03
the
1F
first characters of the senten
03
ce.
instead of
Content-Length: 48
These are the first characters of the sentence.
(I calculated the lengths by eye, they're probably wrong)
The likely cause is that you have some kind of buffering which interferes with Content Encoding. If everything stays in the buffer, all well and good, a Content-Length is available, gets sent and Bob's your uncle. But if you send more than the 8000 bytes of the buffer, the buffer is flushed and something untoward happens. Try looking in the documentation for zlib and output buffering, you might have some conflict in php.ini between what Apache does and what PHP does.
Interesting links that show how exactly 8000 is (was?) a buffer size used in some portions of apache:
https://bugs.php.net/bug.php?id=45945
https://serverfault.com/questions/366996/how-can-the-apache-2-2-deflate-module-length-limit-be-increased
This last link suggests me to suggest you to try checking whether zlib, mod_gzip, ob_gzhandler and/or mod_deflate are enabled, and possibly conflicting, or to try exchanging the one for the other.
Update (as per comment)
Adding the appropriate Transfer-Encoding header fixes the problem.
So what really happened? We know that the output was correctly chunked-encoded (i.e. the chunking itself was correct), and the client was able to interpret it after being told to. So what was missing was simply the knowledge of the content being chunked, which looks absurd and a good bit buggy: for whatever chunked the content had to know it would get chunked (d'oh!), so it had responsibility of adding the appropriate header, and yet it did not (or it did, but something else stripped it), until the OP rectified the situation adding a header himself by hand.
The problem is now solved, but I think there must be still a bug lingering somewhere in the workflow, either in the modules, the application, or in how the headers are processed.
As discussed in the PHP chat room yesterday 9F 1A is the BOM for UTF-16BE Encoding
In my case only two headers are sent from php script:
header("HTTP/1.1 200 OK"); header("Status: 200");
What's going here? Why are you setting both? One is a proper http response, the other is a CGI response. You shouldn't need to set either, as PHP will default to a 200 OK if the script completes. And setting a CGI style header is probably wrong, unless you're definitely running PHP in CGI mode through Apache.
Had this problem for serving HTML as well as plain text.
EXPLICITLY setting header Transfer-Encoding to "chunked"
header("Transfer-Encoding: chunked");
indeed solved the problem (not sure how reliably though).
That Transfer-Encoding header was being added automatically even if i didn't set it - but the problem was still there. Setting the header explicitly in PHP code solved it, but not exactly - see below. Tested using websniffer.cc, PHP 7.4.9, CentOS 7.8
UPDATE: adding the header might confuse some clients. https://securityheaders.com/ failed to connect to the site with this header explicitly set on (regardless of content length)
Background
Part of my application's responsibility is handling requests for static resources (CSS, JavaScript, images) in a controlled manner. Based on some application logic, it will return one from a selection of different files that might be served on that URL at different times and to different users. These are therefore static files, but delivered in a dynamic way.
The application is based on Symfony Components and the serving of these static-ish files is handled by the BinaryFileResponse class.
The bootstrap code calls the trustXSendfileTypeHeader method:
\Symfony\Component\HttpFoundation\BinaryFileResponse::trustXSendfileTypeHeader();
The application uses some internal logic based on configuration and the detection and use of apache_get_modules() to determine availability. If XSendfile is available and the configuration says to use it, it sets the X-Sendfile-Type header:
if ($useHeader === true) {
$request->headers->set('X-Sendfile-Type', $header);
}
$response = new BinaryFileResponse($filename);
Problem
When I run this with the configuration set to never use XSendfile, or through the PHP built-in web server, which obviously does not support XSendfile, everything is perfect.
When I utilise XSendfile, it also works -- most of the time.
Every so often, typically if I press the f5 key 3-4 times in quick succession, "something" wigs out and I get a garbled response. For example, this is supposed to be a JavaScript file (copied from "Response" tab under "Net" in Firebug):
hxYîãx��HTTP/1.1 200 OK Date: Tue, 05 Feb 2013 14:49:10 GMT Server:
Apache/2.2.22 (Ubuntu) X-Powered-By: PHP/5.4.6-1ubuntu1.1
Cache-Control: public Last-Modified: Tue, 29 Jan 2013 13:33:23 GMT
Accept-Ranges: bytes Content-Transfer-Encoding: binary ETag:
"10426f-9f6-0" Vary: Accept-Encoding Content-Encoding: gzip
Content-Length: 1011 Keep-Alive: timeout=5, max=98 Connection:
Keep-Alive Content-Type: application/javascript
������VmoÛ6þ,ÿkÀ²ãIý°~q [Üt]
XÑt¶H¤#Rv¼Àÿ}w(YSÀØ2yïå¹*¾Á>¯¥¥,è) Æ^Ât¸BaÆ\éjgäjí
Î&ð*¸Åí¸tY!³Ç$Óe"jÞ![#,n®®oï®A¨þ¸þù××Þ©¼¼ôÇêÚd¹49mv°ÔrtBÖ^;WÍÓÔg´Y¥´FéôÁR9o°35Îà^º´N=UÐèEµ¢XE¸íÒ%ª°¨Úò7¬KñT¾{;£ÈrTnß³étUè{QÀçÍn·:'üJëQÍÄËZeNjOàyÕÁ:#3wö~4Òét1ù$µeN)RD|
¶FTØJ·ß½¥¨¸õGç >9TyÜxzgl-J:) b«9ûAQ½KXÉ!yÐÓ]
óÆÎ#W¡?¢vún·7j©ÿ¢ðõÖGEÁy\ºp¤÷cKxf?ï*¼Éç0^ïîÌÇ°ñDQ¸mYJ|4t¾ñæËÛ¯Å
¨6:çøp(}þÑò|LÂ;Õ(#v¹* /[¨U|xª
æ]ÍyìjµòÛ¯p?4sI¥"v÷ôp|uQ4ò4&Ï·$eÒc¸ xo%7Ôi´2ñx;TuÙj23 áÊ%ħ¿¹lÌwÀS.&ÏØß7¸}ó
ZXzå k2'Zdùè
�¦ºû-Ù[Ó²ÿU(¯¤¥=pÃjô¾ç]]Øhhô²×ÙãÚÍ4¨[!Õ}'Òþ^Ð�ûxÿ#+ÚVÞ~áÌáy?d
aíD¹·U×ÃÚ] õ5íÃø¨o÷ÂAvUÆmÍaày`¦ä©A?mL[-}®(ÿË
d°öò¬}Ç¢³Çp1À^6%0 hTô^ts´ÞíWô
fO¶ö¢ÎNÜæ·HîUôÔ¶±ÌCµsxh.9åçi Û·_ÈÞØ_ÄãY_Ö}G<ì°ý2wÔ¿aw8/þù\ã±þ"0C
oÂh'tE¶À¤¥7I½éßRt.s?á^d|k/Æ)wRw÷cG¿<Þ
¼´°/^ø*ʤAVZ×y¿zÅΪ¥[²Õ1ò_Vµæï_YXÁÕö ��YXÁÕö ��
Note the presence of the headers in the response body, and the rest of it which is clearly not JavaScript. There are also some spurious characters at the start, which possibly is what leads to the headers being pushed to the body. I have tried to determine if this content is the result of gzipping, but I can't confirm that yet. (See also update below)
Question
Firstly, is BinaryFileResponse even the correct class to use for serving text (non-binary) files? The documentation for the class only says "BinaryFileResponse represents an HTTP response delivering a file." This isn't very detailed but it doesn't say anything about it being exclusively for "binary" files. However the name has its own implications, why didn't Fabien just call this class FileResponse?
Secondly, and more importantly, what could be causing this? I don't believe it is a browser issue because it is repeatable in both Firefox and Chrome. Is this a bug in the XSendfile module or in the BinaryFileResponse class perhaps? (I am likely to think it is not the former because I have used it before in a more "raw" way not via Symfony Components, with no such issues).
Has anyone else experienced this? Any idea where I should even start looking to track this down? I've looked at the BinaryResponseFile source code but it doesn't really do much with XSendfile, just sets the relevant header and prevents content in the response body, from what I can see.
Update
I've just noticed a couple of things about these garbled responses:
There are no actual headers being sent at all, i.e. on the "Headers" tab in Firebug, for the garbled responses, it only lists Request headers and doesn't even show the heading for Response headers.
Even if I set some custom header on the Response in PHP, that header does not appear at all in the garbled responses (as a header or in the response body), but the custom headers appear correctly for the responses that aren't broken.
First, let me say that I don't have any experience with this Apache module, but I'll try to guide you through a general error deduction:
You should check if you can reproduce it more reliably. While a web browser might be ok to try it out, you should go for something like curl and do the request multiple times, for example using a bash for-loop.
for i in `seq 1 5`; do curl -v http://localhost/xsendfile-url; done
The fact that the Connection: Keep-Alive header is set and that there are some weird characters before the actual HTTP header lead me to believe that you won't be able to reproduce this problem with separated curl calls, because it will open a fresh connection each time. So try this to check if that gives you the weird behavior (curl has keep alive on by default):
curl -v http://localhost/xsendfile-url http://localhost/xsendfile-url http://localhost/xsendfile-url
Using this, you could go to the projects github issue page and report your findings. Most probably they will there help you in telling you why mod_xsendfile is behaving the way it is or that you have found a bug.
I know in PHP, it sends the X-Powered-By header to have the PHP version.
I also know by appending some checksums, you can get access to PHP's credits, and some random images (more info here).
I also know in php.ini you can turn expose_php = off.
But here is something I have done on a few sites, and that is use
header('X-Powered-By: Alex');
When I view the headers, I can see that it is now 'Alex' instead of the PHP version. My question is, will this send the previous PHP header first (before it reaches my header(), and is it detectable by any sniffer program? Or are headers 'collected' by PHP, before being sent back to the browser?
By the way, this is not for security by obscurity, just curious how headers work in PHP.
You can set expose_php = Off in your php.ini if you don't want it to send X-Powered-By header.
PHP first compiles everything (including which headers have which values ) and then start the output, not vice-versa.
PHP is also detectable with its own easter eggs, you can read about this topic here : PHP Easter Eggs
See Apache Tips & Tricks: Hide PHP version (X-Powered-By)
Ups… As we can see PHP adds its own
banner:
X-Powered-By: PHP/5.1.2-1+b1…
Let’s see how we can disable it. In
order to prevent PHP from exposing the
fact that it is installed on the
server, by adding its signature to the
web server header we need to locate in
php.ini the variable expose_php and turn it off.
By default expose_php is set to On.
In your php.ini (based on your Linux
distribution this can be found in
various places, like /etc/php.ini,
/etc/php5/apache2/php.ini, etc.)
locate the line containing expose_php
On and set it to Off:
expose_php = Off
After making this change PHP will no
longer add it’s signature to the web
server header. Doing this, will
not make your server more secure… it will just prevent remote hosts to
easily see that you have PHP installed
on the system and what version you are
running.
In PHP, headers aren't sent until PHP encounters its first output statement.
This includes anything before the first <?php.
This is also why setcookie sends throws a warning if you try to use it after something has been output:
Warning: Cannot modify header
information - headers already sent by
(output started at
/path/to/php/file.php:100) in
/path/to/php/file.php on line 150
Note that none of this applies if output buffering is in use, as the output will not be sent until the appropriate output buffering command is run.
To get rid of the X-Powered-By header without having access to php.ini, simply add an empty header.
<?php header('X-Powered-By:'); ?>
This overwrites the default X-Powered-By header with an empty value an though most clients and applications act like this header was not sent at all.
As noticed before, this must be inserted into the code before any output is sent.
And to answer your question:
Only your X-Powered-By header will be sent because it gets replaced by your header with the same name. So it can't be detected by a 'sniffer'.
Headers are "collected" by PHP before being sent back to the browser, so that you can override things like the status header. The way to test it is go to a command prompt, and type:
telnet www.yoursite.com 80
GET /index.php HTTP/1.1
[ENTER]
[ENTER]
And you'll see the headers that are sent in the response (replace /index.php with the URL of your PHP page after the domain.)
To hide X-Powered-By: PHP/7.x.x , if you are using Share Hosting then add the following code in .htaccess file
Header always unset X-Powered-By
Header unset X-Powered-By
Then reload the browser or clear the cache using the LiteSpeed Cache plugin: https://en.wordpress.org/plugins/litespeed-cache/
My question is, will this send the previous PHP header first (before it reaches my header(), and is it detectable by any sniffer program? Or are headers 'collected' by PHP, before being sent back to the browser?
No, it does not send the previous PHP header first. Headers are either sent or not sent (in complete, as one batch) in PHP. By default your headerDocs call replaces a previous header with the same name (unless you specify something different with the second parameter).
Note: If PHP would not collect the headers, it would not be able to replace one.
As it does not sent it earlier, it is not detectable with a sniffer program.
So yes, headers are collected by PHP and are send the moment "the real" output starts (HTTP response body).
See as well headers_sentDocs.
PHP has a built-in function to remove headers: header_remove().
To remove the X-Powered-By header, you can use:
<?php
header_remove(
name: 'X-Powered-By'
);
As you can see, you only have to pass the header name as a string as parameter, and you are done.
Note that name parameter is parsed not case-sensitive, so you are fine calling it with x-powered-by as well.
Since PHP 8.0.0 when calling the function without the name parameter, all previously set headers will be unset.
Here's a strange one:
I've got nginx reverse proxying requests to apache 2 with mod_php.
A user (using firefox 3.1b3) reported that recently, he's started getting sporadic "What should firefox do with this file?" popups during normal navigation. We haven't had any other reports of this issue, and haven't been able to reproduce it ourselves.
I checked Nginx and apache's logs. Nothing in the error logs, and they both show a normal HTTP 200 for the request.
I had him send me the downloaded file, and it's generated HTML, as it should be -- except it has some trailing and leading bytes tacked on.
The opening byte sequence is the magic gzip header: 1F8B08
Here are the opening characters, C-escaped for convenience:
\x1F\x8B\x089608\r\n<!DOCTYPE HTML ...
and the file ends with:
...</html>\n\r\n0\r\n\r\n
When I fetch the same URL via wget, it starts with as expected; the mysterious opening and closing bytes are nowhere to be seen.
Has anyone ever seen anything similar to this? Could this be a FF 3.1b3 bug?
Never seen an issue exactly like that, but I did have an issue once with a transparent proxy that would claim to the web server that it could handle gzip compressed content when, in fact, it received the gzipped content from the server, stripped the gzip headers without decompressing it, and sent the result to the browser. The behavior we saw was what you describe: a save/open file dialog for what should have been a normal web page. In this case the browser in question was IE.
I'm not sure if you're problem is related, but as an experiment, you could look at the requests between the proxy and Apache and see if they are gzipped, or else turn off the gzip compression for requests in Apache and see if that fixes the issue. If so, then you probably have a problem with gzip handling in your proxy.
wget doesn't request a compressed response. Try:
curl --compressed <URL>
You could also try adding a -v to print the response headers, and check that a sensible Content-Type is being returned.