Content-Length header on HHVM FastCGI is intermittent - php

Maybe it's the jetlag, but I'm failing to make PHP/HHVM give me the Content-Type header when I need it.
I've deployed the full stack (MySQL, HHVM, Nginx) on a Vagrant machine and I've managed to reproduce the issue on a test script:
<?php
$file='/usr/share/doc/iptables/html/NAT-HOWTO.html'; # random test file
header('Content-Length: ' . filesize($file));
echo(readfile($file));
?>
If you examine the headers with curl:
hostname:~ jsimpson$ curl -I http://vagrant/test.php
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Tue, 16 Sep 2014 22:09:25 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 2592
Connection: keep-alive
X-Powered-By: HHVM/3.2.0
Content-Encoding: none;
We have a content length header. However if we hit the same URL from Chrome, and get the headers from the Dev tools:
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Tue, 16 Sep 2014 22:14:41 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: HHVM/3.2.0
Content-Encoding: gzip
No Content-Length header. I've also packet sniffed this to verify that the header isn't sent. I can switch to PHP FPM and it sends the header.

I reproduced the issue by hitting the server with:
curl -H 'Accept-Encoding: gzip,deflate' --compressed -v http://foo/bar
HHVM enables compression by default. Disabling that gave me the header back.
Everything was awesome after adding this to /etc/hhvm/server.ini
hhvm.server.gzip_compression_level = 0

I stumbled upon this issue/feature. Not running HHVM though. Pure nginx + PHP-FPM.
The thing is, if your PHP app calculates and sets Content-Lenght header field, and your nginx is configured to gzip content, it will just ditch this information and replace it by Chunked transfer encoding and GZIP headers.
So do not set GZIP to a very small buffers, default is 20 bytes which does quite the opposite (end result is larger then before GZIP compression).
I set it like this:
gzip_min_length 1024;

Related

Immutable Cache is not HIT despite Cache response headers

In an attempt to master HTTP cache, I'm currently trying to craft an HTTP response saying "Once you've got it, cache it forever".
Here's what it looks like (PHP):
<?php
header('Cache-Control: max-age=31536000, immutable, only-if-cached');
sleep(2);
echo date('H:i:s');
exit;
Unfortunately, neither Chrome (when hit directly), neither Cloudflare (when proxied) wants to serve from cache.
Full response headers received from origin:
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Mon, 30 Mar 2020 16:54:17 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Cache-Control: max-age=31536000, immutable, only-if-cached
Content-Encoding: gzip
What am I doing wrong?

php - Output Compression problems

I'm facing a problem trying to use php output compression , I've been searching for many hours and I still have no clues...
Lets see a simple script :
<?php
$response = "abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh";
if(function_exists('ob_gzhandler'))
{
ob_start('ob_gzhandler');
}
else {
ob_start();
}
echo $response;
ob_end_flush();
This is a method I've found all over the internet, and it used to work for me ... but not any more (and I've got no idea why) .
If i look at the http headers when I call this script :
Request :
Host: 192.168.51.191
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0
Response :
Connection: Keep-Alive
Content-Type: text/html
Date: Tue, 26 Jan 2016 15:19:07 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
Transfer-Encoding: chunked
Vary: Accept-Encoding
X-Powered-By: PHP/5.5.12
You can see that the response is NOT zipped (Firebird gives me a 0.06ko response), and the server sends the responses using chunked encoding.
I tried an alternate method to send zipped responses :
<?php
$response = "abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh";
$replyBody = gzencode($response, 9, FORCE_GZIP);
header("Content-Encoding: gzip");
echo $replyBody;
And the response headers are as follow (the request headers are always the same):
Response :
Connection: Keep-Alive
Content-Type: text/html
Date: Tue, 26 Jan 2016 15:29:01 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
Transfer-Encoding: chunked
X-Powered-By: PHP/5.5.12
As you can see, this is basically the same behavior as in the first method.
Then if I try this :
<?php
$response = "abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh";
$replyBody = gzencode($response, 9, FORCE_GZIP);
echo $replyBody;
I receive something that looks like a zipped response (random characters), and the output size is 0.03ko .
Here a the corresponding response http headers :
Connection: Keep-Alive
Content-Length: 31
Content-Type: text/html
Date: Tue, 26 Jan 2016 15:32:46 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
X-Powered-By: PHP/5.5.12
It lets me think that the zipping part is working correctly because the output size has been reduced (it is obviously not readable because the browser can't know that it is zipped content).
That's where I'm lost ...
If I understand it right, when I manually send zipped data (using gzencode) , If I set the header"Content-Encoding: gzip" , the webserver/php seems to UNZIP it before sending it to the browser ??? How is it possible ?
And Why is it sending it as "chunked" data instead of setting the Content-Length header ?
I tried to set the Content-Length manually in the response; it doesn't change anything (it won't appear in the response headers, I'll still have a "chunked" response.
I've seen somewhere that I have to write the "Content-Length" header before sending other data or header to avoid the "chunked" response, I tried it and still had the same results.
I thought it could be a problem with BOM characters at the beginning of my php test script, but it is saved in UTF-8 without BOM encoding so I don't think it 's the problem.
I have this problem on my dev computer (using wampserver) AND in the production environment (IIS), previously it was working on both servers.
I have this problem using several browsers, and I checked the response sizes I wrote previously with fiddler.
Does anyone see where the problem could be ?
Thanks in advance
I'd go through the following checklist if I were you.
1. Check zlib extension is installed or not.
ob_gzhandler needs the zlib extension to work. Without which it just silently falls back to default settings.
2. Verify that you don't have zlib.output_compression enabled in your php.ini.
As explained here, even though zlib.output_compression is preferred over ob_gzhandler(), you can not use both simultaneously. So your code becomes..
if (extension_loaded('zlib') && !ini_get('zlib.output_compression')){
ob_start('ob_gzhandler');
}
3. Check if headers have already been sent eg. something got output-ted before the
ob_start(ob_gzhandler) This will prevent the compressed output from being detected as such. eg. Having some character before <?php or an echo somewhere up in the code.
4. Make sure you aren't using all of the above in addition to the gzipping in apache(mod_deflate).
This will only cause the output to be double gzipped which most probably will confuse the browser.

file_get_contents not working on my server

I have created two files, one on my wamp server (localhost), and one on my ovh.com private space. Both files contains only this content :
echo $search = file_get_contents('https://prod.api.pvp.net/api/lol/euw/v1.3/game/by-summoner/19319907/recent?api_key=6fa73a35-6477-412d-97a6-b6739cb6cf1b');
On my server, there is some wrong characters, like â, þ or ¬, etc ...
How it can happends, and how can I resolve this ?
Edit: It's not about files ! It's about servers, cause files are exactly the same !
You can check the good one here : http://www.dietadom.fr/test.php, and the bad one here : http://82.124.50.144/test.php
headers from requests to both those scripts:
Working:
curl -I http://www.dietadom.fr/test.php
HTTP/1.1 200 OK
Set-Cookie: clusterBAK=R1564861669; path=/; expires=Wed, 19-Mar-2014 16:38:43 GMT
Date: Wed, 19 Mar 2014 15:19:31 GMT
Content-Type: text/html
Connection: keep-alive
Set-Cookie: cluster=R1649376954; path=/; expires=Wed, 19-Mar-2014 16:32:22 GMT
Server: Apache
X-Powered-By: PHP/5.4.24
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Vary: Accept-Encoding
Not Working:
curl -I http://82.124.50.144/test.php
HTTP/1.1 200 OK
Date: Wed, 19 Mar 2014 15:19:48 GMT
Server: Apache/2.4.4 (Win64) OpenSSL/1.0.1d PHP/5.4.12
X-Powered-By: PHP/5.4.12
Content-Type: text/plain; charset: UTF-8
Most likely the default character encoding header provided by your two servers is different. You can fix this by changing the server configurations to make sure they both have the default. Or you can modify your script to over-ride this, adding a content encoding header will make this consistent.
If you modify your PHP file for UTF-8 plain test content the line would be
header('Content-type: text/plain; charset=UTF-8');
Your source URL is responding with:
Content-Type: application/json; charset=UTF-8
retrieved using
curl -I https://prod.api.pvp.net/api/lol/euw/v1.3/game/by-summoner/19319907/recent?api_key=6fa73a35-6477-412d-97a6-b6739cb6cf1b
hence how I know it's UTF-8 encoded.
EDIT
Having checked against your server I can see the header for an error page of:
curl -I http://82.124.50.144/404.html
HTTP/1.1 403 Forbidden
Date: Wed, 19 Mar 2014 17:24:17 GMT
Server: Apache/2.4.4 (Win64) OpenSSL/1.0.1d PHP/5.4.12
Content-Type: text/html; charset=iso-8859-1
The last line showing me the default charecter encoding is iso-8859-1, which means that the utf-8 data from your source is being transmitted with the wrong encoding. Adding the correct header line to your script should fix the issue.
You could change your apache server configuration by adding AddDefaultCharset UTF-8 as well.
I'm sure that your issue is with character encoding, so you should look into that.

php sends charset header but there is no header command

My browser shows page with incorrect encoding. I have figured out that the server sends headers
HTTP/1.1 200 OK
Date: Thu, 02 Jan 2014 18:21:11 GMT
Server: Apache/2.2.4 (Win32) mod_ssl/2.2.4 OpenSSL/0.9.8k PHP/5.2.12
X-Powered-By: PHP/5.2.12
Content-Length: 4
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=windows-1251
but php script doesn't contain encoding set command. I even added exit on the script top and browser still getting Content-Type: text/html; charset=windows-1251. How so?
First look at default_charset in php.ini. Leave it empty if you do not want a Content-Type header.
In PHP one always can do a header('Content-Type', 'text/html; charset=UTF-8');.
Apache also has a config for Content-Type, but the error probably lies at the PHP side.

Nginx/PHP : Nginx is not returning a 304 response when sending Etag header

I'm trying the understand local cache with ETags and a nginx (1.2.1) server which redirects php's request to a php-cgi deamon.
Here my simple index.php :
<?php
header('Cache-Control: public');
header('Etag:"5954c6-10f4-449d11713aac0"');
echo microtime(true);
After a second request, my browser send a If-None-Match header :
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
Cache-Control:max-age=0
Connection:keep-alive
Host:cache.loc
If-None-Match:"5954c6-10f4-449d11713aac0"
But my web server doesn't returns a 304 :
HTTP/1.1 200 OK
Server: nginx/1.2.2
Date: Thu, 12 Jul 2012 11:46:03 GMT
Content-Type: text/html
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.3.12
Cache-Control: public
Etag: "5954c6-10f4-449d11713aac0"
Cache-Control: public
Unless I've misunderstood, my server should compare Etag with the If-None-Match sent and returns a 304 response because they're the same.
Where am I wrong ?
Should I compare Etag with If-None-Match in my PHP script because nginx (or Apache) will not do the job itself ?
Regards,
Manu
If you implement this yourself using php, you are responsible for also sending the 304 Not Modified.
So compare the If-None-Match header with your ETag, and use header() to send back the 304.

Categories