php - Output Compression problems - php

I'm facing a problem trying to use php output compression , I've been searching for many hours and I still have no clues...
Lets see a simple script :
<?php
$response = "abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh";
if(function_exists('ob_gzhandler'))
{
ob_start('ob_gzhandler');
}
else {
ob_start();
}
echo $response;
ob_end_flush();
This is a method I've found all over the internet, and it used to work for me ... but not any more (and I've got no idea why) .
If i look at the http headers when I call this script :
Request :
Host: 192.168.51.191
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0
Response :
Connection: Keep-Alive
Content-Type: text/html
Date: Tue, 26 Jan 2016 15:19:07 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
Transfer-Encoding: chunked
Vary: Accept-Encoding
X-Powered-By: PHP/5.5.12
You can see that the response is NOT zipped (Firebird gives me a 0.06ko response), and the server sends the responses using chunked encoding.
I tried an alternate method to send zipped responses :
<?php
$response = "abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh";
$replyBody = gzencode($response, 9, FORCE_GZIP);
header("Content-Encoding: gzip");
echo $replyBody;
And the response headers are as follow (the request headers are always the same):
Response :
Connection: Keep-Alive
Content-Type: text/html
Date: Tue, 26 Jan 2016 15:29:01 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
Transfer-Encoding: chunked
X-Powered-By: PHP/5.5.12
As you can see, this is basically the same behavior as in the first method.
Then if I try this :
<?php
$response = "abcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefghabcdefgh";
$replyBody = gzencode($response, 9, FORCE_GZIP);
echo $replyBody;
I receive something that looks like a zipped response (random characters), and the output size is 0.03ko .
Here a the corresponding response http headers :
Connection: Keep-Alive
Content-Length: 31
Content-Type: text/html
Date: Tue, 26 Jan 2016 15:32:46 GMT
Keep-Alive: timeout=5, max=100
Server: Apache/2.4.9 (Win64) OpenSSL/1.0.1g PHP/5.5.12
X-Powered-By: PHP/5.5.12
It lets me think that the zipping part is working correctly because the output size has been reduced (it is obviously not readable because the browser can't know that it is zipped content).
That's where I'm lost ...
If I understand it right, when I manually send zipped data (using gzencode) , If I set the header"Content-Encoding: gzip" , the webserver/php seems to UNZIP it before sending it to the browser ??? How is it possible ?
And Why is it sending it as "chunked" data instead of setting the Content-Length header ?
I tried to set the Content-Length manually in the response; it doesn't change anything (it won't appear in the response headers, I'll still have a "chunked" response.
I've seen somewhere that I have to write the "Content-Length" header before sending other data or header to avoid the "chunked" response, I tried it and still had the same results.
I thought it could be a problem with BOM characters at the beginning of my php test script, but it is saved in UTF-8 without BOM encoding so I don't think it 's the problem.
I have this problem on my dev computer (using wampserver) AND in the production environment (IIS), previously it was working on both servers.
I have this problem using several browsers, and I checked the response sizes I wrote previously with fiddler.
Does anyone see where the problem could be ?
Thanks in advance

I'd go through the following checklist if I were you.
1. Check zlib extension is installed or not.
ob_gzhandler needs the zlib extension to work. Without which it just silently falls back to default settings.
2. Verify that you don't have zlib.output_compression enabled in your php.ini.
As explained here, even though zlib.output_compression is preferred over ob_gzhandler(), you can not use both simultaneously. So your code becomes..
if (extension_loaded('zlib') && !ini_get('zlib.output_compression')){
ob_start('ob_gzhandler');
}
3. Check if headers have already been sent eg. something got output-ted before the
ob_start(ob_gzhandler) This will prevent the compressed output from being detected as such. eg. Having some character before <?php or an echo somewhere up in the code.
4. Make sure you aren't using all of the above in addition to the gzipping in apache(mod_deflate).
This will only cause the output to be double gzipped which most probably will confuse the browser.

Related

Content-Length header on HHVM FastCGI is intermittent

Maybe it's the jetlag, but I'm failing to make PHP/HHVM give me the Content-Type header when I need it.
I've deployed the full stack (MySQL, HHVM, Nginx) on a Vagrant machine and I've managed to reproduce the issue on a test script:
<?php
$file='/usr/share/doc/iptables/html/NAT-HOWTO.html'; # random test file
header('Content-Length: ' . filesize($file));
echo(readfile($file));
?>
If you examine the headers with curl:
hostname:~ jsimpson$ curl -I http://vagrant/test.php
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Tue, 16 Sep 2014 22:09:25 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 2592
Connection: keep-alive
X-Powered-By: HHVM/3.2.0
Content-Encoding: none;
We have a content length header. However if we hit the same URL from Chrome, and get the headers from the Dev tools:
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Tue, 16 Sep 2014 22:14:41 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: HHVM/3.2.0
Content-Encoding: gzip
No Content-Length header. I've also packet sniffed this to verify that the header isn't sent. I can switch to PHP FPM and it sends the header.
I reproduced the issue by hitting the server with:
curl -H 'Accept-Encoding: gzip,deflate' --compressed -v http://foo/bar
HHVM enables compression by default. Disabling that gave me the header back.
Everything was awesome after adding this to /etc/hhvm/server.ini
hhvm.server.gzip_compression_level = 0
I stumbled upon this issue/feature. Not running HHVM though. Pure nginx + PHP-FPM.
The thing is, if your PHP app calculates and sets Content-Lenght header field, and your nginx is configured to gzip content, it will just ditch this information and replace it by Chunked transfer encoding and GZIP headers.
So do not set GZIP to a very small buffers, default is 20 bytes which does quite the opposite (end result is larger then before GZIP compression).
I set it like this:
gzip_min_length 1024;

file_get_contents not working on my server

I have created two files, one on my wamp server (localhost), and one on my ovh.com private space. Both files contains only this content :
echo $search = file_get_contents('https://prod.api.pvp.net/api/lol/euw/v1.3/game/by-summoner/19319907/recent?api_key=6fa73a35-6477-412d-97a6-b6739cb6cf1b');
On my server, there is some wrong characters, like â, þ or ¬, etc ...
How it can happends, and how can I resolve this ?
Edit: It's not about files ! It's about servers, cause files are exactly the same !
You can check the good one here : http://www.dietadom.fr/test.php, and the bad one here : http://82.124.50.144/test.php
headers from requests to both those scripts:
Working:
curl -I http://www.dietadom.fr/test.php
HTTP/1.1 200 OK
Set-Cookie: clusterBAK=R1564861669; path=/; expires=Wed, 19-Mar-2014 16:38:43 GMT
Date: Wed, 19 Mar 2014 15:19:31 GMT
Content-Type: text/html
Connection: keep-alive
Set-Cookie: cluster=R1649376954; path=/; expires=Wed, 19-Mar-2014 16:32:22 GMT
Server: Apache
X-Powered-By: PHP/5.4.24
Pragma: no-cache
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Vary: Accept-Encoding
Not Working:
curl -I http://82.124.50.144/test.php
HTTP/1.1 200 OK
Date: Wed, 19 Mar 2014 15:19:48 GMT
Server: Apache/2.4.4 (Win64) OpenSSL/1.0.1d PHP/5.4.12
X-Powered-By: PHP/5.4.12
Content-Type: text/plain; charset: UTF-8
Most likely the default character encoding header provided by your two servers is different. You can fix this by changing the server configurations to make sure they both have the default. Or you can modify your script to over-ride this, adding a content encoding header will make this consistent.
If you modify your PHP file for UTF-8 plain test content the line would be
header('Content-type: text/plain; charset=UTF-8');
Your source URL is responding with:
Content-Type: application/json; charset=UTF-8
retrieved using
curl -I https://prod.api.pvp.net/api/lol/euw/v1.3/game/by-summoner/19319907/recent?api_key=6fa73a35-6477-412d-97a6-b6739cb6cf1b
hence how I know it's UTF-8 encoded.
EDIT
Having checked against your server I can see the header for an error page of:
curl -I http://82.124.50.144/404.html
HTTP/1.1 403 Forbidden
Date: Wed, 19 Mar 2014 17:24:17 GMT
Server: Apache/2.4.4 (Win64) OpenSSL/1.0.1d PHP/5.4.12
Content-Type: text/html; charset=iso-8859-1
The last line showing me the default charecter encoding is iso-8859-1, which means that the utf-8 data from your source is being transmitted with the wrong encoding. Adding the correct header line to your script should fix the issue.
You could change your apache server configuration by adding AddDefaultCharset UTF-8 as well.
I'm sure that your issue is with character encoding, so you should look into that.

php sends charset header but there is no header command

My browser shows page with incorrect encoding. I have figured out that the server sends headers
HTTP/1.1 200 OK
Date: Thu, 02 Jan 2014 18:21:11 GMT
Server: Apache/2.2.4 (Win32) mod_ssl/2.2.4 OpenSSL/0.9.8k PHP/5.2.12
X-Powered-By: PHP/5.2.12
Content-Length: 4
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=windows-1251
but php script doesn't contain encoding set command. I even added exit on the script top and browser still getting Content-Type: text/html; charset=windows-1251. How so?
First look at default_charset in php.ini. Leave it empty if you do not want a Content-Type header.
In PHP one always can do a header('Content-Type', 'text/html; charset=UTF-8');.
Apache also has a config for Content-Type, but the error probably lies at the PHP side.

Unable to Upload File Via POST From Qt to PHP Script

I'm trying to upload a file via POST from a Qt (C++) application to my webserver, running a LAMP stack.
All seems to go well, but for some reason, the script's $_POST and $_FILES arrays are completely empty. I've tried uploading the same file with a plain HTML form, and it works.
Here's how I construct the request inside my Qt program:
request.setUrl(ServerAddress);
request.setHeader(request.ContentTypeHeader, "application/x-www-form-urlencoded");
reply = nam.post(request, file);
And the PHP:
<?php
var_dump($_POST);
var_dump($_FILES);
?>
Where request is a QNetworkRequest, reply is a QNetworkReply, nam is a QNetworkAccessManager, and file is the open QFile for my file.
The request sends fine, and I've even tried to see if somehow the request got messed up by using Wireshark. Here's the output:
POST /upload.php HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 8
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: mysite.com
My data!HTTP/1.1 200 OK
Date: Wed, 03 Jul 2013 15:16:37 GMT
Server: Apache/2.2.22 (Ubuntu)
X-Powered-By: PHP/5.4.17RC1
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 36
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html
array(0) {
}
array(0) {
}
It's a very small text file, and only contains "'My data!". I just don't get why the PHP script isn't seeing the file. Any ideas where I'm going wrong?

PHP Query String Limit

I have PHP 5.1.6 (cli) installed and whenever the GET query string is more than 128 characters it fails with HTTP 406 Not Acceptable error. Any suggestions how I can fix this so can use more than 128 characters? POST is not an option.
The error is being returned by the server so don't think it's browser issue.
And the reason I think it's PHP and not Apache is because it works fine with an HTML file.
GET /test.php?phptestof129characterstring-NEW-WOVEN-FENCE-PANELS-GARDEN_W0QQitemZ200303392512QQihZ010QQcategoryZ139954QQtcZphotoQQcmdZViewItem
HTTP/1.1
Host: *****
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: agent_name=Tim
HTTP/1.1 406 Not Acceptable
Date: Tue, 03 Feb 2009 12:05:33 GMT
Server: Apache/2.2.3 (Red Hat)
X-Powered-By: PHP/5.1.6
Content-Length: 0
Connection: close
Content-Type: text/html
GET /test.html?phptestof129characterstring-NEW-WOVEN-FENCE-PANELS-GARDEN_W0QQitemZ200303392512QQihZ010QQcategoryZ139954QQtcZphotoQQcmdZViewItem
HTTP/1.1
Host: *****
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: agent_name=Tim
HTTP/1.1 200 OK
Date: Tue, 03 Feb 2009 12:18:19 GMT
Server: Apache/2.2.3 (Red Hat)
Last-Modified: Fri, 19 Dec 2008 15:01:17 GMT
ETag: "156960d-221-94be8940"
Accept-Ranges: bytes
Content-Length: 545
Connection: close
Content-Type: text/html
Do you have mod_security enabled on your webserver? It sounds like something it would do. If so, you may be able to disable locally inside your <VirtualHost> block or with an .htaccess file for v1.x
<IfModule mod_security.c>
SecFilterEngine Off
SecFilterScanPOST Off
</IfModule>
Version 2.x has different configuration syntax:
<IfModule mod_security2.c>
SecRuleEngine Off
</IfModule>
That's a bit of a brute force approach, you may want to read the documentation to see how you might allow particular URIs to pass through. See also Handling False Positives and Creating Custom Rules
As a work-around, you can try using Javascript to put the data in cookies. The cookies will be sent automatically with every GET request, and give you an extra 2KB of data space (if I'm not mistaken).
This is very risky if you don't want to transmit that data with every request, so generally speaking I would recommend against it.
It's a long shot, but try adding:
header('Content-Type: text/html');
to your server-side code. If that doesn't help, check your Apache configuration, maybe it's misconfigured so that PHP files can't emit text/html MIME type. If that doesn't help, how about setting up Apache so that .html files are treated as PHP and rename the target script to .html?
BTW, from http://www.checkupdown.com/status/E406.html :
A client (e.g. your Web browser or our CheckUpDown robot) can indicate to the Web server characteristics of the data it will accept back from the Web server. This is done using 'accept headers' of the following types:
Accept: The MIME types accepted by the client. For example, a browser may only accept back types of data (HTML files, GIF files etc.) it knows how to process.
Accept-Charset: The character sets accepted by the client.
Accept-Encoding: The data encoding accepted by the client e.g. the file formats it understands.
Accept-Language: The natural languages (English, German etc.) accepted by the client.
Accept-Ranges: Whether the client accepts ranges of bytes from the resource i.e. a portion of the resource.
If the Web server detects that the data it wants to return is not acceptable to the client, it returns a header containing the 406 error code.
Have found answer thanks to comment from Ben.
Although this generates 406 error:
test.php?129+characters
This works fine:
test.php?data=129+characters
So my guess is that in the first instance PHP is attempting to use the 129 characters as name in $_GET array whereas the second example has only 4 characters for the name and the rest is assigned as value, so array must have 128 character limit for index name.

Categories