Image window in thickbox shows weird characters - php

I am using thickbox on ubercart/drupal 6 on ubuntu. The problem is I moved the site from a windows machine to ubuntu. All problems with paths and permissions sorted and site is working well.
The only problem I'm having now is when I click on a product image, thickbox is supposed to show a preview pop up. Instead, it shows weird characters in the pop up window. A copy/paste of those characters:
�����JFIF�,,�����Exif��MM����� ���������������������������������������(�������1��������2����������������i����������4NIKON CORPORATION�NIKON D70s���,�����,���Adobe Photoshop 7.0�2008:08:21 17:13:50���%�������������������"�������������0221���������������������������֒� �����ޒ������������������������ �������� ����������,�������90��������90��������90��������0100��������������������������������������������������������"���������������������������������������E������������������������������ ��������� ����������������������� ��X������� 2008:08:19 15:40:17�2008:08:19 15:40:17�����������������+��� ������ ASCII��� ���������������������������������(�������������������� W�������H������H��������JFIF��H�H�����Adobe_CM����Adobe�d��������� ������7"�������?���������� ��������� �3�!1AQa . . . . . . and a lot more similar chars
The images are uploaded properly and I can see them under sites/default/files/. Even the thumbnails are generated. These thumbnails appear on the site as well. Also right clicking a thumbnail and open in new tab shows me the whole image properly.
Also, Thickbox sends an ajax GET request for the image to a URL that looks something like this:
http://127.0.0.1/elegancia/?q=system/files/imagecache/product_full/image_1.jpg&random=1299550719133
Copy pasting the same request from firebug into a new browser tab opens the image successfully.
From firebug, these are the request response headers for the ajax request:
Response Headers
view source
Date Tue, 08 Mar 2011 02:18:39 GMT
Server Apache/2.2.16 (Ubuntu)
X-Powered-By PHP/5.3.3-1ubuntu9.3
Expires Tue, 22 Mar 2011 02:18:39 GMT
Last-Modified Tue, 08 Mar 2011 01:21:47 GMT
Cache-Control max-age=1209600, private, must-revalidate
Content-Length 111831
Etag "4dfe0f3d345781ac89aae5c2a10361ad"
Keep-Alive timeout=15, max=92
Connection Keep-Alive
Content-Type image/jpeg
Request Headers
view source
Host 127.0.0.1
User-Agent Mozilla/5.0 (X11; U; Linux i686; en-GB; rv:1.9.2.15) Gecko/20110303 Ubuntu/10.10 (maverick) Firefox/3.6.15
Accept text/html, */*
Accept-Language en-gb,en;q=0.5
Accept-Encoding gzip,deflate
Accept-Charset ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive 115
Connection keep-alive
X-Requested-With XMLHttpRequest
Referer http://127.0.0.1/elegancia/
Cookie SESS7a3e11dd748683d65ee6f3c6a918aa02=bijhrr4tl66t42majfs3702a06; has_js=1

Looks like it was a thickbox (Javascript) issue. PHP and Apache work fine when it comes to recognizing the image using mime.
If there are arguments in the image URL, eg.
(http://127.0.0.1/elegancia/?q=system/files/imagecache/product_full/image_1.jpg&random=1299550719133)
- causes Thickbox to show nonsense characters instead due to thickbox image recognition algorithm.
URLs not ending with an image extension makes the thickbox javascript to treat the image like another mime type that is not an image.
To work around, one needs to modify line 53 of /modules/thickbox/thinkbox.js, by adding " || urlType == '/preview' " to the list of choices in order to make thickbox.js believe in its heart that a Drupal-encoded image link is in fact an image and not an imposter.
Assuming your image size is "preview," change line 53 from:
if(urlType == '.jpg' || urlType == '.jpeg' || urlType == '.png' || urlType == '.gif' || urlType == '.bmp' ){//code to show images
to this:
if(urlType == '.jpg' || urlType == '.jpeg' || urlType == '.png' || urlType == '.gif' || urlType == '.bmp' || urlType == '/preview'){//code to show images
Also, modify line 50 to this:
var urlString = /\.jpg|\.jpeg|\.png|\.gif|\.bmp|\/preview/g;
(substitute "/preview" for "/thumbnail," "/quarter," or whatever you configured your image module to create (and name) various sizes.

Another solution which I've found is to add a path_info addition to the URL to specify the image-type. For example, my URL previously was:
/image.php?foo=bar
I changed it to:
/image.php/image.gif?foo=bar
Note that if you're using a webserver such as Apache, which by default restricts the use of path_info, you may need to turn it on with the AcceptPathInfo directive for the affected path.
I prefer this solution to altering the Thickbox source, because altering modules which may get replaced with updated versions means a possible loss of fixes, whereas altering the path_info should continue to function with any upgrades.

The browser is rendering the file as text, when it should treat it as a JPEG image. You need to send the 'Content-Type: image/jpeg' header to tell the browser how to render the content. Check your web server configuration.
For Apache, your httpd.conf file should have lines like this:
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule mime_module modules/mod_mime.so
...
TypesConfig /etc/mime.types
And then, in /etc/mime.types:
image/jpeg jpeg jpg jpe
This all applies to files which are served by the web server directly. If you can enter the URL in a browser and see the image, then none of this is a problem.
If the files are served by a script, then you need to make sure the header is sent by the script. In PHP:
header('Content-type: image/jpeg');
echo file_get_contents($image_path);

Related

wamp server and production acting differently

My webpage allows the user to drag-and-drop upload an input file, some manipulation and conversion takes place, and then an output file is saved to a publicly accessible location on the Web server. Lastly, a link to the generated file is provided to the user.
The code below (abbreviated) works just fine on my WAMP server but when I upload the code to a production host, it no longer works. The WAMP Apache version is 2.4.9 and the WAMP PHP version is 5.5.12. Production server is Apache version 2.2 and PHP version 5.3.27. I realize the versions aren't the same - is it possible that is the issue? I'd rather not mess with the versions of the production server, but I would if needed.
I have checked the Windows permissions of the folder my script attempts to write to and as part of trouble shooting have opened them to full control for everyone. (this is actually an internal webserver, so I'm not super concerned)
EDIT/UPDATE: Based on #Itay Moav -Malimovka suggestion, I'm included firebug output. I'm new to firebug, so I'm not entirely sure the best way to display results (there is obviously a ton of data).
The dragover and drop events appear to go fine. As do the fileSetup event, sent and inputSetup events (though digging deeper, there is lots of buried red and 'undefined' values). Then, the POST event has a 500 internal server error in filehandler.php.
Response Headers
Connection close
Content-Length 0
Content-Type text/html
Date Sun, 02 Nov 2014 03:05:06 GMT
Server Apache/2.2.22 (Win32) PHP/5.3.27
Request Headers
Accept text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-US,en;q=0.5
Content-Length 34124
Content-Type application/octet-stream
Host domain.com
Referer http://domain.com/path/index.html
User-Agent Mozilla/5.0 (Windows NT 6.3; WOW64; rv:33.0) Gecko/20100101 Firefox/33.0
X-File-Date Tue, 23 Sep 2014 22:10:03 GMT
X-File-Name smallInputFile.rdb
X-File-Size 34124
X-Requested-With FileDrop-XHR-FileAPI
I'm unsure what else to check. What would you check next? I believe it has something to do with server setup rather than the code (due to the fact it works on a default WAMP server) but for completeness, I'm including (abbreviated) relevant code is below:
(index.html) -- note that the alerts don't even fire when on the production server.
<html>
<head>
<script type="text/javascript" src="includes/ddup/filedrop.js"></script>
<link rel="stylesheet" type="text/css" href="style.css" />
</head>
<body>
<fieldset id="zone">
<legend>Drop a file inside…</legend>
<p>Or click here to <em>Browse</em>..</p>
</fieldset>
<span id="status"></span>
<script type="text/javascript">
var zone = new FileDrop('zone', options);
// Do something when a user chooses or drops a file:
zone.event('send', function (files) {
// Depending on browser support files (FileList) might contain multiple items.
files.each(function (file) {
// React on successful AJAX upload:
var p = document.createElement('p');
zone.el.appendChild(p);
file.event('done', function (xhr) {
// 'this' here points to fd.File instance that has triggered the event.
alert('Done uploading ' + this.name);
document.getElementById('status').innerHTML=xhr.responseText;
alert('Done uploading ' + this.name + ', response:\n\n' + xhr.responseText);
});
file.event('progress', function (sent, total) {
p.textContent = 'Uploaded ' + Math.round(sent / total * 100) + '%...';
})
// Send the file:
file.sendTo('fileHandler.php');
});
});
</script>
</body>
</html>
(fileHandler.php)
<?php
/** Error reporting */
error_reporting(E_ALL);
// If an error causes output to be generated before headers are sent - catch it.
ob_start();
/** Include path **/
ini_set('include_path', ini_get('include_path').';../Classes/');
if (!empty($_FILES['fd-file']) and is_uploaded_file($_FILES['fd-file']['tmp_name'])) {
// Regular multipart/form-data upload.
$filename = $_FILES['fd-file']['name'];
$filedata = file_get_contents($_FILES['fd-file']['tmp_name']);
} else {
// Raw POST data.
$filename = urldecode(#$_SERVER['HTTP_X_FILE_NAME']);
$filecontents = file_get_contents("php://input");
$filedata = explode("\n",$filecontents);
}
//**** do stuff to file ****//
// Save Excel 2007 file
$objWriter = new PHPExcel_Writer_Excel2007($objPHPExcel);
$fileBase = explode(".",$filename);
$outFilename = $fileBase[0] . date('His') . ".xlsx";
$url = "xlsxFiles/$outFilename";
$objWriter->save($url);
echo "<br><table class=\"rulesTable\" id=\"linkRow\"><tr><td><span id=\"finalLink\">Link to generated XLS file: $outFilename</span></td></tr></table>";
?>
Is your production server running a *NIX OS?
If it is, this line may well cause it to complain
ini_set('include_path', ini_get('include_path').';../Classes/');
In *NIX the path seperator is a colon : and not a semi colon ;
There is a Predefined constant called PATH_SEPERATOR that allows you to be OS agnostic so try this instead
set_include_path(get_include_path() . PATH_SEPARATOR . '../Classes/');

Image with 'image/pjpeg' mime type not displaying via PHP using GD library

I’m learning PHP & I’m working on a simple upload and display image exercise out of a Apress PHP book. I’m using the latest Eclipse program, XAMPP, Apache, etc…
The problem I’m having is that the image being uploaded is a JPEG but it is being read as type PJPEG. It pushes my IF statement to the FALSE section with my error messages.
If I change the all the criteria to pjpeg, then for some reason I get weird raw code instead of a picture.
How do I ensure that my image being loaded stays a jpeg or how can I rewrite my code to upload and output a pjpeg and actually display in browser.
UPDATE: the code provided by Jake is working but the image itself is not displaying in browsers. Tested in Chrome, Firefox and IE.
<?php
error_reporting(E_ALL);
if ($_SERVER['REQUEST_METHOD'] == 'POST') {
if ( isset($_FILES['photo']) &&
is_uploaded_file($_FILES['photo']['tmp_name']) &&
$_FILES['photo']['error']==UPLOAD_ERR_OK ){
if ($_FILES['photo']['type'] == 'image/jpeg'){
$j = imagecreatefromjpeg($_FILES['photo']['tmp_name']);
header('Content-Type: image/jpeg');
imagejpeg($j);
imagedestroy($j);
} else {
echo "Uploaded file was not a jpeg. <br />";
echo "The file ", $_FILES['photo']['name'], " uploaded was a ", $_FILES['photo']['type'];
}
} else {
echo "No photo upload";
}
} else {
?>
<form action="test.php" method="post" enctype="multipart/form-data">
<label for="photo">User Photo:</label>
<input type="file" name="photo" />
<input type="submit" name="submit" value = "Upload a JPEG" />
</form>
<?php } ?>
Change the code from this:
if ($_FILES['photo']['type'] == 'image/jpeg'){
$j = imagecreatefromjpeg($_FILES['photo']['tmp_name']);
header('Content-Type: image/jpeg');
imagejpeg($j);
imagedestroy($j);
}
To this:
// Set an array of mime types that you accept as valid.
$valid_photo_mime_types = array('image/jpeg', 'image/pjpeg');
// Now check if the mime type of the uploaded file matches your array values of valid photo mime types.
if (in_array($_FILES['photo']['type'], $valid_photo_mime_types)){
// Now let’s ingest the file that was uploaded & assign it to $j using the GD graphics library.
$j = imagecreatefromjpeg($_FILES['photo']['tmp_name']);
// Set the header to be whatever the uploaded file mime-type is.
header('Content-Type: ' . $_FILES['photo']['type']);
// Now send the image to the browser.
imagejpeg($j);
// And finally 'destroy' the stored image to free up memory.
imagedestroy($j);
}
Note that I git rid of the double-spacing of lines in your original example for space. And in my rewrite I am setting an array named $photo_mime_types and then using in_array to check if $_FILES['photo']['type']matches a value in $photo_mime_types. Then for header('Content-Type: '… I am setting that to whatever the value of $_FILES['photo']['type'] since at that point that value is valid, correct? Well programming is about avoiding repetition so let’s just pass what is valid when we know it is valid.
EDIT: This still seems to not be working for the original poster, so another debugging idea is to check the headers sent via curl -I from the command line. Unsure if you can do this on Windows, but if not you should figure out how to since curl is a great debugging tool for things like this. On a Mac OS X machine or a Unix/Linux box just run this command. For example, a call to the image logo on Google right now:
curl -I https://www.google.com/images/srpr/logo11w.png
The output I get is as follows:
HTTP/1.1 200 OK
Content-Type: image/png
Content-Length: 14022
Last-Modified: Wed, 09 Oct 2013 01:35:39 GMT
Date: Thu, 15 May 2014 01:47:25 GMT
Expires: Thu, 15 May 2014 01:47:25 GMT
Cache-Control: private, max-age=31536000
X-Content-Type-Options: nosniff
Server: sffe
X-XSS-Protection: 1; mode=block
Alternate-Protocol: 443:quic
Those are all of the headers that are sent with that image that tells the browser what to do with the image. Note the Content-Type: image/png. Now just substitute the URL to your local PHP code instead of the Google URL & check the output. Also look at this answer on Stack Overflow. As well as this one that explains how to use getimagesize to get data about the image within your PHP code. getimagesize is useful as well to show you what PHP is seeing the data to be.
change
if ($_FILES['photo']['type'] == 'image/jpeg'){
to
if (preg_match('#^image/[a-z\-]jpeg#',$_FILES['photo']['type']) === true) {
this will match any type starting with 'image/' and ending with 'jpeg' and containing only lowercase letter and - characters

How to return an image to an https request with PHP

After many hours,
I nailed down my problem concerning downloading an image with an https request.
When I access an image with the hard path (https://mysite/tmp/file.jpg), apache
returns it with success and I can view it in the browser, without any extra manipulation.
When I try to access it with another path
(https://mysite/files/file.jpg), in order to control its access with php, I get
a response from my php code, but I cannot view the image in the browser.
VirtualHost defined; mysite: set to /var/www/mysite
$app['controllers']->requireHttps();
Description of the environment:
mysite/tmp/file.jpg
mysite/files/.htaccess
//... no files; handled by Silex router. here is the .htaccess:
---
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule ^ ../web/index.php [L]
</IfModule>
---
https:// mysite/tmp/file.jpg, served with https:200 and viewed in browser; ok
https:// mysite/files/file.jpg served with https:200 but not viewed in browser; ?
Here are the 3 methods tried:
Method 1: Silex sendFile() direct method >
$app->get('/files/{onefile}', function ($onefile) use ($app) {
// Validate authorization; if ok, then
return $app->sendFile('/var/www/mysite/tmp/' . $onefile);
});
Method 2: Silex Streaming >
$app->get('/files/{onefile}', function ($onefile) use ($app) {
// Validate authorization; if ok, then
$stream = function () use ($file) {
readfile($file);
};
return $app->stream($stream, 200, array('Content-Type' => 'image/jpeg'));
Method 3: Symfony2 style >
$app->get('/files/{onefile}', function ($onefile) use ($app) {
// Validate authorization; if ok, then
$exactFile="/var/www/mysite/tmp/" . $onefile;
$response = new StreamedResponse();
$response->setCallback(function () use ($exactFile) {
$fp = fopen($exactFile, 'rb');
fpassthru($fp);
});
$response->headers->set('Content-Type', 'image/jpg');
$response->headers->set('Content-length', filesize($exactFile));
$response->headers->set('Connection', 'Keep-Alive');
$response->headers->set('Accept-Ranges','bytes');
$response->send();
This is what Chrome presents:
With this Chrome image this is the Http (Https or not, same result) request
Request URL:https://mysite/files/tmpphp0XkXn9.jpg
Request Method:GET
Status Code:200 OK
Request Headers:
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:keep-alive
Cookie:XDEBUG_SESSION=netbeans-xdebug; _MYCOOKIE=u1k1vafhaqik2d4jknko3c94j1
Host:mysite
Pragma:no-cache
User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36
Response Headers:
Accept-Ranges:bytes
Cache-Control:no-cache
Connection:Keep-Alive, Keep-Alive
Content-Length:39497
Content-Type:image/jpg
Date:Thu, 13 Jun 2013 13:44:55 GMT
Keep-Alive:timeout=15, max=99
Server:Apache/2.2.16 (Debian)
X-Powered-By:PHP/5.3.3-7+squeeze15
Other tests made to eliminate possible undesired behavior:
I checked the BOM and made sure the php code responding to the request is valid
and do not have undesired byte-order marks (BOMs) using Emacs (set-buffer-file-coding-system utf-8). Furthermore, to avoid unknown BOM tagged files, I executed the following command in the mysite directory (grep -rl $'\xEF\xBB\xBF' .). Nothing abnormal appeared.
UPDATE:
Looking at the files (image) received by the browser (save as on each image), this is what the tool (Hex Friend) help me to find, but I still do not understand why:
Comparing the two files
(one with success: mysite/tmp/file.jpg ; served directly by Apache)
(one with NO success: mysite/files/file.jpg; served by a PHP script).
At the beginning of the binary file, I get this difference:
At the end of the binary file, I get this difference:
Question:
How can I return an image (in stream or some other technique) with a php code, and view it in the browser ? All php code methods return the same output, I suspect an environment problem. Is there an environment setting that could produce the error I am experimenting ?
I am not happy with this solution (A PATCH), but the problem is fixed, by adding the following call:
$app->get('/files/{onefile}', function ($onefile) use ($app) {
ob_end_clean(); // ob_clean() gave me the same result.
...
}
Ok, it is fixed, but, can somebody explain to me how to fix it more intelligently ?
UPDATE
What happened ?:
This is what my interpretation:
Extra newlines remains in the orignal php raw files to a unintentionally position! It turns out that when you have a php file
<?php
...
?>
where you left some newlines (or spaces) after the ?>, those newlines will add up in the output buffer. Then trying to stream a file to the output will take these add up newlines and put them where they belong: in the header stream, or at the footer stream. Having a fixed size for the stream (equal to the image size) will take specially the header extra characters and shift accordingly the bytes outputted to the browser. I suspected that I add exactly 5 characters (0A 0A 20 0A 0A) corresponding to (linefeed linefeed space linefeed linefeed) discovered in the received image from the browser. Now the browser do not recognize the image structure, being shift from the offset 0, of 5 non logical character for the binary image. Therefore, the browser can only show a broken image icon.
Also look at: Another helping SO fix
Advise to PHP framework developer:
If you provide a sendFile() equivalent method, maybe you should thrown an Exception, when ob_get_contents() does not return an empty buffer, just before streaming to the output !
For now:
This small linux batch file can at least let you find where your code should be cleaned. After using it, I was able to remove the ob_end_clean()... after analyzing the output of this small script. It tells you suspicious php files that might contain extra space or newline at the end of your php files. Just execute and manually fix files:
#!/bin/bash
for phpfiles in $(ls -1R *.php); do
hexdump -e '1/1 "%.2x"' $phpfiles | tail -1 >endofphpfile;
if [ `cat endofphpfile` = "3f3e" ]; then
echo "OK.................File $phpfiles"
else
thisout=`cat endofphpfile`
echo "File $phpfiles: Suspucious. Check ($thisout) at the EOF; it should end with '?>' (in hex 3f3e) "
fi
done
It can surely be improved, but that should, at least, help anybody !
don't use the closing ?> ... it's against symfony coding standards.
Against PSR-2 to be more concrete.
All PHP files MUST end with a single blank line.
The closing ?> tag MUST be omitted from files containing only PHP.
The reason for that is partly what you experienced.
This behavior is sometimes caused by the BOM aswell.
check none of your files contains the byte-order-mark!

PHPExcel: I need to create two workbooks on one submission [duplicate]

Use case: user clicks the link on a webpage - boom! load of files sitting in his folder.
I tried to pack files using multipart/mixed message, but it seems to work only for Firefox
This is how my response looks like:
HTTP/1.0 200 OK
Connection: close
Date: Wed, 24 Jun 2009 23:41:40 GMT
Content-Type: multipart/mixed;boundary=AMZ90RFX875LKMFasdf09DDFF3
Client-Date: Wed, 24 Jun 2009 23:41:40 GMT
Client-Peer: 127.0.0.1:3000
Client-Response-Num: 1
MIME-Version: 1.0
Status: 200
--AMZ90RFX875LKMFasdf09DDFF3
Content-type: image/jpeg
Content-transfer-encoding: binary
Content-disposition: attachment; filename="001.jpg"
<< here goes binary data >>--AMZ90RFX875LKMFasdf09DDFF3
Content-type: image/jpeg
Content-transfer-encoding: binary
Content-disposition: attachment; filename="002.jpg"
<< here goes binary data >>--AMZ90RFX875LKMFasdf09DDFF3
--AMZ90RFX875LKMFasdf09DDFF3--
Thank you
P.S. No, zipping files is not an option
Zipping is the only option that will have consistent result on all browsers. If it's not an option because you don't know zips can be generated dynamically, well, they can. If it's not an option because you have a grudge against zip files, well..
MIME/multipart is for email messages and/or POST transmission to the HTTP server. It was never intended to be received and parsed on the client side of a HTTP transaction. Some browsers do implement it, some others don't.
As another alternative, you could have a JavaScript script opening windows downloading the individual files. Or a Java Applet (requires Java Runtimes on the machines, if it's an enterprise application, that shouldn't be a problem [as the NetAdmin can deploy it on the workstations]) that would download the files in a directory of the user's choice.
Remember doing this >10 years ago in the netscape 4 days. It used boundaries like what your doing and didn't work at all with other browsers at that time.
While it does not answer your question HTTP 1.1 supports request pipelining so that at least the same TCP connection can be reused to download multiple images.
You can use base64 encoding to embed an (very small) image into a HTML document, however from a browser/server standpoint, you're technically still sending only 1 document. Maybe this is what you intend to do?
Embedd Images into HTML using Base64
EDIT: i just realized that most methods i found in my google search only support firefox, and not iE.
You could make a json with multiple data urls.
Eg:
{
"stamp.png": "data:image/png;base64,...",
"document.pdf": "data:application/pdf;base64,..."
}
(extending trinalbadger587's answer)
You could return an html with multiple clickable, downloadable, inplace data links:
<html>
<body>
<a download="yourCoolFilename.png" href="data:image/png;base64,...">PNG</a>
<a download="theFileGetsSavedWithThisName.pdf" href="data:application/pdf;base64,...">PDF</a>
</body>
</html>

gzcompress() randomly inserting extra data?

I've been researching this all morning and have decided that as a last-ditch effort, maybe someone on Stack Overflow has a "been-there, done-that" type of answer for me.
Background Recently, I implemented compression on our (intranet-oriented) Apache (2.2) server using filters so that all text-based files are compressed (css, js, txt, html, etc.) via mod_deflate, mentioning nothing about php scripts. After plenty of research on how best to compress PHP output, I decided to use the gzcompress() flavor because the PHP documentation suggests that using zlib library and gzip (using the deflate algorithm, blah blah blah) is preferred over ob_gzipwhatever().
So I copied someone else's method like so:
<?php # start each page by enabling output buffering and disabling automatic flushes
ob_start();ob_implicit_flush(0);
(program logic)
print_gzipped_page();
function print_gzipped_page() {
if (headers_sent())
$encoding = false;
elseif(strpos($_SERVER['HTTP_ACCEPT_ENCODING'],'x-gzip') !== false )
$encoding = 'x-gzip';
elseif(strpos($_SERVER['HTTP_ACCEPT_ENCODING'],'gzip') !== false )
$encoding = 'gzip';
else
$encoding = false;
if($encoding){
$contents = ob_get_contents(); # get contents of buffer
ob_end_clean(); # turn off OB and flush buffer
$size = strlen($contents);
if ($size < 512) { # too small to be worth a compression
echo $contents;
exit();
} else {
header("Content-Encoding: $encoding");
header('Vary: Accept-Encoding');
# 8-byte file header: g-zip file (1f 8b) compression type deflate (08), next 5 bytes are padding
echo "\x1f\x8b\x08\x00\x00\x00\x00\x00";
$contents = gzcompress($contents, 9);
$contents = substr($contents, 0,$size); # faster than not using a substr, oddly
echo $contents;
exit();
}
} else {
ob_end_flush();
exit();
}
}
Pretty standard stuff, right?
Problem Between 10% and 33% of all our PHP page requests sent via Firefox go out fine and come back g-zipped, only Firefox displays the compressed ASCII in lieu of decompressing it. AND, the weirdest part, is that the content size sent back is always 30 or 31 bytes larger than the size of the page correctly rendered. As in, when the script is displayed properly, Firebug shows content size of 1044; when Firefox shows a huge screen of binary gibberish, Firebug shows a content size of 1074.
This happened to some of our users on legacy 32-bit Fedora 12s running Firefox 3.3s... Then it happened to a user with FF5, one with FF6, and some with the new 7.1! I've been meaning to upgrade them all to FF7.1, anyway, so I've been updating them as they have issues, but FF7.1 is still exhibiting the same behavior, just less frequently.
Diagnostics I've been installing Firebug on a variety of computers to watch the headers, and that's where I'm getting confused:
Normal, functioning page response headers:
HTTP/1.1 200 OK
Date: Fri, 21 Oct 2011 18:40:15 GMT
Server: Apache/2.2.15 (Fedora)
X-Powered-By: PHP/5.3.2
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Encoding: gzip
Vary: Accept-Encoding
Content-Length: 1045
Keep-Alive: timeout=10, max=75
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
(Notice that content-length is generated automatically)
Same page when broken:
HTTP/1.1 200 OK
(everything else identical)
Content-Length: 1075
The sent headers always include Accept-Encoding: gzip, deflate
Things I've tried to fix the behavior:
Explicitly declare content length with uncompressed and compressed lengths
Not use the substr() of $contents
Remove checksum at the end of $contents
I don't really want to use gzencode because my testing showed it to be significantly slower (9%) than gzcompress, presumably because it's generating extra checksums and whatnot that I (assumed) the web browsers don't need or use.
I cannot duplicate the behavior on my 64-bit Fedora 14 box running Firefox 7.1. Not once in my testing before rolling the compression code live did this happen to me, neither in Chrome nor Firefox. (Edit: Immediately after posting this, one of the windows I'd left open that sends meta refreshes every 30 seconds finally broke after ~60 refreshes in Firefox) Our handful of Windows XP boxes are behaving the same as the Fedora 12s. Searching through Firefox's Bugzilla kicked up one or two bug requests that were somewhat similar to this situation, but that was for versions pre-dating 3.3 and was with all gzipped content, whereas our Apache gzipped css and js files are being downloaded and displayed without error each time.
The fact that the content-length is coming back 30/31 bytes larger each time leads me to think that something is breaking inside my script/gzcompress() that is mangling something in the response that Firefox chokes on. Naturally, if you play with altering the echo'd gzip header, Firefox throws a "Content Encoding Error," so I'm really leaning towards the problem being internal to gzcompress().
Am I doomed? Do I have to scrap this implementation and use the not-preferred ob_start("ob_gzhandler") method?
I guess my "applies to more than one situation" question would be: Are there known bugs in the zlib compression library in PHP that does something funky when receiving very specific input?
Edit: Nuts. I readgzfile()'d one of the broken, non-compressed pages that Firefox downloaded and, lo and behold!, it echo'd everything back perfectly. =( That means this must be... Nope, I've got nothing.
okay 1st of all you don't seem to be setting the content length header, which will cause issues, instead, you are making the gzip content longer so that it matches the content length size that you were receiving in the 1st place. This is going to turn ugly. My suggestion is that you replace the lines
# 8-byte file header: g-zip file (1f 8b) compression type deflate (08), next 5 bytes are padding
echo "\x1f\x8b\x08\x00\x00\x00\x00\x00";
$contents = gzcompress($contents, 9);
$contents = substr($contents, 0,$size); # faster than not using a substr, oddly
echo $contents;
with
$compressed = gzcompress($contents, 9);
$compressed_length = strlen($compressed); /* contains no nulls i believe */
header("Content-length: $compressed_length");
echo "\x1f\x8b\x08\x00\x00\x00\x00\x00", $compressed;
and see if it helps the situation.
Ding! Ding! Ding! After mulling over this problem all weekend, I finally stumbled across the answer after re-reading the PHP man pages for the umpteenth time... From the zlib PHP documentation, "Whether to transparently compress pages." Transparently! As in, nothing else is required to get PHP to compress its output once zlib.output_compression is set to "On". Yeah, embarrassing.
For reasons unknown, the code being called, explicitly, from the PHP script was compressing the already-compressed contents and the browser was simply unwrapping the one layer of compression and displaying the results. Curiously, the strlen() of the content didn't vary when output_compression was on or off, so the transparent compression must occur after the explicit compression, but it occasionally decided not to compress what was already compressed?
Regardless, everything is resolved by simply leaving PHP to its own devices. zlib doesn't need output buffering or anything to compress the output.
Hope this helps others struggling with the wonderful world of HTTP compression.

Categories