I am using a controller in CodeIgniter 2 to generate a resized image based on its URL however the pictures are getting randomly cut off on one of our servers (only the top two thirds or so get shown). I believe the problem is related to GZIP being used when the picture is generated but it is difficult to reproduce consistently and disappears entirely when I turn compression off.
Interestingly, the GZIP compression is only added to the outputted JPEG when the text/html is being compressed.
This is my apache configuration:
<IfModule mod_deflate.c>
<FilesMatch "(?<!\.jpg)$">
AddOutputFilterByType DEFLATE text/html
</FilesMatch>
</IfModule>
<FilesMatch "\.jpg$" >
SetEnv no-gzip dont-vary
RemoveOutputFilter php
</FilesMatch>
My htaccess file contains this
SetEnvIfNoCase Request_URI ".jpg$" no-gzip dont-vary
<FilesMatch "\.(gif|jpe?g|png)$">
SetEnv no-gzip dont-vary
</FilesMatch>
PHP code looks like this
//code to generate the image and save it to the hard drive goes here
//output the file
$binarydata = file_get_contents($directory.$file);
$offset = 5184000;
header("Cache-Control: must-revalidate");
header('Content-Type: image/jpeg');
header('Expires: '. gmdate('D, d M Y H:i:s', $dbrecord->updated_at->format("U") + $offset) .' GMT');
header('Last-Modified: '.gmdate('D, d M Y H:i:s', $dbrecord->updated_at->format("U")) .' GMT');
header('Accept-Ranges: bytes');
header("Content-Disposition: inline; filename=".$file);
header('Content-Length: '.strlen($binarydata));
print($binarydata);
die();
The URL of the photo would be something like http://example.com/photo/100x100-abc.jpg and as you can see, there are four different ways of trying to avoid the compression being set however it is still set.
I have tried all the methods above separately and combined but to no avail. I am at my wits end, can anyone help?
UPDATE:
I removed the content length header and apache did not set it on its own but we have just discovered something interesting, the problem only occurs in Chrome and Safari and not Firefox so I wonder if this is also webkit related?
Related
I have a php script which copies an image to a folder every hour over writing the last image, the name of the image is always the same, image.jpg , im using ifttt to post the image to a website but the image posted is always the same image even though I have checked to see the image is different. so it must be posting the image from the cache? I added clearstatcache(); to the code but it doesn't work. is there any other way of clearing the cache so the actual image is posted? the image is hosted on a shared server. Thanks
$from = '/A1';
$files = scandir($from);
$to = '/A2';
if (!empty($files[2])) {
rename("{$from}/{$files[2]}", "{$to}/image.jpg");
}
clearstatcache();
solved with .htaccess file
<FilesMatch "\.(css|flv|gif|htm|html|ico|jpe|jpeg|jpg|js|mp3|mp4|png|pdf|swf|txt)$">
<IfModule mod_expires.c>
ExpiresActive Off
</IfModule>
<IfModule mod_headers.c>
FileETag None
Header unset ETag
Header unset Pragma
Header unset Cache-Control
Header unset Last-Modified
Header set Pragma "no-cache"
Header set Cache-Control "max-age=0, no-cache, no-store, must-revalidate"
Header set Expires "Thu, 1 Jan 1970 00:00:00 GMT"
</IfModule>
</FilesMatch>
Thanks for all your help!
I have a problem that one of my users, make an insert into my mysql database, but when he goes to another page to list these inserts, it only return old data that was previous there. He can only see the changes after several minutes or even hours.
I read that If I add a random number in my url, I can avoid cache, but still not working.
I tried to put these headers on php.
header("Cache-Control: private, no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
And I tried this one too:
SELECT SQL_NO_CACHE...
None work. Still don't get any visible changes.
Try using the Expires headers in your htaccess file. You will need to ask your user to Ctrl+F5 (Force refresh) for it to correctly take effect;
<IfModule mod_expires.c>
ExpiresActive on
ExpiresByType text/html "access plus 0 seconds"
</IfModule>
This will tell the browser not to cache the text and html within that directory and any subdirectories (unless otherwise specified). I had a similiar issue and resolved it with this when I realised my caching was affecting the update of live data.
Setup
Debian GNU/Linux 7.6 (wheezy) will full access to Apache and virtual
host files.
Concrete5 CMS: http://www.concrete5.org/
Problem
Our website is sending Pragma: no-cache in headers which is stopping a number of optimisations from working - including the CloudFlare service: https://www.cloudflare.com/
Solutions which didn't work
When researching, we either didn't understand the answers or they seemed to be for specific use cases (like Oracle or php frameworks we're not using) but we did try the following:
1. Force caching via the site's .htaccess:
<FilesMatch "\.(ico|jpeg|png|gif|js|css)$">
Header unset Cache-Control
Header unset Pragma
</FilesMatch>
2. Force caching via Concrete5's header.php file
header("Cache-Control: max-age=2592000"); //30days (60sec * 60min * 24hours * 30days)
3. Search site's root for no-cache using grep
$ grep -r "no-cache" * .
backup/databasebackup.php:header('Cache-Control: no-cache, must-revalidate');
concrete/core/controllers/single_pages/login.php: header("Cache-Control: no-store, no-cache, must-revalidate");
concrete/core/controllers/single_pages/login.php: header("Pragma: no-cache");
concrete/js/tiny_mce/plugins/spellchecker/rpc.php:header("Cache-Control: no-store, no-cache, must-revalidate");
concrete/js/tiny_mce/plugins/spellchecker/rpc.php: header("Pragma: no-cache");
concrete/libraries/3rdparty/securimage/securimage.php: header('Cache-Control: no-store, no-cache, must-revalidate');
concrete/libraries/3rdparty/securimage/securimage.php: header("Cache-Control: no-store, no-cache, must-revalidate");
concrete/libraries/3rdparty/securimage/securimage.php: header("Pragma: no-cache");
But after looking at the files/scripts Concrete5 is setting no-cache on (login, database backups, text editor configs etc), we kind of understand why - plus these seem to be for specific files, not the entire site right?
4. Make a blank php file, request it and check the header
The blank file was served with caching on so we suspect that php is the culprit - but have no idea how to isolate the cause sorry.
Question
How do we troubleshoot and fix this issue?
Skill level
We do front-end design and understand the basics on how to setup and serve a CMS but don't have much experience with server configuration or troubleshooting cache issues.
We have command line access to the server and pretty much have full access to Debian, Apache, and the site's install.
Any help would be much appreciated.
Cheers
Ben
Update
To add max-age in a PHP script:
header("Cache-Control: max-age=xxxx");
Where xxxx is the number of seconds to cache, zero for no cache.
OR
If you configure by Content-Type (MIME Type)
header('Content-Type: text/html; charset=utf-8');
Header Set Cache-Control "max-age=0, no-store"
to configure cache by Content-Type (MIME Type):
In .htaccess ot httpd.conf
ExpiresByType text/html "access plus 30 day"
ExpiresByType text/css "access plus 30 day"
ExpiresByType text/javascript "access plus 30 day"
ExpiresByType image/gif "access plus 1 year"
ExpiresByType image/jpg "access plus 1 year"
ExpiresByType image/png "access plus 1 year"
If these methods are not working you need to majke sure the modules are loaded.
You need access to httpd.conf
LoadModule expires_module libexec/mod_expires.so
LoadModule headers_module libexec/mod_headers.so
AddModule mod_expires.c
AddModule mod_headers.c
...
AddModule mod_gzip.c
Note that the load order is important in Apache/1.3x, mod_gzip must load last, after all other modules.
For Apache/2.0:
LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
LoadModule deflate_module modules/mod_deflate.so
end of Update
You should add the same cache based on MIME Type as well as (or rather than) file extension.
The cache should be max-age.
The W3C says max-age takes precedence over all other cache headers.
To troubleshoot you are doing well already if you are not getting "Internal Server Error 500"
In FireFox or Chrome
Right Click page
Select Inspect Element
Go to the "Network Tab"
Change Type from "All" to "HTML"
Click on the HTML page in the list
You should be able to see exactly what is in the HTTP Response Header.
FireFox
Chrome
Finally got to the bottom of this, some of the Concrete5 Blocks on pages are stopping the pages from being cached.
If we turn on "Force full page caching" then we see some weird behaviour because the functionality is frozen by the cache, so we've had to turn it off.
Essentially, we can't cache the site fully because of the functionality in blocks on the page. We can only use APC caching.
I have a rather simple force download script:
$thefile = $_GET['id'];
$file = 'http://www.mywebsite.com/test/'.$thefile;
header('Content-Disposition: attachment; filename='.$thefile);
header("Content-Type: application/force-download");
#readfile($file);
And it works nicely, except it does not show how much time is remaining ("could not calculate time").
I found someone with a similar problem here, by using this code:
# for URL paths that begin with "/foo/bar/"
SetEnvIf Request_URI ^/foo/bar/ no-gzip=1
# for files that end with ".py"
<FilesMatch \.py$>
SetEnv no-gzip 1
</FilesMatch>
But alas, I can not get that to work. My files are media files (avi, wmv, mp4 and so forth), so I assumed I would have to modify the code along these lines:
SetEnvIf Request_URI ^/test/ no-gzip=1
<FilesMatch "\.(avi|wmv|mp4|mpg|mpeg|m4v)$">
SetEnv no-gzip 1
</FilesMatch>
I have also changed the path to a simple test folder, but time is still not being displayed.
How to?
You should add content-length header, for your browser to know total file size.
header("Content-Length: " . filesize($file));
Take a look at this example.
I'm having some trouble prevent mod_deflate from jumping in on this scenario:
user running CodeIgniter (or any other framework that re-directs to index.php)
mod_deflate is active
zip file is served by a CodeIgniter controller (headers + readfile)
The thing is that Apache always detects the content as being php and therefor something like the lines bellow wont work as the server assumes the ZIP file as being a PHP one.
<FilesMatch "\.(xml|txt|html|php)$">
SetOutputFilter DEFLATE
</FilesMatch>
Any ideas on how I can have Apache distinguish from an HTML file or a ZIP file both generated by the same index.php framework file.
Edit:
apache log
[Mon Jun 20 02:14:19 2011] [debug]
mod_deflate.c(602): [client 192.168.0.5]
Zlib: Compressed 50870209 to 50878224 : URL /index.php,
referer: http://demo.dev/
Edit:
CI controller that serves the zip
header('Content-Type: application/zip');
header('Content-Transfer-Encoding: binary');
header("Content-Length: " . filesize($file_location));
header('Content-Disposition: attachment; filename="' . $file_title . '"');
readfile($file_location);
Even tough all answers should have been perfectly valid in a reasonable scenario (and were actually tested prior to making the question) the reason to why I've been unable to instruct Apache to deflate a file by MIME-Type remains unknown.
I was able to have it work as desired by forcing the following instructions into the script
apache_setenv('no-gzip', 1);
ini_set('zlib.output_compression', 0);
I do understand that this is a hot patch and is not addressing the problem's root but so far that will have to suffice. As there are others who may hit the same flag, the above code stays here for reference in what is a dirty fix.
You can either:
use the deprecated AddOutputFilterByType and specify only the content types you do want to filter; or
use the more powerful mod_filter. In FilterProvider you can provide a rule that excludes the filter when the zip content type (application/zip) is found in the response headers.
You can make use of mod_rewrite to change the mime-type of the request on the Apache level:
# Serve .zip request as zip-files
RewriteRule \.zip$ - [T=application/zip,E=no-gzip:1]
Place it above the rules of the framework, however this needs to make DEFLATE as well depended on mime-type and not file-extension as you do with <FilesMatch>.
Probably it works well together with
AddOutputFilterByType DEFLATE text/html
instead of the <FilesMatch> Directive.
Edit: Added the L flag which should be used in .htaccess context and additionally turned DEFLATE off via the no-gzip environment variable.
Try this (since your urls appear to end in .zip it might work for you):
<FilesMatch "\.(xml|txt|html|php)$">
SetEnvIf Request_URI "\.zip$" no-gzip
SetOutputFilter DEFLATE
</FilesMatch>
Instead of using
<FilesMatch "\.(xml|txt|html|php)$">
SetOutputFilter DEFLATE
</FilesMatch>
Use this configuration for setting compression rules.
AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/x-javascript application/javascript
This way, your output will be compressed only if content-type matches with above directives.
CI controller that serves the zip is already sending correct content-type header, so this will not get compressed.
header('Content-Type: application/zip');