The Situation
I am creating a video training site for a client on the RackSpace Cloud using the traditional LAMP stack (RackSpace's cloud has both Windows and LAMP stacks). The videos and other media files I'm serving on this site need to be protected as my client charges money for access to them. There is no DRM or funny business like that, essentially we store the files outside of the web root and use PHP to authenticate user's before they are able to access the files by using mod_rewrite to run the request through PHP.
So let's say the user requests a file at this URL:
http://www.example.com/uploads/preview_image/29.jpg
I am using mod_rewrite to rewrite that url to:
http://www.example.com/files.php?path=%2Fuploads%2Fpreview_image%2F29.jpg
Here is a simplified version of the files.php script:
<?php
// Setups the environment and sets $logged_in
// This part requires $_SESSION
require_once('../../includes/user_config.php');
if (!$logged_in) {
// Redirect non-authenticated users
header('Location: login.php');
}
// This user is authenticated, continue
$content_type = "image/jpeg";
// getAbsolutePathForRequestedResource() takes
// a Query Parameter called path and uses DB
// lookups and some string manipulation to get
// an absolute path. This part doesn't have
// any bearing on the problem at hand
$file_path = getAbsolutePathForRequestedResource($_GET['path']);
// At this point $file_path looks something like
// this: "/path/to/a/place/outside/the/webroot"
if (file_exists($file_path) && !is_dir($file_path)) {
header("Content-Type: $content_type");
header('Content-Length: ' . filesize($file_path));
echo file_get_contents($file_path);
} else {
header('HTTP/1.0 404 Not Found');
header('Status: 404 Not Found');
echo '404 Not Found';
}
exit();
?>
The Problem
Let me start by saying this works perfectly for me. On local test machines it works like a charm. However once deployed to the cloud it stops working. After some debugging it turns out that if a request to the cloud has certain file extensions like .JPG, .PNG, or .SWF (i.e. extensions of typically static media files.) the request is routed to a cache system called Varnish. The end result of this routing is that by the time this whole process makes it to my PHP script the session is not present.
If I change the extension in the URL to .PHP or if I even add a query parameter Varnish is bypassed and the PHP script can get the session. No problem right? I'll just add a meaningless query parameter to my requests!
Here is the rub: The media files I am serving through this system are being requested through compiled SWF files that I have zero control over. They are generated by third-party software and I have no hope of adding or changing the URLs that they request.
Are there any other options I have on this?
Update: I should note that I have verified this behavior with RackSpace support and they have said there is nothing they can do about it.
If the requesting flash app is following redirects, I would try to answer with a redirect on the first request and rewrite the second one, e.g.
GET .../29.jpg
to
header("Status: 302 Moved temporarily");
header("Location: .../r.php?i=29.jpg&random=872938729348");
Then your r.php delivers the file on the second request.
If not (btw. always), I would explicitly send headers along with delivering the static files that Varnish accepts and acts accordingly, something like
header("Cache-Control: no-cache, must-revalidate, max-age=0, post-check=0, pre-check=0");
header("Expires: Sat, 26 Jul 1997 05:00:00 GMT");
And:
I would place the exit(); command after your first header() statement to be sure the rest of the script is not executed. header() sends just headers.
I find it also more reliable to use ob_start() as whitespace in your PHP file may lead to annoying errors when adding headers.
I have the same situation, and I've contacted Rackspace hoping for a better answer.
I got one! They've put together a FAQ outlining half a dozen ways to bypass/modify the caching:
http://cloudsites.rackspacecloud.com/index.php/How_can_I_bypass_the_cache%3F
Related
I have a PHP file get_css.php which generates CSS code more than 60 KB long. This code does not change very often. I want this code to be cached in user's browser.
Now, when i visit a HTML page several times which includes get_css.php url to fetch css, my browser is loading all CSS contents from the server each time i visit the page.
Browsers should get the contents from server only if the CSS code is changed on server side. If the css code is not changed, browser will use the css code from the browser cache.
I cannot use any PHP function which is not allowed in Server Safe Mode.
Is it possible? How can i achieve this?
You cannot force a client to revalidate its cache so easily.
Setting a variable query string to its resource won't play well with proxies, but seems to suffice with browsers. Browsers do tend to only redownload the css file if there's a query string change.
<link rel="stylesheet" type="text/css" href="/get_css.php?v=1.2.3">
Potentially, you could play with the naming of the CSS, such as add numbers, but this isn't a great alternative.
You cannot control browser behaviour from PHP, but you can use HTTP codes to tell the browser something.
If the CSS is not changed, just reply with a 304 Not Modified response code:
if ($css_has_not_changed && $browser_has_a_copy) {
http_response_code(304);
} else {
// regenerate CSS
}
This way, the browser will ask for the document (which you cannot control), but you tell him to use the cached copy.
Of course this needs testing, as I have now idea how it will work 'the first time' a browser requests the file (perhaps the request headers can tell you more). A quick firebug test reveals that Firefox requests Cache-Control: no-cache when it is requesting a fresh copy, and Cache-Control: max-age=0 when it has cache.
add normal GET parameter when you including get_css.php like so
<link rel="stylesheet" type="text/css" href="get_css.php?v=1">
Browser will think that it is new link and will load it again.
and in get_css.php use this to make browser cache data
<?php
header("Content-type: text/css");
header('Cache-Control: public');
header('Expires: ' . gmdate('D, d M Y H:i:s', strtotime('+1 year')) . ' GMT');
ob_start("ob_gzhandler");
//echo css here
The browser wants to cache your document by default, but you have to give it enough info to make that possible. One fairly easy way is to send the Last-Modified header, containing the date/time at which your script was last changed. You'll also need to handle the browser's "revalidation" request correctly by checking the incoming Last-Modified date, comparing it to the actual modified date of your script, and returning a 304 Not Modified response (with an empty response body), if the file is unchanged.
It's also a good idea to be sure that your server isn't "magically" sending any other "no-cache" directives. The easiest way to do this is to send a Cache-Control directive that tells the browser exactly what behavior you expect.
Here is a quick explanation of each Cache-Control option.
Something like the following should do the trick:
<?php
// this must be at the top of your file, no content can be output before it
$modified = filemtime(__FILE__);
if(isset($_SERVER['HTTP_IF_MODIFIED_SINCE'])) {
$if_modified_since=strtotime($_SERVER["HTTP_IF_MODIFIED_SINCE"]);
if( $modified > $if_modified_since ) {
header('HTTP/1.0 304 Not Modified');
exit();
}
}
header('Cache-Control: must-revalidate');
header('Last-Modified: '.date("r",$modified));
// ... and the rest of your file goes here...
The above example was based heavily on the example, and writeup found here.
I'm using a PHP proxy script to load images from Facebook into Flash without any sandbox violations. It is taken from the guide here: http://www.permadi.com/blog/2010/12/loading-facebook-profile-picture-into-flash-swf-using-open-graph-api/. The relevant PHP code is:
<?php
$path=$_GET['path'];
if (stristr($path, "fbcdn.")==FALSE && stristr($path, "facebook.")==FALSE)
{
echo "ERROR";
exit;
}
header("Content-Description: Facebook Proxied File");
header("Content-Type: image");
header("Content-Disposition: attachment; filename=".$path);
#readfile($path);
?>
The guide mentions that additional security measures are recommended for a real world application. What additional measures would be applicable to this? Maybe some kind of key passed from Flash to PHP?
I realise that there's nothing I can do to completely protect the Flash from being decompiled, but can I prevent the script from being used maliciously?
You should restrict the proxy to fetching image files from Facebook. You current "protection" will allow for example this URL: http://virus.provider.com/fbcdn./virus.exe
Make better checks of the domain bname, maybe using the parse_url function.
Check that you are indeed serving only images. Make sure the filename is ending in a image extension (this helps a lot for Windows clients), but also consider doing more thorough checks of the actual file content.
Consider adding a check of the $_SERVER['HTTP_REFERER'] to lower the incentives to use your script for hotlinking. If the HTTP_REFERER is non-empty, check that it's actually your site in there. This will mostly protect you from bandwidth thieves.
Make sure it's actually a remote path. Your current script can be tricked to sending for example your PHP files unparsed, including passwords and other secrets!
The filname in the Content-Disposition header should be set to a filename, not to the entire path.
Also consider caching the file data on your proxy server to speed up multiple calls to the same file.
These are a few of the things to keep in mind. You may reveal more if you put some thought into it.
I am trying to build an application in which i have to stream the media files (audio and video) to the browser. I am reading the file through php and send the data to browser. I am using the following code.
header("Cache-Control: no-cache, must-revalidate"); // HTTP/1.1
header("Expires: Sat, 26 Jul 1997 05:00:00 GMT"); // Date in the past
header("Content-Type: {$file->getMimetype()}");
header("Content-Disposition: inline; filename=".$filename.";");
header("Content-Length: ".strlen($file_content));
echo $file_content;
Every thing is working fine, except when i try to forward the video or audio, (I mean suppose current play location is 0:15 and it directly go to 1:25), media stops and when i press the play button again, it starts from the beginning.
I think the problem is with the buffering, but can't figure it out. Am i doing something wrong in header or something else is required.
Thanks.
I think you need to implement the Range header, so that the client can skip to a specific position in the file. You can probably find out what goes wrong by sniffing the request the player sends.
What you want is called "Content-Range requests"
Have a look here Resumable downloads when using PHP to send the file?
I came across this recently which may help you:
http://www.jasny.net/articles/how-i-php-x-sendfile/
Rather than passing the whole file through PHP (which eats up memory), you can use x-sendfile. This is an Apache module which allows you to run a PHP program, but pass control back to the web server to handle the actual file download once your code has done what it needs to do (authentication, etc).
It means that your PHP code doesn't have to worry about how the file is served; let the web server do what it's designed for.
Hope that helps.
Here is a good tutorial for it, you only want the PHP section but still:
http://www.devshed.com/c/a/PHP/Video-Streaming-PHP-Script-Tutorial/3/
I have a php dynamically generated image which I need to write to file to call later. My problem is that I need this image to have appropriate expiration headers included in it. There are a massive number of these and their headers vary individually file-by-file making .htaccess controls not an option.
I can write expiration headers if I'm outputting the image directly to the browser with this:
header("Content-Type: image/jpeg");
header('Expires: "' . gmdate("D, d M Y H:i:s", $expirationDate) . '"');
imagepng($image, NULL);
Or I can write the image to a file to be used later with this:
imagepng($image, $filepath)
But I can't for the life of me figure out how to combine those two and write the image to a file while including its expiration headers. How would you go about writing an image file with an expires header?
I think your best bet it to server the file just as you are, something like:
header("Content-Type: image/jpeg");
header('Expires: "' . gmdate("D, d M Y H:i:s",
$expirationDate) . '"');
imagepng($image, NULL);
Sure you're using php to serve a static file, but the expire header is going to limit repeat requests.
Update: Since $image is a generated file, on the first request generate and save the image, then output it. On additional requests, just output the already generated image. Essentially the expire headers are controlling the browser's cache, while you need to implement some kind of caching on the server to avoid generating the same output multiple times.
So you're looking at two different kinds of caching. You can do them in the same script, with a combination of two scripts - really however you want.
Unless you can set a standard expire header with apache (which you say you can't, since it varies), I believe this is your best (if not only) choice.
Of course there is the convoluted and complex way:
Set up mod_rewrite to send requests for missing images to your php script.
Append some session id to the image request (so it's unique to the browser).
Have the php script send the expire header, and the image content.
Have the php script link the real static image to the session specific image name.
Or something like that. I'd just serve them all up using php.
Update: Or use mod_asis from VolkerK's great answer.
If you really want to store both the headers and the content in files on the server you could use mod_asis:
In the server configuration file, associate files with the send-as-is handler e.g.
AddHandler send-as-is asis
The contents of any file with a .asis extension will then be sent by Apache to the client with almost no changes. In particular, HTTP headers are derived from the file itself according to mod_cgi rules, so an asis file must include valid headers, and may also use the CGI Status: header to determine the HTTP response code.
Your php script then would write both the headers and the content to files that are handled as send-as-is by the apache webserver.
Perhaps all you have to do is exactly ...nothing, except writing the image data to the disc.
Depending on the webserver you're using some caching mechanisms work out of the box for static files (which you would create with the php script).
If you're using apache's httpd take a look at http://httpd.apache.org/docs/2.2/mod/core.html#fileetag and http://httpd.apache.org/docs/2.2/caching.html. By default httpd will also send a last-modified header and it supports If-Modified-Since request headers.
When your php script changes the image files the ETag changes as well and/or the If-Modified-Since condition would be met and the httpd sends the data. Otherwise it would only send a response saying "nothing has changed" to the client.
On a PHP-based web site, I want to send users a download package after they have filled out a short form. The site-initiated download should be similar to sites like download.com, which say "your download will begin in a moment."
A couple of possible approaches I know about, and browser compatibility (based on a quick test):
1) Do a window.open pointing to the new file.
- FireFox 3 blocks this.
- IE6 blocks this.
- IE7 blocks this.
2) Create an iframe pointing to the new file.
- FireFox 3 seems to think this is OK. (Maybe it's because I already accepted it once?)
- IE6 blocks this.
- IE7 blocks this.
How can I do this so that at least these three browsers will not object?
Bonus: is there a method that doesn't require browser-conditional statements?
(I believe that download.com employs both methods conditionally, but I can't get either one to work.)
Responses and Clarifications:
Q: "Why not point the current window to the file?"
A: That might work, but in this particular case, I want to show them some other content while their download starts - for example, "would you like to donate to this project?"
UPDATE: I have abandoned this approach. See my answer below for reasons.
You can also do a meta refresh, which most browsers support. Download.com places one in a noscript tag.
<meta http-equiv="refresh" content="5;url=/download.php?doc=123.zip"/>
Update: I have decided to abandon this approach, and instead just present the user with a link to the actual file. My reasoning is this:
My initial attempts at a server-initiated download were blocked by the browser. That got me thinking: "the browser is right. How does it know that this is a legitimate download? It should block a download that isn't obviously user-initiated."
Any method that I can use for a server-initiated download could also be used by someone who wants to send malware. Therefore, downloads should only happen when the user specifically requests the file by clicking on a link for it.
You're free to disagree, and if you still want to initiate a download, hopefully this thread will help you do it.
I usually just have a PHP script that outputs the file directly to the browser with the appropriate Content-Type
if(file_exists($filename)) {
header("Pragma: public");
header("Expires: 0");
header("Cache-Control: must-revalidate, pre-check=0");
header("Cache-Control: private", false);
header("Content-Type: " . $content-type);
header("Content-Disposition: attachment; filename=\"" . basename($filename) . "\";" );
header("Content-Transfer-Encoding: binary");
header("Content-Length: " . filesize($filename));
readfile("$filename");
}else{
print "ERROR: the file " . basename($filename) . " could not be downloaded because it did not exist.";
}
The only disadvantage is that, since this sets the HTTP header, it has be called before you have any other output.
But you can have a link to the PHP download page and it will cause the browser to pop up a download box without messing up the content of the current page.
One catch is that you may encounter issues with IE (version 6 in particular) if the headers are not set up "correctly".
Ensure you set the right Content-Type, but also consider setting the Cache options for IE (at least) to allow caching. If the file is one the user can open rather than save (e.g. an MS Word document) early versions of IE need to cache the file, as they hand off the "open" request to the applicable app, pointing to the file that was downloaded in the cache.
There's also a related issue, if the IE6 user's cache is full, it won't properly save the file (thus when the applicable app gets the hand off to open it, it will complain the file is corrupt.
You may also want to turn of any gzip actions on the downloads too (for IE)
IE6/IE7 both have issues with large downloads (e.g. 4.x Gigs...) not a likely scenario since IE doesn't even have a download manager, but something to be aware of.
Finally, IE6 sometimes doesn't nicely handle a download "push" if it was initiated from within a nested iframe. I'm not exactly sure what triggers the issue, but I find it is easier with IE6 to avoid this scenario.
Hoi!
#Nathan:
I decided to do exactly that: Have my "getfile.php" load all necessary stuff and then do a
header("Location: ./$path/$filename");
to let the browser itself and directly do whatever it thinks is correct do with the file. This even works fine in Opera with me.
But this will be a problem in environments, where no direct access to the files is allowed, in that case you will have to find a different way! (Thank Discordia my files are public PDFs!)
Best regards, Basty
How about changing the location to point to the new file? (e.g. by changing window.location)
I've always just made an iframe which points to the file.
<iframe src="/download.exe" frameborder="0" height="0" width="0">Click here to download.</iframe>
Regarding not pointing the current window to the download.
In my experience you can still show your "please donate" page, since downloads (as long as they send the correct headers) don't actually update the browser window.
I do this for csv exports on one of my sites, and as far as the user is concerned it just pops up a safe file window.
So i would recommend a simple meta-redirect as Soldarnal showed.
Just to summarise, you have 2 goals:
start download process
show user a page with a donate options
To achieve this I would do the following:
When your user submits the form, he gets the resulting page with a donate options and a text saying that his download will start in 5 seconds. And in the head section of this page you put the META code as Soldarnal said:
<meta http-equiv="refresh" content="5;url=/download.php?doc=123.zip>
And that's all.
<a href="normaldownload.zip" onclick="use_dhtml_or_ajax_to_display_page()">
Current page is unaffected if download is saved. Just ensure that download doesn't open in the same window (proper MIME type or Content-Disposition) and you'll be able to show anything.
See more complete answer
You can use Javascript/jQuery to initiate the download. Here's an example - you can get rid of the Ajax request and just use the setTimeout() block.
$("btnDownloadCSV").on('click', function() {
$.ajax({
url: "php_backend/get_download_url",
type: 'post',
contentType: "application/x-www-form-urlencoded",
data: {somedata: "somedata"},
success: function(data) {
// If iFrame already exists, remove it.
if($("[id^='iframeTempCSV_"]).length) {
$("[id^='iframeTempCSV_"]).remove();
}
setTimeout(function() {
// If I'm creating an iframe with the same id, it will permit download only the first time.
// So randHashId appended to ID to trick the browser.
var randHashId = Math.random().toString(36).substr(2);
// Create a fresh iFrame for auto-downloading CSV
$('<iframe id="iframeTempCSV_'+randHashId+'" style="display:none;" src="'+data.filepath+'"></iframe>').appendTo('body');
}, 1000);
},
error: function(xhr, textStatus, errorThrown) {
console.error("Error downloading...");
}
});
});