Server setup: Apache 2.2.14, PHP 5.3.1
I use a PHP script to serve files of all types as part of an application with complex access permissions. Things have worked out pretty well so far, but then one of our beta users took a picture with a 10-megapixel digital camera and uploaded it. It's somewhere north of 9 MB, 9785570 bytes.
For some reason, in Safari (and thus far ONLY in Safari, I've reproduced this on 5.0.5) the download will sometimes hang partway through and never finish. Safari just keeps on merrily trying to load forever. I can't consistently reproduce the problem - if I reload over and over sometimes the download will complete and sometimes it won't. There's no apparent pattern.
I'm monitoring the server access logs and in the cases where Safari hangs I see a 200 response of the appropriate filesize after I navigate away from the page, or cancel the page load, but not before.
Here's the code that serves the file, including headers. When the download succeeds and I inspect the headers browser-side I see the content type and size have been set correctly. Is it something in my headers? Something in Safari? Both?
header('Content-Type: ' . $fileContentType);
header('Content-Disposition: filename=' . basename($fpath));
header('Content-Transfer-Encoding: binary');
header('Expires: 0');
header('Cache-Control: must-revalidate, post-check=0, pre-check=0');
header('Pragma: public');
header('Content-Length: ' . filesize($fpath));
ob_clean();
flush();
session_write_close();
readfile($fpath);
exit;
FURTHER BULLETINS AS EVENTS WARRANT:
By artificially throttling download speed to 256k/s -- that is, by chunking the file into 256k pieces and pausing between serving them, as
$chunksize = 1 * (256 * 1024); // how many bytes per chunk
if ($size > $chunksize) {
$handle = fopen($fpath, 'rb');
$buffer = '';
while (!feof($handle)) {
$buffer = fread($handle, $chunksize);
echo $buffer;
ob_flush();
flush();
sleep(1);
}
fclose($handle);
} else {
readfile($fpath);
}
I was able to guarantee a successful display of the image file in Safari under arbitrary conditions.
A chunksize of 512k does not guarantee a successful display.
I am almost certain that the problem here is that Safari's image renderer can't handle data coming in any faster, but:
I would like to know for certain
I would also to know if there's some other kind of workaround like a special CSS webkit property or whatever to handle large images because 256k/second is kind of dire for a 10 MB file.
And just to pile on the weird, setting up a finer-grained sleep with usleep() results in problems at a sleep time of 500 ms but not 750 ms.
I did a little digging and found little specific, but I do see a lot of people asserting that Safari has issues with honoring cache control directives. One person asserts:
You don't need all those Cache Controls, just a max-age with Expires set in the past, does everything all those headers your using does [...] many of those Cache Controls headers your using cause problems for Safari [...] Lastly, some browsers don't understand filename, the only understand name, which must be included in the Content-Type header line, never in the Content-Disposition line. [...]
( see last post in thread: http://www.codingforums.com/archive/index.php/t-114251.html OLD info, but you never know... )
So possibly comment out some of your headers and look to see if there is an improvement.
(anecdotal) I also saw some some older post complaining about safari both resuming an interrupted download by appending the whole file to the end of the partial one, and endless downloading which appears to count bytes beyond the file length being sent. (anecdotal)
You might want to try to "chunk" the file while reading it in.
There a numerous posts here on PHP.net that explain ways to do that: http://php.net/manual/en/function.readfile.php
Try:
ob_start();
readfile($path);
$buffer = ob_get_clean();
echo $buffer;
Related
I've been stuck on this problem for a few days, and have yet to find a solution that fixes the problem I'm having.
What I'm Trying To Do:
I'm attempting to use PHP to download PDFs, and the code works very well for files that can download within about a minute and a half. On my home wifi, I'm able to download a 159MB file within 10 seconds, and it works every time. But when I limit the internet speed to "Fast 3G" (around 170KB/s, in order to simulate slower office speeds), the download fails. And nearly every time, it does so exactly 3 minutes and 24 seconds into the download process, but occasionally it is a lower time of 1 minute and 57 seconds.
What I've Tried:
I've tweaked the php.ini file (setting max_execution_time = 0, and memory_limit at higher intervals than the originally configured 128M)
I've tried other download methods that seem to "chunk" the larger PDFs. This has been mostly unsuccessful. In one instance the download would complete, but there would be an error when trying to open the PDF. According to the poster of this solution, it was only a valid solution for UTF-8 encoded files, and I found the one's I'm dealing with to be UTF-16. (I believe it was some kind of incompatibility with the print() function.
I've made sure the file can download if using a direct link in the URL. It has no problems this way, but it was only done for testing, and cannot be a permanent solution because the PDFs I'm dealing with contain sensitive information. So based off of this result, I was at least able to narrow down the problem to be PHP related and not IIS.
Here's the current code I'm using
header('Content-Description: File Transfer');
header('Content-Type: application/octet-stream');
header("Pragma: public");
header("Expires: 0");
header("Cache-Control:must-revalidate, post-check=0, pre-check=0");
header("Content-Type: application/force-download");
header("Content-Type: application/download");
header('Content-Disposition: attachment; filename="'.basename($file).'"');
header("Content-Transfer-Encoding: binary ");
header('Content-Length: ' . filesize($file));
//$file is a full path to the PDF
while(ob_get_level()) {
ob_end_clean();
}
readfile($file);
flush();
exit;
/*I realize it may be off, but it is at least working for quicker load
times as it currently is, so I'm leaving it alone for now*/
I tried to include any information that seemed relevant, but if any additional information would be useful please let me know! I will also be sure to include the current code that is handling the download process that I mentioned at the top of the post.
Instead of
readfile($file);
flush();
I would try
$handle = fopen($file, 'r');
while (!feof($handle)) {
echo fread($handle, 8192);
flush();
}
fclose($handle);
you may need to adjust the above to handle proper encoding, but that will depend on your environment
I have a script that merges 2 documents and then it shows a final document in a browser.
It works in Firefox, Opera and IE. But it does not work in Chrome.
Chrome only shows loading and it stops in 1/4 of loading.
The code:
exec("pdftk A=$pdfin B=$tmpfname cat B1 A output $tmpfoutput");
$data = file_get_contents($tmpfoutput);
header("Content-type: application/pdf");
header("Content-disposition: inline;filename=GeneratedPdf.pdf");
header('Content-Transfer-Encoding: binary');
header('Content-Length: ' . filesize($tmpfoutput));
header('Accept-Ranges: bytes');
echo $data;
I have been told that it also worked in Chrome before so I think there will be only a problem with the last version.
Thanks for any help.
By appending Accept-Ranges: bytes, your script tells the browser that it will accept range requests, i.e. multiple requests that request a part of the response. Your script obviously does not support range requests because it generates and provides the data at once.
To fix the error, remove header('Accept-Ranges: bytes');
If your PDF files are usually large, then a more user-friendly solution is to actually implement range requests in your script. Odds are that your server does already have an efficient routine that handles range requests, so a smart choice is to save PDF file to a publicly accessible directory, then 302-redirect the request to this URL after the PDF has been generated. Make sure that the URLs are unguessable, e.g. by using UUIDs. And remove the PDF files at some point, e.g. using a cronjob.
I'm trying to pass a large file from an external API to the user, (think 100MB or more)
Currently, I'm using a bit of a paranoid script (due to failures from the past) to get the script downloading ASAP.
By 'downloading', I only mean the download trigger on the browser, not the actual downloading of the file. Just the point where user can select where (s)he wants to save the file.
set_time_limit(0);
apache_setenv('no-gzip', 1);
ini_set('zlib.output_compression', 0);
ini_set('output_buffering', 0);
ini_set('implicit_flush', 1);
for($i = 0; $i < ob_get_level(); $i++) { ob_end_flush(); }
ob_implicit_flush(1);
header('Content-Description: File Transfer');
header('Content-type: application/octet-stream');
header('Content-Transfer-Encoding: Binary');
header('Content-Disposition: attachment; filename="' . $filename . '"');
header('Cache-Control: private');
ob_flush();
flush();
$fh = fopen($external_api_url, 'rb');
while(!feof($fh))
{
echo fread($fh, 512);
ob_flush();
flush();
}
fclose($fh);
Using this script, it still takes 20 seconds for a 50mb file before the download popup shows up, and much longer with bigger files.
Is there any way to start the stream faster?
EDIT:
I've also tried fpassthru() and readfile() but these take 40 seconds for the same 50mb file, making me think this way is better. I've also played around with different read sizes (512, 256, 64, couple of others) but I didn't notice a difference)
The reason it is taking so long for your browser to trigger the download dialog, is due to the API taking that long to return its first chunk of data, for example they might be reading the whole file to memory before starting to send the data to you, which would explain why longer files take longer to start even though you always try to write the first 512 bytes as quickly as possible.
I tried to simulate it locally by reading from a local file, but having a sleep(5); statement right before your while loop.
I was able to get Google Chrome to start the download before any data, by omitting the header('Content-type: application/octet-stream'); line, while still issuing a flush(); call before attempting to read the file. (You are already doing this second part)
This however doesn't seem to work with Firefox 3.6, so you might need different gimmicks for different browsers, unless you can predict the first character of a file (Take a look at BOM) and echo that before anything, subsequently removing it from the beginning of your first fread() call.
I hope it helps! But basically the external API is screwing you over.
I'm serving an .mp3 file (as a download dialog box, where the end-user can rename and save the file). There are a lot of conflicting reccomendations for this on the forums and even within the comments on the php manual.
I hope to solve my issue and create here a reference on how to best execute this goal: i.e. a clear "best practices summary" usable by begining php coders on the most efficient way to get started delivering downloadable files.
I would really appreciate it if you'd check over the code here and suggest your corrections--this is part of a site that helps artists self-publish music, books, etc.
To test, I am uploading this to my server as index.php; perhaps that is also part of the problem.
The current status of this script is that the browser hangs a bit and then loads the binary as text into the browser display window.
I've thought at many points that my problem was syntax in the important "Content-Length" header, or the order of my headers, so I've tried several versions of all that, but none cut off the download.
Here is the exact code that I am now trying on my own,
where ####### means 7 numbers (the actual file size in bytes),
and everything else should be clear:
<?php
header('Server: ');
header('X-Powered-By: ');
header('Cache-Control: no-cache');
header('Expires: -1');
header('Content-Description: File Transfer');
header('Content-Transfer-Encoding: binary');
header('Content-Type: audio/mpeg');
header('Content-Disposition: attachment;filename="suggested-save-name.mp3"');
header('Content-Length: #######["exact-file-name.mp3"]');
#readfile("http://full-public-url.com/exact-file-name.mp3");
ob_clean();
flush();
exit;
?>
Returns these response headers:
>X-Powered-By:PHP/5.4.xy
>Vary:negotiate
>Transfer-Encoding:chunked
>TCN:choice
>Server:Apache
>Keep-Alive:timeout=2, max=200
>Date:Fri, 10 May 2013 16:mm:ss GMT
>Content-Type:text/html
>Content-Location:filename.php
>Connection:Keep-Alive
I hope it is a simple error (or more than one) in the syntax of my script, or the way I created and saved the .php file. I have verified the settings are at default, php is up to date, and there are no .htaccess issues. I have carefully made sure there are no extra spaces at the end of this file, as well as all other files in the web directory, and as well I've tried the script with and without the closing ?>.
Thank you in advance
...
Best script after reading Answers, below:
<?php
$file-variable=('./exact-file-name.mp3')
$size=filesize($file-variable);
header('Server: ');
header('X-Powered-By: ');
header('Cache-Control: no-cache');
header('Expires: -1');
header('Content-Type: application/octet-stream'); //will need to redirect for older IE
header('Content-Disposition: attachment; filename="suggested-save-name.mp3"');
header('Content-Length: "$size"');
#readfile("$file-variable");
ob_clean();
flush();
exit;
Successful response headers, from the better script (& after removing the UTF-8 BOM):
>Vary:negotiate
>TCN:choice
>Server:Apache //How can I hide this?
>Keep-Alive:timeout=2, max=200
>Expires:-1
>Date:Sat, 11 May 2013 12:mm:ss GMT
>Content-Type:audio/mpeg
>content-transfer-encoding:binary
>Content-Location:filename.php //I would also like to obfuscate this
>Content-Length:#######
>Content-Disposition:attachment; filename="suggested-save-name.mp3"
>Content-Description:File Transfer
>Connection:Keep-Alive
>Cache-Control:no-cache
Step 1 would be to remove all nonsense headers that are not defined in the HTTP standard.
This will eliminate Content-Description and Content-Transfer-Encoding. Both are useless at best, and might interfere with normal browser operations in the worst case.
Step 2 is to optimize the file delivery. Do not download the MP3 with a HTTP request, access the FILE on the server. Do not use a URL, do use a file path. If the MP3 is right next to your script, this will work:
readfile('./exact-file-name.mp3');
At this point you should usually end up with a working download. If not, try changing the Content-Type to something more generic. audio/mpeg might trigger the audio player in some browser, application/octet-stream should work in most browsers but older Internet Explorer, which do inappropriate content sniffing on certain mime types including this one. application/x-ms-download is supposed to work then.
Make sure the header is sent. PHP does not send HTTP headers if the HTTP body was already startet, and any whitespace including the UTF-8 BOM will trigger body output.
Some comments on your "final" headers in general:
Content-Length: should only have an integer stating the length in bytes, nothing more. Especially no mention of any filename in square brackets.
Content-Transfer-Encoding and Content-Description are still useless.
Content-Location is not needed. If you don't know what it does, omit it. Obfuscation will not work here, the browser needs to know the URL he is accessing. Duplicating this URL in this header does not change anything, obfuscating it will likely break things somewhere.
The two headers you really only need for a download are: Content-Type and (if you want to pre-define a filename for the user) Content-Disposition.
I'm initializing jplayer with the following parameters:
$jplayer.jPlayer('setMedia',{
mp3: data.audioMP3,
oga: data.audioOGA
});
Assume that data.autdioMP3 (and it's OGA counterpart) are paths to a php script, for example: 'http://myserver.local/playaudio.php?songID=99&format=mp3'
Where I am struggling is with playaudio.php. I would like to read the MP3 file and stream it to jplayer without revealing the path to the audio (this is why I am not initializing jplayer with a path to the audio file).
Something like (taken partially from the example for readfile at php docs):
<?php
$if ($validUser && file_exists($file){
header('Content-Transfer-Encoding: binary');
header('Content-Type: audio/mpeg');
header('Expires: 0');
header('Cache-Control: must-revalidate');
header('Content-Length: ' . filesize($file));
ob_clean();
flush();
readfile($file);
exit;
}
?>
I think this just forces the download of the file though... will this still reveal the path to the file to the user? Is there a better option to stream raw mp3 data to the user without revealing the path to the file that you know of?
Thanks!
Since you are using javascript to get the file, others can do the same by reading the headers/file requests from the browser (firebug, Chrome's default tools, etc).
The best way to protect your files from being downloaded is to make a series of checks to get at the data. It's not perfect, but it will deter all but the most determined. For some ideas, check out this page on Grooveshark's API (This is a bit old and some info is missing, so it's the perfect reference), or just visit Grooveshark/Youtube and see how they protect their content.
It is possible to recreate this kind of security with some clever usage of sessions and a good database.
i think you'll struggle to prevent users discovering your media content URL, i'd say it was pretty much impossible (a quick look at the 'Network' tab on Chrome would reveal your loaded track, regardless how you coded playaudio.php).
I'd say you were better off sticking to publishing media you don't mind sharing with the world.. Or just publish
A n second preview
A lesser quality version, e.g. 90bps / Mono
If you really wanted to protect your resources, you could use oAuth or Access Control at the server side.
Thanks for everyone's answers.
I actually ended up solving the problem without revealing the path to the audio file to the user.
Because this is for an online voice messaging application, it was important to protect the path to the messages in order to protect the user's privacy.
The situation ended up being further complicated by the fact that the messages are stored on a remote server... found this out right after I posted this question.
So all in all the process was:
1) Curl the MP3 file from the remote server
2) Set some headers to force the download of the data, which you then echo to stdout
Because the data is being put on standard out and the headers are set to force a download, the effect for jPlayer is the same as if it had requested a file at /some/dir/message.mp3, except that the request goes to /some/dir/playmessage.php - thus the path is never revealed to the user.
EDIT** - Assume that I have access control, validation running before the snippit below, otherwise exposing the path to the script below is no different than just exposing the path to the mp3. (see comments between me and Lloyd)
Here's the code that ended up getting the job done:
$ch = curl_init($remoteFile);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
$data = curl_exec($ch);
curl_close($ch);
if ($data === false) {
echo 'CURL Failed';
exit;
}
//Get file size
if (preg_match('/Content-Length: (\d+)/', $data, $matches)) {
$contentLength = (int)$matches[1];
}
//force user to download file contents
header('Content-Transfer-Encoding: binary');
header('Content-Type: audio/mpeg');
header('Expires: 0');
header('Cache-Control: must-revalidate');
header('Content-Length: ' . $contentLength);
ob_clean();
flush();
echo $data;
exit;
?>
Assume that $remoteFile is a path to the mp3 file that I want to play on the server where we are storing messages.
I found this resource on reading a remote file using PHP especially helpful. Here's some more information on modifying http headers.
NOTE I am essentially downloading the file twice, which slows the process down for the user. There is probably a better way to do this. :) If you kept the file locally, what I had initially in my question actually works (using readfile() to output the raw mp3 data).