Serve a large file via Zend Framework - php

In our application, authentication is handled via set of Controller Plugins that validate the user etc.
I want to serve a large (video) file only to authenticated users- the obvious way to do this is via readfile() in the controller, but I'm finding it hits the PHP memory limit - presumably the output from the controller is buffered somewhere.
How can I turn off buffering just for this one controller?
EDIT: Thanks for all the useful tips about flushing any existing output buffering - I guess I was specifically looking for a way of doing this within the framework though?

Interesting problem... You could try:
// ...
public function largeFileAction()
{
// this clears all active output buffers
while (ob_get_level()) {
ob_end_clean();
}
readfile('path/to/large/file');
exit(); // to prevent further request handling
}
// ...

Ok, I might be totally wrong here, but I think to have read somewhere OB has to be enabled for ZendLayout and placeholder helpers to work, so you'd have to disable them for the downloadAction (you probably aint gonna need them for serving the file anyway).
Would something like this achieve what you want to do?
class DownloadController
{
public function downloadAction()
{
$this->_helper->layout()->disableLayout();
$this->_helper->viewRenderer->setNoRender(true);
// authenticate user if not done elsewhere already
header( /* ... the usual stuff ... */);
filepassthru(/* some path outside webroot */);
exit;
}
}

As Tyson wrote, your best choice (if you have full control over the server) is to validate users credentials and redirect him (302 temporary redirect) to the URL where he can download the file.
To prevent reuse of this URLs we are using Lighttpd and its mod_secdownload that allows you to generate a hash that is valid for the specified amount of time.
nginx has X-Accel-Redirect and Apache has mod_xsendfile.
If you decide to implement a separate lightweight web server there are other benefits as well (mainly lower memory consumption while serving static files and faster response times).
If you decide to go this route you will either have to add another IP address to the server and bind Apache only to the one IP address, and the other server (lighty of nginx) to the other because they are web servers the both listen on port 80. And changing the port for one of the servers is not a good idea because a lot of people do not have access to higher ports.
If adding another IP address is not an option you can install nginx on port 80 and use it as a reverse proxy to pass the dynamic requests to Apache which can listen on another port and serve all of the static files.

Considering using an external script to output the file, and stream it to the browser using PHP's passthru function.
If on a Linux-based system, you could try something like passthru("cat video_file.flv");
However, a better practice is to avoid this streaming (from within PHP) altogether and issue the client a 301 HTTP redirection to the URL of the actual static resource so that the webserver can handle streaming it directly.

I don't think you can actually. As far as i know php buffers all output before sending it to the requester.
you could increase the memory limit using ini_set()

$handle = fopen('/path/to/file', 'r');
$chunk_size = 8192;
while ($chunk = fread($handle, $chunk_size)) {
echo $chunk;
ob_flush();
}
This will probably need some tweaking, such as adding correct headers and reading in binary mode if necessary, but the basic idea is sound. I have used this method successfully to send 50+ MB files, with a 16 MB PHP memory limit.

Related

Is it possible to start executing PHP script on a multipart/form-data file upload request before file is uploaded?

It should be be a common use-case, but I can't find whether it's achievable at all.
I want to validate a multipart/form-data uploaded file extension on server-side - must I wait for the file to fully upload?
I might be missing something, but it doesn't make sense, especially when handling large files.
Can't I execute PHP before file is uploaded, get metadata and maybe cancel the request altogether?
I know I can split to two separate requests, I'm looking for a single-request solution, if applicable
You should wait until the file is fully uploaded so you can then validate. There is no one single-request solution.
If you use an Apache/Nginx HTTP server, it executes PHP scripts only after it finished loading the whole request from the client - which is too late for your use case, as Sergio in the other answer correctly points out.
There is a single-request solution in PHP, but you need to have control over the HTTP requests in your PHP script.
You can chose to not use Apache, but instead start a HTTP server from your php-cli (either by using the native socket functions or some HTTP server package such as react/socket that uses them in the background).
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server('127.0.0.1:8080', $loop);
$socket->on('connection', function (React\Socket\ConnectionInterface $connection) {
// here you can have $connection->on(...) event handlers
});
$loop->run();
Then you can have handlers that handle each chunk of the incoming request (example from the react/socket package, specifically ReadableResourceStream):
$connection->on('data', function ($chunk) {
echo $chunk;
});
And instead of echoing the chunk, you can validate its contents and call $connection->close() if you need, which effectively terminates the unfinished upload.
But this whole thing is a complex solution, and I'd recommend to use it only for a upload service that is completely separated from the application that generates the form page (which can still run under a regular Apache HTTP server because it's just much easier).
You can validate it before interact with the server in the frontend, in PHP the script executes when request is finished

performance of passthru("cat file")

I'm using passthru("cat filepath") in my download script. My concern is that it might use a lot of server resource.
What is the difference between directly link a file in a public directory and download a file using passthru("cat filepath") in php?
What is the difference between directly link a file in a public directory and download a file using passthru("cat filepath") in php?
The difference is that linking directly to a file does not invoke PHP, while running a PHP script which in turn runs cat causes, well, both PHP and cat to be invoked. This will take up a moderate amount of extra memory, but won't cause server load under most circumstances.
I was using readfile(), but this function can't be used for files larger than 2gb
You might want to find a better solution than passing all of the file contents through PHP, in that case. Look into X-Sendfile support in your web server software of choice.
Don't use passthru() for that, you're opening yourself to CLI Injection and performance is terrible. readfile() exists just for that.
readfile($filepath);
There is a small overhead when passing through PHP compared to a direct link but we are usually talking of milliseconds. However, the browser will not be able to request a 206 Partial when using readfile() unless you code support for it or use something like PEAR::HTTP_Download.
EDIT: Seems you are using passthru() because apparently readfile() doesn't handle >2GB files properly (I never had that problem with readfile(), in fact I just tested it with a 7.2 GB file and it worked fine). In which case, at least escape your parameters.
function readfile_ext($filepath) {
if(!file_exists($filepath))
return false;
passthru('cat ' . escapeshellarg($filepath));
return true;
}
Instead of passthru('cat filepath'), use the PHP native readfile('filepath'), which has better performance.
Both methods will be slower than simply directly linking to the file though, since PHP has a certain overhead.

Is it possible to disconnect an http client from PHP?

In PHP is it possible (and is it safe) to close the http connection without returning any http status code? My server is apache.
Will this be logged to the access log or error log?
I use the following code:
/**
* Make the script run in the background
* return a message to the browser
* #param unknown_type $response
*/
function freeUserBrowser($response)
{
// let's free the user, but continue running the
// script in the background
ignore_user_abort(true);
header("Connection: close");
header("Content-Length: " . mb_strlen($response));
echo $response;
flush();
}
I don't think this is possible: Ultimately it's Apache who closes the connection, and returns a status code if PHP doesn't emit one.
We had a question some time ago about whether http connections can be forcibly cut off from within PHP. IIRC, the consensus was that except for shooting down the web server thread responsible for the current request, this was not possible. Looking for the question now... Update: Can't find it right now, sorry.
I don't think so, short of killing the apache worker itself, which would certainly not be a good idea.
It may be possible if using PHP as an Apache module. There may be some Apache internal function available to modules that you could use for this, but I don't know enough of Apache internals to tell for sure.
In nutshell Apache calling the shots, and you cannot change Apache behaviour from outside, for security reasons.

php most memory efficient way to return files

so i have a bunch of files, some can be up to 30-40mb
and i want to use php to handle security of the files, so i can control who has access to them
that means i have a script sort of like this rough example
$has_permission = check_database_for_permission($user, filename);
if ($has_permission) {
header('Content-Type: image/jpeg');
readfile ($filename);
exit;
} else {
// return 401 error
}
i would hate for every request to load the full file into memory, as it would soon chew up all the memory on my server with a few simultaneous requests
so a couple of questions
is readfile the most memory efficient way of doing this?
is there some better method of achieving the same outcome, that i am overlooking?
server: apache/php5
thanks
readfile is the correct way to do this. By all means don't try to read the file yourself and print it to output--that will consume excessive memory. With the readfile function the contents of the file are buffered directly to output, taking up a trivial amount of transitory memory.
The fastest way is when you can relay this to the webserver. The webserver can use the sendfile() call to ask the operating system kernel to directly copy from a file to the network stream.
for instance when using lighttpd there is a way that PHP can signal the server to take over and do the sendfile trick:
http://redmine.lighttpd.net/projects/lighttpd/wiki/X-LIGHTTPD-send-file

php simultaneous file downloads from the same browser and same php script

<?php
$filename= './get/me/me_'.rand(1,100).'.zip';
header("Content-Length: " . filesize($filename));
header('Content-Type: application/zip');
header('Content-Disposition: attachment; filename=foo.zip');
readfile($filename);
?>
Hi,
I have this simple code that forces a random file download, my problem is that if I call the script two or more times from the same browser the second download won't start until the first is completed or interrupted. Thus I can download only one file per time.
Do you have any clue?
This may be related to PHP's session handling.
Using the default session handler, when a PHP script opens a session it locks it. Subsequent scripts that need to access it have to wait until the first script is finished with it and unlocks it (which happens automatically at shutdown, or by session_write_close() ). This will manifest as the script not doing anything till the previous one finishes in exactly the same way you describe.
Clearly you aren't starting the session explicitly, but there's a config flag that causes the session to start automatically: session.auto_start - http://www.php.net/manual/en/session.configuration.php
Either use phpinfo() to determine if this is set to true, or look in your config. You could also try adding session_write_close() to the top of the script, see if it makes the issue go away.
just guesses. There could be different reasons.
first, your server could restrict the number of connections or childs in paralell. But I guess this sin't the problem
second, it is more likely that the client restricts the number of connections. The "normal" browser opens only two connections at a time to a certain server. Modern browsers allow up to 8 (?) connections. This is a simple restriction in order to avoid problems which could occur with slow servers.
One workaround could be to place every download on a "virtual" subdomain.
give it a try!
Just to say that the session_write_close(); solved the problem for me.
I was using session_destroy(); (that worked) but was not much good if I needed to keep session data :)
All you need to do I place session_write_close(); just before you start streaming the file data.
Example:
<?php
$filename= './get/me/me_'.rand(1,100).'.zip';
session_write_close();
header("Content-Length: " . filesize($filename));
header('Content-Type: application/zip');
header('Content-Disposition: attachment; filename=foo.zip');
readfile($filename);
?>
I'd further investigate Ralf's suggestion about the server restrictions and start with checking the logfiles to ensure that the second request is received by the server at all. Having that knowledge, you can eliminate one of the possibilities and at least see which side the problem resides on.
From the client's browser - you didn't mention which one is it - if Firefox, try to install the Live Http Headers extension to see what happens to request you send and if browser receives any response from the server side.
As far as I can find, there is no php configuration setting that restricts max downloads or anything like that - besides, such a configuration is outside the scope of php.
Therefore, I can come to only two conclusions:
The first is that this is browser behaviour, see if the problem is repeated across multiple browsers (let me know if it is). The HTTP spec does say that only two connections to the same domain should be active at any one time, but I wasn't aware that affected file downloads as well as page downloads. A way of getting round such a limitation is to allocate a number of sub-domains to the same site (or do a catch-all subdomains DNS entry), and when generating a link to the download, select a random sub domain to download from. This should work around the multiple request issue if it is a browser problem.
A second and much more unlikely option is that (and this only applys if you are using Apache), your MaxKeepAliveRequests configuration option is set to something ridiculously low and KeepAlives are enabled. However, I highly doubt that is the issue, so I suggest investigating the browser possibility.
Are you getting an error message from the browser when the second download is initiated, or does it just hang? If it just hangs, this suggests it is a browser issue.

Categories