I am using a variation of the familiar readfile_chunked in attempt of download for larger files:
function readfile_chunked($filename)
{
$chunk_size = 1*(1024*1024); // how many bytes per chunk
$buffer = '';
$handle = fopen($filename, 'rb');
if ($handle === false)
{
return false;
}
while (!feof($handle))
{
$buffer = fread($handle, $chunk_size);
print $buffer;
ob_flush();
flush();
sleep(1);
}
$status = fclose($handle);
return $status;
}
smaller files work fine, but this larger file is missing the last 2830 bytes.
i found out the issue to this. under the php.ini file, make sure you set implicit_flushing to On. i still have the explicit flush code after each line outputted however.
Related
I want to serve huge files from a folder above the public_html.
Currently I do:
<?php
// Authenticate
if ($_GET['key'] !== "MY-API-KEY") {
header('HTTP/1.0 403 Forbidden');
echo "You are not authorized.";
return;
}
define('CHUNK_SIZE', 1024*1024);
$PATH_ROOT_AUTOPILOT_ACTIVITY_STREAMS = "../../../data/csv/";
// Read a file and display its content chunk by chunk
function readfile_chunked($filename, $retbytes = TRUE) {
$buffer = '';
$cnt = 0;
$handle = fopen($filename, 'rb');
if ($handle === false) {
return false;
}
while (!feof($handle)) {
$buffer = fread($handle, CHUNK_SIZE);
echo $buffer;
ob_flush();
flush();
if ($retbytes) {
$cnt += strlen($buffer);
}
}
$status = fclose($handle);
if ($retbytes && $status) {
return $cnt; // return num. bytes delivered like readfile() does.
}
return $status;
}
// Get the file parameter
$file = basename(urldecode($_GET['file']));
$fileDir = $PATH_ROOT_AUTOPILOT_ACTIVITY_STREAMS;
$filePath = $fileDir . $file;
if (file_exists($filePath))
{
// Get the file's mime type to send the correct content type header
$finfo = finfo_open(FILEINFO_MIME_TYPE);
$mime_type = finfo_file($finfo, $filePath);
// Send the headers
header("Content-Disposition: attachment; filename=$file.csv;");
header("Content-Type: $mime_type");
header('Content-Length: ' . filesize($filePath));
// Stream the file
readfile_chunked($filePath);
exit;
}
?>
This currently fails for some reason I don't understand. curl outputs:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 457M 0 457M 0 0 1228k 0 --:--:-- 0:06:21 --:--:-- 1762k
curl: (92) HTTP/2 stream 1 was not closed cleanly: INTERNAL_ERROR (err 2)
Is there a better way to serve big files programatically, via PHP?
Currently about 2/3 of the file is served. The response is not complete because it crashes. There are no logs.
I'm currently having the problem that my download always stops at almost exactly one gigabyte. I have created a PHP download script which checks the user permissions before downloading and then prevents copying the URL. For this I use fread() to pass the data through PHP. However, I have no idea why the download always stops exactly after one gigabyte. The file is about twice as large.
Could someone take a look at this?
// ... Permission check, get download url, etc. ...
try
{
define('CHUNK_SIZE', 8 * 1024);
function readfile_chunked($filename, $retbytes = true)
{
$buffer = '';
$cnt = 0;
$handle = fopen($filename, 'rb');
if ($handle === false)
{
return false;
}
while (!feof($handle))
{
$buffer = fread($handle, CHUNK_SIZE);
echo $buffer;
if(ob_get_level() > 0)
{
ob_flush();
flush();
}
if ($retbytes)
{
$cnt += strlen($buffer);
}
}
$status = fclose($handle);
if ($retbytes && $status)
{
return $cnt;
}
return $status;
}
$mimetype = 'mime/type';
header('Content-Type: application/octet-stream');
header('Content-Disposition: attachment; filename=' . $download->url);
readfile_chunked($download->url);
}
catch(Exception $e)
{
echo "Error while reading the file";
exit;
}
Update:
I have now tested the script on another server. Here the download sometimes terminated after 300 MB and sometimes after 1.3 GB. I then defined the following settings:
ini_set('display_errors', 0);
ini_set('error_reporting', E_ALL);
ini_set("log_errors", 1);
ini_set("error_log", "error.log");
ini_set('memory_limit', '-1');
ini_set('max_execution_time', 7200);
On the other server, the script has now run through several times without any problems. The download was complete and the file was error-free. However, I copied these settings to the original system and the download still stops. I downloaded the file three times in a row. The file size varies slightly:
1.057.983 KB
1.056.192 KB
1.056.776 KB
I'm running out of ideas. What else could that be? They are both Apache web servers.
There are also no noticeable entries in the log.
I am creating a very large table in a single string $output with the data from a MySQL database. In end of the php script, I send two headers:
header("Content-type: application/msexcel");
header("Content-Disposition: attachment; filename=action.xls");
The problem is when the large table has about more than 55000 rows. When this happens, the file sent is 0 bytes and the user opens a empty file. I am sending the file through this after the headers:
echo $output;
When the table has not too many rows, the file sent work. How to send a file in this way that the size of the string $output don't matter?
header("Content-type: application/msexcel");
header("Content-Disposition: attachment; filename=action.xls");
Move these lines to the top the page or Script before html table it will starts working
The solution found in another question worked for this. The file is too large to send, so the solution is send part by part to the user. Instead of use readfile() function, use the readfile_chuncked() customized function:
define('CHUNK_SIZE', 1024*1024); // Size (in bytes) of tiles chunk
// Read a file and display its content chunk by chunk
function readfile_chunked($filename, $retbytes = TRUE) {
$buffer = '';
$cnt = 0;
$handle = fopen($filename, 'rb');
if ($handle === false) {
return false;
}
while (!feof($handle)) {
$buffer = fread($handle, CHUNK_SIZE);
echo $buffer;
ob_flush();
flush();
if ($retbytes) {
$cnt += strlen($buffer);
}
}
$status = fclose($handle);
if ($retbytes && $status) {
return $cnt; // return num. bytes delivered like readfile() does.
}
return $status;
}
I have a script processing large text files.
I am limited however by the size of the files.
I've done some searching on this forum, and i've come to the conclusion that i must process the file line by line, however this brings up quite a bit of issues for me, as i need some detailed info from the file, before i can start processing it.
I've tried adding each line to a variable as below:
$content = "";
$handle = fopen($target_File, "r") or die("Couldn't get handle");
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 256);
// Process buffer here..
$content .= $buffer;
}
fclose($handle);
}
And just as expected, it did not work.
Could anyone help me out ?
add this on top of your file
ini_set("memory_limit", -1);
Be careful, the page can use all the RAM on the server
$content = "";
$handle = fopen($target_File, "r") or die("Couldn't get handle");
if ($handle) {
while (($buffer = fgets($handle, 4096))!== false) {
// Process buffer here..
$content .= $buffer;
}
fclose($handle);
}
OR
$content = "";
$target_File ="/tmp/uploadfile.txt";
$handle = fopen($target_File, "r") or die("Couldn't get handle");
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
// Process buffer here..
//$content .= $buffer;
}
fclose($handle);
}
NOTE:
If the problem is caused by hitting the memory limit, you can try setting it a higher value (this could work or not depending on php's configuration).
this sets the memory limit to 32 Mb
ini_set("memory_limit","32M");
I tried to stream an audio file using this PHP code, but always get cut around 3/4 of the file size, especially for iPad and Android agents.
ob_clean();
flush();
set_time_limit(0);
$size = intval(sprintf("%u", filesize($filename)));
$chunksize = 0.5 * (1024 * 1024);
if ($size > $chunksize) {
$handle = fopen($filename, 'rb');
$buffer = '';
while (!feof($handle)) {
$buffer = fread($handle, $chunksize);
echo $buffer;
ob_flush();
flush();
}
fclose($handle);
} else {
readfile($filename);
}
Is there a better way to do this? I have tried completely with readfile() only, the result is even worse. Thanks in advance.