Sending File in Chunk from Server in PHP - php

I am developing an embedded device which has a simple miro-controller with limited memory. This device will request a file from a server by sending a HTTP (or HTTPS) GET method request to the server. There will be a PHP script which in the server responsible to send the file. Now the PHP script will only send the file continuously to the embedded device. However as the embedded device is not fast enough and do not have enough memory to store the whole file before processing it. I want the PHP script to only sending a chunk of the file in each HTTP GET request. I think it is good that the size of the chunk is determined by the variable in the GET request. And in each chunk it will add a header describing the size, the sequence number, and CRC check of that chunk.
I am a newbie on PHP script. Could you help to guild me to write the PHP script? An example would be really appreciated.
Thank you very much.

I think that your script PHP could read the file and take the chunk you want:
$filename = "YOURFILE.txt";
$chunk_length = 1024; // 1024 chars will be sent
$sequence_number = $_GET['sequence'];
if ($sequence_number>0){
$position = $sequence_number * $chunk_length;
}
else {
$position = 0;
}
$content = file_get_contents($filename);
$data = substr($content, $position, $chunk_length);
header('size:'.strlen($data));
header('sequence_number:'.sequence_number);
header('CRC:'.crc32($data));
echo $data;

Related

PHP read chunk files anomaly

I have a Server part of a Webservice which send to me for parameters: offset and maxLen (see file_get_contents()), an encoded content from a file. From client side i try to get this content and save into another file. My client side code is like that:
for ($i = 1; $i <= $totalFileSize + 1;$i+=50000) {
file_put_contents('ex.zip', $server->download($i, 50000), FILE_APPEND);
}
On the service side i just use file_get_contents to give that part of file which is requested. The problem is that dimension of two files are not the same and also md5 is different.
I know what is the size of server file.
Can you tell me, please, what is it wrong?

Keep track of download progress in PHP

All,
I have built a form, if the users fills in the form, my code can determine what (large) files need to be downloaded by that user. I display each file with a download button, and I want to show the status of the download of that file next to the button (download in progress, cancelled/aborted or completed). Because the files are large (200 MB+), I use a well-known function to read the file to be downloaded in chuncks.
My idea was to log the reading of each (100 kbyte) chunk in a database, so I can keep track of the progress of the download.
My code is below:
function readfile_chunked($filename, $retbytes = TRUE)
{
// Read the personID from the cookie
$PersonID=$_COOKIE['PERSONID'];
// Setup connection to database for logging download progress per file
include "config.php";
$SqlConnectionInfo= array ( "UID"=>$SqlServerUser,"PWD"=>$SqlServerPass,"Database"=>$SqlServerDatabase);
$SqlConnection= sqlsrv_connect($SqlServer,$SqlConnectionInfo);
ChunckSize=100*1024; // 100 kbyte buffer
$buffer = '';
$cnt =0;
$handle = fopen($filename, 'rb');
if ($handle === false)
{
return false;
}
$BytesProcessed=0;
while (!feof($handle))
{
$buffer = fread($handle, $ChunckSize);
$BytesSent = $BytesSent + $ChunckSize;
// update database
$Query= "UPDATE [SoftwareDownload].[dbo].[DownloadProgress] SET LastTimeStamp = '" . time() . "' WHERE PersonID='$PersonID' AND FileName='$filename'";
$QueryResult = sqlsrv_query($SqlConnection, $Query);
echo $buffer;
ob_flush();
flush();
if ($retbytes)
{
$cnt += strlen($buffer);
}
}
$status = fclose($handle);
if ($retbytes && $status)
{
return $cnt; // return num. bytes delivered like readfile() does.
}
return $status;
}
The weird thing is that my download runs smoothly at a constant rate, but my database gets updated in spikes: sometimes I see a couple of dozen of database updates (so a couple of dozens of 100 kilobyte blocks) within a second, the next moment I see no database updates (so no 100 k blocks being fed by PHP) for up to a minute (this timeout depends on the bandwidth available for the download). The fact that no progress is written for a minute makes my code believe that the download was aborted.
In the meantime I see the memory usage of IIS increasing, so I get the idea that IIS buffers the chuncks of data that PHP delivers.
My webserver is running Windows 2008 R2 datacenter (64 bits), IIS 7.5 and PHP 5.3.8 in FastCGI mode. Is there anything I can do (in either my code, in my PHP config or in IIS config) to prevent IIS from caching the data that PHP delivers, or is there any way to see in PHP if the data generated was actually delivered to the downloading client?
Thanks in advance!
As no usefull answers came up, I've created a real dodgy workaround by using Apache strictly for the download of the file(s). The whole form, CSS etc is still delivered through IIS, just the downloads links are server by Apache (a reverse proxy server sits in between, so users don't have to fiddle with port numbers, the proxy takes care of this).
Apache doesn't cache PHP output, so the timestamps I get in my database are reliable.
I've asked a similar question once, and got very good results with the answers there.
Basically, you use a APC cache to cache a variable, then retrieve it in a different page with an Ajax call from the client.
Another possibility is to do the same with a database. Write the progress to a database and have the client to test for that (let's say every second) using an Ajax call.

How to get the size of a binary data posted to a php?

My 3rd party app creates an image and store it in a variable and then post it to a php on the server. for security matters, I need to limit the upload size. Now that the binary data is not written to a file yet, how would I go around getting its size?
Here is the code:
<?php
$_data = $_POST['data']; //get the binary data from the client side.
if($_data !=""){
$handle = fopen($_path."/".$_ref.".png", "w");
fwrite($handle,$_data);
}
?>
Use
$length = strlen($_POST['data']);
use $_FILES['filename']['size'] to get the uploaded filesize

Cannot resume downloads bigger than 300M

I am working on a program with php to download files.
the script request is like: http://localhost/download.php?file=abc.zip
I use some script mentioned in Resumable downloads when using PHP to send the file?
it definitely works for files under 300M, either multithread or single-thread download, but, when i try to download a file >300M, I get a problem in single-thread downloading, I downloaded only about 250M data, then it seems like the http connection is broken. it doesnot break in the break-point ..Why?
debugging the script, I pinpointed where it broke:
$max_bf_size = 10240;
$pf = fopen("$file_path", "rb");
fseek($pf, $offset);
while(1)
{
$rd_length = $length < $max_bf_size? $length:$max_bf_size;
$data = fread($pf, $rd_length);
print $data;
$length = $length - $rd_length;
if( $length <= 0 )
{
//__break-point__
break;
}
}
this seems like every requested document can only get 250M data buffer to echo or print..But it works when i use a multi-thread to download a file
fread() will read up to the number of bytes you ask for, so you are doing some unnecessary work calculating the number of bytes to read. I don't know what you mean by single-thread and multi-thread downloading. Do you know about readfile() to just dump an entire file? I assume you need to read a portion of the file starting at $offset up to $length bytes, correct?
I'd also check my web server (Apache?) configuration and ISP limits if applicable; your maximum response size or time may be throttled.
Try this:
define(MAX_BUF_SIZE, 10240);
$pf = fopen($file_path, 'rb');
fseek($pf, $offset);
while (!feof($pf)) {
$data = fread($pf, MAX_BUF_SIZE);
if ($data === false)
break;
print $data;
}
fclose($pf);

Server-side, get progress on sending file

Basically, what I want to do is to check how much of a file my webserver has sent to a client, when the client is downloading one. Is this even possible?
Does apache provide any module/extension that would help me accomplish my task?
I use a linux distro, apache2 and php5. Regards.
Browser provides this functionality if file has correct "Content-length" header set. Why do you want to implement this in your page?
Solved it.
I simply open the file with PHP that I want to send to the client.
$fh = fopen($filePath, 'r');
Then I calculate 60% of the filesize by writing
$fileSize = filesize($filePath);
$sizeFirst = floor(($fileSize / 100) * 60);
Now the $sizeFirst variable contains the length of the first 60% of the file, in a numeric value.
To calculate the rest 40% I use:
$sizeLast = $fileSize - $sizeFirst;
Now I can write out the first 60%, do my action, and then write outh the rest 40%.
$dataFirst = fread($fh, $sizeFirst);
echo($dataDirst);
// Do my action here.
$dataSecond = fread($fh, $sizeSecond);
echo($dataSecond);
exit();
I need to set the header(); before writing out this, the Content-length, Content-type and Content-Disposition must be set in order to send a valid header and filecontent to the client.
Hope it helps someone.

Categories