PHP read chunk files anomaly - php

I have a Server part of a Webservice which send to me for parameters: offset and maxLen (see file_get_contents()), an encoded content from a file. From client side i try to get this content and save into another file. My client side code is like that:
for ($i = 1; $i <= $totalFileSize + 1;$i+=50000) {
file_put_contents('ex.zip', $server->download($i, 50000), FILE_APPEND);
}
On the service side i just use file_get_contents to give that part of file which is requested. The problem is that dimension of two files are not the same and also md5 is different.
I know what is the size of server file.
Can you tell me, please, what is it wrong?

Related

How to receive an uploaded file which is not sent as multipart/form? PHP

I got the docs from the third party that is sending me a file over http protocol and I need to write a script that will successfully receive the sent file. Content-type is set as application/gzip so I can't pick up the file uploaded using a $_FILES variable as it would be easy with multipart/form-data.
This link gave me a hint: http://php.net/manual/en/features.file-upload.post-method.php
Note:
Be sure your file upload form has attribute enctype="multipart/form-data" otherwise the file upload will not work.
I tried to reproduce their "client" side to test my server using the example in the following url http://blog.derakkilgo.com/2009/06/07/send-a-file-via-post-with-curl-and-php/
And to ensure crosdomain posting is available, I used a function posted and explained by #slashingweapon CORS with php headers
There must be a way to do it - Halp!
Hi what I understand is you just need to download the file that has been uploaded to a server, try getting the file in binary mode using the below code:
ftp_fget($conn_id, $file, FTP_BINARY, 0);
Ref: http://php.net/manual/en/function.ftp-get.php
The connection details must be shared with you by the 3rd party who is sharing the file.
Thanks to #fusion3k who pointed me to the right direction.
$rawData = file_get_contents("php://input") gave the the RAW data from which I had to parse the file from. It was painful to extract it because it was a binary file. I used
Explode the data with "\n" as a delimiter $rawData = explode("\n", $rawData)
Skip first few lines (it was 3 lines of header data for me) and take the rest into $valuableData
Convert the last line of data from string to binary using $convertedLine = unpack("H*", $valuableData[count($valuableData)-1])
Cut out the last byte of data from the last row $convertedLine[1] = substr($convertedLine[1], 0, -2) ( have no idea why is `unpack returning an array)
$valuableData[count($valuableData)-1] = pack("H*", $convertedLine[1])
implode("\n", $valuableData) and write that to a file

Sending binary file content using JSON

I've written a REST interface for my ownCloud app. I've method getFileFromRemote($path) which should return a JSON object with the file content.
Unfortunately this only works when the file that I've specified in $path is a plaintext file. When I try to call the method for an image or PDF the status code is 200 but the response is empty. For returning the file contents I use file_get_contents for retrieving the content.
Note: I know ownCloud has a WebDAV interface, but I want to solve this with REST only.
EDIT
This is the code server side (ownCloud):
public function synchroniseDown($path)
{
$this->_syncService->download(array($path));//get latest version
$content = file_get_contents($this->_homeFolder.urldecode($path));
return new DataResponse(['path'=>$path, 'fileContent'=>$content]);
}
The first line retrieves downloades the content on the ownCloud server and works completely.
You probably have to base64_encode your file content to make json_encode/decode handle it properly:
return new DataResponse([
'path'=>$path,
'fileContent' => base64_encode($content) // convert binary data to alphanum
]);
and then when receiving file via second side, you will have to always:
$fileContent = base64_decode($response['fileContent']);
It's only one, but ones of easiest way to handle that. Btw, sooner or later you will find that Mime-Type would be useful in response.

Sending File in Chunk from Server in PHP

I am developing an embedded device which has a simple miro-controller with limited memory. This device will request a file from a server by sending a HTTP (or HTTPS) GET method request to the server. There will be a PHP script which in the server responsible to send the file. Now the PHP script will only send the file continuously to the embedded device. However as the embedded device is not fast enough and do not have enough memory to store the whole file before processing it. I want the PHP script to only sending a chunk of the file in each HTTP GET request. I think it is good that the size of the chunk is determined by the variable in the GET request. And in each chunk it will add a header describing the size, the sequence number, and CRC check of that chunk.
I am a newbie on PHP script. Could you help to guild me to write the PHP script? An example would be really appreciated.
Thank you very much.
I think that your script PHP could read the file and take the chunk you want:
$filename = "YOURFILE.txt";
$chunk_length = 1024; // 1024 chars will be sent
$sequence_number = $_GET['sequence'];
if ($sequence_number>0){
$position = $sequence_number * $chunk_length;
}
else {
$position = 0;
}
$content = file_get_contents($filename);
$data = substr($content, $position, $chunk_length);
header('size:'.strlen($data));
header('sequence_number:'.sequence_number);
header('CRC:'.crc32($data));
echo $data;

PHP: fseek() for large file (>2GB)

I have a very large file (about 20GB), how can I use fseek() to jump around and read its content.
The code looks like this:
function read_bytes($f, $offset, $length) {
fseek($f, $offset);
return fread($f, $length);
}
The result is only correct if $offset < 2147483647.
Update: I am running on windows 64,
phpinfo - Architecture: x64,
PHP_INT_MAX: 2147483647
WARNING: as noted in comments, fseek uses INT internally and it simply cant work
with such large files on 32bit PHP compilations. Following solution
wont work. It is left here just for reference.
a little bit of searching led me to comments on PHP manual page for fseek:
http://php.net/manual/en/function.fseek.php
problem is maximum int size for offset parameter but seems that you can work around it by doing multiple fseek calls with SEEK_CUR option and mix it with one of big numbers processing library.
example:
function fseek64(&$fh, $offset)
{
fseek($fh, 0, SEEK_SET);
$t_offset = '' . PHP_INT_MAX;
while (gmp_cmp($offset, $t_offset) == 1)
{
$offset = gmp_sub($offset, $t_offset);
fseek($fh, gmp_intval($t_offset), SEEK_CUR);
}
return fseek($fh, gmp_intval($offset), SEEK_CUR);
}
fseek64($f, '23456781232');
for my project, i needed to READ blocks of 10KB from a BIG offset in a BIG file (>3 GB). Writes were always append, so no offsets needed.
this will work, irrespective of which PHP version and OS you are using.
Pre-requisite = your server should support Range-retrieval queries. Apache & IIS already support this, as do 99% of other webservers (shared hosting or otherwise)
// offset, 3GB+
$start=floatval(3355902253);
// bytes to read, 100 KB
$len=floatval(100*1024);
// set up the http byte range headers
$opts = array('http'=>array('method'=>'GET','header'=>"Range: bytes=$start-".($start+$len-1)));
$context = stream_context_create($opts);
// bytes ranges header
print_r($opts);
// change the URL below to the URL of your file. DO NOT change it to a file path.
// you MUST use a http:// URL for your file for a http request to work
// this will output the results
echo $result = file_get_contents('http://127.0.0.1/dir/mydbfile.dat', false, $context);
// status of your request
// if this is empty, means http request didnt fire.
print_r($http_response_header);
// Check your file URL and verify by going directly to your file URL from a web
// browser. If http response shows errors i.e. code > 400 check you are sending the
// correct Range headers bytes. For eg - if you give a start Range which exceeds the
// current file size, it will give 406.
// NOTE - The current file size is also returned back in the http response header
// Content-Range: bytes 355902253-355903252/355904253, the last number is the file size
...
...
...
SECURITY - you must add a .htaccess rule which denies all requests for this database file except those coming from local ip 127.0.0.1.

Writing a filename to a OutputStream when uploading to php server so the file will be named properly

this should be real quick. Currently I have this code to write an image to an output stream and upload it to a server:
OutputStream os = connection.getOutputStream();
BufferedInputStream fis = new BufferedInputStream(new FileInputStream(file));
int totalBytes = fis.available();
for(int i = 0; i < totalBytes; i++) {
os.write(fis.read());
}
Etc. (I know that for loop for writing the data is super basic, I'm gonna make it a bit better in the future I'm just trying to get things working at a basic level here).
Anyways, as you can see I'm writing a file to the stream and sending it out, this works just fine. My problem is, I have no way as of yet to transmit the proper file name as well. On the php end I just set it to a generic timestamp file name, and I don't want to use the default image name on the phone either, the image uploaded will include the user's name who uploaded, including a timestamp which I can add on the php end just fine. I just need to know how to include something like a namevaluepair which has the user's name in it, so on the php side the filename can include that string. Thanks!
edit: dont know why i was saying data, its just an output stream...
The file name is usually sent in the headers. For POST (which is usually what you are using when you obtain an output stream), you should have a content-type of multipart/form-data and each part will have a content-disposition header. For a part corresponding to the file (the only part, in your case), the content-disposition header should have a filename="foo.txt" component. The server then saves the contents in a temp file and hands your script the info contained in the content-disposition header (along with a means of getting to the uploaded file itself).
See RFC 1867 for details.

Categories