I have got a large file in PHP of which I would like to replace the first 512 bytes with some other 512 bytes. Is there any PHP function that helps me with that?
If you want to optionally create a file and read and write to it (without truncating it), you need to open the file with the fopen() function in 'c+' mode:
$handle = fopen($filename, 'c+');
PHP then has the stream_get_contents() function which allows to read a chunk of bytes with a specific length (and from a specific offset in the file) into a string variable:
$buffer = stream_get_contents($handle, $length = 512, $offset = 0);
However, there is no stream_put_contents() function to write the string buffer back to the stream at a specific position/offset. A related function is file_put_contents() but it does not allow to write to a file-handle resource at a specific offset. But there is fseek() and fwrite() to do that:
$bytes_written = false;
if (0 === fseek($handle, $offset)) {
$bytes_written = fwrite($handle, $buffer, $length);
}
Here is the full picture:
$handle = fopen($filename, 'c+');
$buffer = stream_get_contents($handle, $length = 512, $offset = 0);
// ... change $buffer ...
$bytes_written = false;
if (0 === fseek($handle, $offset)) {
$bytes_written = fwrite($handle, $buffer, $length);
}
fclose($handle);
If the length of $buffer is not fixed this will not properly work. In that case it's better to work with two files and to use stream_copy_to_stream() as outlined in How to update csv column names with database table header or if the file is not large it is also possible to do that in memory:
$buffer = file_get_contents($filename);
// ... change $buffer ...
file_put_contents($filename, $buffer);
Related
I'm trying to encode/decode a file that is several MBs or sometimes GBs in base64 encoding however some pieces of data gets encoded/decoded in a wrong way which results in strange characters like: � �̴.
I'm reading the file chunk by chunk encoding and saving each individually (Probably that's the problem however i cannot figure it out).
Here is what i have tried so far:
<?php
function encode_file($Ifilename, $Efilename){
$handle = fopen($Ifilename, 'rb');
$outHandle = fopen($Efilename, 'wb');
$bufferSize = 8151;
while(!feof($handle)){
$buffer = fread($handle, $bufferSize);
$ebuffer = base64_encode($buffer);
fwrite($outHandle, $ebuffer);
}
fclose($handle);
fclose($outHandle);
}
function decode_file($Ifilename, $Efilename){
$handle = fopen($Ifilename, 'rb');
$outHandle = fopen($Efilename, 'wb');
$bufferSize = 8151;
while(!feof($handle)){
$buffer = fread($handle, $bufferSize);
$dbuffer = base64_decode($buffer);
fwrite($outHandle, $dbuffer);
}
fclose($handle);
fclose($outHandle);
}
encode_file('input.txt', 'out.bin');//Big text file ~4MBs
decode_file('out.bin', 'out.txt');
After reading the whole Wikipedia article on base64, I found that every 3 characters encodes to 4 base64 characters, this is what was causing the file corruption.
The fix is to simply set the buffer to n when encoding, where n is a multiple of 3.
When decoding set the buffer to N, where N is a multiple of 4.
The working code:
<?php
function encode_file($Ifilename, $Efilename){
$handle = fopen($Ifilename, 'rb');
$outHandle = fopen($Efilename, 'wb');
$bufferSize = 3 * 256;// 3 bytes of ASCII encodes to 4 bytes of base64
while(!feof($handle)){
$buffer = fread($handle, $bufferSize);
$ebuffer = base64_encode($buffer);
fwrite($outHandle, $ebuffer);
}
fclose($handle);
fclose($outHandle);
}
function decode_file($Ifilename, $Efilename){
$handle = fopen($Ifilename, 'rb');
$outHandle = fopen($Efilename, 'wb');
$bufferSize = 4 * 256; // 4 bytes of base64 decodes to 3 bytes of ASCII
while(!feof($handle)){
$buffer = fread($handle, $bufferSize);
$dbuffer = base64_decode($buffer);
fwrite($outHandle, $dbuffer);
}
fclose($handle);
fclose($outHandle);
}
encode_file('input.txt', 'out.bin');
decode_file('out.bin', 'output.txt');
If I have three get parameters:
$filename = $_GET['filename'];
$start = $_GET['start'];
$size = $_GET['size'];
And I am reading a chunk of the file like so:
$handle = fopen($basepath . $filename, "rb");
fseek($handle, $start);
$contents = fread($handle, $size);
echo md5($contents);
How may I read large portions of a file (anywhere from 1mb to 1gb) and create a hash or checksum of its contents without needing to allocate enough memory for the entire read?
At the moment if I try to hash a too large of part of the file I get a memory error since php can not allocated enough memory (roughly 400mb).
Is there a hashing function in which I can digest parts of the file at a time rather than the entire contents at once (for example starting at $start read 100kb blocks and feed it to the function until $size is met)? And how would I read the file in chunks so that I would start at $start and read $size bytes?
If there is not such a hashing or checksum function that supports feeding chunks of the data pieces at a time, would file_get_contents() fix the issue of allocating memory for large reads? I am not entirely sure how that function works.
Thanks.
http://php.net/manual/en/function.hash-update.php
<?php
define('CHUNK', 65536);
//$file = 'alargefile.img';
//$start = 256 * 1024 * 1024;
//$size = 512 * 1024 * 1024;
$fp = fopen($file, "r");
fseek($fp, $start);
$ctx = hash_init('md5');
while ($size > 0) {
$buffer = fread($fp, min($size, CHUNK));
hash_update($ctx, $buffer);
$size -= CHUNK;
}
$hash = hash_final($ctx);
fclose($fp);
print $hash;
?>
I am trying to delete last line from my text file in PHP using ftruncate, but the problem is the length if line will keep varying most of the times. And I dont want to use file_put_contents, so that I write the file again because the file could be in Megabytes.
Please any suggestions ?
To get you an idea what I meant something like this:
$fp = fopen('file.txt', 'r+');
$pos = filesize('file.txt');
while ($pos > 0) {
$pos = max($pos - 1024, 0);
fseek($fp, $pos);
$tmp = fread($fp, 1024);
$tmppos = strrpos($tmp, "\n");
if ($tmppos !== false) {
ftruncate($fp, $pos + $tmppos);
break;
}
}
It will read the last 1024 bytes. Find the last newline in that buffer if it exists. If it exists truncate to that position. If is doesn't exists it reads the next 1024 bytes and check them. And so on.
In PHP if you write to a file it will write end of that existing file.
How do we prepend a file to write in the beginning of that file?
I have tried rewind($handle) function but seems overwriting if current content is larger than existing.
Any Ideas?
$prepend = 'prepend me please';
$file = '/path/to/file';
$fileContents = file_get_contents($file);
file_put_contents($file, $prepend . $fileContents);
The file_get_contents solution is inefficient for large files. This solution may take longer, depending on the amount of data that needs to be prepended (more is actually better), but it won't eat up memory.
<?php
$cache_new = "Prepend this"; // this gets prepended
$file = "file.dat"; // the file to which $cache_new gets prepended
$handle = fopen($file, "r+");
$len = strlen($cache_new);
$final_len = filesize($file) + $len;
$cache_old = fread($handle, $len);
rewind($handle);
$i = 1;
while (ftell($handle) < $final_len) {
fwrite($handle, $cache_new);
$cache_new = $cache_old;
$cache_old = fread($handle, $len);
fseek($handle, $i * $len);
$i++;
}
?>
$filename = "log.txt";
$file_to_read = #fopen($filename, "r");
$old_text = #fread($file_to_read, 1024); // max 1024
#fclose(file_to_read);
$file_to_write = fopen($filename, "w");
fwrite($file_to_write, "new text".$old_text);
Another (rough) suggestion:
$tempFile = tempnam('/tmp/dir');
$fhandle = fopen($tempFile, 'w');
fwrite($fhandle, 'string to prepend');
$oldFhandle = fopen('/path/to/file', 'r');
while (($buffer = fread($oldFhandle, 10000)) !== false) {
fwrite($fhandle, $buffer);
}
fclose($fhandle);
fclose($oldFhandle);
rename($tempFile, '/path/to/file');
This has the drawback of using a temporary file, but is otherwise pretty efficient.
When using fopen() you can set the mode to set the pointer (ie. the begginng or end.
$afile = fopen("file.txt", "r+");
'r' Open for reading only; place
the file pointer at the beginning of
the file.
'r+' Open for reading and
writing; place the file pointer at the
beginning of the file.
$file = fopen('filepath.txt', 'r+') or die('Error');
$txt = "/n".$string;
fwrite($file, $txt);
fclose($file);
This will add a blank line in the text file, so next time you write to it you replace the blank line. with a blank line and your string.
This is the only and best trick.
I am using a variation of the familiar readfile_chunked in attempt of download for larger files:
function readfile_chunked($filename)
{
$chunk_size = 1*(1024*1024); // how many bytes per chunk
$buffer = '';
$handle = fopen($filename, 'rb');
if ($handle === false)
{
return false;
}
while (!feof($handle))
{
$buffer = fread($handle, $chunk_size);
print $buffer;
ob_flush();
flush();
sleep(1);
}
$status = fclose($handle);
return $status;
}
smaller files work fine, but this larger file is missing the last 2830 bytes.
i found out the issue to this. under the php.ini file, make sure you set implicit_flushing to On. i still have the explicit flush code after each line outputted however.