I need to base64 encode big file with PHP.
file() and file_get_contents() are not options since them loads whole file into memory.
I got idea to use this:
$handle = #fopen("/tmp/inputfile.txt", "r");
if ($handle) {
while (($buffer = fgets($handle, 4096)) !== false) {
echo $buffer;
}
fclose($handle);
}
SOURCE: Read and parse contents of very large file
This working well for reading, but is it possible to do it like this:
Read line -> base64 encode -> write back to file
And then repeat for each line in file.
Would be nice if it could do it directly, without need to write to temporary file.
Base64 encodes 3 bytes of raw data into 4 bytes on 7-bit safe text. If you feed it less than 3 bytes padding will occur, and you can't have that happen in the middle of the string. However, so long as you're dealing in multiples of 3 you're golden, sooo:
$base_unit = 4096;
$handle = #fopen("/tmp/inputfile.txt", "r");
if ($handle) {
while (($buffer = fread($handle, $base_unit*3)) !== false) {
echo base64_encode($buffer);
}
fclose($handle);
}
http://en.wikipedia.org/wiki/Base64#Examples
Related
I would like to read a file until X bytes. But the last line should be NOT cut off like in my current code:
$file = fopen("test.txt", "r");
while(! feof($file)) {
$contents = fread($file,10000);
Right now, fread reads until 10000 bytes are reached. Then cuts the line off and creates a new file. The line basically is completely stores but is split into two files. I dont want do only stop and the end of a line.
Any solutions? Thanks!
I think I got it:
$file = fopen("test.txt", "r");
while(! feof($file)) {
$contents = fread($file,$eachFileSize);
$contents = $contents . fgets($file);
Can someone confirm this is the intended way (e.g. LawrenceCherone suggested)?
I have got a large file in PHP of which I would like to replace the first 512 bytes with some other 512 bytes. Is there any PHP function that helps me with that?
If you want to optionally create a file and read and write to it (without truncating it), you need to open the file with the fopen() function in 'c+' mode:
$handle = fopen($filename, 'c+');
PHP then has the stream_get_contents() function which allows to read a chunk of bytes with a specific length (and from a specific offset in the file) into a string variable:
$buffer = stream_get_contents($handle, $length = 512, $offset = 0);
However, there is no stream_put_contents() function to write the string buffer back to the stream at a specific position/offset. A related function is file_put_contents() but it does not allow to write to a file-handle resource at a specific offset. But there is fseek() and fwrite() to do that:
$bytes_written = false;
if (0 === fseek($handle, $offset)) {
$bytes_written = fwrite($handle, $buffer, $length);
}
Here is the full picture:
$handle = fopen($filename, 'c+');
$buffer = stream_get_contents($handle, $length = 512, $offset = 0);
// ... change $buffer ...
$bytes_written = false;
if (0 === fseek($handle, $offset)) {
$bytes_written = fwrite($handle, $buffer, $length);
}
fclose($handle);
If the length of $buffer is not fixed this will not properly work. In that case it's better to work with two files and to use stream_copy_to_stream() as outlined in How to update csv column names with database table header or if the file is not large it is also possible to do that in memory:
$buffer = file_get_contents($filename);
// ... change $buffer ...
file_put_contents($filename, $buffer);
Im trying to extract the values between eng_tid
and eng_data for http://fdguirhgeruih.x10.mx/html.txt and I keep getting T string errors.
why do I keep getting errors
<? php
//First, open the file. Change your filename
$file = "http://fdguirhgeruih.x10.mx/html.txt";
$word1='tid';
$word2='data';
$handle = fopen($file, "r");
$contents = fread($handle, filesize($file));
fclose($handle);
$between=substr($contents, strpos($contents, $word1), strpos($contents, $word2) - strpos($contents, $word1));
echo $between;
?>
UPDATE after seeing error messages...
Instead of fread() and attempting to use the size in bytes of your target file, you may simply use file_get_contents() to retrieve the remote file. Your error is because PHP wants to read the filesize of the file as though it is local, but it is a remote file over HTTP. filesize() reports 0 and an error. Instead do
// Don't do this...
//$handle = fopen($file, "r");
//$contents = fread($handle, filesize($file));
//fclose($handle);
// Instead do this...
$contents = file_get_contents($file);
I need to scan through a 30MB text file - it's a list of world cities - How can I access this file, I feel like a File_Get_Contents will give my server a stroke
Just fopen it and then use fgets.
Filesystem functions come handy in this situation.
Example
$filename = "your_file_path";
// to open file
$fp = fopen($filename, 'r'); // use 'rw' to open file in read/write mode
// to output entire file
echo fread($fp, filesize($filename));
// to close file
fclose($fp);
References
(some handy functions)
All Filesystem Functions
fopen() - open file
fread() - read file content
fgets() - to get line
fwrite() - write content to file
fseek() - change file pointer's position
rewind() - rewind file pointer to pos 0
fclose() - close file
...
<?php
$fh = #fopen("inputfile.txt", "r");
if ($fh) {
while (($line = fgets($fh)) !== false) {
echo $line;
// do something with $line..
}
fclose($fh);
}
?>
More information/examples on http://pt.php.net/manual/en/function.fgets.php
This question already has answers here:
Reading very large files in PHP
(8 answers)
Closed 8 years ago.
I have few text files with size more than 30MB.
How can i read such giant text files from PHP?
Unless you need to work with all the data at the same moment, you can read them in pieces. Example for binary files:
<?php
$handle = fopen("/foo/bar/somefile", "rb");
$contents = '';
while (!feof($handle)) {
$block = fread($handle, 8192);
do_something_with_block($block);
}
fclose($handle);
?>
The above example might break multibyte encodings (in case there's a multibyte character across the 8192-byte boundary - e.g. วพ in UTF-8), so for files that have meaningful endlines (e.g. text), try this:
<?php
$handle = fopen("/foo/bar/somefile", "rb");
$contents = '';
while (!feof($handle)) {
$line = fgets($handle);
do_something_with_line($line);
}
fclose($handle);
?>
You can open the file using fopen, read the lines using fgets.
$fh = fopen("file", "r"); // open file to read.
while (!feof($fh)) { // loop till lines are left in the input file.
$buffer = fgets($fh); // read input file line by line.
.....
}
}
fclose($fh);