Split big file everytime </byebye> occurs - php

Below code splits my file every 10 lines, but I want it to split everytime
</byebye>
occurs. That way, I will get multiple files each containing;
<byebye>
*stuff here*
</byebye>
Code:
<?php
/**
*
* Split large files into smaller ones
* #param string $source Source file
* #param string $targetpath Target directory for saving files
* #param int $lines Number of lines to split
* #return void
*/
function split_file($source, $targetpath='files/', $lines=10){
$i=0;
$j=1;
$date = date("m-d-y");
$buffer='';
$handle = #fopen ($source, "r");
while (!feof ($handle)) {
$buffer .= #fgets($handle, 4096);
$i++;
if ($i >= $lines) {
$fname = $targetpath.".part_".$date.$j.".xml";
if (!$fhandle = #fopen($fname, 'w')) {
echo "Cannot open file ($fname)";
exit;
}
if (!#fwrite($fhandle, $buffer)) {
echo "Cannot write to file ($fname)";
exit;
}
fclose($fhandle);
$j++;
$buffer='';
$i=0;
$line+=10; // add 10 to $lines after each iteration. Modify this line as required
}
}
fclose ($handle);
}
split_file('testxml.xml')
?>
Any ideas?

If I understand you right, this should do it.
$content = file_get_contents($source);
$parts = explode('</byebye>', $content);
$parts = array_map('trim', $parts);
Then just write the parts to the different files
$dateString = date('m-d-y');
foreach ($parts as $index => $part) {
file_put_contents("{$targetpath}part_{$dateString}{$index}.xml", $part);
}
But I assume (without knowing your source), that this will result in invalid xml. You should use one of the XML-Parser (SimpleXML, DOM, ..) to handle xml-files.
Sidenote: You use # much much too much.

If you are worried about sizes you can switch to a file resource and use fread or fgets to control the amount of memory you are hitting.
$f = fopen($source, "r");
$out = '';
while (!feof($f))
{
$line .= fgets($f);
$arr = explode('</byebye>', $line);
$out .= $arr[0];
if (count($arr) == 1)
continue;
else
{
// file_put_contents here
// will need to handle lines with multiple </byebye> entries here,
// outputting as necessary
// replace $out with the final entry of the $arr array onto
}
}
You can also save more memory by opening up the file for output, and as you parse, pipe the contents to it. When you encounter a entry you would close the file and open the next one.

Related

create multiple directories using loop in php

I am taking data from text file( data is: daa1 daa2 daa3 on separate lines) then trying to make folders with exact name but only daa3 folders is created. Also when i use integer it creates all folders, same is the case with static string i.e "faraz".
$file = __DIR__."/dataFile.txt";
$f = fopen($file, "r");
$line =0;
while ( $line < 5 )
{
$a = fgets($f, 100);
$nl = mb_strtolower($line);
$nl = "checkmeck/".$nl;
$nl = $nl."faraz"; // it works for static value i.e for faraz
//$nl = $nl.$a; // i want this to be the name of folder
if (!file_exists($nl)) {
mkdir($nl, 0777, true);
}
$line++;
}
kindly help
use feof function its much better to get file content also line by line
Check this full code
$file = __DIR__."/dataFile.txt";
$linecount = 0;
$handle = fopen($file, "r");
$mainFolder = "checkmeck";
while(!feof($handle))
{
$line = fgets($handle);
$foldername = $mainFolder."/".trim($line);
//$line is line name daa1,daa2,daa3 etc
if (!file_exists($foldername)) {
mkdir($foldername, 0777, true);
}
$linecount++;
unset($line);
}
fclose($handle);
output folders
1countfaraz
2countfaraz
3countfaraz
Not sure why you're having trouble with your code, but I find it to be more straightforward to use file_get_contents() instead of fopen() and fgets():
$file = __DIR__."/dataFile.txt";
$contents = file_get_contents($file);
$lines = explode("\n", $contents);
foreach ($lines as $line) {
$nl = "checkmeck/". $line;
if (!file_exists($nl)) {
echo 'Creating file '. $nl . PHP_EOL;
mkdir($nl, 0777, true);
echo 'File '. $nl .' has been created'. PHP_EOL;
} else {
echo 'File '. $nl .' already exists'. PHP_EOL;
}
}
The echo statements above are for debugging so that you can see what your code is doing. Once it is working correctly, you can remove them.
So you get the entire file contents, split it (explode()) by the newline character (\n), and then loop through the lines in the file. If what you said is true, and the file looks like:
daa1
daa2
daa3
...then it should create the following folders:
checkmeck/daa1
checkmeck/daa2
checkmeck/daa3

Read and iterate .txt in PHP

I sense that I am almost there.
Here is a .txt file, which is about 60 Kbytes and full of German words. Every word is on a new line.
I want to iterate through it with this code:
<?php
$file = "GermanWords.txt";
$f = fopen($file,"r");
$parts = explode("\n", $f);
foreach ($parts as &$v)
{
echo $v;
}
?>
When I execute this code, I get: Resourceid#2
The word resource is not in the .txt, I do not know where it comes from.
How can I manage to show up all words in the txt?
No need for fopen just use file_get_contents:
$file = "GermanWords.txt";
$contents = file_get_contents($file);
$lines = explode("\n", $contents); // this is your array of words
foreach($lines as $word) {
echo $word;
}
fopen() just opens the file, it doesn't read it -- In your code, $f contains a file handle, not the file contents. This is where the word "Resource" comes from; it's PHP's internal name for the file handle.
One answer would be to replace fopen() with file_get_contents(). This opens and reads the file in one action. This would solve the problem, but if the file is big, you probably don't want to read the whole thing into memory in one go.
So I would suggest instead using SplFileObject(). The code would look like this:
<?php
$file = "GermanWords.txt";
$parts = new SplFileObject($file);
foreach ($parts as $line) {
echo $line;
}
?>
It only reads into memory one line at at time, so you don't have to worry about the size of the file.
Hope that helps.
See the PHP manual for more info: http://php.net/manual/en/splfileobject.construct.php
$f, the result of fopen is a resource, not the contents of the file. If you just want an array of the lines contained in the file, you can use file:
$parts = file('GermanWords.txt');
foreach($parts as $v){
echo $v;
}
Alternatively, if you want to stick with fopen you can use fread to read the content:
$f = fopen('GermanWords.txt', 'r');
// read the entire file into $contents
$contents = fread($f, filesize('GermanWords.txt'));
fclose($handle);
$parts = explode("\n", $contents);
The SplFileObject provides a way to do that :
$file = new SplFileObject("file.txt");
while (!$file->eof()) {
echo $file->fgets();
}
And if you prefer the foreach loop, you can create a generator function for that :
function lines($filename) {
$file = new SplFileObject($filename);
while (!$file->eof()) {
yield $file->fgets();
}
}
foreach (lines('German.txt') as $line) {
echo $line;
}
Reading the entire content of the file (with file_get_contents) before treating it can be memory consuming.
If you want to treat a file line by line, this class might help you.
It implements an Iterator (see phpdoc about it), that can be walked through in a foreach loop. Only the last line read is stored in memory.
class TxtFileIterator implements \Iterator{
protected $fileHandler;
protected $key;
protected $current;
protected $fileName;
function __construct($fileName){
$this->fileHandler = fopen($fileName, "r") or die("Unable to open file!");
$this->fileName = $fileName;
$this->key = 0;
}
function __destruct(){
fclose( $this->fileHandler );
}
//Iterator interface
public function current (){
return $this->current;
}
public function key (){
return $this->key;
}
public function next (){
if ( $this->valid() ){
$this->current = fgets( $this->fileHandler );
$this->key++;
}
}
public function rewind (){
$this->__destruct();
$this->__construct( $this->fileName );
}
public function valid (){
return !feof( $this->fileHandler );
}
Usage :
$iterator = new TxtFileIterator("German.txt");
foreach ($iterator as $line) {
echo $line;// or do whatever you want with line
}

Reading large files from end

Can I read a file in PHP from my end, for example if I want to read last 10-20 lines?
And, as I read, if the size of the file is more than 10mbs I start getting errors.
How can I prevent this error?
For reading a normal file, we use the code :
if ($handle) {
while (($buffer = fgets($handle, 4096)) !== false) {
$i1++;
$content[$i1]=$buffer;
}
if (!feof($handle)) {
echo "Error: unexpected fgets() fail\n";
}
fclose($handle);
}
My file might go over 10mbs, but I just need to read the last few lines. How do I do it?
Thanks
You can use fopen and fseek to navigate in file backwards from end. For example
$fp = #fopen($file, "r");
$pos = -2;
while (fgetc($fp) != "\n") {
fseek($fp, $pos, SEEK_END);
$pos = $pos - 1;
}
$lastline = fgets($fp);
It's not pure PHP, but the common solution is to use the tac command which is the revert of cat and loads the file in reverse. Use exec() or passthru() to run it on the server and then read the results. Example usage:
<?php
$myfile = 'myfile.txt';
$command = "tac $myfile > /tmp/myfilereversed.txt";
exec($command);
$currentRow = 0;
$numRows = 20; // stops after this number of rows
$handle = fopen("/tmp/myfilereversed.txt", "r");
while (!feof($handle) && $currentRow <= $numRows) {
$currentRow++;
$buffer = fgets($handle, 4096);
echo $buffer."<br>";
}
fclose($handle);
?>
It depends how you interpret "can".
If you wonder whether you can do this directly (with PHP function) without reading the all the preceding lines, then the answer is: No, you cannot.
A line ending is an interpretation of the data and you can only know where they are, if you actually read the data.
If it is a really big file, I'd not do that though.
It would be better if you were to scan the file starting from the end, and gradually read blocks from the end to the file.
Update
Here's a PHP-only way to read the last n lines of a file without reading through all of it:
function last_lines($path, $line_count, $block_size = 512){
$lines = array();
// we will always have a fragment of a non-complete line
// keep this in here till we have our next entire line.
$leftover = "";
$fh = fopen($path, 'r');
// go to the end of the file
fseek($fh, 0, SEEK_END);
do{
// need to know whether we can actually go back
// $block_size bytes
$can_read = $block_size;
if(ftell($fh) < $block_size){
$can_read = ftell($fh);
}
// go back as many bytes as we can
// read them to $data and then move the file pointer
// back to where we were.
fseek($fh, -$can_read, SEEK_CUR);
$data = fread($fh, $can_read);
$data .= $leftover;
fseek($fh, -$can_read, SEEK_CUR);
// split lines by \n. Then reverse them,
// now the last line is most likely not a complete
// line which is why we do not directly add it, but
// append it to the data read the next time.
$split_data = array_reverse(explode("\n", $data));
$new_lines = array_slice($split_data, 0, -1);
$lines = array_merge($lines, $new_lines);
$leftover = $split_data[count($split_data) - 1];
}
while(count($lines) < $line_count && ftell($fh) != 0);
if(ftell($fh) == 0){
$lines[] = $leftover;
}
fclose($fh);
// Usually, we will read too many lines, correct that here.
return array_slice($lines, 0, $line_count);
}
Following snippet worked for me.
$file = popen("tac $filename",'r');
while ($line = fgets($file)) {
echo $line;
}
Reference: http://laughingmeme.org/2008/02/28/reading-a-file-backwards-in-php/
If your code is not working and reporting an error you should include the error in your posts!
The reason you are getting an error is because you are trying to store the entire contents of the file in PHP's memory space.
The most effiicent way to solve the problem would be as Greenisha suggests and seek to the end of the file then go back a bit. But Greenisha's mecanism for going back a bit is not very efficient.
Consider instead the method for getting the last few lines from a stream (i.e. where you can't seek):
while (($buffer = fgets($handle, 4096)) !== false) {
$i1++;
$content[$i1]=$buffer;
unset($content[$i1-$lines_to_keep]);
}
So if you know that your max line length is 4096, then you would:
if (4096*lines_to_keep<filesize($input_file)) {
fseek($fp, -4096*$lines_to_keep, SEEK_END);
}
Then apply the loop I described previously.
Since C has some more efficient methods for dealing with byte streams, the fastest solution (on a POSIX/Unix/Linux/BSD) system would be simply:
$last_lines=system("last -" . $lines_to_keep . " filename");
For Linux you can do
$linesToRead = 10;
exec("tail -n{$linesToRead} {$myFileName}" , $content);
You will get an array of lines in $content variable
Pure PHP solution
$f = fopen($myFileName, 'r');
$maxLineLength = 1000; // Real maximum length of your records
$linesToRead = 10;
fseek($f, -$maxLineLength*$linesToRead, SEEK_END); // Moves cursor back from the end of file
$res = array();
while (($buffer = fgets($f, $maxLineLength)) !== false) {
$res[] = $buffer;
}
$content = array_slice($res, -$linesToRead);
If you know about how long the lines are, you can avoid a lot of the black magic and just grab a chunk of the end of the file.
I needed the last 15 lines from a very large log file, and altogether they were about 3000 characters. So I just grab the last 8000 bytes to be safe, then read the file as normal and take what I need from the end.
$fh = fopen($file, "r");
fseek($fh, -8192, SEEK_END);
$lines = array();
while($lines[] = fgets($fh)) {}
This is possibly even more efficient than the highest rated answer, which reads the file character by character, compares each character, and splits based on newline characters.
Here is another solution. It doesn't have line length control in fgets(), you can add it.
/* Read file from end line by line */
$fp = fopen( dirname(__FILE__) . '\\some_file.txt', 'r');
$lines_read = 0;
$lines_to_read = 1000;
fseek($fp, 0, SEEK_END); //goto EOF
$eol_size = 2; // for windows is 2, rest is 1
$eol_char = "\r\n"; // mac=\r, unix=\n
while ($lines_read < $lines_to_read) {
if (ftell($fp)==0) break; //break on BOF (beginning...)
do {
fseek($fp, -1, SEEK_CUR); //seek 1 by 1 char from EOF
$eol = fgetc($fp) . fgetc($fp); //search for EOL (remove 1 fgetc if needed)
fseek($fp, -$eol_size, SEEK_CUR); //go back for EOL
} while ($eol != $eol_char && ftell($fp)>0 ); //check EOL and BOF
$position = ftell($fp); //save current position
if ($position != 0) fseek($fp, $eol_size, SEEK_CUR); //move for EOL
echo fgets($fp); //read LINE or do whatever is needed
fseek($fp, $position, SEEK_SET); //set current position
$lines_read++;
}
fclose($fp);
Well while searching for the same thing, I can across the following and thought it might be useful to others as well so sharing it here:
/* Read file from end line by line */
function tail_custom($filepath, $lines = 1, $adaptive = true) {
// Open file
$f = #fopen($filepath, "rb");
if ($f === false) return false;
// Sets buffer size, according to the number of lines to retrieve.
// This gives a performance boost when reading a few lines from the file.
if (!$adaptive) $buffer = 4096;
else $buffer = ($lines < 2 ? 64 : ($lines < 10 ? 512 : 4096));
// Jump to last character
fseek($f, -1, SEEK_END);
// Read it and adjust line number if necessary
// (Otherwise the result would be wrong if file doesn't end with a blank line)
if (fread($f, 1) != "\n") $lines -= 1;
// Start reading
$output = '';
$chunk = '';
// While we would like more
while (ftell($f) > 0 && $lines >= 0) {
// Figure out how far back we should jump
$seek = min(ftell($f), $buffer);
// Do the jump (backwards, relative to where we are)
fseek($f, -$seek, SEEK_CUR);
// Read a chunk and prepend it to our output
$output = ($chunk = fread($f, $seek)) . $output;
// Jump back to where we started reading
fseek($f, -mb_strlen($chunk, '8bit'), SEEK_CUR);
// Decrease our line counter
$lines -= substr_count($chunk, "\n");
}
// While we have too many lines
// (Because of buffer size we might have read too many)
while ($lines++ < 0) {
// Find first newline and remove all text before that
$output = substr($output, strpos($output, "\n") + 1);
}
// Close file and return
fclose($f);
return trim($output);
}
As Einstein said every thing should be made as simple as possible but no simpler. At this point you are in need of a data structure, a LIFO data structure or simply put a stack.
A more complete example of the "tail" suggestion above is provided here. This seems to be a simple and efficient method -- thank-you. Very large files should not be an issue and a temporary file is not required.
$out = array();
$ret = null;
// capture the last 30 files of the log file into a buffer
exec('tail -30 ' . $weatherLog, $buf, $ret);
if ( $ret == 0 ) {
// process the captured lines one at a time
foreach ($buf as $line) {
$n = sscanf($line, "%s temperature %f", $dt, $t);
if ( $n > 0 ) $temperature = $t;
$n = sscanf($line, "%s humidity %f", $dt, $h);
if ( $n > 0 ) $humidity = $h;
}
printf("<tr><th>Temperature</th><td>%0.1f</td></tr>\n",
$temperature);
printf("<tr><th>Humidity</th><td>%0.1f</td></tr>\n", $humidity);
}
else { # something bad happened }
In the above example, the code reads 30 lines of text output and displays the last temperature and humidity readings in the file (that's why the printf's are outside of the loop, in case you were wondering). The file is filled by an ESP32 which adds to the file every few minutes even when the sensor reports only nan. So thirty lines gets plenty of readings so it should never fail. Each reading includes the date and time so in the final version the output will include the time the reading was taken.

How to delete a line from the file with php?

I have a file named $dir and a string named $line, I know that this string is a complete line of that file but I don't know its line number and I want to remove it from file, what should I do?
Is it possible to use awk?
$contents = file_get_contents($dir);
$contents = str_replace($line, '', $contents);
file_put_contents($dir, $contents);
Read the lines one by one, and write all but the matching line to another file. Then replace the original file.
this will just look over every line and if it not what you want to delete, it gets pushed to an array that will get written back to the file. see this
$DELETE = "the_line_you_want_to_delete";
$data = file("./foo.txt");
$out = array();
foreach($data as $line) {
if(trim($line) != $DELETE) {
$out[] = $line;
}
}
$fp = fopen("./foo.txt", "w+");
flock($fp, LOCK_EX);
foreach($out as $line) {
fwrite($fp, $line);
}
flock($fp, LOCK_UN);
fclose($fp);
It can be solved without the use of awk:
function remove_line($file, $remove) {
$lines = file($file, FILE_IGNORE_NEW_LINES);
foreach($lines as $key => $line) {
if($line === $remove) unset($lines[$key]);
}
$data = implode(PHP_EOL, $lines);
file_put_contents($file, $data);
}
Another approach is to read the file line by line until you find a match, then truncate the file to that point, and then append the rest of the lines.
This is also good if you're looking for a substring (ID) in a line and want to replace the old line with the a new line.
Code:
$contents = file_get_contents($dir);
$new_contents = "";
if (strpos($contents, $id) !== false) { // if file contains ID
$contents_array = explode(PHP_EOL, $contents);
foreach ($contents_array as &$record) { // for each line
if (strpos($record, $id) !== false) { // if we have found the correct line
continue; // we've found the line to delete - so don't add it to the new contents.
} else {
$new_contents .= $record . "\r"; // not the correct line, so we keep it
}
}
file_put_contents($dir, $new_contents); // save the records to the file
echo json_encode("Successfully updated record!");
}
else {
echo json_encode("failed - user ID ". $id ." doesn't exist!");
}
Example:
input: "123,student"
Old file:
ID,occupation
123,student
124,brick layer
Running the code will change file to:
New file:
ID,occupation
124,brick layer
All answeres here have in common, that they load the complete file into the memory. Here is an implementation of removing one (or more) line(s) without coyping the files content into a variable.
The idea is to iterate over the files lines. If a line should be removed, the lines length is added to the $byte_offset. The next line is then moved $byte_offset bytes "upwards". This is done with all following lines. If all lines are processed, the files last $byte_offset bytes are removed.
I guess that this is faster for bigger files because nothing is copied. And I guess that at some file size the other answers do not work at all while this one should. But I didn't test it.
Usage:
$file = fopen("path/to/file", "a+");
// remove lines 1 and 2 and the line containing only "line"
fremove_line($file, 1, 2, "line");
fclose($file);
The code of the fremove_line() function:
/**
* Remove the `$lines` by either their line number (as an int) or their content
* (without trailing new-lines).
*
* Example:
* ```php
* $file = fopen("path/to/file", "a+"); // must be opened writable
* // remove lines 1 and 2 and the line containing only "line"
* fremove_line($file, 1, 2, "line");
* fclose($file);
* ```
*
* #param resource $file The file resource opened by `fopen()`
* #param int|string ...$lines The one-based line number(s) or the full line
* string(s) to remove, if the line does not exist, it is ignored
*
* #return boolean True on success, false on failure
*/
function fremove_line($file, ..$lines): bool{
// set the pointer to the start of the file
if(!rewind($file)){
return false;
}
// get the stat for the full size to truncate the file later on
$stat = fstat($file);
if(!$stat){
return false;
}
$current_line = 1; // change to 0 for zero-based $lines
$byte_offset = 0;
while(($line = fgets($file)) !== false){
// the bytes of the lines ("number of ASCII chars")
$line_bytes = strlen($line);
if($byte_offset > 0){
// move lines upwards
// go back the `$byte_offset`
fseek($file, -1 * ($byte_offset + $line_bytes), SEEK_CUR);
// move the line upwards, until the `$byte_offset` is reached
if(!fwrite($file, $line)){
return false;
}
// set the file pointer to the current line again, `fwrite()` added `$line_bytes`
// already
fseek($file, $byte_offset, SEEK_CUR);
}
// remove trailing line endings for comparing
$line_content = preg_replace("~[\n\r]+$~", "", $line);
if(in_array($current_line, $lines, true) || in_array($line_content, $lines, true)){
// the `$current_line` should be removed so save to skip the number of bytes
$byte_offset += $line_bytes;
}
// keep track of the current line
$current_line++;
}
// remove the end of the file
return ftruncate($file, $stat["size"] - $byte_offset);
}
Convert text to array, remove first line and reconvert to text
$line=explode("\r\n",$text);
unset($line[0]);
$text=implode("\r\n",$line);
I think the best way to work with files is to act them like strings:
/**
* Removes the first found line inside the given file.
*
* #param string $line The line content to be searched.
* #param string $filePath Path of the file to be edited.
* #param bool $removeOnlyFirstMatch Whether to remove only the first match or
* the whole matches.
* #return bool If any matches found (and removed) or not.
*
* #throw \RuntimeException If the file is empty.
* #throw \RuntimeException When the file cannot be updated.
*/
function removeLineFromFile(
string $line,
string $filePath,
bool $removeOnlyFirstMatch = true
): bool {
// You can wrap it inside a try-catch block
$file = new \SplFileObject($filePath, "r");
// Checks whether the file size is not zero
$fileSize = $file->getSize();
if ($fileSize !== 0) {
// Read the whole file
$fileContent = $file->fread($fileSize);
} else {
// File is empty
throw new \RuntimeException("File '$filePath' is empty");
}
// Free file resources
$file = null;
// Divide file content into its lines
$fileLineByLine = explode(PHP_EOL, $fileContent);
$found = false;
foreach ($fileLineByLine as $lineNumber => $thisLine) {
if ($thisLine === $line) {
$found = true;
unset($fileLineByLine[$lineNumber]);
if ($removeOnlyFirstMatch) {
break;
}
}
}
// We don't need to update file either if the line not found
if (!$found) {
return false;
}
// Join lines together
$newFileContent = implode(PHP_EOL, $fileLineByLine);
// Finally, update the file
$file = new \SplFileObject($filePath, "w");
if ($file->fwrite($newFileContent) !== strlen($newFileContent)) {
throw new \RuntimeException("Could not update the file '$filePath'");
}
return true;
}
Here is a brief description of what is being done: Get the whole file content, split the content into its lines (i.e. as an array), find the match(es) and remove them, join all lines together, and save the result back to the file (only if any changes happened).
Let's now use it:
// $dir is your filename, as you mentioned
removeLineFromFile($line, $dir);
Notes:
You can use fopen() family functions instead of SplFileObject, but I do recommend the object form, as it's exception-based, more robust and more efficient (in this case at least).
It's safe to unset() an element of an array being iterated using foreach (There's a comment here showing it can lead unexpected results, but it's totally wrong: As you can see in the example code, $value is copied (i.e. it's not a reference), and removing an array element does not affect it).
$line should not have new line characters like \n, otherwise, you may perform lots of redundant searches.
Don't use
$fileLineByLine[$lineNumber] = "";
// Or even
$fileLineByLine[$lineNumber] = null;
instead of
unset($fileLineByLine[$key]);
The reason is, the first case doesn't remove the line, it just clears the line (and an unwanted empty line will remain).
Hope it helps.
Like this:
file_put_contents($filename, str_replace($line . "\r\n", "", file_get_contents($filename)));

Is this the most efficient way to get and remove first line in file?

I have a script which, each time is called, gets the first line of a file. Each line is known to be exactly of the same length (32 alphanumeric chars) and terminates with "\r\n".
After getting the first line, the script removes it.
This is done in this way:
$contents = file_get_contents($file));
$first_line = substr($contents, 0, 32);
file_put_contents($file, substr($contents, 32 + 2)); //+2 because we remove also the \r\n
Obviously it works, but I was wondering whether there is a smarter (or more efficient) way to do this?
In my simple solution I basically read and rewrite the entire file just to take and remove the first line.
I came up with this idea yesterday:
function read_and_delete_first_line($filename) {
$file = file($filename);
$output = $file[0];
unset($file[0]);
file_put_contents($filename, $file);
return $output;
}
There is no more efficient way to do this other than rewriting the file.
No need to create a second temporary file, nor put the whole file in memory:
if ($handle = fopen("file", "c+")) { // open the file in reading and editing mode
if (flock($handle, LOCK_EX)) { // lock the file, so no one can read or edit this file
while (($line = fgets($handle, 4096)) !== FALSE) {
if (!isset($write_position)) { // move the line to previous position, except the first line
$write_position = 0;
} else {
$read_position = ftell($handle); // get actual line
fseek($handle, $write_position); // move to previous position
fputs($handle, $line); // put actual line in previous position
fseek($handle, $read_position); // return to actual position
$write_position += strlen($line); // set write position to the next loop
}
}
fflush($handle); // write any pending change to file
ftruncate($handle, $write_position); // drop the repeated last line
flock($handle, LOCK_UN); // unlock the file
}
fclose($handle);
}
This will shift the first line of a file, you dont need to load the entire file in memory like you do using the 'file' function. Maybe for small files is a bit more slow than with 'file' (maybe but i bet is not) but is able to manage largest files without problems.
$firstline = false;
if($handle = fopen($logFile,'c+')){
if(!flock($handle,LOCK_EX)){fclose($handle);}
$offset = 0;
$len = filesize($logFile);
while(($line = fgets($handle,4096)) !== false){
if(!$firstline){$firstline = $line;$offset = strlen($firstline);continue;}
$pos = ftell($handle);
fseek($handle,$pos-strlen($line)-$offset);
fputs($handle,$line);
fseek($handle,$pos);
}
fflush($handle);
ftruncate($handle,($len-$offset));
flock($handle,LOCK_UN);
fclose($handle);
}
you can iterate the file , instead of putting them all in memory
$handle = fopen("file", "r");
$first = fgets($handle,2048); #get first line.
$outfile="temp";
$o = fopen($outfile,"w");
while (!feof($handle)) {
$buffer = fgets($handle,2048);
fwrite($o,$buffer);
}
fclose($handle);
fclose($o);
rename($outfile,$file);
I wouldn't usually recommend opening up a shell for this sort of thing, but if you're doing this infrequently on really large files, there's probably something to be said for:
$lines = `wc -l myfile` - 1;
`tail -n $lines myfile > newfile`;
It's simple, and it doesn't involve reading the whole file into memory.
I wouldn't recommend this for small files, or extremely frequent use though. The overhead's too high.
You could store positional info into the file itself. For example, the first 8 bytes of the file could store an integer. This integer is the byte offset of the first real line in the file.
So, you never delete lines anymore. Instead, deleting a line means altering the start position. fseek() to it and then read lines as normal.
The file will grow big eventually. You could periodically clean up the orphaned lines to reduce the file size.
But seriously, just use a database and don't do stuff like this.
Here's one way:
$contents = file($file, FILE_IGNORE_NEW_LINES);
$first_line = array_shift($contents);
file_put_contents($file, implode("\r\n", $contents));
There's countless other ways to do that also, but all the methods would involve separating the first line somehow and saving the rest. You cannot avoid rewriting the whole file. An alternative take:
list($first_line, $contents) = explode("\r\n", file_get_contents($file), 2);
file_put_contents($file, implode("\r\n", $contents));
My problem was large files. I just needed to edit, or remove the first line. This was a solution I used. Didn't require to load the complete file in a variable. Currently echos, but you could always save the contents.
$fh = fopen($local_file, 'rb');
echo "add\tfirst\tline\n"; // add your new first line.
fgets($fh); // moves the file pointer to the next line.
echo stream_get_contents($fh); // flushes the remaining file.
fclose($fh);
I think this is best for any file size
$myfile = fopen("yourfile.txt", "r") or die("Unable to open file!");
$ch=1;
while(!feof($myfile)) {
$dataline= fgets($myfile) . "<br>";
if($ch == 2){
echo str_replace(' ', ' ', $dataline)."\n";
}
$ch = 2;
}
fclose($myfile);
The solutions here didn't work performantly for me.
My solution grabs the last line (not the first line, in my case it was not relevant to get the first or last line) from the file and removes that from that file.
This is very quickly even with very large files (>150000000 lines).
function file_pop($file)
{
if ($fp = #fopen($file, "c+")) {
if (!flock($fp, LOCK_EX)) {
fclose($fp);
}
$pos = -1;
$found = 0;
while ($found < 2) {
if (fseek($fp, $pos--, SEEK_END) < 0) { // can not seek to position
rewind($fp); // rewind to the beginnung of the file
break;
};
if (ord(fgetc($fp)) == 10) { // newline
$found++;
}
}
$lastpos = ftell($fp); // get current position of file
$lastline = fgets($fp); // get current line
ftruncate($fp, $lastpos); // truncate file to last position
flock($fp, LOCK_UN); // unlock
fclose($fp); // close the file
return trim($lastline);
}
}
You could use file() method.
Gets the first line
$content = file('myfile.txt');
echo $content[0];

Categories