PHP get multiple specific line using fseek - php

I'm trying to get using only fopen() and fseek() to get specific lines of code (not only one lines, i need to get line above and below of current seek line).
To improve performance, I know how to get specific line to seek and then exit. If I need line 5 then should be get seekable into 4 and 6.
Here is a code to get bytes of each lines then put into array as lines as key and value as bytes to EOF.
$fh = fopen($source, 'r');
$meta = stream_get_meta_data($fh);
if (!$meta['seekable']) {
throw new Exception(sprintf("A source is not seekable: %s", print_r($source, true)));
}
$line = fgets($fh, 4096);
$pos = -1;
$i = 0;
$result = null;
$linenum = 10;
var_dump('Line num:'.$linenum);
$total_lines = null;
// Get seek byte end of each line
while (!feof($fh)) {
$char = fgetc($fh);
if ($char != "\n" && $char != "\r") {
$total_lines[$i] = $pos;
$pos++;
} else {
$i++;
}
//var_dump(fgets($fh).' _ '.$pos);
}
// Now get specific lines (line 5, line 6 and line 7)
$seekssearch = array($total_lines[5], $total_lines[6], $total_lines[7]);
$result = null;
$posr = 0;
foreach ($seekssearch as $sk) {
while (!feof($fh)) {
if ($char != "\n" && $char != "\r") {
fseek($fh, $sk, SEEK_SET);
$posr++;
} else {
$ir++;
}
}
// Merge result of line 5,6 and 7
$result .= fgets($fh);
}
echo $result;
exit;
while (!feof($fh) && $i<($linenum)) {
$char = fgetc($fh);
if ($char != "\n" && $char != "\r") {
fseek($fh, $pos, SEEK_SET);
$pos++;
}
else {
$i++;
}
}
$line = trim(fgets($fh));
var_dump($line);
exit;
exit;
while (!feof($fh) && $i<($linenum-1)) {
$char = fgetc($fh);
if ($char != "\n" && $char != "\r") {
//fseek($fh, $pos);
fseek($fh, $pos);
$pos++;
}
else {
if ($pos == 3) {
$line = fgets($fh);
}
$i++;
}
}
//$line = fgets($fh);
var_dump($line); exit;
How to merge this lines?
Note: I don't want using splFileInfo or any tricks like arrays. Just want to seek then exit.

I've created a function that read a file and count lines and store into arrays each lines bytes to seek. If maximum specified by linenum is set, it will break from while to keep performance than in a new loop function to seek a position in bytes to get a content of file.
I believe that can this function improve.
function readFileSeek($source, $linenum = 0, $range = 0)
{
$fh = fopen($source, 'r');
$meta = stream_get_meta_data($fh);
if (!$meta['seekable']) {
throw new Exception(sprintf("A source is not seekable: %s", print_r($source, true)));
}
$pos = 2;
$result = null;
if ($linenum) {
$minline = $linenum - $range - 1;
$maxline = $minline+$range+$range;
}
$totalLines = 0;
while (!feof($fh)) {
$char = fgetc($fh);
if ($char == "\n" || $char == "\r") {
++$totalLines;
} else {
$result[$totalLines] = $pos;
}
$pos++;
if ($maxline+1 == $totalLines) {
// break from while to not read entire file
break;
}
}
$buffer = '';
for ($nr=$minline; $nr<=$maxline; $nr++) {
if (isset($result[$nr])) {
fseek($fh, $result[$nr], SEEK_SET);
while (!feof($fh)) {
$char = fgetc($fh);
if ($char == "\n" || $char == "\r") {
$buffer .= $char;
break;
} else {
$buffer .= $char;
}
}
}
}
return $buffer;
}
Test results (1.3 GB file, 100000000 lines of codes, seek to 300000 line a code):
string(55) "299998_abc
299999_abc
300000_abc
300001_abc
300002_abc
"
Time: 612 ms, Memory: 20.00Mb
$ ll -h /tmp/testReadSourceLines_27151460344/41340913936
-rw-rw-r-- 1 1,3G /tmp/testReadSourceLines_27151460344/41340913936

Related

How can I most efficiently read the first n lines of a large text file in php?

So far I'm able to return the entire file, or an empty string. But not the first n lines of a file as expected.
echo head('/path/to/my_file.txt',100); // returns '', or in other versions `1\n1\n...`
function head($filepath, $lines = 1, $adaptive = true) {
// Open file
$f = #fopen($filepath, "rb");
if ($f === false) return false;
// Sets buffer size
if (!$adaptive) $buffer = 4096;
else $buffer = ($lines < 2 ? 64 : ($lines < 10 ? 512 : 4096));
// Start reading
$output = '';
while($line = fgets($f,$buffer) !== false) {
if (feof($f)) break;
$output .= $line . "\n";
}
// Close file and return
fclose($f);
return trim($output);
}
In contrast, the following tail function (source) works perfectly:
function tail($filepath, $lines = 1, $adaptive = true) {
// Open file
$f = #fopen($filepath, "rb");
if ($f === false) return false;
// Sets buffer size
if (!$adaptive) $buffer = 4096;
else $buffer = ($lines < 2 ? 64 : ($lines < 10 ? 512 : 4096));
// Jump to last character
fseek($f, -1, SEEK_END);
// Read it and adjust line number if necessary
// (Otherwise the result would be wrong if file doesn't end with a blank line)
if (fread($f, 1) != "\n") $lines -= 1;
// Start reading
$output = '';
$chunk = '';
// While we would like more
while (ftell($f) > 0 && $lines >= 0) {
// Figure out how far back we should jump
$seek = min(ftell($f), $buffer);
// Do the jump (backwards, relative to where we are)
fseek($f, -$seek, SEEK_CUR);
// Read a chunk and prepend it to our output
$output = ($chunk = fread($f, $seek)) . $output;
// Jump back to where we started reading
fseek($f, -mb_strlen($chunk, '8bit'), SEEK_CUR);
// Decrease our line counter
$lines -= substr_count($chunk, "\n");
}
// While we have too many lines
// (Because of buffer size we might have read too many)
while ($lines++ < 0) {
// Find first newline and remove all text before that
$output = substr($output, strpos($output, "\n") + 1);
}
// Close file and return
fclose($f);
return trim($output);
}
// here the correction => apply parenthesis before ...
while(($line = fgets($f)) !== false) {
if (feof($f)) break;
if ($lines-- == 0) break;
$output .= $line . "\n";
}
You just need to keep a count of the number of lines read, and abort after $lines is reached.
$i = 0;
while($line = fgets($file) !== false) {
if (feof($file)) break;
$output .= $line . "\n";
if ($i++ >= $lines) {
break;
}
}

Get Last row value and N row value before last row

How to read and echo the N rows before the last row in TEXT file using PHP?
Example in file.txt
test1
test2
test3
test4
test5
test6
test7
I want to get the last row value and 3 row before last row.
So the result will be :
test4
test7
Here is my code so far (just show last row)
$line = '';
$f = fopen('\\\\192.168.183.28\\wk$\\sbin\\file.txt', 'r');
$cursor = -1;
fseek($f, $cursor, SEEK_END);
$char = fgetc($f);
while ($char === "\n" || $char === "\r") {
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
}
while ($char !== false && $char !== "\n" && $char !== "\r")
{
$line = $char . $line;
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
}
$future_sn = substr($line, 28, 36);
Any advice?
try
$array = explode("\n", file_get_contents('file.txt'));
// counting array and get key for prev
$prev_third = count($array)-4;
echo end($array); // test7
echo $array[$prev_third]; //test4
You can use fgets() to read file and keep a counter to keep track of last row. You can get 3 row ahead from last by mod(%) the counter.
$handle = fopen("input.txt", "r");
$lastRow = "";
$last3Ahead = "";
$counter = 0;
$a = "";
$b = "";
$c = "";
if ($handle) {
while (($line = fgets($handle)) !== false) {
if($counter == 0) $a = $line;
if($counter == 1) $b = $line;
if($counter == 2) $c = $line;
$lastRow = $line ;
if($counter >= 3) {
$last3Ahead = $a;
$a = $b;
$b = $c;
$c = $line
}
$counter++;
} else {
// error opening the file.
}
fclose($handle);
echo $lastRow. "<br>" ; //last row
echo $last3Ahead . "<br>"; //3 row before last;

What function should I use to read the 5th line of a large file?

What php function should I use to count the 5th line of a large file as the 5th line is the bandwidth?
Example of data:
103.239.234.105 -- [2007-04-01 00:42:21] "GET articles/learn_PHP_basics HTTP/1.0" 200 12729 "Mozilla/4.0"
If you want to read every 5th line, you could use an SplFileObject to make life a little easier (than the fopen/fgets/fclose family of functions).
$f = new SplFileObject('myreallybigfile.txt');
// Read ahead so that if the last line in the file is a 5th line, we echo it.
$f->setFlags(SplFileObject::READ_AHEAD);
// Loop over every 5th line starting at line 5 (offset 4).
for ($f->rewind(), $f->seek(4); $f->valid(); $f->seek($f->key()+5)) {
echo $f->current();
}
Open the file into a handle:
$handle = fopen("someFilePath", "r");
Then read the first 5 lines, and only save the fifth:
$i = 0;
$fifthLine = null;
while (($buffer = fgets($handle, 4096)) !== false) {
if $i++ >= 5) {
$fifthLine = $buffer;
break;
}
}
fclose($handle);
if ($i < 5); // uh oh, there weren't 5 lines in the file!
//$fifthLine should contain the 5th line in the file
Note that this is streamed so it doesn't load the whole file.
http://tekkie.flashbit.net/php/tail-functionality-in-php
<?php
// full path to text file
define("TEXT_FILE", "/home/www/default-error.log");
// number of lines to read from the end of file
define("LINES_COUNT", 10);
function read_file($file, $lines) {
//global $fsize;
$handle = fopen($file, "r");
$linecounter = $lines;
$pos = -2;
$beginning = false;
$text = array();
while ($linecounter > 0) {
$t = " ";
while ($t != "\n") {
if(fseek($handle, $pos, SEEK_END) == -1) {
$beginning = true;
break;
}
$t = fgetc($handle);
$pos --;
}
$linecounter --;
if ($beginning) {
rewind($handle);
}
$text[$lines-$linecounter-1] = fgets($handle);
if ($beginning) break;
}
fclose ($handle);
return array_reverse($text);
}
$fsize = round(filesize(TEXT_FILE)/1024/1024,2);
echo "<strong>".TEXT_FILE."</strong>\n\n";
echo "File size is {$fsize} megabytes\n\n";
echo "Last ".LINES_COUNT." lines of the file:\n\n";
$lines = read_file(TEXT_FILE, LINES_COUNT);
foreach ($lines as $line) {
echo $line;
}

Read last line from file

I've been bumping into a problem. I have a log on a Linux box in which is written the output from several running processes. This file can get really big sometimes and I need to read the last line from that file.
The problem is this action will be called via an AJAX request pretty often and when the file size of that log gets over 5-6MB it's rather not good for the server. So I'm thinking I have to read the last line but not to read the whole file and pass through it or load it in RAM because that would just load to death my box.
Is there any optimization for this operation so that it run smooth and not harm the server or kill Apache?
Other option that I have is to exec('tail -n 1 /path/to/log') but it doesn't sound so good.
Later edit: I DO NOT want to put the file in RAM because it might get huge. fopen() is not an option.
This should work:
$line = '';
$f = fopen('data.txt', 'r');
$cursor = -1;
fseek($f, $cursor, SEEK_END);
$char = fgetc($f);
/**
* Trim trailing newline chars of the file
*/
while ($char === "\n" || $char === "\r") {
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
}
/**
* Read until the start of file or first newline char
*/
while ($char !== false && $char !== "\n" && $char !== "\r") {
/**
* Prepend the new char
*/
$line = $char . $line;
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
}
fclose($f);
echo $line;
Note that this solution will repeat the last character of the line unless your file ends in a newline. If your file does not end in a newline, you can change both instances of $cursor-- to --$cursor.
Use fseek. You seek to the last position and seek it backward (use ftell to tell the current position) until you find a "\n".
$fp = fopen(".....");
fseek($fp, -1, SEEK_END);
$pos = ftell($fp);
$LastLine = "";
// Loop backword util "\n" is found.
while((($C = fgetc($fp)) != "\n") && ($pos > 0)) {
$LastLine = $C.$LastLine;
fseek($fp, $pos--);
}
fclose($fp);
NOTE: I've not tested. You may need some adjustment.
UPDATE: Thanks Syntax Error for pointing out about empty file.
:-D
UPDATE2: Fixxed another Syntax Error, missing semicolon at $LastLine = ""
You're looking for the fseek function. There are working examples of how to read the last line of a file in the comments section there.
this the code of IonuČ› G. Stan
i modified your code a little and made it a function for reuseability
function read_last_line ($file_path){
$line = '';
$f = fopen($file_path, 'r');
$cursor = -1;
fseek($f, $cursor, SEEK_END);
$char = fgetc($f);
/**
* Trim trailing newline chars of the file
*/
while ($char === "\n" || $char === "\r") {
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
}
/**
* Read until the start of file or first newline char
*/
while ($char !== false && $char !== "\n" && $char !== "\r") {
/**
* Prepend the new char
*/
$line = $char . $line;
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
}
return $line;
}
echo read_last_line('log.txt');
you will get that last line
If you know the upper bound of line length you could do something like this.
$maxLength = 1024;
$fp = fopen('somefile.txt', 'r');
fseek($fp, -$maxLength , SEEK_END);
$fewLines = explode("\n", fgets($fp, $maxLength));
$lastLine = $fewLines[count($fewLines) - 1];
In response to the edit: fopen just acquires a handle to the file (i.e. make sure it exists, process has permission, lets os know a process is using the file, etc...). In this example only 1024 characters from the file will be read into memory.
function readlastline()
{
$fp = #fopen("/dosmnt/LOGFILE.DAT", "r");
$pos = -1;
$t = " ";
while ($t != "\n") {
fseek($fp, $pos, SEEK_END);
$t = fgetc($fp);
$pos = $pos - 1;
}
$t = fgets($fp);
fclose($fp);
return $t;
}
Source: http://forums.devshed.com/php-development-5/php-quick-way-to-read-last-line-156010.html
Your problem looks similar to this one
The best approach to avoid loading the whole file into memory seems to be:
$file = escapeshellarg($file); // for the security concious (should be everyone!)
$line = `tail -n 1 $file`;
Would it be possible to optimize this from the other side?
If so, just let the logging application always log the line to a file while truncating it (i.e. > instead of >>)
Some optimization might be achieved by "guessing" though, just open the file and with the average log line width you could guess where the last line would be. Jump to that position with fseek and find the last line.
Untested code from the comments of http://php.net/manual/en/function.fseek.php
jim at lfchosting dot com 05-Nov-2003 02:03
Here is a function that returns the last line of a file. This should be quicker than reading the whole file till you get to the last line. If you want to speed it up a bit, you can set the $pos = some number that is just greater than the line length. The files I was dealing with were various lengths, so this worked for me.
<?php
function readlastline($file)
{
$fp = #fopen($file, "r");
$pos = -1;
$t = " ";
while ($t != "\n") {
fseek($fp, $pos, SEEK_END);
$t = fgetc($fp);
$pos = $pos - 1;
}
$t = fgets($fp);
fclose($fp);
return $t;
}
?>
This is my solution with only one loop
$line = '';
$f = fopen($file_path, 'r');
$cursor = 0 ;
do {
fseek($f, $cursor--, SEEK_END);
$char = fgetc($f);
$line = $char.$line;
} while (
$cursor > -1 || (
ord($char) !== 10 &&
ord($char) !== 13
)
);
Here is a compilation of the answers here wrapped into a function which can specify how many lines should be returned.
function getLastLines($path, $totalLines) {
$lines = array();
$fp = fopen($path, 'r');
fseek($fp, -1, SEEK_END);
$pos = ftell($fp);
$lastLine = "";
// Loop backword until we have our lines or we reach the start
while($pos > 0 && count($lines) < $totalLines) {
$C = fgetc($fp);
if($C == "\n") {
// skip empty lines
if(trim($lastLine) != "") {
$lines[] = $lastLine;
}
$lastLine = '';
} else {
$lastLine = $C.$lastLine;
}
fseek($fp, $pos--);
}
$lines = array_reverse($lines);
return $lines;
}
This function will let you read last line or (optionally) entire file line-by-line from end, by passing $initial_pos which will tell from where to start reading a file from end (negative integer).
function file_read_last_line($file, $initial_pos = -1) {
$fp = is_string($file) ? fopen($file, 'r') : $file;
$pos = $initial_pos;
$line = '';
do {
fseek($fp, $pos, SEEK_END);
$char = fgetc($fp);
if ($char === false) {
if ($pos === $initial_pos) return false;
break;
}
$pos = $pos - 1;
if ($char === "\r" || $char === "\n") continue;
$line = $char . $line;
} while ($char !== "\n");
if (is_string($file)) fclose($file);
return $line;
}
Then, to read last line:
$last_line = file_read_last_line('/path/to/file');
To read entire file line-by-line from end:
$fp = fopen('/path/to/file', 'r');
$pos = -1;
while (($line = file_read_last_line($fp, $pos)) !== false) {
$pos += -(strlen($line) + 1);
echo 'LINE: ' . $line . "\n";
}
fclose($fp);
Traverse backward a file chunk by chunk, stops after find a new line, and returns everything after the last "\n".
function get_last_line($file) {
if (!is_file($file)) return false;
$fileSize = filesize($file);
$bufferSize = 1024; // <------------------ Change buffer size here.
$bufferSize = ($fileSize > $bufferSize) ? $bufferSize : $fileSize;
$fp = fopen($file, 'r');
$position = $fileSize - $bufferSize;
$data = "";
while (true) {
fseek($fp, $position);
$chunk = fread($fp, $bufferSize);
$data = $chunk.$data;
$pos = strrchr($data, "\n");
if ($pos !== false) return substr($pos, 1);
if ($position <= 0) break;
$position -= $bufferSize;
if ($position <=0) $position = 0;
}
// whole file is 'the last line' :)
return $data;
}
You can define the length of the chunk your self.
Smaller chunk = smaller memory usage, more iteration.
Bigger chunk = bigger memory usage, less iteration.

PHP - Returning the last line in a file?

I'm guessing it's fgets, but I can't find the specific syntax. I'm trying to read out (in a string I'm thinking is easier) the last line added to a log file.
The simplest naive solution is simply:
$file = "/path/to/file";
$data = file($file);
$line = $data[count($data)-1];
Though, this WILL load the whole file into memory. Possibly a problem (or not). A better solution is this:
$file = escapeshellarg($file); // for the security concious (should be everyone!)
$line = `tail -n 1 $file`;
This looks like it is what you are looking for:
tekkie.flashbit.net: Tail functionality in PHP
It implements a function that uses fseek() with a negative index to roll up the file from the end. You can define how many lines you want to be returned.
The code also is available as a Gist on GitHub:
// full path to text file
define("TEXT_FILE", "/home/www/default-error.log");
// number of lines to read from the end of file
define("LINES_COUNT", 10);
function read_file($file, $lines) {
//global $fsize;
$handle = fopen($file, "r");
$linecounter = $lines;
$pos = -2;
$beginning = false;
$text = array();
while ($linecounter > 0) {
$t = " ";
while ($t != "\n") {
if(fseek($handle, $pos, SEEK_END) == -1) {
$beginning = true;
break;
}
$t = fgetc($handle);
$pos --;
}
$linecounter --;
if ($beginning) {
rewind($handle);
}
$text[$lines-$linecounter-1] = fgets($handle);
if ($beginning) break;
}
fclose ($handle);
return array_reverse($text);
}
$fsize = round(filesize(TEXT_FILE)/1024/1024,2);
echo "<strong>".TEXT_FILE."</strong>\n\n";
echo "File size is {$fsize} megabytes\n\n";
echo "Last ".LINES_COUNT." lines of the file:\n\n";
$lines = read_file(TEXT_FILE, LINES_COUNT);
foreach ($lines as $line) {
echo $line;
}
define('YOUR_EOL', "\n");
$fp = fopen('yourfile.txt', 'r');
$pos = -1; $line = ''; $c = '';
do {
$line = $c . $line;
fseek($fp, $pos--, SEEK_END);
$c = fgetc($fp);
} while ($c != YOUR_EOL);
echo $line;
fclose($fp);
This is better, since it does not load the complete file into memory...
Set YOUR_EOL to your correct line endings, if you use the same line endings as the default line endings of the OS where your script resides, you could use the constant PHP_EOL.
function seekLastLine($f) {
$pos = -2;
do {
fseek($f, $pos--, SEEK_END);
$ch = fgetc($f);
} while ($ch != "\n");
}
-2 because last char can be \n
You either have to read the file in line by line and save the last read line to get it.
Or if on unix/linux you might consider using the shell command tail
tail -n 1 filename
This one wont break for a 1 or 0 line file.
function readlastline($fileName)
{
$fp = #fopen($fileName, "r");
$begining = fseek($fp, 0);
$pos = -1;
$t = " ";
while ($t != "\n") {
fseek($fp, $pos, SEEK_END);
if(ftell($fp) == $begining){
break;
}
$t = fgetc($fp);
$pos = $pos - 1;
}
$t = fgets($fp);
fclose($fp);
return $t;
}
If you want to read a file line by line the file function reads the contents of a file, line by line and returns each line as an element of an array.
So you could do something simple like:
$lines = file('log.txt');
$lastLine = array_pop($lines);
...Why just read the last line?
function readLines($fp, $num) {
$line_count = 0; $line = ''; $pos = -1; $lines = array(); $c = '';
while($line_count < $num) {
$line = $c . $line;
fseek($fp, $pos--, SEEK_END);
$c = fgetc($fp);
if($c == "\n") { $line_count++; $lines[] = $line; $line = ''; $c = ''; }
}
return $lines;
}
$filename = "content.txt";
$fp = #fopen($filename, "r");
print_r(readLines($fp, 2));
fclose($fp);
#unique_stephen, your answer is flawed. PHP fseek returns 0 for success and -1 for failure. Storing the result in $begining (sic) and then later using it in a filter for ftell() isn't correct. If my reputation were higher I would have voted you down and left a comment. Here is a modified version of unique_stephen's function.
function readlastline($fileName)
{
$fp = #fopen($fileName, "r");
if (fseek($fp, 0) == -1)
exit('Cannot seek to beginning of the file');
$pos = -1;
$t = " ";
while ($t != "\n") {
if (fseek($fp, $pos, SEEK_END) == -1)
exit('Cannot seek to the end of the file');
if (ftell($fp) == 0) {
break;
}
$t = fgetc($fp);
$pos = $pos - 1;
}
$t = fgets($fp);
fclose($fp);
return $t;
}
NOTE: PHP's fseek cannot manage to seek to the end of files larger than PHP_MAX_INT which is 32bit signed even on 64bit binaries.
function readlastline($fileName)
{
$fp = #fopen($fileName, "r");
$begining = fseek($fp, 0);
$pos = -1;
$t = " ";
while ($t != "\n") {
fseek($fp, $pos, SEEK_END);
if(ftell($fp) == $begining){
break;
}
$t = fgetc($fp);
$pos = $pos - 1;
}
$t = fgets($fp);
fclose($fp);
return $t;
}

Categories