Search for text in a 5GB+ file then get whole line - php

I want to search for the text Hello (example) in a TXT file whose size is 5GB+ then return the whole line.
I've tried using SplFileObject but what I know is that the line number is required to use SplFileObject, like that:
$linenumber = 2094;
$file = new SplFileObject('myfile.txt');
$file->seek($linenumber-1);
echo $file->current();
But as previously mentioned, I want to search for a string then get the whole line, I don't know the line number.
Any help would be appreciated.

this should work:
<?php
$needle = 'hello';
$count = 1;
$handle = fopen("inputfile.txt", "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
// process the line read.
$pos = strpos($line, $needle);
if ($pos !== false) {
echo $line . PHP_EOL;
echo "in line: ".$count . PHP_EOL;
break;
}
$count++;
}
fclose($handle);
} else {
// error opening the file.
}

This is the answer that I can use. Thanks a lot to #user3783243
For Linux:
exec('grep "Hello" myfile.txt', $return);
For Windows:
exec('findstr "Hello" "myfile.txt"', $return);
Now $return should contain the whole line.
Unfortunately, this doesn't work if exec() and system() functions are disabled by your server administrator in the php.ini file. But for me it works fine.
If someone have a better solution I'd be glad to know it :)

Related

php getting lines of text from a text file based on a # delimiter at the beginning of each sentence

I have a text file
test.txt
and in the file it has lines that begin with '#'
example
#this is a test
#this is another test
this is not a test
When I run my php script I need it to only grab the lines with the '#' and then remove it so it doesn't show in the echo.
So the output would be....
this is a test
this is another test
I'm just learning php but this is what I have... It reads one line...
<?php
$f=fopen("alertmon_user.txt", "r");
// is this where I need to set the conditions?
echo fgets($f);
fclose($f);
?>
What can I add to this to make this work? Am I on the right track?
Read the file line by line and check if line starts with #.
$handle = fopen("alertmon_user.txt", "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
if(substr($line, 0, 1) === "#") {
//process line here
echo $line . "</br>"; // add </br> for new line
}
}
fclose($handle);
} else {
// error opening the file.
}
Update : To remove # from line.
If you want to remove # from line then use substr method.
for example.
$updatedLine = substr($line, 1, strlen($line));
echo $updatedLine . "</br>";
Hope it'll help.

PHP opening a large file

What's the best way to read a .TXT file (The file size is 225mb). I want to open the file and loop thru it and find information via REGEX.
Example data in the file :
00424333060001410100100BILLLLOYD BRRUSSELL & 12675 MAKALISO AVE WEST WORKS TOWN KS 23456-1035 3341310350630200500004200000001887800001789IWD QM1214200400003367250001799900001287IWD QM 000000000000000000000000000000 000000000000000000000000000000
The problem I am having is the file taking forever to open. And to search thru takes a while. My loop could have 75 items I need to search.
$name2 = "BILLLLOYD BRRUSSELL ";
$RE21 = "/[0-9]{23}.$name2/";
$file = fopen("MYFILE.TXT", "r");
while(!feof($file)){
$line = fget($file);
for ($row = 0; $row = 75; $row++{
$name2 = data i am getting from another file...;
$RE21 = "/[0-9]{23}.$name2/"; //Not sure if this works!!
$a = preg_match($RE21, $line, $matches);
foreach($matches as $x => $x_value) {
I will $x_value and store it.} //$x_value should be 00424333060001410100100BILLLLOYD BRRUSSELL
} //foreach
} //for
}//while
fclose($file);
Maybe you should try a different approach and use command line grep? Generate the regexps from your "another file" and the execute a grep command using your generated pattern and the file you want to search?
Use the -o flag to only get your matches from the result
You can read it line by line:
$handle = fopen("bla.txt", "r");
while (($buffer = fgets($handle, 4096)) !== false) {
// ...
}
fclose($handle);

How can I add multiple strings to one text file, and read the all?

I am trying to make a program where I can add things to a list, read things, and clear the list. I have the clear function working perfectly, however I can't seem to add or read more than 1 line at a time. I am using fwrite($handle, $MyString); but that replaces everything in the entire file with $MyString. To get the information from the file I am using $list = fgets($handle); and then using echo to print it. This reads the first line in the file only.
Any help?
Thanks!
Getlist code:
<?php
$myFile = "needlist.txt";
$fh = fopen($myFile, 'r');
$theData = fgets($fh);
fclose($fh);
echo $theData;
?>
Add to the list code:
<?php
$neededlist = "needlist.txt";
$fh = fopen($neededlist, 'w');
$user_message = $_REQUEST['txtweb-message'];
$needed .= $user_message;
$needed .= "\n";
fwrite($fh, $needed);
fclose($fh);
echo "You have successfully added ", $user_message;
?>
When you write to the file are you opening your filehandle with the "a" mode option? Opening with "w" or "x" truncates it so you start with a clean file (http://php.net/fopen)
fgets(); reads only until the end of the line ( http://php.net/fgets ). To get the whole file you can try:
var $list = "";
var $line = "";
while ($line = fgets($handle)) {
$list = $list . "\n" . $line;
}
echo $list;
You want to add the "\n" because fread doesn't read the linefeeds IIRC. There're also a couple functions that might be more appropriate in this situation like file_get_contents and fread.
Fgets returns only one string. You should use it in cycle like that:
while (($buffer = fgets($handle, 4096)) !== false) {
echo $buffer;
}

print contents of file until found the word hi

I want the program to print the document contents line by line while not reaching neither the end of file or found the word hi
The problem is when it found the word hi, it prints nothing although it is at position 22. Why not print the previous words how to solve this issue.
My file contain "Php is a special case hi. You will use less memory using the iterative solution. Moreover, function calls in PHP are costly, so it's better to avoid function calls when you can." string.
Here is my code
<?php
$contents = file_get_contents('m.txt');
$search_keyword = 'hi';
// check if word is there
$file=fopen("m.txt","r+");
while(!feof($file)&&strpos($contents, $search_keyword) == FALSE)
{
echo fgets($file)."<br>";
}
?>
change this condition
while(!feof($file)&&strpos($contents, $search_keyword) == FALSE)
to
while(!feof($file)) {
if(strpos($contents, $search_keyword) === FALSE) {
echo fgets($file)."<br>";
} else
break;
}
}
You mean print the file line by line until the word 'hi' is found?
<?php
$search_keyword = 'hi';
$handle = #fopen("m.txt", "r");
if ( $handle )
{
// Read file one line at a time
while ( ($buffer = fgets($handle, 4096)) !== false )
{
echo $buffer . '<br />';
if ( preg_match('/'.$search_keyword.'/i', $subject) )
break;
}
fclose($handle);
}
?>
You can replace the preg_match to strpos if you like.

PHP: Read Specific Line From File

I'm trying to read a specific line from a text file using php.
Here's the text file:
foo
foo2
How would I get the content of the second line using php?
This returns the first line:
<?php
$myFile = "4-24-11.txt";
$fh = fopen($myFile, 'r');
$theData = fgets($fh);
fclose($fh);
echo $theData;
?>
..but I need the second.
Any help would be greatly appreciated
$myFile = "4-24-11.txt";
$lines = file($myFile);//file in to an array
echo $lines[1]; //line 2
file — Reads entire file into an array
omg I'm lacking 7 rep to make comments. This is #Raptor's & #Tomm's comment, since this question still shows up way high in google serps.
He's exactly right. For small files file($file); is perfectly fine. For large files it's total overkill b/c php arrays eat memory like crazy.
I just ran a tiny test with a *.csv with a file size of ~67mb (1,000,000 lines):
$t = -microtime(1);
$file = '../data/1000k.csv';
$lines = file($file);
echo $lines[999999]
."\n".(memory_get_peak_usage(1)/1024/1024)
."\n".($t+microtime(1));
//227.5
//0.22701287269592
//Process finished with exit code 0
And since noone mentioned it yet, I gave the SplFileObject a try, which I actually just recently discovered for myself.
$t = -microtime(1);
$file = '../data/1000k.csv';
$spl = new SplFileObject($file);
$spl->seek(999999);
echo $spl->current()
."\n".(memory_get_peak_usage(1)/1024/1024)
."\n".($t+microtime(1));
//0.5
//0.11500692367554
//Process finished with exit code 0
This was on my Win7 desktop so it's not representative for production environment, but still ... quite the difference.
If you wanted to do it that way...
$line = 0;
while (($buffer = fgets($fh)) !== FALSE) {
if ($line == 1) {
// This is the second line.
break;
}
$line++;
}
Alternatively, open it with file() and subscript the line with [1].
I would use the SplFileObject class...
$file = new SplFileObject("filename");
if (!$file->eof()) {
$file->seek($lineNumber);
$contents = $file->current(); // $contents would hold the data from line x
}
you can use the following to get all the lines in the file
$handle = #fopen('test.txt', "r");
if ($handle) {
while (!feof($handle)) {
$lines[] = fgets($handle, 4096);
}
fclose($handle);
}
print_r($lines);
and $lines[1] for your second line
$myFile = "4-21-11.txt";
$fh = fopen($myFile, 'r');
while(!feof($fh))
{
$data[] = fgets($fh);
//Do whatever you want with the data in here
//This feeds the file into an array line by line
}
fclose($fh);
This question is quite old by now, but for anyone dealing with very large files, here is a solution that does not involve reading every preceding line. This was also the only solution that worked in my case for a file with ~160 million lines.
<?php
function rand_line($fileName) {
do{
$fileSize=filesize($fileName);
$fp = fopen($fileName, 'r');
fseek($fp, rand(0, $fileSize));
$data = fread($fp, 4096); // assumes lines are < 4096 characters
fclose($fp);
$a = explode("\n",$data);
}while(count($a)<2);
return $a[1];
}
echo rand_line("file.txt"); // change file name
?>
It works by opening the file without reading anything, then moving the pointer instantly to a random position, reading up to 4096 characters from that point, then grabbing the first complete line from that data.
If you use PHP on Linux, you may try the following to read text for example between 74th and 159th lines:
$text = shell_exec("sed -n '74,159p' path/to/file.log");
This solution is good if your file is large.
You have to loop the file till end of file.
while(!feof($file))
{
echo fgets($file). "<br />";
}
fclose($file);
Use stream_get_line: stream_get_line — Gets line from stream resource up to a given delimiter
Source: http://php.net/manual/en/function.stream-get-line.php
You could try looping until the line you want, not the EOF, and resetting the variable to the line each time (not adding to it). In your case, the 2nd line is the EOF. (A for loop is probably more appropriate in my code below).
This way the entire file is not in the memory; the drawback is it takes time to go through the file up to the point you want.
<?php
$myFile = "4-24-11.txt";
$fh = fopen($myFile, 'r');
$i = 0;
while ($i < 2)
{
$theData = fgets($fh);
$i++
}
fclose($fh);
echo $theData;
?>
I like daggett answer but there is another solution you can get try if your file is not big enough.
$file = __FILE__; // Let's take the current file just as an example.
$start_line = __LINE__ -1; // The same with the line what we look for. Take the line number where $line variable is declared as the start.
$lines_to_display = 5; // The number of lines to display. Displays only the $start_line if set to 1. If $lines_to_display argument is omitted displays all lines starting from the $start_line.
echo implode('', array_slice(file($file), $start_line, lines_to_display));
I searched for a one line solution to read specific line from a file.
Here my solution:
echo file('dayInt.txt')[1]

Categories