What's the best way to read a .TXT file (The file size is 225mb). I want to open the file and loop thru it and find information via REGEX.
Example data in the file :
00424333060001410100100BILLLLOYD BRRUSSELL & 12675 MAKALISO AVE WEST WORKS TOWN KS 23456-1035 3341310350630200500004200000001887800001789IWD QM1214200400003367250001799900001287IWD QM 000000000000000000000000000000 000000000000000000000000000000
The problem I am having is the file taking forever to open. And to search thru takes a while. My loop could have 75 items I need to search.
$name2 = "BILLLLOYD BRRUSSELL ";
$RE21 = "/[0-9]{23}.$name2/";
$file = fopen("MYFILE.TXT", "r");
while(!feof($file)){
$line = fget($file);
for ($row = 0; $row = 75; $row++{
$name2 = data i am getting from another file...;
$RE21 = "/[0-9]{23}.$name2/"; //Not sure if this works!!
$a = preg_match($RE21, $line, $matches);
foreach($matches as $x => $x_value) {
I will $x_value and store it.} //$x_value should be 00424333060001410100100BILLLLOYD BRRUSSELL
} //foreach
} //for
}//while
fclose($file);
Maybe you should try a different approach and use command line grep? Generate the regexps from your "another file" and the execute a grep command using your generated pattern and the file you want to search?
Use the -o flag to only get your matches from the result
You can read it line by line:
$handle = fopen("bla.txt", "r");
while (($buffer = fgets($handle, 4096)) !== false) {
// ...
}
fclose($handle);
Related
I want to search for the text Hello (example) in a TXT file whose size is 5GB+ then return the whole line.
I've tried using SplFileObject but what I know is that the line number is required to use SplFileObject, like that:
$linenumber = 2094;
$file = new SplFileObject('myfile.txt');
$file->seek($linenumber-1);
echo $file->current();
But as previously mentioned, I want to search for a string then get the whole line, I don't know the line number.
Any help would be appreciated.
this should work:
<?php
$needle = 'hello';
$count = 1;
$handle = fopen("inputfile.txt", "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
// process the line read.
$pos = strpos($line, $needle);
if ($pos !== false) {
echo $line . PHP_EOL;
echo "in line: ".$count . PHP_EOL;
break;
}
$count++;
}
fclose($handle);
} else {
// error opening the file.
}
This is the answer that I can use. Thanks a lot to #user3783243
For Linux:
exec('grep "Hello" myfile.txt', $return);
For Windows:
exec('findstr "Hello" "myfile.txt"', $return);
Now $return should contain the whole line.
Unfortunately, this doesn't work if exec() and system() functions are disabled by your server administrator in the php.ini file. But for me it works fine.
If someone have a better solution I'd be glad to know it :)
I need to dynamically add in the img src full file path to a file from a .txt file. So far i have been using the below code to populate titles, and descriptions:
< ?php
$myFile = "film_music/feature1.txt";
$lines = file($myFile);//file in to an array<br />
echo $lines[0]; //line 1
? >
The feature file contains all details for movies that will eventually be displayed here:
http://www.londonosophy.com/film_music2.php
Currently the file contains the following rows:
Sightseers (2012)
Dark comedy, featuring Alice Lowe
images/Sightseers-TinaPencil.jpg
Does anyone know php code that can read line X (or line 3 in this case) and dynamically populate the img src="" file path?
Many thanks in advance for your suggestions!
If you need to iterate over lines and need to check if line contains file path (for example, if sometimes there are white lines between blocks and sometimes they are missing), then:
<?php
$myFile = 'film_music/feature1.txt';
$lines = file($myFile);
$needle = 'images/';
$needleLen = strlen($needle);
foreach ($lines AS $line) {
$line = trim($line);
if (substr($line, 0, $needleLen) == $needle) {
echo '<img src="' . $line . '" alt="" />';
}
}
?>
You need to fetch 3rd line of the file every time, because it contains that path of the image.
<?php
global var $raw;
$myFile = "film_music/feature1.txt";
// open file...
$lines = file($myFile);//file in to an array<br />
for($i=2;$i<=no_of_lines;$i+3){
$raw = $lines[$i];
}
// Populate $raw variable where you need.
?>
Use $fh = fopen(...) and loop every line.
while (!feof($fh)) {
$line = fgets($fh);
if ($line === false) {
throw new Exception("File read error");
}
[do your things.]
}
http://www.php.net/manual/en/function.fopen.php
You can read each line of text line as:
$lines=file('file.txt');
$lines=array();
$fp=fopen('file.txt', 'r');
while (!feof($fp))
{
$line=fgets($fp);
//process line however you like
$line=trim($line);
//add to array
$lines[]=$line;
}
fclose($fp);
if you don't need any special processing, this should do what you're looking for
$lines = file($lines, FILE_IGNORE_NEW_LINES);
may this help you... :)
you can check all line of text file with in_array("you want", $lines) as you desire...if its not dynamically....
OK so let's say I have a .txt file with the following lines:
Mark
Jane
Ann
How could I make a php code which reads these names line by line and then saves them in a new text file like this:
Hi my name is Mark, what's yours?
Hi my name is Jane, what's yours?
hi my name is Ann, what's yours?
P.S - This is a simplified version of my problem, I actually have a list of URLs that I need to sorround by some XML code and then stack those one on top of another, so it turns out to be one big XML file. The surrounding blocks of XML text are static, they don't change, the only thing that should change is the part where the URL needs to be inserted, of course.
How about this: (result.txt contains your result, and input.txt contains your names)
<?php
$filename = "result.txt";
if (!$handleRes = fopen($filename, 'a')) {
echo "Cannot open file ($filename)";
exit;
}
$handle = #fopen("input.txt", "r");
if ($handle) {
while (($buffer = fgets($handle, 4096)) !== false) {
fwrite($handleRes, "Hi my name is ".chop($buffer).", what's yours?\n");
}
if (!feof($handle)) {
echo "Error: unexpected fgets() fail\n";
}
fclose($handle);
}
?>
$aString = file_get_contents("text.txt"); // get text from file
$aString = explode("\n", $aString); // split this text with new line character
$aString = array_map('trim',$aString); // remove spacing between left and right
foreach($aString as $index => $strString){ // browse each line
$aString[$index] = "Hi my name is $strString, what's yours?"; // change to new format
}
$aString = implode("\r\n", $aString); // join array to string
echo $aString;
Open the file, and then read.
$names = file("names.txt");
foreach ($names as $name) {
printf("Hi My name is %s, What's yours?\n", $name);
}
I'm trying to read a specific line from a text file using php.
Here's the text file:
foo
foo2
How would I get the content of the second line using php?
This returns the first line:
<?php
$myFile = "4-24-11.txt";
$fh = fopen($myFile, 'r');
$theData = fgets($fh);
fclose($fh);
echo $theData;
?>
..but I need the second.
Any help would be greatly appreciated
$myFile = "4-24-11.txt";
$lines = file($myFile);//file in to an array
echo $lines[1]; //line 2
file — Reads entire file into an array
omg I'm lacking 7 rep to make comments. This is #Raptor's & #Tomm's comment, since this question still shows up way high in google serps.
He's exactly right. For small files file($file); is perfectly fine. For large files it's total overkill b/c php arrays eat memory like crazy.
I just ran a tiny test with a *.csv with a file size of ~67mb (1,000,000 lines):
$t = -microtime(1);
$file = '../data/1000k.csv';
$lines = file($file);
echo $lines[999999]
."\n".(memory_get_peak_usage(1)/1024/1024)
."\n".($t+microtime(1));
//227.5
//0.22701287269592
//Process finished with exit code 0
And since noone mentioned it yet, I gave the SplFileObject a try, which I actually just recently discovered for myself.
$t = -microtime(1);
$file = '../data/1000k.csv';
$spl = new SplFileObject($file);
$spl->seek(999999);
echo $spl->current()
."\n".(memory_get_peak_usage(1)/1024/1024)
."\n".($t+microtime(1));
//0.5
//0.11500692367554
//Process finished with exit code 0
This was on my Win7 desktop so it's not representative for production environment, but still ... quite the difference.
If you wanted to do it that way...
$line = 0;
while (($buffer = fgets($fh)) !== FALSE) {
if ($line == 1) {
// This is the second line.
break;
}
$line++;
}
Alternatively, open it with file() and subscript the line with [1].
I would use the SplFileObject class...
$file = new SplFileObject("filename");
if (!$file->eof()) {
$file->seek($lineNumber);
$contents = $file->current(); // $contents would hold the data from line x
}
you can use the following to get all the lines in the file
$handle = #fopen('test.txt', "r");
if ($handle) {
while (!feof($handle)) {
$lines[] = fgets($handle, 4096);
}
fclose($handle);
}
print_r($lines);
and $lines[1] for your second line
$myFile = "4-21-11.txt";
$fh = fopen($myFile, 'r');
while(!feof($fh))
{
$data[] = fgets($fh);
//Do whatever you want with the data in here
//This feeds the file into an array line by line
}
fclose($fh);
This question is quite old by now, but for anyone dealing with very large files, here is a solution that does not involve reading every preceding line. This was also the only solution that worked in my case for a file with ~160 million lines.
<?php
function rand_line($fileName) {
do{
$fileSize=filesize($fileName);
$fp = fopen($fileName, 'r');
fseek($fp, rand(0, $fileSize));
$data = fread($fp, 4096); // assumes lines are < 4096 characters
fclose($fp);
$a = explode("\n",$data);
}while(count($a)<2);
return $a[1];
}
echo rand_line("file.txt"); // change file name
?>
It works by opening the file without reading anything, then moving the pointer instantly to a random position, reading up to 4096 characters from that point, then grabbing the first complete line from that data.
If you use PHP on Linux, you may try the following to read text for example between 74th and 159th lines:
$text = shell_exec("sed -n '74,159p' path/to/file.log");
This solution is good if your file is large.
You have to loop the file till end of file.
while(!feof($file))
{
echo fgets($file). "<br />";
}
fclose($file);
Use stream_get_line: stream_get_line — Gets line from stream resource up to a given delimiter
Source: http://php.net/manual/en/function.stream-get-line.php
You could try looping until the line you want, not the EOF, and resetting the variable to the line each time (not adding to it). In your case, the 2nd line is the EOF. (A for loop is probably more appropriate in my code below).
This way the entire file is not in the memory; the drawback is it takes time to go through the file up to the point you want.
<?php
$myFile = "4-24-11.txt";
$fh = fopen($myFile, 'r');
$i = 0;
while ($i < 2)
{
$theData = fgets($fh);
$i++
}
fclose($fh);
echo $theData;
?>
I like daggett answer but there is another solution you can get try if your file is not big enough.
$file = __FILE__; // Let's take the current file just as an example.
$start_line = __LINE__ -1; // The same with the line what we look for. Take the line number where $line variable is declared as the start.
$lines_to_display = 5; // The number of lines to display. Displays only the $start_line if set to 1. If $lines_to_display argument is omitted displays all lines starting from the $start_line.
echo implode('', array_slice(file($file), $start_line, lines_to_display));
I searched for a one line solution to read specific line from a file.
Here my solution:
echo file('dayInt.txt')[1]
I have a script which, each time is called, gets the first line of a file. Each line is known to be exactly of the same length (32 alphanumeric chars) and terminates with "\r\n".
After getting the first line, the script removes it.
This is done in this way:
$contents = file_get_contents($file));
$first_line = substr($contents, 0, 32);
file_put_contents($file, substr($contents, 32 + 2)); //+2 because we remove also the \r\n
Obviously it works, but I was wondering whether there is a smarter (or more efficient) way to do this?
In my simple solution I basically read and rewrite the entire file just to take and remove the first line.
I came up with this idea yesterday:
function read_and_delete_first_line($filename) {
$file = file($filename);
$output = $file[0];
unset($file[0]);
file_put_contents($filename, $file);
return $output;
}
There is no more efficient way to do this other than rewriting the file.
No need to create a second temporary file, nor put the whole file in memory:
if ($handle = fopen("file", "c+")) { // open the file in reading and editing mode
if (flock($handle, LOCK_EX)) { // lock the file, so no one can read or edit this file
while (($line = fgets($handle, 4096)) !== FALSE) {
if (!isset($write_position)) { // move the line to previous position, except the first line
$write_position = 0;
} else {
$read_position = ftell($handle); // get actual line
fseek($handle, $write_position); // move to previous position
fputs($handle, $line); // put actual line in previous position
fseek($handle, $read_position); // return to actual position
$write_position += strlen($line); // set write position to the next loop
}
}
fflush($handle); // write any pending change to file
ftruncate($handle, $write_position); // drop the repeated last line
flock($handle, LOCK_UN); // unlock the file
}
fclose($handle);
}
This will shift the first line of a file, you dont need to load the entire file in memory like you do using the 'file' function. Maybe for small files is a bit more slow than with 'file' (maybe but i bet is not) but is able to manage largest files without problems.
$firstline = false;
if($handle = fopen($logFile,'c+')){
if(!flock($handle,LOCK_EX)){fclose($handle);}
$offset = 0;
$len = filesize($logFile);
while(($line = fgets($handle,4096)) !== false){
if(!$firstline){$firstline = $line;$offset = strlen($firstline);continue;}
$pos = ftell($handle);
fseek($handle,$pos-strlen($line)-$offset);
fputs($handle,$line);
fseek($handle,$pos);
}
fflush($handle);
ftruncate($handle,($len-$offset));
flock($handle,LOCK_UN);
fclose($handle);
}
you can iterate the file , instead of putting them all in memory
$handle = fopen("file", "r");
$first = fgets($handle,2048); #get first line.
$outfile="temp";
$o = fopen($outfile,"w");
while (!feof($handle)) {
$buffer = fgets($handle,2048);
fwrite($o,$buffer);
}
fclose($handle);
fclose($o);
rename($outfile,$file);
I wouldn't usually recommend opening up a shell for this sort of thing, but if you're doing this infrequently on really large files, there's probably something to be said for:
$lines = `wc -l myfile` - 1;
`tail -n $lines myfile > newfile`;
It's simple, and it doesn't involve reading the whole file into memory.
I wouldn't recommend this for small files, or extremely frequent use though. The overhead's too high.
You could store positional info into the file itself. For example, the first 8 bytes of the file could store an integer. This integer is the byte offset of the first real line in the file.
So, you never delete lines anymore. Instead, deleting a line means altering the start position. fseek() to it and then read lines as normal.
The file will grow big eventually. You could periodically clean up the orphaned lines to reduce the file size.
But seriously, just use a database and don't do stuff like this.
Here's one way:
$contents = file($file, FILE_IGNORE_NEW_LINES);
$first_line = array_shift($contents);
file_put_contents($file, implode("\r\n", $contents));
There's countless other ways to do that also, but all the methods would involve separating the first line somehow and saving the rest. You cannot avoid rewriting the whole file. An alternative take:
list($first_line, $contents) = explode("\r\n", file_get_contents($file), 2);
file_put_contents($file, implode("\r\n", $contents));
My problem was large files. I just needed to edit, or remove the first line. This was a solution I used. Didn't require to load the complete file in a variable. Currently echos, but you could always save the contents.
$fh = fopen($local_file, 'rb');
echo "add\tfirst\tline\n"; // add your new first line.
fgets($fh); // moves the file pointer to the next line.
echo stream_get_contents($fh); // flushes the remaining file.
fclose($fh);
I think this is best for any file size
$myfile = fopen("yourfile.txt", "r") or die("Unable to open file!");
$ch=1;
while(!feof($myfile)) {
$dataline= fgets($myfile) . "<br>";
if($ch == 2){
echo str_replace(' ', ' ', $dataline)."\n";
}
$ch = 2;
}
fclose($myfile);
The solutions here didn't work performantly for me.
My solution grabs the last line (not the first line, in my case it was not relevant to get the first or last line) from the file and removes that from that file.
This is very quickly even with very large files (>150000000 lines).
function file_pop($file)
{
if ($fp = #fopen($file, "c+")) {
if (!flock($fp, LOCK_EX)) {
fclose($fp);
}
$pos = -1;
$found = 0;
while ($found < 2) {
if (fseek($fp, $pos--, SEEK_END) < 0) { // can not seek to position
rewind($fp); // rewind to the beginnung of the file
break;
};
if (ord(fgetc($fp)) == 10) { // newline
$found++;
}
}
$lastpos = ftell($fp); // get current position of file
$lastline = fgets($fp); // get current line
ftruncate($fp, $lastpos); // truncate file to last position
flock($fp, LOCK_UN); // unlock
fclose($fp); // close the file
return trim($lastline);
}
}
You could use file() method.
Gets the first line
$content = file('myfile.txt');
echo $content[0];