The script outputs a list and must write the entire list to a file. Why does the script write only the last line to the .txt file, and not the entire list? How do I fix a script to write the entire list to a .txt file?
$dir = "./radio/radio_stantion/";
$name = scandir($dir);
for($i=2; $i<=(sizeof($name)-1); $i++) {
$fopen=file( $dir.$name[$i] );
$line = $fopen[3];
$url = $fopen[2];
$radio_name = explode(") ", $line);
$radio_url = explode("//", $url);
$link = $radio_url[1];
$http = explode(":", $link);
$fff = $http[1];
$port = explode("/", $fff);
$zzz = $radio_name[1];
$finalname = preg_replace ("/[^a-zа-я\s]/si","",$zzz);
$sss = $finalname."-".$http[0]."-".$port[0];
--------------------------
A list is displayed:
FRESH FM IBADAN s4.voscast.com 8442
Bestfriend FM 178.32.62.172 8217
COOLfahrenheit 111.223.51.7 8005
RThess 37.59.32.115 6156
SmoothJazzcom 149.56.155.209 80
--------------------------
$f = fopen("../top_100/radio_top_100_v2.txt", "w");
fwrite($f, $sss);
fclose($f);
}
I barely understand whats going on there but the problem is that you are opening the file in write mode on each loop iteration.
Using fopen with w mode clears the file and writes to it:
'w' Open for writing only; place the file pointer at the beginning of
the file and truncate the file to zero length. If the file does not
exist, attempt to create it.
You should fopen the file before the loop. fwrite on each iteration. and fclose after the loop.
$f = fopen("../top_100/radio_top_100_v2.txt", "w");
for($i=2; $i<=(sizeof($name)-1); $i++) {
...
fwrite($f, $sss);
...
}
fclose($f);
I think you are overwriting the previous writes in every iteration. Try writing everything to a string variable first(and append a \n to each line), then write the string contents to the file one time. If you are dealing with a lot of data, then it would make sense to use fwrite every iteration, so not to consume too much memory.
File IO is expensive, so only opening the file once, and writing once, should improve performance as well.
Related
I have a text file like this:
1
2
3
4
5
6
7
8
9
10
And I want to remove specific lines which numbers are in an array like this:
$myfile='txt.txt';
$remove=array(1,3,6,7,10);
//wanna remove these lines
So I tried this code but It didn't work and It just doubles the text and ruins everything:
<?php
$myfile='txt.txt';
$remove=array(1,3,5,7,10);
$lines=file($myfile);
$countline=sizeof($lines);
$data=file_get_contents($myfile);
for ($i=0; $i < $countline+1; $i++) {
if (in_array($i, $remove)) {
$editeddata=str_replace($lines[$i], "", $data);
$removeline = file_put_contents($myfile, $editeddata.PHP_EOL , FILE_APPEND | LOCK_EX);
}
}
?>
I couldn't use ((for)) properly and I think it will just ruin the text because it deletes lines one after another have been deleted and it changes the order so I should have a code to remove them all at once.
And please don't give a code to just replace numbers because the main text file is not only numbers and contains word,etc...
Thanks A lot!
You're reading the file twice (with file and file_get_contents), which I think is confusing the later code. You have everything you need with the first call - an array of all the lines in the file. You're also using str_replace to remove the content, which seems a bit dangerous if any of the content is repeated.
I'd refactor this to simply filter the array of lines based on their line-number, then write it back to the file in a single operation:
$myfile = 'txt.txt';
$remove = [1, 3, 5, 7, 10];
// Read file into memory
$lines = file($myfile);
// Filter lines based on line number (+1 because the array is zero-indexed)
$lines = array_filter($lines, function($lineNumber) use ($remove) {
return !in_array($lineNumber + 1, $remove);
}, ARRAY_FILTER_USE_KEY);
// Re-assemble the output (the lines already have a line-break at the end)
$output = implode('', $lines);
// Write back to file
file_put_contents($myfile, $output);
If the file fits in memory then you can do the simple:
$myfile='txt.txt';
$remove=array(1,3,6,7,10);
file_put_contents($myfile, implode(PHP_EOL,array_diff($file($myfile,FILE_IGNORE_NEW_LINES), $remove)));
Note: Because it's a bit ambiguous whether $remove has the content or the lines you want to remove, the above code removes the content . If you want to remove lines change array_diff($file($myfile,FILE_IGNORE_NEW_LINES), $remove) to array_diff_keys($file($myfile,FILE_IGNORE_NEW_LINES), array_flip($remove))
If your file is large then you need to resort to some sort of streaming. I suggest against reading and writing to the same file and doing something like:
$myfile='txt.txt';
$remove=array(1,3,6,7,10);
$h = fopen($myfile,"r");
$tmp = fopen($myfile.".tmp", "w");
while (($line = fgets($h)) !== false) {
if (!in_array(rtrim($line, PHP_EOL), $remove)) {
fwrite($tmp, $line);
}
}
fclose($h);
fclose($tmp);
unlink($myfile);
rename($myfile.".tmp", $myfile);
I am working on reading a file in php.
I need to read specific lines of the file.
I used this code:
fseek($file_handle,$start);
while (!feof($file_handle))
{
///Get and read the line of the file pointed at.
$line = fgets($file_handle);
$lineArray .= $line."LINE_SEPARATOR";
processLine($lineArray, $linecount, $logger, $xmlReply);
$counter++;
}
fclose($file_handle);
However I realized that the fseek() takes the number of bytes and not the line number.
Does PHP have other function that bases its pointer in line numbers?
Or do I have to read the file from the start every time, and have a
counter until my desired line number is read?
I'm looking for an efficient algorithm, stepping over 500-1000 Kb file to get to the desired line seems inefficient.
Use SplFileObject::seek
$file = new SplFileObject('yourfile.txt');
$file->seek(123); // seek to line 124 (0-based)
Does this work for you?
$file = "name-of-my-file.txt";
$lines = file( $file );
echo $lines[67]; // echos line 68 (lines numbers start at 0 (replace 68 with whatever))
You would obviously need to check the lines exists before printing though. Any good?
You could do like:
$lines = file($filename); //file in to an array
echo $lines[1]; //line 2
OR
$line = 0;
$fh = fopen($myFile, 'r');
while (($buffer = fgets($fh)) !== FALSE) {
if ($line == 1) {
// $buffer is the second line.
break;
}
$line++;
}
You must read from the beginning. But if the file never/rarely changes, you could cache the line offsets somewhere else, perhaps in another file.
Try this,the simplest one
$buffer=explode("\n",file_get_contents("filename"));//Split data to array for each "\n"
Now the buffer is an array and each array index contain each lines;
To get the 5th line
echo $buffer[4];
You could use function file($filename) . This function reads data from the file into array.
I am trying add HTML to a file using fwrite(). My final goal is to get it to add it 15 lines above the end of the file. Here is what I have so far:
<?php
$file = fopen("index.html", "r+");
// Seek to the end
fseek($file, SEEK_END, 0);
// Get and save that position
$filesize = ftell($file);
// Seek to half the length of the file
fseek($file, SEEK_SET, $filesize + 15);
// Write your data
$main = <<<MAIN
//html goes here
MAIN;
fwrite($file, $main);
// Close the file handler
fclose($file);
?>
This just keeps overwriting the top of the file.
Thanks.
The sample code in the question does not operate based on lines, since you're working with file size (unless there is an assumption about definition of lines in the application that is not mentioned in here). If you want to work with lines, then you'd need to search for new line characters (which separates each line with the next).
If the target file is not a large file (so we could load the whole file into memory), we could use PHP built-in file() to read all the lines of the file into an array, and then insert the data after the 15th element. something like this:
<?php
$lines = file($filename);
$num_lines = count($lines);
if ($num_lines > 15) {
array_splice($lines, $num_lines - 15, 0, array($content));
file_put_contents($filename, implode('', $lines));
} else {
file_put_contents($filename, PHP_EOL . $content, FILE_APPEND);
}
I'm trying to make (as immature as this sounds) an application online that prints random insults. I have a list that is 140 lines long, and I would like to print one entire line. There is mt_rand(min,max) but when I use that alongside fgets(file, "line") It doesn't give me the line of the random number, it gives me the character. Any help? I have all the code so far below.
<?php
$file = fopen("Insults.txt","r");
echo fgets($file, (mt_rand(1, 140)));
fclose($file);
?>
Try this, it's easier version of what you want to do:
$file = file('Insults.txt');
echo $file[array_rand($file)];
$lines = file("Insults.txt");
echo $lines[array_rand($lines)];
Or within a function:
function random_line($filename) {
$lines = file($filename) ;
return $lines[array_rand($lines)] ;
}
$insult = random_line("Insults.txt");
echo $insult;
use file() for this. it returns an array with the lines of the file:
$lines = file($filename);
$line = mt_rand(0, count($lines));
echo $lines[$line];
First: You totally screwed on using fgets() correctly, please refer to the manual about the meaning of the second parameter (it just plainly not what you think it is).
Second: the file() solution will work... until the filesize exceeds a certain size and exhaust the complete PHP memory. Keep in mind: file() reads the complete file into an array.
You might be better off with reading line-by-line, even if that means you'll have to discard most of the read data.
$fp = fopen(...);
$line = 129;
// read (and ignore) the first 128 lines in the file
$i = 1;
while ($i < $line) {
fgets($fp);
$i++;
}
// at last: this is the line we wanted
$theLine = fgets($fp);
(not tested!)
I have just found out that my script gives me a fatal error:
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 440 bytes) in C:\process_txt.php on line 109
That line is this:
$lines = count(file($path)) - 1;
So I think it is having difficulty loading the file into memeory and counting the number of lines, is there a more efficient way I can do this without having memory issues?
The text files that I need to count the number of lines for range from 2MB to 500MB. Maybe a Gig sometimes.
Thanks all for any help.
This will use less memory, since it doesn't load the whole file into memory:
$file="largefile.txt";
$linecount = 0;
$handle = fopen($file, "r");
while(!feof($handle)){
$line = fgets($handle);
$linecount++;
}
fclose($handle);
echo $linecount;
fgets loads a single line into memory (if the second argument $length is omitted it will keep reading from the stream until it reaches the end of the line, which is what we want). This is still unlikely to be as quick as using something other than PHP, if you care about wall time as well as memory usage.
The only danger with this is if any lines are particularly long (what if you encounter a 2GB file without line breaks?). In which case you're better off doing slurping it in in chunks, and counting end-of-line characters:
$file="largefile.txt";
$linecount = 0;
$handle = fopen($file, "r");
while(!feof($handle)){
$line = fgets($handle, 4096);
$linecount = $linecount + substr_count($line, PHP_EOL);
}
fclose($handle);
echo $linecount;
Using a loop of fgets() calls is fine solution and the most straightforward to write, however:
even though internally the file is read using a buffer of 8192 bytes, your code still has to call that function for each line.
it's technically possible that a single line may be bigger than the available memory if you're reading a binary file.
This code reads a file in chunks of 8kB each and then counts the number of newlines within that chunk.
function getLines($file)
{
$f = fopen($file, 'rb');
$lines = 0;
while (!feof($f)) {
$lines += substr_count(fread($f, 8192), "\n");
}
fclose($f);
return $lines;
}
If the average length of each line is at most 4kB, you will already start saving on function calls, and those can add up when you process big files.
Benchmark
I ran a test with a 1GB file; here are the results:
+-------------+------------------+---------+
| This answer | Dominic's answer | wc -l |
+------------+-------------+------------------+---------+
| Lines | 3550388 | 3550389 | 3550388 |
+------------+-------------+------------------+---------+
| Runtime | 1.055 | 4.297 | 0.587 |
+------------+-------------+------------------+---------+
Time is measured in seconds real time, see here what real means
True line count
While the above works well and returns the same results as wc -l, if the file ends without a newline, the line number will be off by one; if you care about this particular scenario, you can make it more accurate by using this logic:
function getLines($file)
{
$f = fopen($file, 'rb');
$lines = 0; $buffer = '';
while (!feof($f)) {
$buffer = fread($f, 8192);
$lines += substr_count($buffer, "\n");
}
fclose($f);
if (strlen($buffer) > 0 && $buffer[-1] != "\n") {
++$lines;
}
return $lines;
}
Simple Oriented Object solution
$file = new \SplFileObject('file.extension');
while($file->valid()) $file->fgets();
var_dump($file->key());
#Update
Another way to make this is with PHP_INT_MAX in SplFileObject::seek method.
$file = new \SplFileObject('file.extension', 'r');
$file->seek(PHP_INT_MAX);
echo $file->key();
If you're running this on a Linux/Unix host, the easiest solution would be to use exec() or similar to run the command wc -l $path. Just make sure you've sanitized $path first to be sure that it isn't something like "/path/to/file ; rm -rf /".
There is a faster way I found that does not require looping through the entire file
only on *nix systems, there might be a similar way on windows ...
$file = '/path/to/your.file';
//Get number of lines
$totalLines = intval(exec("wc -l '$file'"));
If you're using PHP 5.5 you can use a generator. This will NOT work in any version of PHP before 5.5 though. From php.net:
"Generators provide an easy way to implement simple iterators without the overhead or complexity of implementing a class that implements the Iterator interface."
// This function implements a generator to load individual lines of a large file
function getLines($file) {
$f = fopen($file, 'r');
// read each line of the file without loading the whole file to memory
while ($line = fgets($f)) {
yield $line;
}
}
// Since generators implement simple iterators, I can quickly count the number
// of lines using the iterator_count() function.
$file = '/path/to/file.txt';
$lineCount = iterator_count(getLines($file)); // the number of lines in the file
If you're under linux you can simply do:
number_of_lines = intval(trim(shell_exec("wc -l ".$file_name." | awk '{print $1}'")));
You just have to find the right command if you're using another OS
Regards
This is an addition to Wallace Maxter's solution
It also skips empty lines while counting:
function getLines($file)
{
$file = new \SplFileObject($file, 'r');
$file->setFlags(SplFileObject::READ_AHEAD | SplFileObject::SKIP_EMPTY |
SplFileObject::DROP_NEW_LINE);
$file->seek(PHP_INT_MAX);
return $file->key() + 1;
}
The most succinct cross-platform solution that only buffers one line at a time.
$file = new \SplFileObject(__FILE__);
$file->setFlags($file::READ_AHEAD);
$lines = iterator_count($file);
Unfortunately, we have to set the READ_AHEAD flag otherwise iterator_count blocks indefinitely. Otherwise, this would be a one-liner.
private static function lineCount($file) {
$linecount = 0;
$handle = fopen($file, "r");
while(!feof($handle)){
if (fgets($handle) !== false) {
$linecount++;
}
}
fclose($handle);
return $linecount;
}
I wanted to add a little fix to the function above...
in a specific example where i had a file containing the word 'testing' the function returned 2 as a result. so i needed to add a check if fgets returned false or not :)
have fun :)
Based on dominic Rodger's solution,
here is what I use (it uses wc if available, otherwise fallbacks to dominic Rodger's solution).
class FileTool
{
public static function getNbLines($file)
{
$linecount = 0;
$m = exec('which wc');
if ('' !== $m) {
$cmd = 'wc -l < "' . str_replace('"', '\\"', $file) . '"';
$n = exec($cmd);
return (int)$n + 1;
}
$handle = fopen($file, "r");
while (!feof($handle)) {
$line = fgets($handle);
$linecount++;
}
fclose($handle);
return $linecount;
}
}
https://github.com/lingtalfi/Bat/blob/master/FileTool.php
Counting the number of lines can be done by following codes:
<?php
$fp= fopen("myfile.txt", "r");
$count=0;
while($line = fgetss($fp)) // fgetss() is used to get a line from a file ignoring html tags
$count++;
echo "Total number of lines are ".$count;
fclose($fp);
?>
You have several options. The first is to increase the availble memory allowed, which is probably not the best way to do things given that you state the file can get very large. The other way is to use fgets to read the file line by line and increment a counter, which should not cause any memory issues at all as only the current line is in memory at any one time.
There is another answer that I thought might be a good addition to this list.
If you have perl installed and are able to run things from the shell in PHP:
$lines = exec('perl -pe \'s/\r\n|\n|\r/\n/g\' ' . escapeshellarg('largetextfile.txt') . ' | wc -l');
This should handle most line breaks whether from Unix or Windows created files.
TWO downsides (at least):
1) It is not a great idea to have your script so dependent upon the system its running on ( it may not be safe to assume Perl and wc are available )
2) Just a small mistake in escaping and you have handed over access to a shell on your machine.
As with most things I know (or think I know) about coding, I got this info from somewhere else:
John Reeve Article
public function quickAndDirtyLineCounter()
{
echo "<table>";
$folders = ['C:\wamp\www\qa\abcfolder\',
];
foreach ($folders as $folder) {
$files = scandir($folder);
foreach ($files as $file) {
if($file == '.' || $file == '..' || !file_exists($folder.'\\'.$file)){
continue;
}
$handle = fopen($folder.'/'.$file, "r");
$linecount = 0;
while(!feof($handle)){
if(is_bool($handle)){break;}
$line = fgets($handle);
$linecount++;
}
fclose($handle);
echo "<tr><td>" . $folder . "</td><td>" . $file . "</td><td>" . $linecount . "</td></tr>";
}
}
echo "</table>";
}
I use this method for purely counting how many lines in a file. What is the downside of doing this verses the other answers. I'm seeing many lines as opposed to my two line solution. I'm guessing there's a reason nobody does this.
$lines = count(file('your.file'));
echo $lines;
this is a bit late but...
Here is my solution for a text log file I have which uses \n to separate each line.
$data = file_get_contents("myfile.txt");
$numlines = strlen($data) - strlen(str_replace("\n","",$data));
It does load the file into memory but doesn't need to cycle through an unknown number of lines. It may be unsuitable if the file is GB in size but for smaller files with short lines of data it works a treat for me.
It just removes the "\n" from the file and compares how many have been removed by comparing the length of the data in the file to the length after removing all the line breaks ("\n" chars n my case). If your line delineator is a different char, replace the "\n" with whatever is your line delineation character.
I know it is not the best answer for all occasions but is something I have found quick and simple for my purposes where each line of the log is only a few hundred chars and total log file is not too large.
For just counting the lines use:
$handle = fopen("file","r");
static $b = 0;
while($a = fgets($handle)) {
$b++;
}
echo $b;