How to apply code on multiple files at once? - php

I couldn't find a solution to this. I'm sorry if this is a silly question.
I have 4 log files and I need to remove all log except last 10 lines.
I'm able to do it for 1 file but how to apply it on 4 files using once simple php code?
My current code:
<?php
$lines_array = file("log.txt");
$lines = count($lines_array);
$new_output = "";
for ($i=$lines - 10; $i < $lines; $i++) {
$new_output .= $lines_array[$i];
}
$filename = "log.txt";
file_put_contents($filename,$new_output);
What is the best way to achieve this?

Functional programming to the rescue:
function rotate(string $filename)
{
$lines_array = file($filename);
$lines = count($lines_array);
$new_output = "";
for ($i=$lines - 10; $i < $lines; $i++) {
$new_output .= $lines_array[$i];
}
file_put_contents($filename,$new_output);
}
rotate('log1.txt');
rotate('someOtherLog.txt');
rotate('third/log/file.txt);
//etc.
// or,
$logs = [
'log1.txt',
'someOtherLog.txt',
'third/log/file.txt'
];
foreach($logs as $file) {
rotate($file);
}
This allows you to write the code for rotating your logs one time, which makes your code better by being DRY (Don’t Repeat Yourself)

List your logfiles in an array, and loop over it, rewriting the log files as you go:
$logs = [
'log1.txt',
'log2.txt',
'log3.log'
];
foreach($logs as $log) {
// Only do this if we read the file.
if ($logData = file($log)) {
// array_slice takes a portion of the array
file_put_contents($log, array_slice($logData,-10));
}
}

Please try this code:
<?php
$fp = fopen("log.txt","ab+");
$data = fread($fp,filesize("log.txt"));
$data_array = explode("\n",$data);
$new_data = array();
for($i = count($data_array) - 1; $i >= count($data_array) - 10;$i--)
{
array_push($new_data , $data_array[$i]);
}
fclose($fp);
$new_data_array = array_reverse($new_data);
$data = implode("\n",$new_data_array);
$fp = fopen("log.txt","w");
fwrite($fp,$data);
fclose($fp);
?>

Related

Read Specific Lines From Big File Fast with Low Memory Usage

I got inspiration here to read line from specific line of file.
But when I tested it to get range of line from big file: I got 2 different result
Here's the benchmark result reading 100 lines from 10mb file:
Function v1 via file(): in 35ms with memory usage 12.00Mb
Function v2 via SplFileObject: in 956ms with memory usage 2.00Mb
My question, is there other way to do this so its fast like using file() but with low memory like using SplFileObject?
My current functions:
function get_line_content_range_v1($line_number_start, $line_number_end) {
$content = array();
$data = file('10mb.txt');
for($i = $line_number_start; $i <= $line_number_end; $i++) {
$content[] = $data[$i];
}
return $content;
}
function get_line_content_range_v2($line_number_start, $line_number_end) {
$content = array();
$file = new SplFileObject("10mb.txt", "r");
for($i = $line_number_start; $i <= $line_number_end; $i++) {
$file->seek($i);
$content[] = $file->current();
}
return $content;
}
Use a generator to save memory. There is no need to have all contents in RAM.
function get_line_content_range_v3($line_number_start, $line_number_end)
{
$filehandle = fopen('10mb.txt', 'r');
$line_number = 0;
while (++$line_number <= $line_number_end) {
$line = fgets($filehandle);
if ($line_number < $line_number_start) {
continue;
}
yield $line;
}
fclose($filehandle);
}
foreach (get_line_content_range_v3(12, 15) as $line) {
echo $line;
}

Modify PHP script to read from CSV file

I have script which was reading two variable from CSV and those variable was at column 0 and column 5 and it was fine:
$file = fopen($qtyfile,"r");
output("reading file $qtyfile");
$i=0;
$imported = 0;
$failed = 0;
while(! feof($file))
{
$i++;
$line = (fgetcsv($file));
if($i==1) continue;
$cols = explode(';',$line[0]);
$pcode = $cols[0];
$stock = $cols[5];
The CSV file has been orginzed in different way, and the variable now at column 16 & 19 i tried to modify the code to the following but it's not working :
$file = fopen($qtyfile,"r");
output("reading file $qtyfile");
$i=0;
$imported = 0;
$failed = 0;
while(! feof($file))
{
$i++;
$line = (fgetcsv($file));
if($i==1) continue;
$cols = explode(';',$line[0]);
$pcode = $cols[16];
$stock = $cols[19];
can you please help to make the script ready from new column.
any help will be appreciated.
Link to full script and the csv:
Here's an awesome little function which will parse any csv file into an associative array with column titles from row 1 as array keys.
function csvArray($file) {
$csv = array_map('str_getcsv', file($file));
array_walk($csv, function(&$a) use ($csv) {
$a = array_combine($csv[0], $a);
});
array_shift($csv);
return $csv;
}
Usage:
$output = csvArray($file);
foreach ($output as $o) {
echo $o['pcode'];
// whatever
}
I have to deal with quite a lot of csv files and I use this all the time. Once the data is in an array it's much easier to deal with, and it doesn't matter if columns get moved around as long as the name stays the same it won't break your code.

PHP and csv report calculations

I have a csv file that I would like to generate a summary report from. The csv looks like this :
The csv has in each row an activity and the coresponding time when it starts.
The summary I'm trying to generate has to look like this :
Basically I need to show each activity and the times when it starts and it ends
I did as following in PHP, I'm almost done but the result I get is not really what I want :
$csvFileName = "The csv path";
$report = array();
$file = fopen($csvFileName, "r");
while (($data = fgetcsv($file, 8000, "\n")) !== FALSE) {
$num = count($data);
for ($c = 0; $c < $num; $c++) {
$t = explode(',', $data[$c]);
$time = $t[0];
$activity = $t[1];
$report[] = array($activity, $time);
}
}
fclose($file);
//I'm reading the whole file content and copying it into an array.
$summaryReport = array();
$j = 1;
for($i=0; $i<sizeof($report); $i++){
if($report[$i][0] !== $report[$j][0]){
array_push($summaryReport,array($report[$i][0],$report[$i][1],$report[$j][1]));
}
$j++;
}
echo json_encode($summaryReport);
The output json looks like this :
[["Start","10:42","10:59"],["Driving route","11:10","11:50"],["Lunch-Rest Break","11:50","11:57"],["Driving route","11:57","12:03"],["Break","12:11","12:41"],["Driving route","13:05","14:09"],["Waiting","14:14","14:28"]]
What I'm looking for as result is something like that:
[["Start","10:42","10:59"],["Driving route","10:59","11:50"],["Lunch-Rest Break","11:50","11:57"],["Driving route","11:57","12:03"],["Break","12:03","12:41"],["Driving route","12:41","14:09"],["Waiting","14:09","14:28"],["End","14:28"]]
my coding logic is not really working well, does anyone see how can I do a simple loop to do what I'm looking for?
Thank you in advance.
The result can be achieved much easier. Look at my code, I got rid of all your inner loops, fixed syntax errors and there is no need to store the whole csv file in memory:
PHP code
<?php
$csvFileName = "./test.csv";
$file = fopen($csvFileName, "r");
$summaryReport = array();
$i = 0;
$previous_name = null;
while ($data = fgetcsv($file, 8000)) {
if ($previous_name !== $data[1])
{
$summaryReport[$i] = array($data[1], $data[0]);
if ($i > 0)
{
$summaryReport[$i-1][2] = $data[0];
}
$previous_name = $data[1];
++$i;
}
}
fclose($file);
echo json_encode($summaryReport);
Test csv file
10:41,Start
10:59,Driving
11:29,Driving
11:11,End
Output
[["Start","10:41","10:59"],["Driving","10:59","11:11"],["End","11:11"]]

remove All lines except first 20 using php

how to remove every line except the first 20 using php from a text file?
If loading the entire file in memory is feasible you can do:
// read the file in an array.
$file = file($filename);
// slice first 20 elements.
$file = array_slice($file,0,20);
// write back to file after joining.
file_put_contents($filename,implode("",$file));
A better solution would be to use the function ftruncate which takes the file handle and the new size of the file in bytes as follows:
// open the file in read-write mode.
$handle = fopen($filename, 'r+');
if(!$handle) {
// die here.
}
// new length of the file.
$length = 0;
// line count.
$count = 0;
// read line by line.
while (($buffer = fgets($handle)) !== false) {
// increment line count.
++$count;
// if count exceeds limit..break.
if($count > 20) {
break;
}
// add the current line length to final length.
$length += strlen($buffer);
}
// truncate the file to new file length.
ftruncate($handle, $length);
// close the file.
fclose($handle);
For a memory efficient solution you can use
$file = new SplFileObject('/path/to/file.txt', 'a+');
$file->seek(19); // zero-based, hence 19 is line 20
$file->ftruncate($file->ftell());
Apologies, mis-read the question...
$filename = "blah.txt";
$lines = file($filename);
$data = "";
for ($i = 0; $i < 20; $i++) {
$data .= $lines[$i] . PHP_EOL;
}
file_put_contents($filename, $data);
Something like:
$lines_array = file("yourFile.txt");
$new_output = "";
for ($i=0; $i<20; $i++){
$new_output .= $lines_array[$i];
}
file_put_contents("yourFile.txt", $new_output);
This should work as well without huge memory usage
$result = '';
$file = fopen('/path/to/file.txt', 'r');
for ($i = 0; $i < 20; $i++)
{
$result .= fgets($file);
}
fclose($file);
file_put_contents('/path/to/file.txt', $result);

Using file() incrementally?

I'm not sure if this is possible, I've been googling for a solution... But, essentially, I have a very large file, the lines of which I want to store in an array. Thus, I'm using file(), but is there a way to do that in batches? So that every,say, 100 lines it creates, it "pauses"?
I think there's likely to be something I can do with a foreach loop or something, but I'm not sure that I'm thinking about it the right way...
Like
$i=0;
$j=0;
$throttle=100;
foreach($files as $k => $v) {
if($i < $j+$throttle && $i > $j) {
$lines[] = file($v);
//Do some other stuff, like importing into a db
}
$i++;
$j++;
}
But, I think that won't really work because $i & $j will always be equal... Anyway, feeling muddled... Can someone help me think a lil' clearer?
Read the file in line by line for however many lines you need, appending each line to an array. When the array gets to the desired length, process it, and empty the array. E.g.:
$handle = #fopen("/tmp/inputfile.txt", "r");
$throttle = 100;
$data = array();
if ($handle) {
while(!feof($handle)) {
$buffer = fgets($handle, 4096);
$data[] = $buffer;
if(count($data) == $throttle) {
doSomething($data);
$data = array();
}
}
fclose($handle);
}
You never incremented $i or $j... What you can do, is something like:
$data = array();
$chunk = 100;
$f = fopen($file, 'r');
while (!feof($f)) {
for ($i = 0; $i < $chunk; $i++) {
$tmp = fgets($f);
if ($tmp !== false) {
$data[] = $tmp;
} else {
//No more data, break out of the inner loop
break;
}
}
//Process your data
$data = array();
}
fclose($f);
If by "pause", you mean that you really want to pause execution of your script, use sleep or some of its variants: http://php.net/manual/en/function.sleep.php

Categories