I'm not sure if this is possible, I've been googling for a solution... But, essentially, I have a very large file, the lines of which I want to store in an array. Thus, I'm using file(), but is there a way to do that in batches? So that every,say, 100 lines it creates, it "pauses"?
I think there's likely to be something I can do with a foreach loop or something, but I'm not sure that I'm thinking about it the right way...
Like
$i=0;
$j=0;
$throttle=100;
foreach($files as $k => $v) {
if($i < $j+$throttle && $i > $j) {
$lines[] = file($v);
//Do some other stuff, like importing into a db
}
$i++;
$j++;
}
But, I think that won't really work because $i & $j will always be equal... Anyway, feeling muddled... Can someone help me think a lil' clearer?
Read the file in line by line for however many lines you need, appending each line to an array. When the array gets to the desired length, process it, and empty the array. E.g.:
$handle = #fopen("/tmp/inputfile.txt", "r");
$throttle = 100;
$data = array();
if ($handle) {
while(!feof($handle)) {
$buffer = fgets($handle, 4096);
$data[] = $buffer;
if(count($data) == $throttle) {
doSomething($data);
$data = array();
}
}
fclose($handle);
}
You never incremented $i or $j... What you can do, is something like:
$data = array();
$chunk = 100;
$f = fopen($file, 'r');
while (!feof($f)) {
for ($i = 0; $i < $chunk; $i++) {
$tmp = fgets($f);
if ($tmp !== false) {
$data[] = $tmp;
} else {
//No more data, break out of the inner loop
break;
}
}
//Process your data
$data = array();
}
fclose($f);
If by "pause", you mean that you really want to pause execution of your script, use sleep or some of its variants: http://php.net/manual/en/function.sleep.php
Related
I got inspiration here to read line from specific line of file.
But when I tested it to get range of line from big file: I got 2 different result
Here's the benchmark result reading 100 lines from 10mb file:
Function v1 via file(): in 35ms with memory usage 12.00Mb
Function v2 via SplFileObject: in 956ms with memory usage 2.00Mb
My question, is there other way to do this so its fast like using file() but with low memory like using SplFileObject?
My current functions:
function get_line_content_range_v1($line_number_start, $line_number_end) {
$content = array();
$data = file('10mb.txt');
for($i = $line_number_start; $i <= $line_number_end; $i++) {
$content[] = $data[$i];
}
return $content;
}
function get_line_content_range_v2($line_number_start, $line_number_end) {
$content = array();
$file = new SplFileObject("10mb.txt", "r");
for($i = $line_number_start; $i <= $line_number_end; $i++) {
$file->seek($i);
$content[] = $file->current();
}
return $content;
}
Use a generator to save memory. There is no need to have all contents in RAM.
function get_line_content_range_v3($line_number_start, $line_number_end)
{
$filehandle = fopen('10mb.txt', 'r');
$line_number = 0;
while (++$line_number <= $line_number_end) {
$line = fgets($filehandle);
if ($line_number < $line_number_start) {
continue;
}
yield $line;
}
fclose($filehandle);
}
foreach (get_line_content_range_v3(12, 15) as $line) {
echo $line;
}
I couldn't find a solution to this. I'm sorry if this is a silly question.
I have 4 log files and I need to remove all log except last 10 lines.
I'm able to do it for 1 file but how to apply it on 4 files using once simple php code?
My current code:
<?php
$lines_array = file("log.txt");
$lines = count($lines_array);
$new_output = "";
for ($i=$lines - 10; $i < $lines; $i++) {
$new_output .= $lines_array[$i];
}
$filename = "log.txt";
file_put_contents($filename,$new_output);
What is the best way to achieve this?
Functional programming to the rescue:
function rotate(string $filename)
{
$lines_array = file($filename);
$lines = count($lines_array);
$new_output = "";
for ($i=$lines - 10; $i < $lines; $i++) {
$new_output .= $lines_array[$i];
}
file_put_contents($filename,$new_output);
}
rotate('log1.txt');
rotate('someOtherLog.txt');
rotate('third/log/file.txt);
//etc.
// or,
$logs = [
'log1.txt',
'someOtherLog.txt',
'third/log/file.txt'
];
foreach($logs as $file) {
rotate($file);
}
This allows you to write the code for rotating your logs one time, which makes your code better by being DRY (Don’t Repeat Yourself)
List your logfiles in an array, and loop over it, rewriting the log files as you go:
$logs = [
'log1.txt',
'log2.txt',
'log3.log'
];
foreach($logs as $log) {
// Only do this if we read the file.
if ($logData = file($log)) {
// array_slice takes a portion of the array
file_put_contents($log, array_slice($logData,-10));
}
}
Please try this code:
<?php
$fp = fopen("log.txt","ab+");
$data = fread($fp,filesize("log.txt"));
$data_array = explode("\n",$data);
$new_data = array();
for($i = count($data_array) - 1; $i >= count($data_array) - 10;$i--)
{
array_push($new_data , $data_array[$i]);
}
fclose($fp);
$new_data_array = array_reverse($new_data);
$data = implode("\n",$new_data_array);
$fp = fopen("log.txt","w");
fwrite($fp,$data);
fclose($fp);
?>
I'm trying to post 3 arrays using foreach and for some reason the break at the end isn't working and it outputs the whole list (40+) on to the page.
$file = fopen('names.csv', 'r');
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
shuffle($line);
$i = 0;
foreach ($line as $number) {
{
if($i==3){ break; } else {
$rtime = mt_rand(1, 7);
echo $number; }
$i++;
}
}
}
fclose($file);
This is kind of how it looks: take.ms/cLgIh, instead it should only show 3 of these usernames.
<?php
$i = 0;
//I have opened my contact.csv :P
$file = fopen('contact.csv', 'r');
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
shuffle($line);
foreach ($line as $number) {
{
if($i==3){ exit(); } else {
$rtime = mt_rand(1, 7);
echo "<br/> i = ".$i.$number.", "; }
}
$i++;
}
}
fclose($file);
?>
I have downloade first sample CSV from here:-http://www.sample-videos.com/download-sample-csv.php
And this code works for me:-
<?php
$file = fopen('SampleCSVFile_2kb.csv', 'r');
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
shuffle($line);
$i = 0;
foreach ($line as $number) {
if($i==3){
exit;
} else {
$rtime = mt_rand(1, 7);
echo $number.'<br/>';
echo $i.'<br/>'; // you can remove this line
}
$i++;
}
}
fclose($file);
?>
Output on each page refresh:-
http://prntscr.com/cln2ju
http://prntscr.com/cln2nf
Note:- if still not work then check your CSV file. May be it is corrupted.
Conclusion:- And after all discussion it comes to an end that your CSV file is corrupted. But yes code improvement is needed too
You need to increment $i otherwise it's value will always be 0
$i = 0;
foreach ($line as $number) {
$rtime = mt_rand(1, 7);
echo "$number";
if($i==3) break;
$i++;
}
Also you need to check if your while statement is closing
$file = fopen('names.csv', 'r');
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
shuffle($line);
$i = 0;
foreach ($line as $number) {
$rtime = mt_rand(1, 7);
echo $number;
if($i==3) break;
$i++;
}
} //check for this
As I can see in your code example it is missing the closing brace
Also, remove the double quotes from your $number, it's not necessary.
echo $number
You have a mistake is that the break are in the if($i==3) and you declared $i =0, but never increment this. So $i never arrive at 3.
I have a csv file that I would like to generate a summary report from. The csv looks like this :
The csv has in each row an activity and the coresponding time when it starts.
The summary I'm trying to generate has to look like this :
Basically I need to show each activity and the times when it starts and it ends
I did as following in PHP, I'm almost done but the result I get is not really what I want :
$csvFileName = "The csv path";
$report = array();
$file = fopen($csvFileName, "r");
while (($data = fgetcsv($file, 8000, "\n")) !== FALSE) {
$num = count($data);
for ($c = 0; $c < $num; $c++) {
$t = explode(',', $data[$c]);
$time = $t[0];
$activity = $t[1];
$report[] = array($activity, $time);
}
}
fclose($file);
//I'm reading the whole file content and copying it into an array.
$summaryReport = array();
$j = 1;
for($i=0; $i<sizeof($report); $i++){
if($report[$i][0] !== $report[$j][0]){
array_push($summaryReport,array($report[$i][0],$report[$i][1],$report[$j][1]));
}
$j++;
}
echo json_encode($summaryReport);
The output json looks like this :
[["Start","10:42","10:59"],["Driving route","11:10","11:50"],["Lunch-Rest Break","11:50","11:57"],["Driving route","11:57","12:03"],["Break","12:11","12:41"],["Driving route","13:05","14:09"],["Waiting","14:14","14:28"]]
What I'm looking for as result is something like that:
[["Start","10:42","10:59"],["Driving route","10:59","11:50"],["Lunch-Rest Break","11:50","11:57"],["Driving route","11:57","12:03"],["Break","12:03","12:41"],["Driving route","12:41","14:09"],["Waiting","14:09","14:28"],["End","14:28"]]
my coding logic is not really working well, does anyone see how can I do a simple loop to do what I'm looking for?
Thank you in advance.
The result can be achieved much easier. Look at my code, I got rid of all your inner loops, fixed syntax errors and there is no need to store the whole csv file in memory:
PHP code
<?php
$csvFileName = "./test.csv";
$file = fopen($csvFileName, "r");
$summaryReport = array();
$i = 0;
$previous_name = null;
while ($data = fgetcsv($file, 8000)) {
if ($previous_name !== $data[1])
{
$summaryReport[$i] = array($data[1], $data[0]);
if ($i > 0)
{
$summaryReport[$i-1][2] = $data[0];
}
$previous_name = $data[1];
++$i;
}
}
fclose($file);
echo json_encode($summaryReport);
Test csv file
10:41,Start
10:59,Driving
11:29,Driving
11:11,End
Output
[["Start","10:41","10:59"],["Driving","10:59","11:11"],["End","11:11"]]
I am attempting to assign the string returned by the fgets() function to an array in PHP. I have tried test strings and they work fine. I have also made sure that fgets() is returning items, but still no joy. Thinking that it may be a timing issue, I had the function run onload and that didn't work. My code is below; any help on this would be much appreciated.
function createDataArray()
{
global $resultsArray;
$i = 0;
$file = fopen("downloads/E0.csv","r");
while(! feof($file))
{
$line = fgets($file, 4096);
$resultsArray[$i] = $line; //This isn't working. Something is wrong with $line. It is a string, but it doesn't get assigned to the array.
$i = $i + 1;
}
fclose($file);
}
PLEASE return the array; do not use globals.
This fix should work:
function createDataArray()
{
$resultsArray = array();
$file = fopen("downloads/E0.csv","r");
while(! feof($file))
{
$line = fgets($file, 4096);
$resultsArray[] = $line;
}
fclose($file);
return $resultsArray;
}