SplFileObject + LimitIterator + offset - php
I have data file with two lines (two lines just for my example, in real, that file can contain millions of lines) and I use SplFileObject and LimitIterator with offseting. But this combination have strange behaviour in some cases:
$offset = 0;
$file = new \SplFileObject($filePath);
$fileIterator = new \LimitIterator($file, $offset, 100);
foreach ($fileIterator as $key => $line) {
echo $key;
}
Output is: 01
But with $offset set to 1, output is blank (foreach doesn't iterate any line).
My data file contain this:
{"generatedAt":1434665322,"numRecords":"1}
{"id":"215255","code":"NB000110"}
What I'm doing wrong?
Thanks
Required:
Use SplFileObject to process a number of records from:
a given start record number
for a given number of records or until EOF.
The issue is that SplFileObject gets confused as regards the last record in the file. This prevents it working correctly in foreach loops.
This code uses the SplFileObject and 'skip records' and 'processes records'. Alas, It cannot use foreach loops.
Skip a number of records from the start of the file ($offset).
Process a given number of records or unit the end of file ($recordsToProccess)
The code:
<?php
$filePath = __DIR__ . '/Q30932555.txt';
// $filePath = __DIR__ . '/Q30932555_1.txt';
$offset = 1;
$recordsToProcess = 100;
$file = new \SplFileObject($filePath);
// skip the records
$file->seek($offset);
$recordsProcessed = 0;
while ( ($file->valid() || strlen($file->current()) > 0)
&& $recordsProcessed < $recordsToProcess
) {
$recordsProcessed++;
echo '<br />', 'current: ', $file->key(), ' ', $file->current();
$file->next();
}
Reading the related PHP bug 65601 suggests adding the READ_AHEAD flag will fix this. Tested and works as you expected it to.
$offset = 0;
$file = new \SplFileObject($filePath);
$file->setFlags(SplFileObject::READ_AHEAD);
$fileIterator = new \LimitIterator($file, $offset, 100);
foreach ($fileIterator as $key => $line) {
echo $key;
}
Related
looping through txt file to use specific part of a string
I am new to Php and can't seem to figure this out no matter how much I've googled. So I've opened the txt file (which consists of multiple lines of this type of string unique Identifier IMEI in bold: Rx:00:39:54 06/09/2015:+RESP:GTEPS,210101,863286020022449,,8296,01,1,3,0.0,0,1031.1,29.367950,-30.799161,20150906003710,,,,,,2857.9,20150906003710,8038$) There are different strings with different IMEIs but i only want to use a specific one. My question is, how do I extract/only use the string with the same Unique identifier and then loop through those to use in another function? My function has different cases and each case has different calculations, so I'll need to loop through the txt file (with e.g. 863286020022449 as Identifier, ignoring other identifiers/IMEIs) in order to use the string in my function as below: This is my starter function: function GetParam($unknownFunction, $numberCommas) { $returnString = ""; $foundSting = 0; $numberFound = 0; $len = strlen($unknownFunction); for ($i = 0; $i < $len; ++$i) { if ($Rawline[$i] == ",") { ++$numberFound; if ($numberFound > $numberCommas) break; if ($numberFound == $numberCommas) $foundSting = 1; } else if ($foundSting == 1) { $returnString .= $unknownFunction[$i]; } } return $returnString; echo $returnString; } $i = strpos($unknownFunction, ":GT"); $p = substr($unknownFunction, $i+3,3); $Protocol = GetParam($unknownFunction, 1); //this switch reads the differences in the message types (e.g. HBD- in this case is a heartbeat message type and would thus have a different amount of commas in the string and has different definitions of the characters within the commas) switch ($p) { case 'HBD': //+ACK:GTHBD,220100,135790246811220,,20100214093254,11F0$ //This is an example of an HBD message $result2["Type"] = 'Heart beat'; $IMEI = GetParam($unknownFunction, 2); $mDate = GetParam($unknownFunction, 4); $mDate = substr($mDate,0,4).'-'.substr($mDate,4,2).'- '.substr($mDate,6,2).' '.substr($mDate,8,2).':'.substr($mDate,10,2).':'.substr($mDate,12,2); break; This is the biggest problem I am facing at the moment and when I print the different lines, it indicates the correct IMEI but it does not loop through the whole file to use each string that belongs to that IMEI. Your assistance would be greatly appreciated. Thank you so much. Example of input file: Rx:00:00:00 28/02/2018:+RESP:GTFRI,3C0103,862045030241360,,14067,11,1,1,29.7,320,151.1,30.949307,-29.819685,20180227235959,0655,0001,013A,87B6,00,35484.1,01500:51:31,,,100,220101,,,,20180228000000,3461$ Rx:00:00:01 28/02/2018:+RESP:GTERI,380201,869606020047340,gv65,00000002,14076,10,1,1,119.0,119,24.3,18.668516,-34.016808,20180227235955,0655,0001,00F7,2DC9,00,98912.0,02235:20:25,0,100,220101,0,0,20180227235958,FF20$ Rx:00:00:03 28/02/2018:+RESP:GTERI,380201,869606020162990,,00000002,12912,10,1,1,0.0,230,1127.3,30.846671,-27.674206,20180227235956,0655,0001,013E,88B0,00,106651.1,03546:44:42,0,100,210101,0,0,20180227235959,6190$ Rx:00:00:03 28/02/2018:+ACK:GTHBD,450102,865084030005340,gb100,20180228000003,CC61$ Rx:00:00:03 28/02/2018:+RESP:GTERI,380201,869606020115980,,00000002,13640,10,1,1,12.1,353,1663.1,28.580726,-28.162208,20180227235957,,,,,,37599.6,02422:07:24,0,100,220101,0,0,20180228000000,1937$ Rx:00:00:04 28/02/2018:+RESP:GTERI,380502,869606020276840,gv65,00000002,12723,10,1,1,0.0,106,1232.8,22.878013,-27.951762,20180227235952,0655,0001,0204,63C5,00,13808.9,00778:32:20,0,100,210100,0,0,20180228000002,2C50$ Rx:00:00:04 28/02/2018:+RESP:GTERI,380502,869606020274530,gv65,00000002,12683,10,1,1,0.0,91,1213.7,24.863444,-28.174319,20180227235956,0655,0001,0203,69F1,00,9753.2,00673:49:21,0,100,210100,0,0,20180228000003,8AC7$ Rx:00:00:05 28/02/2018:+ACK:GTHBD,380201,863286023083810,,20180228000003,0D87$ Rx:00:00:06 28/02/2018:+RESP:GTFRI,3C0103,862045030241360,,14086,10,1,1,34.0,327,152.0,30.949152,-29.819501,20180228000002,0655,0001,013A,87B6,00,35484.1,01500:51:36,,,100,220101,,,,20180228000005,3462$ Rx:00:00:06 28/02/2018:+ACK:GTHBD,060228,862894021626380,,20180228000007,F9A5$ Rx:00:00:07 28/02/2018:+RESP:GTERI,380201,869606020019430,,00000002,12653,10,1,1,0.0,219,1338.7,26.882063,-28.138099,20180228000002,,,,,,86473.7,05645:48:34,0,93,210101,0,0,20180228000003,0FA5$ Rx:00:00:09 28/02/2018:+ACK:GTHBD,380502,869606020233940,gv65,20180228000008,7416$ Rx:00:00:10 28/02/2018:+RESP:GTAIS,380201,869606020171710,,11,11,1,1,0.0,95,281.2,30.855164,-29.896575,20180228000009,0655,0001,0156,9A9F,00,156073.7,20180228000008,F9A4$ Each GT message means something which is why i need to extract only one specific IMEI and use the result in my function as a breakdown of what every set of numbers between the commas actually mean. The end result needs to be populated in an excel spreadsheet but that's a different issue.
Nested foreach, keeping tracking of the IMEIs you've already gone through. Or something like this. <?php $filename = 'info.txt'; $contents = file($filename); foreach ($contents as $line) { $doneAlreadyArray = array(); $IMEI = GetParam($line, 2); foreach ($contents as $IMEIline){ $thisIMEI = GetParam($IMEIline,2); //check if already done the IMEI previously if (!in_array($thisIMEI, $doneAlreadyArray)){ //matching IMEIs? if ($thisIMEI == $IMEI){ //run new function with entire $IMEIline new_function($IMEIline); } } } //add IMEI to doneAlreadyArray array_push($doneAlreadyArray,$IMEI); } ?>
If I've understood your question right and you want to extract the string(line) with the same Unique identifier, this may be useful for your needs as a strating point. The example is very basic, and use data from your question: <?php // Read the file. $filename = 'input.txt'; $file = file($filename, FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES); // Each item of $output will contain an array of lines: $output = array(); foreach ($file as $row) { $a = explode(',', $row); $imei = $a[2]; if (!array_key_exists($imei, $output)) { $output[$imei] = array(); } $output[$imei][] = $row; } // Then do what you want ... foreach ($output as $key=>$value) { echo 'IMEI: '.$key.'</br>'; foreach($value as $row) { // Here you can call your functions. I just echo the row: echo $row.'</br>'; } } ?>
thank you for the feedback. Ryan Dewberry ended up helping me. The fix was simpler than I thought too :) //Unknownfunction is now $line function GetParam($line, $numberCommas) { $returnString = ""; $foundSting = 0; $numberFound = 0; $len = strlen($line); for ($i = 0; $i < $len; ++$i) { if ($line[$i] == ",") { ++$numberFound; if ($numberFound > $numberCommas) break; if ($numberFound == $numberCommas) $foundSting = 1; } else if ($foundSting == 1) { $returnString .= $line[$i]; } } return $returnString; // print $returnString; } //this is new - makes sure I use the correct IMEI $contents = file($fileName); foreach ($contents as $line){ $haveData = 0; $IMEI = GetParam($line, 2); if ($IMEI == $gprsid){ $i = strpos($line, ":GT"); $p = substr($line, $i+3,3); $Protocol = GetParam($line, 1); //this is the part I struggled with as well - This is an array of all of my //calculation //results and in printing it out I can see that everything is working $superResult = array(); array_push($superResult,$result2); print_r($superResult); } } Much appreciated. Thank you!
PHP display random n images from directory
I want to display random n number of images from a folder. Currently i am using this script to display images <?php $dir = './images/gallery/'; foreach(glob($dir.'*.jpg') as $file) { ?> <div class="item"><img src="<?php=$file;?>"></div> <?php } ?> I want only 10 (or n number) images, that too randomly. How to do this?
The shuffle() method will put the elements of a given array in a random order: <?php $dir = './images/gallery/'; function displayImgs($dir, $n=10){ $files = glob($dir.'*.jpg'); shuffle($files); $files = array_slice($files, 0, $n); foreach($files as $file) { ?> <div class="item"><img src="<?php=$file;?>"></div> <?php } } ?> Usage: displayImgs("/dir/temp/path", 20);
Well, this might be overkill, but you can also use a directory iterator and some randomness to achieve this. I used a modified version of the random numbers generation function from this answer. make sure that the path you give to the function is relative to the directory in which the script resides, with a slash at the beginning. The __DIR__ constants will not change would you happen to call this script from different places in your file hierarchy. <?php function randomImages($path,$n) { $dir = new DirectoryIterator(__DIR__. $path); // we need to know how many images we can range on // but we do not want the two special files . and .. $count = iterator_count($dir) - 2; // slightly modified function to create an array containing n random position // within our range $positionsArray = UniqueRandomNumbersWithinRange(0,$count-1,$n); $i = 0; foreach ($dir as $file) { // those super files seldom make good images if ($file->getFilename() === '.' || $file->getFilename() === '..') continue; if (isset($positionsArray[$i])) echo '<div class="item"><img src="'.$file->getPathname().'"></div>'; $i++; // change the count after the check of the filename, // because otherwise you might overflow } } function UniqueRandomNumbersWithinRange($min, $max, $quantity) { $numbers = range($min, $max); shuffle($numbers); return array_flip(array_slice($numbers, 0, $quantity)); }
Let us first create a array and push some random numbers into it. And as per you let $n be 10. $n = 10; $arr = array(); for($i = 1; $i <= $n; $i++){ /* Where $n is the limit */ $rand = rand($n); array_push($arr, $rand); } So now we have an array containing the random digits and now we have to echo out the images by iterating over the array: foreach($arr as $image){ $intToStr = (string) $image; foreach(glob($dir. $intToStr . '.jpg') as $file){ echo "<div class='item'>$file</div>"; } } This would echo out your images.
PHP CSVImporter
I had a script called CSVimporter V3 for PHP that I used to run on a website and it worked fine. A couple of years later I've now dug out the same script to use on another project, all works okay except the CSV files are being read as one long line, as opposed to header row and multiple lines. Here is part of the script. Any ideas why it would be being read as a long line? <?php // Reference session variable for short-hand $f = &$_SESSION['csv_file']; // Open file - fp = file pointer if (!$fp = #fopen($f, 'r')) { error(L_ERROR_PREVIEW_NO_FILE); } else { // Array to store each row to be inserted $batch = array(); // Row counter $rc = 0; // Work out starting row switch ($_SESSION['csv_first_row']) { case 'data': $start_row = 0; break; default: $start_row = 1; } // Get contents, while below preview limit and there's data to be read while ($data = fgetcsv($fp, 1024, delimiter_to_char($_SESSION['csv_delimiter']))) { if ($rc < $start_row) { // Incremement counter $rc++; // Go to next loop continue; } else { // Array to store data to be inputted $values = array(); // Loop data for ($i = 0; $i < count($data); $i++) { // If data is wanted, put data into array if (array_key_exists($i, $column_index)) { $values[$column_index[$i]] = $data[$i]; } } // Sort array into correct order by index ksort($values); // Join values together and store in array $batch[] = '("' . implode('", "', str_replace('"', '\"', $values)) . '","'.$accti.'","'.$impt.'")'; } } } // Close the file fclose($fp);
I added this at the top of the code and it all works now! ini_set('auto_detect_line_endings', true);
PHP_Count total lines in all files under a given folder
Just wanted to count total number of line from all the files from the folder. following php function helps me to count line num for only particular file. just wondering what is the way to cont total number of lines from the folder. $lines = COUNT(FILE($file)); Thank you.!
You could iterate the directory and count each file and sum them all. And you are using file() function, which will load the whole content into memory, if the file is very large, your php script will reach the memory limit of your config. If you could use external command, there is a solution with one line. (If you are using windows, just omit it.) $total = system("find $dir_path -type f -exec wc -l {} \; | awk '{total += $1} END{print total}'");
Same as one above (salathe's answer), except this one prints the number of lines (now in php7) rather than a bunch of error messages. $files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator(__DIR__)); $lines = 0; foreach ($files as $fileinfo) { if (!$fileinfo->isFile()) { continue; } $read = $fileinfo->openFile(); $read->setFlags(SplFileObject::READ_AHEAD); $lines += iterator_count($read) - 1; // -1 gives the same number as "wc -l" } echo ("Found :$lines");
Something like this perhaps: <?php $line_count = 0; if ($handle = opendir('some/dir/path')) { while (false !== ($entry = readdir($handle))) { if (is_file($entry)) { $line_count += count(file($entry)); } } closedir($handle); } var_dump($line_count); ?>
Check out the Standard PHP Library (aka SPL) for DirectoryIterator: $dir = new DirectoryIterator('/path/to/dir'); foreach($dir as $file ){ $x += (isImage($file)) ? 1 : 0; } (FYI there is an undocumented function called iterator_count() but probably best not to rely on it for now I would imagine. And you'd need to filter out unseen stuff like . and .. anyway.) or try this:-- see url :- http://www.brightcherry.co.uk/scribbles/php-count-files-in-a-directory/ $directory = "../images/team/harry/"; if (glob($directory . "*.jpg") != false) { $filecount = count(glob($directory . "*.jpg")); echo $filecount; } else { echo 0; }
A very basic example of counting the lines might look something like the following, which gives the same numbers as xdazz's answer. <?php $files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator(__DIR__)); $lines = $files = 0; foreach ($files as $fileinfo) { if (!$fileinfo->isFile()) { continue; } $files++; $read = $fileinfo->openFile(); $read->setFlags(SplFileObject::READ_AHEAD); $lines += iterator_count($read) - 1; // -1 gives the same number as "wc -l" } printf("Found %d lines in %d files.", $lines, $files); See also RecursiveDirectoryIterator SplFileInfo SplFileObject RecursiveIteratorIterator iterator_count()
Invalid argument supplied for foreach() + Logic of Script
I posted a similar question earlier today, however I recently downloaded xampp and can now get syntax errors. I'm very new to all of this. Please see this example to see the current state of my .php file: http://rich2233.comoj.com/file2.php I can get the .txt file to extract and display the content from the .txt file, but any attempt of modifying the .txt file does not happen. When I ran this on xampp, I got this error: Invalid argument supplied for foreach() in file2.php on line 30
After your most recent update of the code: The answer to the error message in the question title is: The code is missing the line $users = file("test.txt");
Here's a quick tip: keep track of your iterator. Confusing? Think of each row number as unique offset in the file. You need to iterate (foreach) and increment it by +1 each time. When you're editing a row later on, you just need to submit the offset number, iterate again and check if $i == $_POST['iterator']. // Iterator example $users = file('text.txt'); $hours_taken = array(); foreach ($users as $user) { list($time, $name) = explode('|', $user); array_push($hours_taken, $time); } // Assuming same order of rows in users.txt $hours = array('8:00 am', '9:00 am', '10:00 am'); // ...5:00 pm $i = 1; echo '<select name="time"><option selected>-- Select time --</option>'; foreach ($hours as $hour) { if (in_array($hour, $hours_taken)) { echo '<option disabled=disabled>'. $hour .'</option>'; } else { echo '<option value='. $i .'>'. $hour .'</option>'; } $i++; }
Edited as that last function wouldn't have worked function editTxt(){ $find_name = $_POST['name']; $find_time = $_POST['time']; $filename = getcwd() . "/test.txt"; $lines = file( $filename , FILE_IGNORE_NEW_LINES ); foreach($lines as $key => $line) { list($time, $name) = explode('|', $line); if($time == $find_time && $name == "Available") { $lines[$key] = $time."|".$find_name; file_put_contents( $filename , $lines ); break; } } } If you are planning to use this for something other than just an exercise I suggest using a database.
Try: $users = file($filename); I suspect $users === false as you aren't using the same "calculated" file name.