I am reading the file and getting the particular line if there is a match for the searched string. There are bunch of strings to be searched which are stored in a array. I cant be opening the file every time when i loop through the array to get the string. But want to go to the first line of the file and start searching again. The file contains around 15k lines. If i open the file every time(inside the loop) its working fine. but if the open the file outside the loop. Only the first matched string line is returned.
$scheme_code =
array("106212","112422","114239","104685","100122","118191","131666");
foreach($scheme_code as $searchthis) {
$handle = #fopen("myfile", "r");
//DONT WANT TO DO THE ABOVE LINE FOR EVERY ITERATION
if ($handle)
{
//echo "handle open"."<br>";
while (!feof($handle))
{
$buffer = fgets($handle,4096);
if(strpos($buffer, $searchthis) !== FALSE){
$matches[] = $buffer;
}
}
}
}
But want to do something like this
$handle = #fopen("Myfile", "r");
foreach(){
// inside foreach
//go to the first line of the file
}
fclose($handle);
EDIT - I tried rewind(). I got the notice "rewind(): stream does not support seeking"
Here you can use file() function which will give you complete array of lines and after that you can match line by line without using your IO resource everytime by fopen.
<?php
$linesArray = file("/path/to/your/file.txt");
foreach($linesArray as $line){
// do the stuff or matching you want to perform line by line on $line
}
Related
The target is how to read a range of rows/lines from large CSV file into a JSON array in order to handle large files and read the data in pagination method, each page fetches a range of lines ( e.x. page number 1 fetch from line 1 to 10, page number 2 fetch from line 11 to line 20, and so and ).
the below PHP script read from the being CSV file to the desired line ($desired_line), My question is how we can determine the starting line to read from a specific line ($starting_line)
<?php
// php function to convert csv to json format
function csvToJson($fname, $starting_line, $desired_line) {
// open csv file
if (!($fp = fopen($fname, 'r'))) {
die("Can't open file...");
}
//read csv headers
$key = fgetcsv($fp,"1024","\t");
$line_counter = 0;
// parse csv rows into array
$json = array();
while (($row = fgetcsv($fp,"1024","\t")) && ($line_counter < $desired_line)) {
$json[] = array_combine($key, $row);
$line_counter++;
}
// release file handle
fclose($fp);
// encode array to json
return json_encode($json);
}
// Define the path to CSV file
$csv = 'file.csv';
print_r(csvToJson($csv, 20, 30));
?>
You should use functions like:
fgets() to read the file line by line
fseek() to move to the position of the last fgets() of the chunk
ftell() to read the position for fseek()
Something like this (it's only a schema):
<?php
...
$line_counter = 0;
$last_pos = ...
$fseek($fp,$last_pos);
while($line = fgets($fp)){ // read a line of the file
$line_counter++;
(...) // parse line of csv here
if($line_counter == 100){
$lastpos = ftell($fp);
(...) // save the $lastpos for next reading cycle
break;
}
}
...
?>
You can also skip the fseek() and ftell() part and just count the lines every time from the beginning, but that will generally have to go through the whole file from the beginning till the desired lines.
I need help processing files holding about 46k lines or more than 30MB of data.
My original idea was to open the file and turn each line into an array element. This worked the first time as the array held about 32k values total.
The second time, the process was repeated, the array only held 1011 elements, and finally, the third time it could only hold 100.
I'm confused and don't know much about the backend array processes. Can someone explain what is happening and fix the code?
function file_to_array($cvsFile){
$handle = fopen($cvsFile, "r");
$path = fread($handle, filesize($cvsFile));
fclose($handle);
//Turn the file into an array and separate lines to elements
$csv = explode(",", $path);
//Remove common double spaces
foreach ($csv as $key => $line){
$csv[$key] = str_replace(' ', '', str_getcsv($line));
}
array_filter($csv);
//get the row count for the file and array
$rows = count($csv);
$filerows = count(file($cvsFile)); //this no longer works
echo "File has $filerows and array has $rows";
return $csv;
}
The approach here can be split in 2.
Optimized file reading and processing
Proper storage solution
Optimized file processing can be done like so:
$handle = fopen($cvsFile, "r");
$rowsSucceed = 0;
$rowsFailed = 0;
if ($handle) {
while (($line = fgets($handle)) !== false) { // Reading file by line
// Process CSV line and check if it was parsed correctly
// And count as you go
if (!empty($parsedLine)) {
$csv[$key] = ... ;
$rowsSucceed++;
} else {
$rowsFailed++;
}
}
fclose($handle);
} else {
// Error handling
}
$totalLines = $rowsSucceed + $rowsFailed;
Also you can avoid array_filter() simply by not adding processed line if its empty.
It will allow to optimize memory usage during script execution.
Proper storage
Proper storage here is needed for performing operations on certain amount of data. File reading are ineffective and expensive. Using simple file based database like sqlite can help you a lot and increase overall performance of your script.
For this purpose you probably should process your CSV directly to database and than perform count operation on parsed data avoiding excessive file line counts etc.
Also it gives you further advantage on working with data not keeping it all in memory.
Your question says you want to "turn each line into an array element" but that is definitely not what you are doing. The code is quite clear; it reads the entire file into $path and then uses explode() to make one massive flat array of every element on every line. Then later you're trying to run str_getcsv() on each item, which of course isn't going to work; you've already exploded all the commas away.
Looping over the file using fgetcsv() makes more sense:
function file_to_array($cvsFile) {
$filerows = 0;
$handle = fopen($cvsFile, "r");
while ($line = fgetcsv($handle)) {
$filerows++;
// skip empty lines
if ($line[0] === null) {
continue;
}
//Remove common double spaces
$csv[] = str_replace(' ', '', $line);
}
//get the row count for the file and array
$rows = count($csv);
echo "File has $filerows and array has $rows";
fclose($handle);
return $csv;
}
My text file sample.txt. I want to exclude the first row from the text file and store the other rows into mysql database.
ID Name EMail
1 Siva xyz#gmail.com
2 vinoth xxx#gmail.com
3 ashwin yyy#gmail.com
Now I want to read this data from the text file except the first row(ID,name,email) and store into the MYsql db.Because already I have created a filed in database with the same name.
I have tried
$handle = #fopen($filename, "r"); //read line one by one
while (!feof($handle)) // Loop till end of file.
{
$buffer = fgets($handle, 4096); // Read a line.
}
print_r($buffer); // It shows all the text.
Please let me know how to do this?
Thanks.
Regards,
Siva R
It's easier if you use file() since it will get all rows in an array instead:
// Get all rows in an array (and tell file not to include the trailing new lines
$rows = file($filename, FILE_IGNORE_NEW_LINES);
// Remove the first element (first row) from the array
array_shift($rows);
// Now do what you want with the rest
foreach ($rows as $lineNumber => $row) {
// do something cool with the row data
}
If you want to get it all as a string again, without the first row, just implode it with a new line as glue:
// The rows still contain the line break, since we only trimmed the copy
$content = implode("\n", $rows);
Note: As #Don'tPanic pointed out in his comment, using file() is simple and easy but not advisable if the original file is large, since it will read the whole thing into memory as an array (and arrays take more memory than strings). He also correctly recommended the FILE_IGNORE_NEW_LINES-flag, just so you know :-)
You can just call fgets once before your while loop to get the header row out of the way.
$firstline = fgets($handle, 4096);
while (!feof($handle)) // Loop till end of file.
{ ...
I have a program that searches a text file to see if a certain string is in that file, which I have gotten to work fine. What I need to know is how to print a particular line of that text file. For example, if the file lists three street names each on its own line, and the program searches for one of them, I would want it to print out only the line that has that street name on it.
If the file looked like this: and the word being searched for was Rose Road,
I want it to only print out 6784 Rose Road
4543 Drock Drive
1254 HeadHill Road
6784 Rose Road
This is what I have so far, which checks if it's in the file, but I am just unsure how to print out a particular line.
$roadName = "Rose";
$handle = fopen("streets.txt", "r");
if(strpos(file_get_contents("streets.txt"),$roadName) !== false) //Checks to make sure its in the file.
{
echo fgets($handle); //This is what I was trying, but it only prints the 1st line.
}
file_get_contents and strpos have no effect on the file handle, so fgets just read from the beginning of the file.
You can read each line with fgets, test if it matches the string, and then print it.
while ($line = fgets($handler)) {
if (strpos($line, $roadName) !== false) {
echo $line;
break;
}
}
If the file is very large, this is better than the solution that uses file_get_contents, because it doesn't have to read the entire file into memory and then create a huge array of all the lines.
I would explode the lines into an array and the check every line:
$roadName = "Rose";
$file=file_get_contents("streets.txt")
if(strpos($file,$roadName) !== false) //Checks to make sure its in the file.
{
$lines = explode(PHP_EOL,$file);
foreach($lines as $line) {
if (strpos($line,$roadName) !== false) echo($line);
}
I am attempting to insert the data from an uploaded file into a single dimension array.
The file is as such that there is one student number to a line like so:
392232,392231,etc
this is the most common way I've found online:
while (($line = fgetcsv($file, 25, ',')) !== FALSE) {
//$line is an array of the csv elements
print_r($line);
}
However form what I understand this will create an array ($line) for each row. Which is not what I want.
that aside I tried this to see if it is working and my code is not printing out the array after using ftgetcsv(). The file is successfully uploading.
here is my code:
if(isset($_FILES['csv_file']) && is_uploaded_file($_FILES['csv_file']['tmp_name'])){
//create file name
$file_path = "csv_files/" . $_FILES['csv_file']['name'];
//move uploaded file to upload dir
if (!move_uploaded_file($_FILES['csv_file']['tmp_name'], $file_path)) {
//error moving upload file
echo "Error moving uploaded file";
}
print_r($_FILES['csv_file']);
$file = fopen('$file_path', 'r');
while (($line = fgetcsv($file, 25, ',')) !== FALSE) {
//$line is an array of the csv elements
print_r($line);
}
//delete csv file
unlink($file_path);
}
First off, can anyone obviously see why it wouldnt work to at least print them as seperate arrays of data (each row).
Second, is it possible to set it so that it creates a 1d array of all rows in the file?
Many thanks,
Question 1 is because of
print_r($_FILES['csv_file']);
$file = fopen('$file_path', 'r');
should be:
$file = fopen($file_path, 'r');
and for Question 2, check out the array_push
1st Question:
This line will actually try to open a file called '$file_path' because you're using single quotes (so it doesn't expand to the value of the variable). You can just remove the quotes.
$file = fopen('$file_path', 'r');
$file is null after this.
2nd Question:
If all you want to do is convert a file into an array by lines you can use one of these instead:
file() - get whole file into a 1D array of lines of the file (closest to what you want)
fgets() - get a string per line per call; keep calling this until it returns false to get each line one at a time
file_get_contents() - get the whole file into a string and process as you like
According to PHP.net $line has to return as array.
"returns an array containing the fields read."
But if you are sure it's contains only one student number you can use $line[0] to get the first line value (Ignoring the ",")
Here are some general comments on your code:
You are passing the file path into the fopen() function incorrectly. The variable should not be surrounded with single quotes.
Since you are deleting the CSV file after processing it, moving it is unnecessary. Simply use $_FILES['csv_file']['tmp_name'] as the path to the file.
Since there is only one entry per row in your CSV file, simply access the first element of the array that is returned from fgetcsv(): $numbers[] = $line[0];