Let's consider we got following format text file with a size around 1Gb:
...
li1
li2
li3
...
My task is to update line li2 to line2.
Following will not work:
$fd = fopen("file", 'c+');
// ..
// code that loops till we reach li2 line..
// ..
$offset = ftell($fd);
// ..
fseek($fd, $offset );
fwrite($fd, "line2" . PHP_EOL);
Since it produces:
...
li1
line2
3
...
I'm expecting to have as result:
...
li1
line2
li3
...
Thanks
In my opinion we need a temporary file here where we copy the non-changed result, add the change and continue with non-changed result. At the end the original file may be renamed and the temp file to take its name.
As I am familiar with file systems you cannot just delete/insert part of the file. Or it is just too complicated and it is not worthy.
Second - you can copy the file in the memory and make your modifications there but here you are resource dependent and your code may not pass on some servers (e.g. dedicated hosting).
Good luck!
If you know what you must change in text just use this:
$file_data=file_get_contents(some_name.txt);
$text = str_replace("line2", "li2", $file_data);
file_put_contents('some_name_changed.txt', $text);
Or read line by line and load in $text then replace what you want.
Related
What is the best way to overwrite a specific line in a file? I basically want to search a file for the string '#parsethis' and overwrite the rest of that line with something else.
If the file is really big (log files or something like this) and you are willing to sacrifice speed for memory consumption you could open two files and essentially do the trick Jeremy Ruten proposed by using files instead of system memory.
$source='in.txt';
$target='out.txt';
// copy operation
$sh=fopen($source, 'r');
$th=fopen($target, 'w');
while (!feof($sh)) {
$line=fgets($sh);
if (strpos($line, '#parsethis')!==false) {
$line='new line to be inserted' . PHP_EOL;
}
fwrite($th, $line);
}
fclose($sh);
fclose($th);
// delete old source file
unlink($source);
// rename target file to source file
rename($target, $source);
If the file isn't too big, the best way would probably be to read the file into an array of lines with file(), search through the array of lines for your string and edit that line, then implode() the array back together and fwrite() it back to the file.
Your main problem is the fact that the new line may not be the same length as the old line. If you need to change the length of the line, there is no way out of rewriting at least all of the file after the changed line. The easiest way is to create a new, modified file and then move it over the original. This way there is a complete file available at all times for readers. Use locking to make sure that only one script is modifying the file at once, and since you are going to replace the file, do the locking on a different file. Check out flock().
If you are certain that the new line will be the same length as the old line, you can open the file in read/write mode (use r+ as the second argument to fopen()) and call ftell() to save the position the line starts at each time before you call fgets() to read a line. Once you find the line that you want to overwrite, you can use fseek() to go back to the beginning of the line and fwrite() the new data. One way to force the line to always be the same length is to space pad it out to the maximum possible length.
This is a solution that works for rewriting only one line of a file in place with sed from PHP. My file contains only style vars and is formatted:
$styleVarName: styleVarProperty;\n
For this I first add the ":" to the ends of myStyleVarName, and sed replaces the rest of that line with the new property and adds a semicolon.
Make sure characters are properly escaped in myStyleVarProp.
$command = "pathToShellScript folder1Name folder2Name myStyleVarName myStyleVarProp";
shell_exec($command);
/* shellScript */
#!/bin/bash
file=/var/www/vhosts/mydomain.com/$1/$2/scss/_variables.scss
str=$3"$4"
sed -i "s/^$3.*/$str;/" $file
or if your file isn't too big:
$sample = file_get_contents('sample');
$parsed =preg_replace('##parsethis.*#', 'REPLACE TO END OF LINE', $sample);
You'll have to choose delimiters '#' that aren't present in the file though.
If you want to completely replace the contents of one file with the contents of another file you can use this:
rename("./some_path/data.txt", "./some_path/data_backup.txt");
rename("./some_path/new_data.txt", "./some_path/data.txt");
So in the first line you backup the file and in the second line you replace the file with the contents of a new file.
As far as I can tell the rename returns a boolean. True if the rename is successful and false if it fails. One could, therefore, only run the second step if the first step is successful to prevent overwriting the file unless a backup has been made successfully. Check out:
https://www.php.net/manual/en/function.rename.php
Hope that is useful to someone.
Cheers
Adrian
I'd most likely do what Jeremy suggested, but just for an alternate way to do it here is another solution. This has not been tested or used and is for *nix systems.
$cmd = "grep '#parsethis' " . $filename;
$output = system($cmd, $result);
$lines = explode("\n", $result);
// Read the entire file as a string
// Do a str_repalce for each item in $lines with ""
This question already has answers here:
PHP what is the best way to write data to middle of file without rewriting file
(3 answers)
Closed 6 years ago.
EDIT: For anyone else looking for an answer, I used this. Probably the fastest solution to the problem.
Suppose I have a very large test.txt file containing:
line1
line4
line5
line6
I wish to add line2 and line3 after line1. I use the following code to achieve this:
$file = fopen('test.txt', 'r+');
fseek($file, 5);
fwrite($file, "\r\nline2\r\nline3\r\n");
However, the txt file now becomes, with line4 and line5 being overwritten:
line1
line2
line3
line6
I get the same result with c+ mode in fopen. I can't use a or a+ as they just append content to the end of the file. I also can't read the whole file into a string and analyse it to make changes, as this file is really huge.
Any way to fix this problem?
Note: Any solutions that involve reading the whole file as a string and then making changes from there will not be feasible due to the large amount of content in the file. If there are any cheeky workarounds to add content to a few lines between the first and second line, it will be good enough. :D
Sadly, fopen and friends do not allow inserts, only appends. From their perspective, the file is a large, growable array. Therefore, there is no call to insert data, only to overwrite.
However, you can get around this in several ways. One easy way is to make a temporary file opened with a, fwrite line 1 into it from the original file, append lines 2 and 3, then write the rest of the original file into the temporary file, in chunks. Once you've transferred the whole file (without loading the whole thing into memory at once, notice), then you can move the temporary file on top of the original one.
You can do this by creating an array from each line. Then replace searched line with whatever you want.
$searchLine = 'line1';
$file = getcwd()."test.txt";
$lineArray = file( $file , FILE_IGNORE_NEW_LINES );
$lineArray[$searchLine] = $searchLine."\r\nline2\r\nline3\r\n"; // Replace the line with new ones
file_put_contents( $filename , implode( PHP_EOL, $lineArray) ); // Put the content
I do have two text files and want to loop through both files then combine both line (line 1 of first test file and line1 of second text file. like that for thousands of lines) and do some function
I am familiar with loop through one file and for that code is given below:
$lines = file('data.txt');
foreach ($lines as $line) {
//some function
}
but how will I do for two files and combine bothe lines?
Not sure what you mean by search through the table, but to open both files and do stuff with them:
$file1 = fopen("/path/to/file1.txt","r"); //Open file with read only access
$file2 = fopen("/path/to/file2.txt","r");
$combined = fopen("/path/to/combined.txt","w"); //in case you want to write the combined lines to a new file
while(!feof($file1) && !feof($file2))
{
$line1 = trim(fgets($file1)); //Grab a line of the first file, note the trim will clip off the carriage return/new line at the end of the line, can remove it if you don't need it.
$line2 = trim(fgets($file2)); //Grab a line of the second file
$combline = $line1 . $line2;
fwrite($combined,$combline . "\r\n"); //Write to new combined file, and add a new carriage return/newline at the end of the combined line to replace the one trimmed off.
//You can do whatever with data from $line1, $line2, or the combined $combline after getting them.
}
Note: You might run into trouble if you hit the end of file on one file before the other, which would only happen if they aren't the same length, might need some if control statements to set $line1 or $line2 to "" or something else if feof() their respective files, once both hit the end of file, the while loop will end.
You can do this programmatically as Crayon and Tim have shown. If both files have the same number of lines, it should work. If the line number is different you will have to loop over the larger file to make sure you get all lines or check EOF on both.
To combine line by line, I often use the unix command paste which is very fast. This also accounts for files with different lengths. Run this on the command line:
paste file1 file2 > output.txt
See the manpage for paste for command line options, field delimiters.
man paste
Example:
$file1 = fopen("file1.txt", "rb");
$file2 = fopen("file2.txt", "rb");
while (!feof($file1)) {
$combined = fread($file1, 8192) . " " . fread($file2, 8192);
// now insert $combined into db
}
fclose($file1);
fclose($file2);
you will want to use the longer of the two files in the while condition.
you may need to adjust the bytes read in fread depending on how long your lines are
change " " to whatever delimiter you want
I want to merge two large CSV files with PHP. This files are too big to even put into memory all at once. In pseudocode, I can think of something like this:
for i in file1
file3.write(file1.line(i) + ',' + file2.line(i))
end
But when I'm looping through a file using fgetcsv, it's not really clear how I would grab line n from a certain file without loading the whole thing into memory first.
Any ideas?
Edit: I forgot to mention that each of the two files has the same number of lines and they have a one-to-one relationship. That is, line 62,324 in file1 goes with line 62,324 in file2.
Not sure what operating system you're on, but if you're using Linux, using the paste command is probably a lot easier than trying to do this in PHP.
If this is a viable solution and you don't absolutely need to do it in PHP, you could try the following:
paste -d ',' file1 file2 > combined_file
Take a look at the fgets function. You could read a single line of each file, process them, and write them to your new file, then move on to the next line until you've reached the end of your file.
PHP: fgets
Specifically look at the example titled Example #1 Reading a file line by line in the PHP manual. It's also important to note the return value of the the fgets functions.
Returns a string of up to length - 1
bytes read from the file pointed to by
handle. If there is no more data to
read in the file pointer, then FALSE
is returned.
So, if it doesn't return FALSE you know you still have more lines to process.
You can use fgets().
$file1 = fopen('file1.txt', 'r');
$file2 = fopen('file2.txt', 'r');
$merged = fopen('merged.txt', 'w');
while (
($line1 = fgets($file1)) !== false
&& ($line2 = fgets($file2)) !== false) {
fwrite($merged, $line1 . ',' . $line2);
}
fgets() reads one line from a file. As you can see, this code uses it on both files at the same time, writing the merged lines to a third file. The manual here:
http://php.net/fgets
http://php.net/fopen
http://php.net/fwrite
Try using fgets() to read one line from each file at a time.
I think the solution for this is to map first line begins for each line ( and some kind of key if you need ) and then make a new csv using fread and fwrite ( we know beginning and ending of each line now , so we need just seek and read )
Another way is to put it into MySQL ( if it is possible ) and then back to new CSV
What is the best way to overwrite a specific line in a file? I basically want to search a file for the string '#parsethis' and overwrite the rest of that line with something else.
If the file is really big (log files or something like this) and you are willing to sacrifice speed for memory consumption you could open two files and essentially do the trick Jeremy Ruten proposed by using files instead of system memory.
$source='in.txt';
$target='out.txt';
// copy operation
$sh=fopen($source, 'r');
$th=fopen($target, 'w');
while (!feof($sh)) {
$line=fgets($sh);
if (strpos($line, '#parsethis')!==false) {
$line='new line to be inserted' . PHP_EOL;
}
fwrite($th, $line);
}
fclose($sh);
fclose($th);
// delete old source file
unlink($source);
// rename target file to source file
rename($target, $source);
If the file isn't too big, the best way would probably be to read the file into an array of lines with file(), search through the array of lines for your string and edit that line, then implode() the array back together and fwrite() it back to the file.
Your main problem is the fact that the new line may not be the same length as the old line. If you need to change the length of the line, there is no way out of rewriting at least all of the file after the changed line. The easiest way is to create a new, modified file and then move it over the original. This way there is a complete file available at all times for readers. Use locking to make sure that only one script is modifying the file at once, and since you are going to replace the file, do the locking on a different file. Check out flock().
If you are certain that the new line will be the same length as the old line, you can open the file in read/write mode (use r+ as the second argument to fopen()) and call ftell() to save the position the line starts at each time before you call fgets() to read a line. Once you find the line that you want to overwrite, you can use fseek() to go back to the beginning of the line and fwrite() the new data. One way to force the line to always be the same length is to space pad it out to the maximum possible length.
This is a solution that works for rewriting only one line of a file in place with sed from PHP. My file contains only style vars and is formatted:
$styleVarName: styleVarProperty;\n
For this I first add the ":" to the ends of myStyleVarName, and sed replaces the rest of that line with the new property and adds a semicolon.
Make sure characters are properly escaped in myStyleVarProp.
$command = "pathToShellScript folder1Name folder2Name myStyleVarName myStyleVarProp";
shell_exec($command);
/* shellScript */
#!/bin/bash
file=/var/www/vhosts/mydomain.com/$1/$2/scss/_variables.scss
str=$3"$4"
sed -i "s/^$3.*/$str;/" $file
or if your file isn't too big:
$sample = file_get_contents('sample');
$parsed =preg_replace('##parsethis.*#', 'REPLACE TO END OF LINE', $sample);
You'll have to choose delimiters '#' that aren't present in the file though.
If you want to completely replace the contents of one file with the contents of another file you can use this:
rename("./some_path/data.txt", "./some_path/data_backup.txt");
rename("./some_path/new_data.txt", "./some_path/data.txt");
So in the first line you backup the file and in the second line you replace the file with the contents of a new file.
As far as I can tell the rename returns a boolean. True if the rename is successful and false if it fails. One could, therefore, only run the second step if the first step is successful to prevent overwriting the file unless a backup has been made successfully. Check out:
https://www.php.net/manual/en/function.rename.php
Hope that is useful to someone.
Cheers
Adrian
I'd most likely do what Jeremy suggested, but just for an alternate way to do it here is another solution. This has not been tested or used and is for *nix systems.
$cmd = "grep '#parsethis' " . $filename;
$output = system($cmd, $result);
$lines = explode("\n", $result);
// Read the entire file as a string
// Do a str_repalce for each item in $lines with ""