I have multiple files. My script searches the file for the sequence name and the sequence. If it's found, the format is changed from gb to fasta and only the sequence name and sequence is kept and written back into the file. But sometimes the file doesn't contain a sequences name. In that case and I don't write anything into the file, the file is empty. This files should be removed because at the end of my script a multifasta is created from all these files.
# Find all gb files
$files = glob("*.gb");
foreach ($files as $filename){
$newname = basename($filename, ".gb"). ".fasta";
rename($filename, $newname);
$condition = false;
$lines = file($newname);
foreach($lines as $line) {
if (strstr($line, "ACCESSION") ) {
# Find the line containing the sequence name
$head = str_replace("ACCESSION ","",$line);
$final = "> " . $head;
# check if $head contains text
if ($head == ""){
$condition = true;
}
}
$sequence = trim($line);
# Find the sequence and check the condition
if (preg_match('/^\d/', $sequence) && $condition == false){
$sequence = preg_replace('/[0-9]+/', '', $sequence);
$sequence = preg_replace('/\s/',"",$sequence);
# Store in string
$out .= $sequence;
}
}
# Read lines into file
$f = fopen($newname, "w");
fwrite($f, $t);
fclose($f);
}
# Create multifasta
exec('for f in *fasta; do cat "$f"; echo; done > db', $return);
How could I best remove the file when it's empty so it wouldn't be inserted in the multifasta. I am sure it's something simple but I can't figure out how to do it.
The simplest way is to use the filesize command, I think:
if (filesize ( $filename) === 0){
unlink ($filename); //This will delete the file.
continue; //carry on with next file
}
The unlink command will generate an error message if it can't delete the file for whatever reason. I don't know if you need to check that.
Related
i got 35 second to execution this code. how to reduce the execution time? what should i change in this source code.
$file_handle = fopen("WMLG2_2017_07_11.log", "r");
while (!feof($file_handle)) {
$line = fgets($file_handle);
if (strpos($line, 'root#CLA-0 [WMLG2] >') !== false) {
$namafileA = explode('> ', $line);
$namafile = str_replace(' ', '_', $namafileA[1]);
$filenameExtension = $namafile.".txt";
$file = preg_replace('/[^A-Za-z0-9\-_.]/', '', $filenameExtension); // hapus special character kecuali "." dan "_"
} else {
$newfile = fopen("show_command_file_Tes2/$file", "a");
fwrite($newfile, $line);
}
}
fclose($file_handle);
I found some mistakes you did with the original code that could impact your performance, but I am not sure how much.
If I understand correctly, you are opening a log file and sorting the messages out to separate files.
You have not pasted an example from the log file, but I assume you have duplicate file targets, not every line of the log file has individual file targets.
Your code opens, but never closes the handles and it stays open during the script run. The file handles are not closing on outer-scope by garbage collector, you have to do it manually to release the resources.
Based on that you should store the file pointers (or at least close them) and re-use that handle what is already open. You are opening at least X line of handle during the execution and not closing it / reusing it where X is the line count in the file.
Other thing I noticed, your lines may be long ones, an that is a rare case where php's strpos() function could be slower than a regex matching the correct position of the string. Without the log file, I can't say for sure because preg_match() is pretty expensive function on simple / short strings (strpos() is way faster.)
If its a log file, most likely starts with that "root#CLA"... string, you should try to match that if you can specify the string position with ^ (begining of the string) or $ (end of string).
<?php
$file_handle = fopen("WMLG2_2017_07_11.log", "r");
//you 'll store your handles here
$targetHandles = [];
while (!feof($file_handle))
{
$line = fgets($file_handle);
if (strpos($line, 'root#CLA-0 [WMLG2] >') !== false)
{
$namafileA = explode('> ', $line);
$namafile = str_replace(' ', '_', $namafileA[1]);
$filenameExtension = $namafile . ".txt";
$file = preg_replace('/[^A-Za-z0-9\-_.]/', '', $filenameExtension); // hapus special character kecuali "." dan "_"
}
else
{
//no $file defined, most likely nothing to write yet
if (empty($file))
{
continue;
}
//if its not open, we'll make them open
if (empty($targetHandles[$file]))
{
$targetHandles[$file] = fopen("show_command_file_Tes2/$file", "a");
}
//writing the line to target
fwrite($targetHandles[$file], $line);
}
}
//you should close your handles every time
foreach ($targetHandles as $handle)
{
fclose($handle);
}
fclose($file_handle);
Im trying to make a Php file that receives nothing and checks every file on the folder, searching for a string inside them. it echos a array of filenames that have the string inside. Any way to do it, possibly with low memory usage?
Thank you a lot.
To achieve something like this, I recommend you read about the DirectoryIterator class, file_get_contents, and about strings in PHP.
Here is an example of how you can read the contents of a a given directory ($dir) and use strstr to search for a specific string occurrence in each file's contents ($contents):
<?php
$dir = '.';
if (substr($dir, -1) !== '/') {
$dir .= '/';
}
$matchedFiles = [];
$dirIterator = new \DirectoryIterator($dir);
foreach ($dirIterator as $item) {
if ($item->isDot() || $item->isDir()) {
continue;
}
$file = realpath($dir . $item->getFilename());
// Skip this PHP file.
if ($file === __FILE__) {
continue;
}
$contents = file_get_contents($file);
// Seach $contents for what you're looking for.
if (strstr($contents, 'this is what I am looking for')) {
echo 'Found something in ' . $file . PHP_EOL;
$matchedFiles[] = $file;
}
}
var_dump($matchedFiles);
There is some extra code in this example (adding a trailing slash to $dir, skipping dot files and directories, skipping itself, etc.) that I encourage you to read and learn about.
<?php
$folderPath = '/htdocs/stock/tae';
$searchString = 'php';
$cmd = "grep -r '$searchString' $folderPath";
$output = array();
$files = array();
$res = exec($cmd, $output);
foreach ($output as $line) {
$files[] = substr($line, 0, strpos($line, ':'));
}
print_r($files);
I have a csv file with multiple columns, in some of these columns there are some HTML tags that looks like StreamHandler.ashx?SubscriptionID=6348. Then there is a folder that contains all images renamed with ID.Extension i.e. 6348.jpg.
I would like to create a script that searches for StreamHandler.ashx?SubscriptionID=6348 and replaces it with http://newdomain.com/images/6348.jpg.
Note that not all the files are .jpg so the extension needs to be checked when the file is found.
$file = fopen("images.csv", "r");
$lines = array();
while (($line = fgetcsv($file)) !== FALSE) {
//$line is an array of the csv elements
$lines[] = $line;
}
foreach ($lines as $line => $data) {
$data = preg_replace_callback('/StreamHandler\.ashx\?SubscriptionID=([0-9]+)/', function($matches) {
$img = $matches[1]; //get the filename
$img = glob("/Users/sandro/Sites/test/destination/" . $img . ".*"); //find the file in the fileserver (in the current directory)
$img[0] = str_replace ( "/Users/sandro/Sites/test/destination/", 'http://newdomain.com/images/', $img[0] );
if( isset($img[0]) ) { //was there a match?
return $img[0]; //replace
}
return $matches[0]; //dont replace because file doesnt exist
}, $data);
print_r($data);
}
fclose($file);
I've written the part to open and read the csv file but the search is still missing. Any thoughts?
Thanks
You can do a preg_replace_callback to find the string you want to search, then look the file extension up, and replace.
Find matches to replace
See if the file exists by looking for the filename
If it exists, replace string
If it doesn't exist, keep current string
$csv = preg_replace_callback('/StreamHandler\.ashx\?SubscriptionID=([0-9]+)/', function($matches) {
$file = $matches[1]; //get the filename
$file = glob($file .".*"); //find the file in the fileserver (in the current directory)
if( isset($file[0]) ) { //was there a match?
return 'http://newdomain.com/images/'. $file[0]; //replace
}
return $matches[0]; //dont replace because file doesnt exist
}, $csv);
Example
I have the file 6348.png.
My csv file holds: StreamHandler.ashx?SubscriptionID=6348,StreamHandler.ashx?SubscriptionID=6349,StreamHandler.ashx?SubscriptionID=635,StreamHandler.ashx?SubscriptionID=64
The output:
http://newdomain.com/images/6348.png,StreamHandler.ashx?SubscriptionID=6349,StreamHandler.ashx?SubscriptionID=635,StreamHandler.ashx?SubscriptionID=64
I need a script that is finding and then replacing a sertain line in a CSV like file.
The file looks like this:
18:110327,98414,127500,114185,121701,89379,89385,89382,92223,89388,89366,89362,89372,89369
21:82297,79292,89359,89382,83486,99100
98:110327,98414,127500,114185,121701
24:82297,79292,89359,89382,83486,99100
Now i need to change the line 21.
This is wat i got so far.
The first 2 to 4 digits folowed by : ar a catergory number. Every number after this(followed by a ,) is a id of a page.
I acces te id's i want (i.e. 82297 and so on) from database.
//test 2
$sQry = "SELECT * FROM artikelen WHERE adviesprijs <>''";
$rQuery = mysql_query ($sQry);
if ( $rQuery === false )
{
echo mysql_error ();
exit ;
}
$aResult = array ();
while ( $r = mysql_fetch_assoc ($rQuery) )
{
$aResult[] = $r['artikelid'];
}
$replace_val_dirty = join(",",$aResult);
$replace_val= "21:".$replace_val_dirty;
// file location
$file='../../data/articles/index.lst';
// read the file index.lst
$file1 = file_get_contents($file);
//strip eerde artikel id van index.lst
$file3='../../data/articles/index_grp21.lst';
$file3_contents = file_get_contents($file3);
$file2 = str_replace($file3_contents, $replace_val, $file1);
if (file_exists($file)) {
echo "The file $filename exists";
} else {
echo "The file $filename does not exist";
}
if (file_exists($file3)) {
echo "The file $filename exists";
} else {
echo "The file $filename does not exist";
}
// replace the data
$file_val = $file2;
// write the file
file_put_contents($file, $file_val);
//write index_grp98.lst
file_put_contents($file3, $replace_val);
mail('info#', 'Aanbieding catergorie geupdate', 'Aanbieding catergorie geupdate');
Can anyone point me in the right direction to do this?
Any help would be appreciated.
You need to open the original file and go through each line. When you find the line to be changed, change that line.
As you can not edit the file while you do that, you write a temporary file while doing this, so you copy over line-by-line and in case the line needs a change, you change that line.
When you're done with the whole file, you copy over the temporary file to the original file.
Example Code:
$path = 'file';
$category = 21;
$articles = [111182297, 79292, 89359, 89382, 83486, 99100];
$prefix = $category . ':';
$prefixLen = strlen($prefix);
$newLine = $prefix . implode(',', $articles);
This part is just setting up the basics: The category, the IDs of the articles and then building the related strings.
Now opening the file to change the line in:
$file = new SplFileObject($path, 'r+');
$file->setFlags(SplFileObject::DROP_NEW_LINE | SplFileObject::SKIP_EMPTY);
$file->flock(LOCK_EX);
The file is locked so that no other process can edit the file while it gets changed. Next to that file, the temporary file is needed, too:
$temp = new SplTempFileObject(4096);
After setting up the two files, let's go over each line in $file and compare if it needs to be replaced:
foreach ($file as $line) {
$isCategoryLine = substr($line, 0, $prefixLen) === $prefix;
if ($isCategoryLine) {
$line = $newLine;
}
$temp->fwrite($line."\n");
}
Now the $temporary file contains already the changed line. Take note that I used UNIX type of EOF (End Of Line) character (\n), depending on your concrete file-type this may vary.
So now, the temporary file needs to be copied over to the original file. Let's rewind the file, truncate it and then write all lines again:
$file->seek(0);
$file->ftruncate(0);
foreach ($temp as $line) {
$file->fwrite($line);
}
And finally you need to lift the lock:
$file->flock(LOCK_UN);
And that's it, in $file, the line has been replaced.
Example at once:
$path = 'file';
$category = 21;
$articles = [111182297, 79292, 89359, 89382, 83486, 99100];
$prefix = $category . ':';
$prefixLen = strlen($prefix);
$newLine = $prefix . implode(',', $articles);
$file = new SplFileObject($path, 'r+');
$file->setFlags(SplFileObject::DROP_NEW_LINE | SplFileObject::SKIP_EMPTY);
$file->flock(LOCK_EX);
$temp = new SplTempFileObject(4096);
foreach ($file as $line) {
$isCategoryLine = substr($line, 0, $prefixLen) === $prefix;
if ($isCategoryLine) {
$line = $newLine;
}
$temp->fwrite($line."\n");
}
$file->seek(0);
$file->ftruncate(0);
foreach ($temp as $line) {
$file->fwrite($line);
}
$file->flock(LOCK_UN);
Should work with PHP 5.2 and above, I use PHP 5.4 array syntax, you can replace [111182297, ...] with array(111182297, ...) in case you're using PHP 5.2 / 5.3.
I have a file named $dir and a string named $line, I know that this string is a complete line of that file but I don't know its line number and I want to remove it from file, what should I do?
Is it possible to use awk?
$contents = file_get_contents($dir);
$contents = str_replace($line, '', $contents);
file_put_contents($dir, $contents);
Read the lines one by one, and write all but the matching line to another file. Then replace the original file.
this will just look over every line and if it not what you want to delete, it gets pushed to an array that will get written back to the file. see this
$DELETE = "the_line_you_want_to_delete";
$data = file("./foo.txt");
$out = array();
foreach($data as $line) {
if(trim($line) != $DELETE) {
$out[] = $line;
}
}
$fp = fopen("./foo.txt", "w+");
flock($fp, LOCK_EX);
foreach($out as $line) {
fwrite($fp, $line);
}
flock($fp, LOCK_UN);
fclose($fp);
It can be solved without the use of awk:
function remove_line($file, $remove) {
$lines = file($file, FILE_IGNORE_NEW_LINES);
foreach($lines as $key => $line) {
if($line === $remove) unset($lines[$key]);
}
$data = implode(PHP_EOL, $lines);
file_put_contents($file, $data);
}
Another approach is to read the file line by line until you find a match, then truncate the file to that point, and then append the rest of the lines.
This is also good if you're looking for a substring (ID) in a line and want to replace the old line with the a new line.
Code:
$contents = file_get_contents($dir);
$new_contents = "";
if (strpos($contents, $id) !== false) { // if file contains ID
$contents_array = explode(PHP_EOL, $contents);
foreach ($contents_array as &$record) { // for each line
if (strpos($record, $id) !== false) { // if we have found the correct line
continue; // we've found the line to delete - so don't add it to the new contents.
} else {
$new_contents .= $record . "\r"; // not the correct line, so we keep it
}
}
file_put_contents($dir, $new_contents); // save the records to the file
echo json_encode("Successfully updated record!");
}
else {
echo json_encode("failed - user ID ". $id ." doesn't exist!");
}
Example:
input: "123,student"
Old file:
ID,occupation
123,student
124,brick layer
Running the code will change file to:
New file:
ID,occupation
124,brick layer
All answeres here have in common, that they load the complete file into the memory. Here is an implementation of removing one (or more) line(s) without coyping the files content into a variable.
The idea is to iterate over the files lines. If a line should be removed, the lines length is added to the $byte_offset. The next line is then moved $byte_offset bytes "upwards". This is done with all following lines. If all lines are processed, the files last $byte_offset bytes are removed.
I guess that this is faster for bigger files because nothing is copied. And I guess that at some file size the other answers do not work at all while this one should. But I didn't test it.
Usage:
$file = fopen("path/to/file", "a+");
// remove lines 1 and 2 and the line containing only "line"
fremove_line($file, 1, 2, "line");
fclose($file);
The code of the fremove_line() function:
/**
* Remove the `$lines` by either their line number (as an int) or their content
* (without trailing new-lines).
*
* Example:
* ```php
* $file = fopen("path/to/file", "a+"); // must be opened writable
* // remove lines 1 and 2 and the line containing only "line"
* fremove_line($file, 1, 2, "line");
* fclose($file);
* ```
*
* #param resource $file The file resource opened by `fopen()`
* #param int|string ...$lines The one-based line number(s) or the full line
* string(s) to remove, if the line does not exist, it is ignored
*
* #return boolean True on success, false on failure
*/
function fremove_line($file, ..$lines): bool{
// set the pointer to the start of the file
if(!rewind($file)){
return false;
}
// get the stat for the full size to truncate the file later on
$stat = fstat($file);
if(!$stat){
return false;
}
$current_line = 1; // change to 0 for zero-based $lines
$byte_offset = 0;
while(($line = fgets($file)) !== false){
// the bytes of the lines ("number of ASCII chars")
$line_bytes = strlen($line);
if($byte_offset > 0){
// move lines upwards
// go back the `$byte_offset`
fseek($file, -1 * ($byte_offset + $line_bytes), SEEK_CUR);
// move the line upwards, until the `$byte_offset` is reached
if(!fwrite($file, $line)){
return false;
}
// set the file pointer to the current line again, `fwrite()` added `$line_bytes`
// already
fseek($file, $byte_offset, SEEK_CUR);
}
// remove trailing line endings for comparing
$line_content = preg_replace("~[\n\r]+$~", "", $line);
if(in_array($current_line, $lines, true) || in_array($line_content, $lines, true)){
// the `$current_line` should be removed so save to skip the number of bytes
$byte_offset += $line_bytes;
}
// keep track of the current line
$current_line++;
}
// remove the end of the file
return ftruncate($file, $stat["size"] - $byte_offset);
}
Convert text to array, remove first line and reconvert to text
$line=explode("\r\n",$text);
unset($line[0]);
$text=implode("\r\n",$line);
I think the best way to work with files is to act them like strings:
/**
* Removes the first found line inside the given file.
*
* #param string $line The line content to be searched.
* #param string $filePath Path of the file to be edited.
* #param bool $removeOnlyFirstMatch Whether to remove only the first match or
* the whole matches.
* #return bool If any matches found (and removed) or not.
*
* #throw \RuntimeException If the file is empty.
* #throw \RuntimeException When the file cannot be updated.
*/
function removeLineFromFile(
string $line,
string $filePath,
bool $removeOnlyFirstMatch = true
): bool {
// You can wrap it inside a try-catch block
$file = new \SplFileObject($filePath, "r");
// Checks whether the file size is not zero
$fileSize = $file->getSize();
if ($fileSize !== 0) {
// Read the whole file
$fileContent = $file->fread($fileSize);
} else {
// File is empty
throw new \RuntimeException("File '$filePath' is empty");
}
// Free file resources
$file = null;
// Divide file content into its lines
$fileLineByLine = explode(PHP_EOL, $fileContent);
$found = false;
foreach ($fileLineByLine as $lineNumber => $thisLine) {
if ($thisLine === $line) {
$found = true;
unset($fileLineByLine[$lineNumber]);
if ($removeOnlyFirstMatch) {
break;
}
}
}
// We don't need to update file either if the line not found
if (!$found) {
return false;
}
// Join lines together
$newFileContent = implode(PHP_EOL, $fileLineByLine);
// Finally, update the file
$file = new \SplFileObject($filePath, "w");
if ($file->fwrite($newFileContent) !== strlen($newFileContent)) {
throw new \RuntimeException("Could not update the file '$filePath'");
}
return true;
}
Here is a brief description of what is being done: Get the whole file content, split the content into its lines (i.e. as an array), find the match(es) and remove them, join all lines together, and save the result back to the file (only if any changes happened).
Let's now use it:
// $dir is your filename, as you mentioned
removeLineFromFile($line, $dir);
Notes:
You can use fopen() family functions instead of SplFileObject, but I do recommend the object form, as it's exception-based, more robust and more efficient (in this case at least).
It's safe to unset() an element of an array being iterated using foreach (There's a comment here showing it can lead unexpected results, but it's totally wrong: As you can see in the example code, $value is copied (i.e. it's not a reference), and removing an array element does not affect it).
$line should not have new line characters like \n, otherwise, you may perform lots of redundant searches.
Don't use
$fileLineByLine[$lineNumber] = "";
// Or even
$fileLineByLine[$lineNumber] = null;
instead of
unset($fileLineByLine[$key]);
The reason is, the first case doesn't remove the line, it just clears the line (and an unwanted empty line will remain).
Hope it helps.
Like this:
file_put_contents($filename, str_replace($line . "\r\n", "", file_get_contents($filename)));