echo "sed -i 's/NULL/\\N/g' ".$_REQUEST['para'].".sql";
The above statement works. But it fail when I use it in exec like this...
exec("sed -i 's/NULL//\/\/\N/g' ".$_REQUEST['para'].".sql");
You should escape backslashes with backslashes, not with forward slashes, like this:
exec("sed -i 's/NULL/\\\\N/g' ".$_REQUEST['para'].".sql");
EDIT I wrote the answer without looking at what the code actually does. Don't do this, because $_REQUEST['para'] can be whatever the user wants, which can be used for code injection. Use the PHP functions as the other answer suggests.
Although it's entirely up to you, but my advice is not to call system commands unnecessarily. In PHP, you can use preg_replace() to do the functionality of sed.
preg_replace("/NULL/","\\N",file_get_contents("$_REQUEST['para']"."sql") )
Building on ghostdog's idea, here's code that will actually do what you want (the original code he posted didn't actually read content of the file in):
//basename protects against directory traversal
//ideally we should also do a is_writable() check
$file = basename($_REQUEST['para'].".sql");
$text = file_get_contents($file);
$text = str_replace('NULL', '\\N', $text); //no need for a regex
file_put_contents($file, $text);
Admittedly, however, if the file in question is more than a few meg, this is inadvisable as the whole file will be read into memory. You could read it in chunks, but that'd get a bit more complicated:
$file = basename($_REQUEST['para'].".sql");
$tmpFile = tempnam("/tmp", "FOO");
$in = fopen($file, 'r');
$tmp = fopen($tmpFile, 'w');
while($line = fgets($in)) {
$line = str_replace('NULL', '\\N', $line);
fputs($tmp, $line);
}
fclose($tmp);
fclose($in);
rename($tmpFile, $file);
If the file is 100+ meg, honestly, calling sed directly like you are will be faster. When it comes to large files, the overhead of trying to reproduce a tool like sed/grep with its PHP equivalent just isn't worth it. However, you need to at least take some steps to protect yourself if you're going to do so:
Taking some basic steps to secure amnom's code:
$file = basename($_REQUEST['para'].".sql");
if(!is_writable($file))
throw new Exception('bad filename');
exec("sed -i 's/NULL/\\\\N/g' ".escapeshellarg($file));
First, we call basename, which
strips any path from our filename
(e.g., if an attacker submitted the
string '/etc/passwd', we'd at least
now be limiting them to the file
'passwd' in the current working
directory
Next, we ensure that the file is, in
fact, writable. If not, we
shouldn't continue
Finally, we escapeshellarg() on the file. Failure to do so allows arbitrary command execution. e.g., if the attacker submitted the string /etc/passwd; rm -rf /; #, you'd end up with the command sed 's/blah/blah/' /etc/passwd; rm -rf /; #.sql. It should be clear that while that exact command may not work, finding one that actually would is trivial.
Related
What is the best way to overwrite a specific line in a file? I basically want to search a file for the string '#parsethis' and overwrite the rest of that line with something else.
If the file is really big (log files or something like this) and you are willing to sacrifice speed for memory consumption you could open two files and essentially do the trick Jeremy Ruten proposed by using files instead of system memory.
$source='in.txt';
$target='out.txt';
// copy operation
$sh=fopen($source, 'r');
$th=fopen($target, 'w');
while (!feof($sh)) {
$line=fgets($sh);
if (strpos($line, '#parsethis')!==false) {
$line='new line to be inserted' . PHP_EOL;
}
fwrite($th, $line);
}
fclose($sh);
fclose($th);
// delete old source file
unlink($source);
// rename target file to source file
rename($target, $source);
If the file isn't too big, the best way would probably be to read the file into an array of lines with file(), search through the array of lines for your string and edit that line, then implode() the array back together and fwrite() it back to the file.
Your main problem is the fact that the new line may not be the same length as the old line. If you need to change the length of the line, there is no way out of rewriting at least all of the file after the changed line. The easiest way is to create a new, modified file and then move it over the original. This way there is a complete file available at all times for readers. Use locking to make sure that only one script is modifying the file at once, and since you are going to replace the file, do the locking on a different file. Check out flock().
If you are certain that the new line will be the same length as the old line, you can open the file in read/write mode (use r+ as the second argument to fopen()) and call ftell() to save the position the line starts at each time before you call fgets() to read a line. Once you find the line that you want to overwrite, you can use fseek() to go back to the beginning of the line and fwrite() the new data. One way to force the line to always be the same length is to space pad it out to the maximum possible length.
This is a solution that works for rewriting only one line of a file in place with sed from PHP. My file contains only style vars and is formatted:
$styleVarName: styleVarProperty;\n
For this I first add the ":" to the ends of myStyleVarName, and sed replaces the rest of that line with the new property and adds a semicolon.
Make sure characters are properly escaped in myStyleVarProp.
$command = "pathToShellScript folder1Name folder2Name myStyleVarName myStyleVarProp";
shell_exec($command);
/* shellScript */
#!/bin/bash
file=/var/www/vhosts/mydomain.com/$1/$2/scss/_variables.scss
str=$3"$4"
sed -i "s/^$3.*/$str;/" $file
or if your file isn't too big:
$sample = file_get_contents('sample');
$parsed =preg_replace('##parsethis.*#', 'REPLACE TO END OF LINE', $sample);
You'll have to choose delimiters '#' that aren't present in the file though.
If you want to completely replace the contents of one file with the contents of another file you can use this:
rename("./some_path/data.txt", "./some_path/data_backup.txt");
rename("./some_path/new_data.txt", "./some_path/data.txt");
So in the first line you backup the file and in the second line you replace the file with the contents of a new file.
As far as I can tell the rename returns a boolean. True if the rename is successful and false if it fails. One could, therefore, only run the second step if the first step is successful to prevent overwriting the file unless a backup has been made successfully. Check out:
https://www.php.net/manual/en/function.rename.php
Hope that is useful to someone.
Cheers
Adrian
I'd most likely do what Jeremy suggested, but just for an alternate way to do it here is another solution. This has not been tested or used and is for *nix systems.
$cmd = "grep '#parsethis' " . $filename;
$output = system($cmd, $result);
$lines = explode("\n", $result);
// Read the entire file as a string
// Do a str_repalce for each item in $lines with ""
I am using PHP file which execute sed:
shell_exec("C:\\cygwin64\\bin\\bash.exe --login -c 'sed -i -r \'s/.{2}//\' $text_files_path/File.txt 2>&1'");
This statement will delete the first 2 character from file.txt.
How to delete the first 2 char from (each line) in the file?
File.text:
< TTGCATGCAAAAATTT
< AAAAAAATTTTGCTGA
< AAGGTTCCCCCTTAGT
Edit 1:
shell_exec("C:\\cygwin64\\bin\\bash.exe --login -c 'sed -i -r 's/^..//' $text_files_path/File.txt 2>&1'");
This works but, it concatenate all lines together:
File.text after above command:
TTGCATGCAAAAATTTAAAAAAATTTTGCTGAAAGGTTCCCCCTTAGT
Please don't call sed via bash to do something that PHP can do natively. It's a complete anti-pattern. Worryingly, I have seen the exact same thing in another question quite recently...
I hope you've got plenty of free disk space:
$input_filename = "$text_files_path/File.txt";
$output_filename = 'path/to/temp/output.txt';
$input_file = fopen($input_filename, 'rb');
$output_file = fopen($output_filename, 'wb');
while (($line = fgets($input_file)) !== false) {
fwrite($output_file, substr($line, 2));
}
fclose($input_file);
fclose($output_file);
rename($output_filename, $input_filename);
Open the input file for reading and the temporary output file for writing. Use binary mode in both cases to avoid issues related to different line endings on different systems.
Read each line of the input and write the substring from the second character to the temporary output.
Close both files and then overwrite the input with the temporary file.
Technically this could actually be implemented in-place but the resulting script would be much more complicated and you would run further risk of corrupting your input file if things went wrong.
If you just want to use PHP, then you can explode() the file into individual lines and then use substr() to drop the first two characters before joining the lines back into a single string separated with a new line:
// Set the results array.
$result = array();
// Split the file into lines.
$file = $text_files_path . '/File.txt';
$lines = explode("\n", $file);
// Cut the first two characters of each line and add to the results array.
foreach($lines AS $line) {
$result[] = substr($line, 2);
}
// Split the result back into lines.
$result = implode("\n", $result);
s/^..// That should give you the result you need.
^ points to the start of the line then the . will match any character
I have many files containing php serialized data in which I have to replace some strings by another one. The linux host doesn't have any php installed. The problem is to adjust the modified string to correct size.
I tried something like to replace /share path to /opt:
sed -re 's~s:([0-9]+):"/share([^"]*)~s:int(\1-2):/opt\2~g' file
but the result file is bad: lengths are litteral expression int(size - 2)
Any idea ?
This solution isn't ideal, but you could use perl:
my $line;
while ($line = <STDIN>) {
$line =~ s~s:([0-9]+):"/share([^"]*)~"s:".($1-2).":\"/opt$2"~ge;
print $line;
}
Hopefully I've understood your requirements correctly. Here's an example:
php -r 'echo serialize(array("/share/foo")) . "\n";'
a:1:{i:0;s:10:"/share/foo";}
php -r 'echo serialize(array("/share/foo")) . "\n";' | perl replace.pl
a:1:{i:0;s:8:"/opt/foo";}
EDIT: Here's a modified script to edit the file in-place with variable search and replace strings.
It is possible to pipe data using unix pipes into a command-line php script? I've tried
$> data | php script.php
But the expected data did not show up in $argv. Is there a way to do this?
PHP can read from standard input, and also provides a nice shortcut for it: STDIN.
With it, you can use things like stream_get_contents and others to do things like:
$data = stream_get_contents(STDIN);
This will just dump all the piped data into $data.
If you want to start processing before all data is read, or the input size is too big to fit into a variable, you can use:
while(!feof(STDIN)){
$line = fgets(STDIN);
}
STDIN is just a shortcut of $fh = fopen("php://stdin", "r");.
The same methods can be applied to reading and writing files, and tcp streams.
As I understand it, $argv will show the arguments of the program, in other words:
php script.php arg1 arg2 arg3
But if you pipe data into PHP, you will have to read it from standard input. I've never tried this, but I think it's something like this:
$fp = readfile("php://stdin");
// read $fp as if it were a file
If your data is on one like, you can also use either the -F or -R flag (-F reads & executes the file following it, -R executes it literally) If you use these flags the string that has been piped in will appear in the (regular) global variable $argn
Simple example:
echo "hello world" | php -R 'echo str_replace("world","stackoverflow", $argn);'
You can pipe data in, yes. But it won't appear in $argv. It'll go to stdin. You can read this several ways, including fopen('php://stdin','r')
There are good examples in the manual
This worked for me:
stream_get_contents(fopen("php://stdin", "r"));
Came upon this post looking to make a script that behaves like a shell script, executing another command for each line of the input... ex:
ls -ln | awk '{print $9}'
If you're looking to make a php script that behaves in a similar way, this worked for me:
#!/usr/bin/php
<?php
$input = stream_get_contents(fopen("php://stdin", "r"));
$lines = explode("\n", $input);
foreach($lines as $line) {
$command = "php next_script.php '" . $line . "'";
$output = shell_exec($command);
echo $output;
}
If you want it to show up in $argv, try this:
echo "Whatever you want" | xargs php script.php
That would covert whatever goes into standard input into command line arguments.
Best option is to use -r option and take the data from the stdin. Ie I use it to easily decode JSON using PHP.
This way you don't have to create physical script file.
It goes like this:
docker inspect $1|php -r '$a=json_decode(stream_get_contents(STDIN),true);echo str_replace(["Array",":"],["Shares"," --> "],print_r($a[0]["HostConfig"]["Binds"],true));'
This piece of code will display shared folders between host & a container.
Please replace $1 by the container name or put it in a bash alias like ie displayshares() { ... }
I needed to take a CSV file and convert it to a TSV file. Sure, I could import the file into Excel and then re-export it, but where's the fun in that when piping the data through a converter means I can stay in the commandline and get the job done easily!
So, my script (called csv2tsv) is
#!/usr/bin/php
<?php
while(!feof(STDIN)){
echo implode("\t", str_getcsv(fgets(STDIN))), PHP_EOL;
}
I chmod +x csv2tsv.
I can then run it cat data.csv | csv2tsv > data.tsv and I now have my data as a TSV!
OK. No error checking (is the data an actual CSV file?), etc. but the principle works well.
And of course, you can chain as many commands as you need.
If you are wanting more to expand on this idea, then how about the ability to include additional options to your command?
Simple!
#!/usr/bin/php
<?php
$separator = $argv[1] ?? "\t";
while(!feof(STDIN)){
echo implode($separator, str_getcsv(fgets(STDIN))), PHP_EOL;
}
Now I can overwrite the default separator from being a tab to something else. A | maybe!
cat data.csv | csv2tsv '|' > data.psv
Hope this helps and allows you to see how much more you can do!
What is the best way to overwrite a specific line in a file? I basically want to search a file for the string '#parsethis' and overwrite the rest of that line with something else.
If the file is really big (log files or something like this) and you are willing to sacrifice speed for memory consumption you could open two files and essentially do the trick Jeremy Ruten proposed by using files instead of system memory.
$source='in.txt';
$target='out.txt';
// copy operation
$sh=fopen($source, 'r');
$th=fopen($target, 'w');
while (!feof($sh)) {
$line=fgets($sh);
if (strpos($line, '#parsethis')!==false) {
$line='new line to be inserted' . PHP_EOL;
}
fwrite($th, $line);
}
fclose($sh);
fclose($th);
// delete old source file
unlink($source);
// rename target file to source file
rename($target, $source);
If the file isn't too big, the best way would probably be to read the file into an array of lines with file(), search through the array of lines for your string and edit that line, then implode() the array back together and fwrite() it back to the file.
Your main problem is the fact that the new line may not be the same length as the old line. If you need to change the length of the line, there is no way out of rewriting at least all of the file after the changed line. The easiest way is to create a new, modified file and then move it over the original. This way there is a complete file available at all times for readers. Use locking to make sure that only one script is modifying the file at once, and since you are going to replace the file, do the locking on a different file. Check out flock().
If you are certain that the new line will be the same length as the old line, you can open the file in read/write mode (use r+ as the second argument to fopen()) and call ftell() to save the position the line starts at each time before you call fgets() to read a line. Once you find the line that you want to overwrite, you can use fseek() to go back to the beginning of the line and fwrite() the new data. One way to force the line to always be the same length is to space pad it out to the maximum possible length.
This is a solution that works for rewriting only one line of a file in place with sed from PHP. My file contains only style vars and is formatted:
$styleVarName: styleVarProperty;\n
For this I first add the ":" to the ends of myStyleVarName, and sed replaces the rest of that line with the new property and adds a semicolon.
Make sure characters are properly escaped in myStyleVarProp.
$command = "pathToShellScript folder1Name folder2Name myStyleVarName myStyleVarProp";
shell_exec($command);
/* shellScript */
#!/bin/bash
file=/var/www/vhosts/mydomain.com/$1/$2/scss/_variables.scss
str=$3"$4"
sed -i "s/^$3.*/$str;/" $file
or if your file isn't too big:
$sample = file_get_contents('sample');
$parsed =preg_replace('##parsethis.*#', 'REPLACE TO END OF LINE', $sample);
You'll have to choose delimiters '#' that aren't present in the file though.
If you want to completely replace the contents of one file with the contents of another file you can use this:
rename("./some_path/data.txt", "./some_path/data_backup.txt");
rename("./some_path/new_data.txt", "./some_path/data.txt");
So in the first line you backup the file and in the second line you replace the file with the contents of a new file.
As far as I can tell the rename returns a boolean. True if the rename is successful and false if it fails. One could, therefore, only run the second step if the first step is successful to prevent overwriting the file unless a backup has been made successfully. Check out:
https://www.php.net/manual/en/function.rename.php
Hope that is useful to someone.
Cheers
Adrian
I'd most likely do what Jeremy suggested, but just for an alternate way to do it here is another solution. This has not been tested or used and is for *nix systems.
$cmd = "grep '#parsethis' " . $filename;
$output = system($cmd, $result);
$lines = explode("\n", $result);
// Read the entire file as a string
// Do a str_repalce for each item in $lines with ""