Get filename from Unix "ls -la" command with regexp? - php

How can I produce a regular expressions pattern that returns the filename from any one of these lines? (I will search one line at a time).
drwxrwxrwx 4 apache apache 4096 Oct 14 09:40 .
drwxrwxrwx 11 apache apache 4096 Oct 13 11:33 ..
-rwxrwxrwx 1 apache apache 16507 Oct 17 10:16 .bash_history
-rwxrwxrwx 1 apache apache 33 Sep 1 09:36 .bash_logout
-rwxrwxrwx 1 apache apache 176 Sep 1 09:36 .bash_profile
-rwxrwxrwx 1 apache apache 124 Sep 1 09:36 .bashrc
-rwxrwxrwx 1 apache apache 515 Sep 1 09:36 .emacs
-rw------- 1 christoffer christoffer 11993 Sep 18 10:00 .mysql_history
drwxrwxrwx 3 apache apache 4096 Sep 1 09:48 .subversion
-rwxrwxrwx 1 christoffer christoffer 9204 Oct 14 09:40 .viminfo
drwxrwxrwx 14 apache apache 4096 Oct 12 07:39 www
The search is done using PHP, but I guess that doesn't really make a difference. :)
EDIT: The file listing is retrieved by a SSH connection and that is why I don't use a built in PHP-function. I need this full listing to see whether or not a file is actually a directory.

Try ls -a1F instead. That will list you all entries (-a), one per line (-1), with additional information about the file type appended to the name (-F).
You will then probably get something like this for your directory:
./
../
.bash_history
.bash_logout
.bash_profile
.bashrc
.emacs
.mysql_history
.subversion/
.viminfo
www/
The directories have a slash / at the end.

The main question is... Why? Use readdir and stat instead.
<?php
$directory = './';
$dh = opendir($directory);
while (($file = readdir($dh)) !== false)
{
$stat = stat($directory.$file);
echo '<b>'.$directory.$file.':</b><br/>';
var_dump($stat);
}

If you are looking for directories, rather than parsing ls output, just use find.
find -maxdepth 1 -mindepth 1 -type d
This will list the directories like this:
./Documents
./.gnupg
./Download
You no longer have to parse the data to determine what is a directory and what isn't.
If you're actually wanting the files, and not the directories, you use -type f instead.
Your parsing of the ls output may very well break on symlinks...

I wouldn't use regex
Given a line, you could explode and pop the last element from the array
if (preg_match('/^d/', $line)) {
$name = array_pop(explode(' ', $line));
}
EDIT: none of your examples have embedded spaces but a later comment suggests that it IS important to find filenames

Adding to what Matthew said, there's plenty of reasons to not parse ls output. You might have spaces in file names - or even delete characters. The format of the date part of the listing, especially for older files, is different, the size of the large files can break the listing.
If you must use regex, and you really have no spaces in file names, then just tie to the end of the line and get the non-spaces you find there
(\S+)$

There's a nicer way to do this in php5 using the spl and DirectoryIterator
$dir = '.';
foreach (new DirectoryIterator($dir) as $fileInfo) {
echo $fileInfo->getFilename() . "<br>\n";
}

Given your constraint of using the full directory listing I would do it this way:
ls -l | egrep '^d' | awk '{print $NF}'
Egrep command would search for the letter "d" at the beginning of the line. Awk by default uses spaces as seperators and the $NF will get you the last element. The only edge case I can think of where this wouldn't always work 100% of the time is when the file name would have spaces in it.
I would suggest using the find command:
find . -maxdepth 1 -type d | awk -F '/' '{print $NF}'
The find command above will get you only the files/directories in your current directory (b/c of -maxdepth 1 arg). The awk command will split the line using the '/' and will only retrieve the last token ($NF).
Because, the awk command
awk -F '/' '{print $NF}'
will get you the last element you can essentially use:
find . -maxdepth x -type d
where x is a number of your choice >= 1, you'll still get what you want, the filename and/or the directory name.

\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+\S+\s+(\S+)
Each string is built of 9 parts separated by whitespace. You are looking for the 9th part.

Use glob('*') instead?

Instead of trying to parse difficult output, how about generating some more helpful output in the first place. For example:
ssh user#machine 'cd /etc; for a in *; do [ -f "$a" ] && echo "$a"; done'
will generate a list of non-directory files in /etc on the remote machine. This should be much easier for you to parse.

Displays hidden files too, try it if you don't believe me.
glob('{,.}*', GLOB_BRACE);

Related

Permissions problems with proc_open

I'm having permissions problems when running the following PHP script as root:
#!/usr/bin/php
<?php
$ph = proc_open('whoami', [['pipe','r'],['pipe','w'],['file','/tmp/foo.bar', 'w']], $fds);
if ($ph) {
echo 'command output: ' . stream_get_contents($fds[1]);
proc_close($ph);
} else {
echo 'proc_open failed' . PHP_EOL;
}
The script itself runs fine if /tmp/foo.bar doesn't exist, or is owned by root. But if ownership is changed to another user, proc_open will fail regardless of permissions on the file.
SELinux is disabled, and we are not using ACLs. I'm using PHP 7.4.33 (I know it's old and unsupported, but it's a requirement for FreePBX) on Alma Linux 9.1.
Output:
$ ./test.php
command output: root
$ ls -lah /tmp/
total 12K
drwxrwxrwt. 18 root root 4.0K Dec 14 16:57 .
dr-xr-xr-x. 18 root root 4.0K Dec 14 16:48 ..
-rw-r--r-- 1 root root 0 Dec 14 16:57 foo.bar
$ chown admin /tmp/foo.bar
$ ./test.php
proc_open failed
$ chmod 777 /tmp/foo.bar
$ ./test.php
proc_open failed
$ ls -lah /tmp/
total 12K
drwxrwxrwt. 18 root root 4.0K Dec 14 16:57 .
dr-xr-xr-x. 18 root root 4.0K Dec 14 16:48 ..
-rwxrwxrwx 1 admin root 0 Dec 14 16:57 foo.bar
$ tail -2 /var/log/php.log
[14-Dec-2022 16:57:17 America/Toronto] PHP Warning: proc_open(/tmp/foo.bar): failed to open stream: Permission denied in /test.php on line 3
[14-Dec-2022 16:57:28 America/Toronto] PHP Warning: proc_open(/tmp/foo.bar): failed to open stream: Permission denied in /test.php on line 3
Even disregarding the fact that I'm root, group permissions should allow me full access to the file. So what's going on?
This is due to the permissions on the /tmp directory. When PHP tries to open the file for writing, it gets the EACCES error. From the documentation of open(2):
EACCES
Where O_CREAT is specified, the protected_fifos or protected_regular sysctl is enabled, the file already exists and is a FIFO or regular file, the owner of the file is neither the current user nor the owner of the containing directory, and the containing directory is both world- or group-writable and sticky. For details, see the descriptions of /proc/sys/fs/protected_fifos and /proc/sys/fs/protected_regular in proc(5).
/tmp has the sticky bit set so that anyone can create files there, but users can only delete their own files. Although root can bypass this deletion restriction, it can't bypass the above check in open().
Ok I tried this in a different directory than /tmp, as suggested in comments, and it worked as expected. Using that to hone my search terms I was able pretty quickly to find this U&L answer. Beginning with kernel 4.19 the fs.protected_regular kernel parameter was made available. This parameter:
Disallows open of FIFOs or regular files not owned by the user in world writable sticky directories, unless the owner is the same as that of the directory or the file is opened without the O_CREAT flag. The purpose is to make data spoofing attacks harder.
Apparently it's enabled by default. So because /tmp is world-writable and sticky, I can't touch files that aren't mine – even if I'm root. For the record, if I have to disable this feature:
sysctl fs.protected_regular=0
echo 'fs.protected_regular=0' > /etc/sysctl.d/90-writabletemp.conf
But I'll be better off trying to work around it in the code somehow.

Search & replace issue on mPDF, CAM'PDF on Ubuntu 18.04

On Ubuntu 18.04 I have a problem with editing PDF files - specifically search & replace strings.
I tried:
PHP mPDF Overwrite () do nothing.
perl CAM :: PDF 1.60 changepagestring.pl do nothing
sed, do nothing.
Does not work with compressed or decompressed PDF, Does not even work with PDF generated from mPDF.
UPDATE: after reinstalling libsodium mPDF works fine with PDF files generated fromm mPDF. For other PDF files issue still exist.
Also tried in var / www folders user / group www-data: www / data and in other folders / home e.g.
Any idea for bulk search & replace because I have over 1000 files to process?
The text in the files is readable. Check.
P.S. Search / Replace from the program and online service works with the same files.
Permission on files 0755 i 0777
root#sasa-ubuntu-1:/var/www/website.local/wp-content/test/2018/12# ls -la *.pdf
-rwxr-xr-x 1 www-data www-data 847451 Oct 18 12:21 clean.pdf
-rwxrwxrwx 1 www-data www-data 395527 Oct 17 21:41 My-First.pdf
-rwxr-xr-x 1 www-data www-data 838307 Oct 17 23:30 My.pdf
-rwxr-xr-x 1 www-data www-data 838167 Oct 18 12:24 New2.pdf
-rwxr-xr-x 1 www-data www-data 838167 Oct 18 01:20 New.pdf
-rwxrwxrwx 1 www-data www-data 270340 Oct 17 16:39 Test2.pdf
-rwxrwxrwx 1 www-data www-data 274022 Oct 17 16:39 Test1.pdf
-rwxr-xr-x 1 www-data www-data 838000 Oct 18 00:55 Test2.pdf
-rwxrwxrwx 1 www-data www-data 205679 Oct 17 23:44 test.pdf
Perl script allways return "Could not find title" nevermind of readability of file when I print $page variable (see images)
use CAM::PDF;
my $pdf = CAM::PDF->new('test.pdf'); # existing document
my $nump = $pdf->numPages();
#print $nump;
my $page = $pdf->getPageContent(1);
print $page;
# $page now holds the uncompressed page content as a string
# replace the text part
if ($page =~ s/Wagner/SoundTech/g) {
$pdf->setPageContent(1, $page);
}
else {
die "Could not find title\n";
}
$pdf->cleanoutput('Test2.pdf');
A lot of files ends on this way.
The string that I try to find is "Wagner International Music Examinations" or only "Wagner"
mPDF and CAM-PDF are properly installed without warnings and erros and with all dependencies, I hope.
Ubuntu 18.04
mPDF version 8.0
PHP 7.2
Perl 5.26.1
CAM-PDF version 1.60
mPDF occasionally have bug with Overwrite() function, I found on their github community.
Any suggestion or another way for bulk search & replace in PDF files?
Here is a hack that currently works almost for your case (I will come back later and try improve this):
use feature qw(say);
use strict;
use warnings;
# the PDF uses a non-standard encoding so it does not help to use UTF-8
# use open qw(:std :encoding(UTF-8));
use utf8;
use CAM::PDF;
my $fn = 'test.pdf'; # uncompressed file..
my $save_fn = 'test2.pdf';
my $pdf = CAM::PDF->new($fn);
my $nump = $pdf->numPages();
my $match = 0;
my $replace = '[(\x{a9} SoundTech International Music Examinations)]TJ';
for my $i (1..$nump) {
my $page = $pdf->getPageContent( $i );
# replace the text part
if ($page =~ s/\[\(\x{a9}\).*?\]TJ/$replace/g) {
$match = 1;
$pdf->setPageContent($i, $page);
}
}
if ( $match ) {
$pdf->cleanoutput($save_fn);
say "Save $save_fn ..";
}
else {
say "No match";
}

php shell_exec and exec doesn't work with shell script

So this is my code for the raspberry pi to get a still shot from the raspicam and save it on a directory,
<?php
exec('raspistill -n -hf -o /var/www/img/image.jpg --timeout 1');
?>
I have given the ownership and the permission to read/write in that forlder using -R. so my ls -al in /var/www is this
drwxr-xr-x 3 www-data www-data 4096 Jun 19 08:05 .
drwxr-xr-x 12 root root 4096 Jun 19 05:54 ..
-rwxrwxrwx 1 www-data www-data 74 Jun 19 08:30 getImg
drwxrwxrwx 2 www-data www-data 4096 Jun 19 09:21 img
-rw-r--r-- 1 root root 70 Jun 19 10:07 index.php
getImg is the script i tried to run the script as a file like shell_exec('/bin/bash ./getImg'); that also doesn't work.
i have added /bash/bin and tried to run the script without using the script file too but that doesn't get the results.
How ever when i try to run the php file in the terminal, it creates the image as it normally should. So i figure this must be a permission issue, but what else should i do with the permissions? I have given all the rights to the directory.
EDIT
So I have found a workaround to this. since I don't know what the cause for the problem, i'd not mark this as an answer, but please vote it to appear at the top.
I now execute the script using the cgi scripts. I have created a shell script in the /usr/lib/cgi-bin/
#!/bin/bash
echo "Content-type:text/html\n"
sudo raspistill -vf -n -o /var/www/img/image.jpg --timeout 1200 --metering matrix
echo "Status: 204"
I saved this as capture and made this executable, did nothing with the permissions though.
sudo chmod +x capture
now when i open the link http://192.168.1.85/cgi-bin/capture the browser will still get me a 500 internal server error message. how ever, the image would still be created.
I would now need to get the 500 internal server error to be fixed.
[I'd add this as a comment but don't have enough points for it]
if you use the optional parameters $output and $return_var to capture the output and return value what do you get?
string exec ( string $command [, array &$output [, int &$return_var ]] )
does your command rely on environment variables that may be available when you run it as your user but not as www-data? (you can use the env command to check that)
does it still work if you run it via terminal after switching user to www-data?

ls -ltr using PHP exec()

as the problem states..
when i do
exec("ls -ltr > output.txt 2>&1",$result,$status);
its different from the normal output. An extra column gets added. something like
-rw-r--r-- 1 apache apache 211 Jul 1 15:52 withoutsudo.txt
-rw-r--r-- 1 apache apache 0 Jul 1 15:53 withsudo.txt
where as when executed from the command prompt its like
-rw-r--r-- 1 apache apache 211 2010-07-01 15:52 withoutsudo.txt
-rw-r--r-- 1 apache apache 274 2010-07-01 15:53 withsudo.txt
-rw-r--r-- 1 apache apache 346 2010-07-01 15:55 sudominusu.txt
-rw-r--r-- 1 apache apache 414 2010-07-01 15:58 sudominusu.txt
See the difference. So in the first output , my usual awk '{print $8}' fails.
I was facing the same problem with cron. But solved it by calling
./$HOME/.bashrc
in the script. But not happening using php. If somehow i can "tell" php to "exec" from the usual environment. Any help would be appreciated.
In your login shell, ls is probably aliased so that it prints another date. This would be in your .basrc or .bash_profile.
Explicitly pass the --time-style= option to ls to ensure that it prints the date in the expected format when using PHP.
I guess you are only interested in the file names and you want to sort with reverse time.
Try this:
ls -tr1 > output.txt 2>&1
You'll get a list with only the file names, so you don't need awk at all.
Another solution is to specify the time format with "--time-style iso". Have a look at the man page
That's not an extra output, that's a difference in formatting the date. Apparently you have a different locale set in PHP and in bash ("command prompt").
(in bash, running export LANG=C or export LANG=en_US gives the result with three-letter month name)
The output of ls is heavily dependent on the environment (e.g., LANG being the important variable here). Why not use a combination of scandir, stat, and krsort?
function ls($dir_name) {
$finfo = array();
foreach (scandir($dir_name) as $file_name) {
$s = stat(join('/', array($dir_name,$file_name)));
$finfo[$file_name] = $s['mtime'];
}
krsort($finfo);
return array_keys($finfo);
}
This will be safer and a lot more efficient than shelling out to ls. Not to mention that you get the benefit of being about to customize the sorting and filter the results in ways that are difficult to do inside of an exec.
BTW: I am by no means a PHP expert, so the above snippet is likely to be incredibly unsafe and full of errors.

PHP mkdir and apache ownership

Is there a way to set php running under apache to create folders with the folder owned by the owner of the program that creates it instead of being owned by apache?
Using word press it creates new folders to upload into but these are owned by apache.apache and not by the site that they are running in. This also happens using ostickets. For now we have to SSH into the server and chmod the folder, but it would seem there would be a setting somewhere to override the ownership outside of any program that does it.
Safe_mode is turn on on your server. The function mkdir() creates folder with owner ("apache", "none", ..) that different of the current script owner. And scripts couldn't upload (move, copy) files into that folder with another owner (that is not like current script owner).
Disable safe_mode and that would be work.
See http://php.net/manual/en/features.safe-mode.php for details.
P.S. With enable safe_mode you can't use chmod() function in php.
Another way is to put the apache user and the "customer users" in a new group. Additional the directory should use the sticky bit SGID so each new file got the group assignment to this new group. This way the webserver and the "customer users" can work with the files without any problems
[17:57] progman#proglap /tmp/test $ ls -al /tmp/test
total 9
drwxrwsr-x 2 root users 48 Apr 1 17:55 .
drwxrwxrwt 36 root root 9264 Apr 1 17:53 ..
As you see the directory got the stick bit SGID and the owner is the "users" group in which I (progman) am. No if another user adds a file the group automatically get set to this group
[17:55] proglap ~ # touch /tmp/test/x
This is executed from root. Now we get:
[17:57] progman#proglap /tmp/test $ ls -la /tmp/test
total 9
drwxrwsr-x 2 root users 72 Apr 1 17:59 .
drwxrwxrwt 36 root root 9264 Apr 1 17:53 ..
-rw-r--r-- 1 root users 0 Apr 1 17:59 x
As you see the added file is from root, but the group is set to users and this way I can remove it
[18:00] progman#proglap /tmp/test $ rm x
rm: remove write-protected regular empty file `x'? y
[18:01] progman#proglap /tmp/test $ ls -la /tmp/test
total 9
drwxrwsr-x 2 root users 48 Apr 1 18:01 .
drwxrwxrwt 36 root root 9264 Apr 1 17:53 ..
Keep in mind that you still need to change the chmod if you want to edit the file as rw-r--r-- is just group read access. But changing the chmod, maybe even working with umask, is better than dealing with root-access and using chown.
Not directly, no. You can't "give away" ownership of a file to another user, unless you're root. You could investigate using the "AssignUserID" apache directive to force that particular vhost to run as a particular user/group. With that Apache/PHP would create any files with the appropriate ownership
Check out PHP chown() function

Categories