I'm trying to implement Mercurial to use to manage my personal websites. Here's my current setup...
I have a repository on Bitbucket
I cloned the repo to my home PC for development
I cloned the repo to my web server (Hostgator VPS) for deployment
My problem is that I can only execute "hg pull" (or any other commands) on the web as the root user. I can't su or sudo to the actual user on the web server that the account belongs to.
As a result, all files are created on my server with the user/group as 'root', and the server throws an internal server error when trying to load those files. I can use 'chown' to change the user/group on the files, and get them working. However, doing an 'hg pull' fails to copy down the changed files. It seems as though it won't overwrite the files after the owner has been changed.
Any suggestions on what to do?
This is usually handled using a group and the group sticky bit.
For example create a group 'dev' and add both 'root' (really, they make you login as root?!) and 'apache' (where apache may actually by wwwrun or www, etc) to it:
% groupadd dev
% usermod --add-to-group dev root
% usermod --add-to-group dev apache
Then chgrp all the files over to that group:
% chgrp -R dev /path/to/repos
Then make sure all group members can read and write those file:
% chmod -R g+rwx /path/to/repos
The make sure any newly created files are automatically in that group:
% find /path/to/repos -type d -print0 | xargs -0 chmod g+s
Then any new files created by apache will be owned by the 'dev' group which both it and root can read. Root will need to make sure the files it creates from the command line had read/write permissions for group too.
Note: Doing anything like this with root is unsafe -- use a dedicated account whenever possible, I always create 'hg'.
This is explained here in the Mercurial wiki.
Related
I've recently been learning to build images and containers with Docker. I was getting fairly confident with it when using a Mac, but recently switched to Ubuntu, I'm fairly new to this side of development.
I'm using a standard new Laravel project as my "code", and am currently just using a php container and nginx container.
I'm using a docker-compose.yml file to create my containers:
version: "3.1"
services:
nginx:
image: nginx:latest
volumes:
- ./code:/var/www
- ./nginx_conf.conf:/etc/nginx/conf.d/default.conf
ports:
- "80:80"
php:
image: php:7.3-fpm
ports:
- 9000
volumes:
- ./code:/var/www
There may or may not be a mistake in the code above just because I've just typed it out rather than copy and pasting - but it works on my machine.
The problem is:
php-fpm is configured with --with-fpm-user=www-data and --with-fpm-group=www-data, and that's set in the php:7.3-fpm Dockerfile (see here).
The files on my host machine, are saved with my user name and group as owner / group.
When I go into the container, the files are owned by 1000 and group 1000 (I assume a mapping to my user account and group on the host machine?)
However, when I access the application through the browser, I get a permission denied error on start up (when Laravel tries to create an error log file in storage). I think this is because php-fpm is running as www-data, but the storage directory has permissions drwxr-xr-x for owner / group phil:phil - my host owner and group.
I've tried the following, after hours of googling and trials:
Recursively change the owner and group of the code directory on the host machine to www-data:www-data. This allows the Laravel application to work, but I now cant create or edit etc files on the host using PHPStorm, because the directory is read-only (I guess because phpstorm is running as my user, and directory is owned by a different user / group).
I've added my host user account to the www-data group, and granted write permissions to the group using sudo chmod -R g+w ./code, which now allows the application to run the application, and for phpstorm to write, execute etc files, but when i create or edit a file, the files ownership and group change back to my host phil:phil, and I guess this would break the application again.
I've tried to create a php image, and set the env (as described in the link above) to configure with --with-fpm-user=phil --with-fpm-group=phil, but after building, it doesn't change anything - it's still running with www-data (after reading a github issue I think this is because envs cant be changed until later, at which point php is already configured?) (see github issue here)
I'm running out of ideas to try. The only other thing I can think of, is to recursively set owner and group of the code directory on my host to www-data and try run phpstorm as www-data instead, but that feels weird (Update: I tried to open phpstorm as www-data user, using sudo -u www-data phpstorm.sh, but i get a java exception - something to do with graphics -so this approach is unfeasible as well)
Now the only thing I can think of to try is to create a new php image from alpine base image and bypass php's images completely - which seems like an awful lot of inconvenience just because the maintainers want to use ENV instead of ARG?
I'm not sure of best practice for this scenario. Should I be trying to change how php-fpm is run (user/group)? should I be updating the directory owner/group on my host? should I be running phpstorm as a different user?
Literally any advice will be greatly appreciated.
#bnoeafk I'll just post this as a new answer although it has basically been said already. I don't think this is hacky, it works basically like ntfsusermap, certainly more elegant than changing all file permissions.
For the Dockerfile:
FROM php:7.4-apache
# do stuff...
ARG UNAME=www-data
ARG UGROUP=www-data
ARG UID=1000
ARG GID=1001
RUN usermod --uid $UID $UNAME
RUN groupmod --gid $GID $UGROUP
Every user using this image can pass himself into it while building: docker-compose build --build-arg UID=$(id -u) --build-arg GID=$(id -g)
ran into the same problem a few weeks ago.
what actually happens is that your host and your container are sharing the same files via the volume, therefore, they also share the permissions.
in production, everything is fine - your server (the www-data user) should be the owner of the files, so no problem here. things get complicated in development - when you are trying to access those files from the host.
i know a few workarounds, the most hacky one seems to be to set www-data uid in the container to 1000, so it will match your uid in the host.
another simple one is to open 777 full permissions on the shared directory, since its only needed in the development build - (should never be done in production though, but as i mentioned before, in production you dont have any problem, so you must seperate the 2 processes and do it only in development mode)
to me, the most elegant solution seems to be to allow all group members to access the files (set 770 permissions), and add www-data to your group:
usermod www-data -a -G phill #// add it to your group
chown -r phill ./code #// make yourself the owner. might need sudo.
chmod 770 ./code #//grunt permissions to all group members
You have many options depending on your system to do this, but keep in mind you may have to restart your running process (php-fpm for example)
Some examples on how to achieve this: (you can run the commands outside the container with: docker container exec ...)
Example 1:
usermod -g 1007 www-data
It will update the uid of the user www-data to 1007
Example 2:
deluser www-data
adduser -u 1007 -D -S -G www-data www-data
It will delete the user www-data and recreate it with the uid 1007
Get pid and restart process
To restart a a running process, for example php-fpm, you can do it that way:
First get the pid, with one of the following command:
pidof php-fpm
ps -ef | grep -v grep | grep php-fpm | awk '{print $2}'
find /proc -mindepth 2 -maxdepth 2 -name exe -lname '*/php-fpm' -printf %h\\n 2>/dev/null | sed s+^/proc/++
Then restart the process with the pid(s) you got just before (if your process support USR2 signal):
kill -USR2 pid <-- replace pid by the number you got before
I found that the easiest way is to update the host or to build your container knowing the right pid (not always doable if you work with different environments)
Let's assume that you want to set the user of your PHP container and the owner of your project files to www-data. This can be done inside Dockerfile:
FROM php
.
.
.
RUN chown -R www-data:www-data /var/www
USER www-data # next instruction might face permission error if this line is not at the end of the dockerfile
The important fact here is that the original permissions in the Docker host are corresponded to the permission inside the container. Thus, if you now add your current user to www-data group (which probably needs a logout/reboot to take effect), you will have sufficient permission to edit the files outside the container (for instance in your IDE):
sudo usermod -aG www-data your_user
This way, the PHP code is permitted to run executables or write new files while you can edit the files on the host environment.
On my development machine I have a public directory to view a test site. Sometimes I change files using command line PHP and sometimes the files are changed by Apache. This leads to endless conflicts (apache can't write because user owns its OR user can't write because www-data owns it).
I have added myself to the www-data group, but I am still getting errors.
What is the best approach here - i.e. who should own the file, what group, and what should the permissions be (e.g. 0777 - obviously not, but you get the idea).
THANK YOU
# jump into the project's folder
cd /var/www/www.my-site.com
# exec dev/build as user www-data to avoid conflicts
# (www-data has no shell configured, so you must define it via -s option)
su www-data -s /bin/bash -c "php vendor/silverstripe/framework/cli-script.php dev/build \"flush=all\""
My PHP runs "exec" under the user "www-data"
my git repository is owned by "admin"
so it cannot access.
If I change ownership to "www-data" it passes the access problem but I can't create SSH keys because www-data is not a "real" user. I don't even know what that is. where is the home folder for such user?
also, when i change the ownership to "www-data" I cannot use git as admin on that machine anymore. gives me error: cannot open .git/FETCH_HEAD: Permission denied
Where is the home fold for user www-data ?
how can I get it to be able to use git with SSH keys set up?
Transferring ownership to www-data is the right decision. I mean www-data is indeed the owner and since it needs full read/write permissions to that folder anyway - do it!.
The following information is at least true for Debian based systems
www-data is a real user. Also it has a home dir. You can find it using:
cat /etc/passwd | awk -F: '/www-data/{print $6}'
On Debian/Ubuntu it is /var/www for example.
To make the github access possible you can create keys for www-data and place them in /var/www/.ssh. !!!Make sure that this folder is not accessible from web!!!. Then create a machine-user on github add use the public key for www-data.
Creating the keys:
sudo -u www-data ssh-keygen -t rsa
Once you are finished, create a new user on github, name it your-app-machine-user (or whatever), copy /var/www/.ssh/id_rsa.puband add it to that github account.
Read access for www-data should work now.
Try: chown -R www-data:www-data .git/
My PHP application and all the files are owned by www-data however I am currently logged in to the server using my username ruser. I'm developing on the same machine so everytime I want to test changes to my php code I have to go back and forth between file owners.
Is this the best practice or is there a different way I can set this up to make my development smoother?
Make your user part of the www-data group, or make the www-data user part of your group. Then give group access to your files:
usermod -a -G www-data yourusername
In terms of best practice, you should be developing on a different machine anyway. Then deploy your code and use scripts to set up whatever secure configuration is right for your project.
There are a few options open to you.
1) You can change all the files to the www-data group while keeping them owned by you
from your site's root directory run:
chmod -R ruser:www-data ./*
2) The best method is to set up an ACL if your distribution supports it: http://bit.ly/Lat25d
3) Or the simplest method (on a dev machine only - don't do this on a live server) is to chmod everything to 777 from the site's root directory
chmod -R 777 ./*
You can change the files to be owned by any user, i.e use the following command: chown username:groupname file.php
Only require Apache to own the files if Apache is going to write to the files or overwrite the files.
So change the ownership to the FTP user to avoid constant ownership changes.
I'm real new to Mac and Apache. I my development machine my website cannot access any files under the web roots /images or /css folders.
The apache log gives the following error:
(13)Permission denied: file permissions deny server access:
The site is hosted up under the 'Sites' folder. I checked in 'Get Info' on this folder and it seems that 'Everyone' has read access. What gives?
Thanks!
The problem is that Apache runs with a user different to the user owner of files, and the Apache's user doesn't have read/write/execute permissions. In my case the user was _www and is member of the _www group.
I solved this issue changing the group of the files to the _www:
Look for the apache's user and group. I used this php script:
<?php
echo exec('whoami') . '<br>';
echo exec('groups') . '<br>';
?>
Login with the user owner of the files.
Add the user owner of files to the _www group.
$ sudo dseditgroup -o edit -a userOwnerOfFiles -t user _www
Change the group of files needed to _www
$ chgrp -R _www path/containing/files
Change file permissions for the group
$ chmod -R g+rwx path/containing/files
This was a tough one for me today. It turned out that I needed to give permissions to the web server to the entire directory tree all the way up to the doc root.
It came up for me today because I'm using a virtual host and storing the files pretty far up a tree in my user directory.
I did not want to recursively change all the thousands of files in my Documents directory so I just chmod ed each folder in the path. In my home directory:
$ chmod 755 Documents
$ chmod 755 Documents/projects
$ chmod 755 Documents/projects/dev
$ chmod 755 Documents/projects/dev/someglamorousclientname/
$ chmod 755 Documents/projects/dev/someglamorousclientname/docroot
Another alternative way of solving this is using extended attributes in MacOSX
chmod +a "_www allow list,read,search,readattr,readsecurity,file_inherit,directory_inherit" /path/to/document_root
I've found 2 things did the trick for me (I was specifically trying to get apache to have access to the Downloads folder):
In System Preferences -> Security & Privacy -> Privacy scroll to Full Disk Access on the left, make sure you unlock at bottom, and then click the + to add an app. Navigate to /usr/sbin and find the executable httpd and add that, making sure it has full disk access enabled. Re-lock the preferences
Right click the particular folder in Finder and choose Get Info, then under Sharing & Permissions, allow access for the "everyone" user (or if you are trying to be more security conscious, perhaps only allow for "_www" user - but I did not test this).
That solved it for me
This method is safe & fast to test, and easy to switch back if it's not working (it won't mess up things even more, which is ofter a problem when fixing these kind of issues:
Locate httpd.conf (you can do it with httpd -V in terminal)
Open this file in Brackets or any text editor
In this file, find:
User _www
Group _www
Change it to
User {your username}
Group staff
Maybe you will have to add something else to your User and Group:
In this httpd.conf file, you can also find a path to your webserver, just search for DocumentRoot. Copy this path, and navigate to it in terminal with cd command, for example: cd /Library/WebServer/Documents
When you are in, do a ls -l. This will give you info about webroot folder ownership. Adjust your User and Group in the httpd.conf regarding this
You can also enter the webroot folder and check the sites ownerships as well with ls -l, and update httpd.conf regarding that.
If this is not working, don't forget to switch back to:
User _www
Group _www