TravisCI Docker Permissions - php

I use Docker to run my PHP app. Now I want Travis CI to test my app. But all my builds fail, because containers can not open files in /temp directory of my app.
I have data container:
FROM ubuntu
COPY ./ /project
VOLUME /project
CMD ["true"]
I use my own PHP fpm container based on php:7.0-fpm - only with some extensions installed. Same for nginx.
My docker compose looks like this
version: '2'
services:
data:
build: ./
volumes:
- .:/project
command: "true"
nginx:
image: myNginx
ports:
- "80:80"
- "443:443"
volumes_from:
- data
links:
- php
php:
image: myPhp
ports:
- "9000:9000"
volumes_from:
- data
You can see that I use shared volume with host. Everything works on my local machine (windows) but not on travis.
And finaly my travis.yml
sudo: required
language: php
services:
- docker
before_script:
- docker-compose up --build -d
# Run firefox
- docker run -d -p 4444:4444 -p 5900:5900 --name firefox --link my_nginx:nginx --net myapp_default selenium/standalone-firefox-debug:2.53.0
script:
# Run Codeception
- docker run --rm --volumes-from my_data --link firefox --net myapp_default --name codeception codeception/codeception run accept
All codeception tests fails because app can not write to /log and can not open files in /temp. It is interesting because it writes some files but later it can not open it.
I ls -la with result:
# ./temp
drwxrwxr-x 3 travis travis 4096 Nov 4 15:55 .
drwxrwxr-x 14 travis travis 4096 Nov 4 15:56 ..
drwxr-xr-x 4 root root 4096 Nov 4 15:55 cache
-rw-rw-r-- 1 travis travis 14 Nov 4 15:49 .gitignore
# ./temp/cache
drwxr-xr-x 4 root root 4096 Nov 4 15:55 .
drwxrwxr-x 3 travis travis 4096 Nov 4 15:55 ..
drwxr-xr-x 2 root root 4096 Nov 4 15:55 Nette.Configurator
drwxr-xr-x 2 root root 4096 Nov 4 15:55 _Nette.RobotLoader
# ./temp/cache/Nette.Configurator
drwxr-xr-x 2 root root 4096 Nov 4 15:55 .
drwxr-xr-x 4 root root 4096 Nov 4 15:55 ..
-rw-r--r-- 1 root root 116093 Nov 4 15:55 Container_70d15d6361.php
-rw-r--r-- 1 root root 0 Nov 4 15:55 Container_70d15d6361.php.lock
-rw-r--r-- 1 root root 52913 Nov 4 15:55 Container_70d15d6361.php.meta
I ma pretty sure that I have bad permissions set in travis or in container, but I don`t know how to fix it.

The files that Travis pulls will be owned by the travis user and group, while the processes running inside the container expect the active user to be the owner. I had this issue with a docker-compose file that was running fine on my Mac, but failed on Travis.
For me, this was fixed by adding this to the install step in my .travis.yaml (edit: you'll probably want it in your before_script section instead):
install
- docker-compose run --user='root' --entrypoint chown worker_test -R myuser:myuser .
This writeup on UID/GID bits was helpful: Understanding user file ownership in docker: how to avoid changing permissions of linked volumes

Related

Docker: Forbidden You don't have permission to access this resource

Im trying to reach http://localhost:8100 and all i get is: "Forbidden
You don't have permission to access this resource.
Apache/2.4.38 (Debian) Server at localhost Port 8100"
In the src/ i have the docker file and a php folder in which i have both index.php and mysql.php files.
The docker-compose.yml looks like this:
version: '3'
services:
db:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: test_db
MYSQL_USER: devuser
MYSQL_PASSWORD: devpass
ports:
- "9906:3306"
web:
image: php:7.4.0-apache
container_name: php_web
depends_on:
- db
volumes:
- ./php/:/var/www/html/
links:
- db
ports:
- "8100:80"
stdin_open: true
tty: true
Commands on the terminal:
docker-compose up -d
docker exec -ti php_web sh
docker-php-ext-install mysqli
When I try to reach localhost:8100/mysql.php it works fine but when i try to open either localhost:8100 or localhost:8100/index.php I get the error.
Any clues on what could be causing it? I hope Im being clear with my question.
Thanks in advance!
As per output of docker exec -ti php_web ls -la /var/www/html/, i can see space in the name of the file :
-rw-r--r-- 1 root root 70 Jul 11 23:06 ' index.php'
drwxr-xr-x 5 root root 160 Jul 11 22:40 .
drwxr-xr-x 1 root root 4096 Nov 22 2019 ..
drwxr-xr-x 4 root root 128 Jul 11 22:31 includes
-rwxr-xr-x 1 root root 118 Jul 11 22:43 mysql.php
rename it and remove the space
mv ' index.php' index.php

laravel fails to load videos from directory mapped as a volume in a docker container

I'm building a laravel app packaged as a docker container. Our home and product pages play HD videos and I want to avoid storing these videos in source control as they might change a bit frequently.
The idea is to put updated videos in a directory, add updated video names as a config property which can be picked up by blade when rendering the webpage. Adding hardcoded video file names in the code shared the but the intro-xxxxxxxx would come from a variable read from the .env file later on
<div class="background-holder overlay overlay-2 parallax d-none d-sm-block">
<video autoplay="autoplay" loop="loop" muted="muted">
<source src="{{ asset('videos/intro-72126f2acfb477064fe31fcfaea14e79.webm') }}" type="video/webm" />
<source src="{{ asset('videos/intro-01be5dedca0a6a6259c107e69d61af8d.mp4') }}" type="video/mp4" />
<source src="{{ asset('videos/intro-213e0ed58dcc68f1089d0fc13408e586.ogv') }}" type="video/ogv" />
</video>
</div>
now I'm mapping the directory with all these videos as a docker volume like so; in the Dockerfile I define a working directory
# Set working directory
WORKDIR /var/www
and then in Docker compose I mount volumes
version: "3.8"
services:
app:
build:
args:
user: www
uid: 1000
context: ./
dockerfile: Dockerfile
container_name: onex-webapp
restart: unless-stopped
tty: true
environment:
SERVICE_NAME: app
SERVICE_TAGS: dev
working_dir: /var/www
volumes:
- ./:/var/www
- /Users/anadi/Movies/OneX/:/var/www/public/videos
networks:
- onexweb
nginx:
image: nginx:alpine
container_name: onex-webserver
restart: unless-stopped
tty: true
ports:
- 8000:80
volumes:
- ./:/var/www
- ./nginx/dev/conf.d/:/etc/nginx/conf.d/
networks:
- onexweb
networks:
onexweb:
driver: bridge
when I check contents of the container I see this directory and the videos
❯ docker exec onex-webapp ls -l /var/www/public/videos
total 987168
-rw-r--r-- 1 www www 64285377 Apr 23 08:18 fr-intro.mp4
-rw-r--r-- 1 www www 78065607 Apr 27 07:27 fr-intro.ogv
-rw-r--r-- 1 www www 4212950 Apr 27 07:16 fr-intro.webm
-rw-rw-r-- 1 www www 11110738 Oct 23 2018 fr3-intro.mp4
-rw-r--r-- 1 www www 22746906 Jun 23 07:43 isc-intro.mp4
-rw-r--r-- 1 www www 30409396 Jun 23 07:43 intro-01be5dedca0a6a6259c107e69d61af8d.mp4
-rw-r--r-- 1 www www 36319165 Jun 23 07:43 intro-213e0ed58dcc68f1089d0fc13408e586.ogv
-rw-r--r-- 1 www www 22733111 Jun 23 07:43 intro-72126f2acfb477064fe31fcfaea14e79.webm
-rw-r--r-- 1 www www 29432716 Apr 27 15:00 ta-intro.mov
-rw-r--r-- 1 www www 18688528 Jul 17 2018 ta-intro.mp4
-rw-r--r-- 1 www www 21402515 Apr 29 05:30 ta-intro.ogv
-rw-r--r-- 1 www www 9233096 Apr 29 05:38 ta-intro.webm
But when I access the pages I get a 404 error for these video URLs
GET http://localhost:8000/videos/intro-72126f2acfb477064fe31fcfaea14e79.webm net::ERR_ABORTED 404 (Not Found)
If I give up the volumes way and add a symlink to these videos' directory under public/videos then I do not get a 404 error; curious what's causing this, and possible solutions. In production I will be packaging contents into docker container layers instead of mounting volumes hence the need to solve this.
Moved /videos as a directory config to nginx and mounted directory with videos as volume to nginx container, it works now.

How to reflect host permission in the container docker?

I know that it is impossible to change the permissions of a file shared via volume, because a matter of default, and from what I understand, the permissions of the docker container, reflect the permissions of the host, however, this is not happening in my case, the docker simply changes permissions on all files to 755, and some files must have specific permissions.
HOST:
Docker Container:
Docker File:
FROM ubuntu:18.04
ARG DEBIAN_FRONTEND=noninteractive
#Updating operating system
RUN apt-get update && apt-get -y upgrade && apt-get -y dist-upgrade
##Installing essential packages
RUN apt-get -y install apt-utils software-properties-common curl bash-completion vim git supervisor
## Add Scripts
ADD ./start.sh /start.sh
EXPOSE 80
STOPSIGNAL SIGTERM
#CMD ["/start.sh"]
ENTRYPOINT echo $XDEBUG_CONFIG >> /etc/php/7.3/fpm/php.ini && service php7.3-fpm start && nginx -g "daemon off;"
docker-compose.yml
volumes:
- ${DOCUMENT_ROOT-./www}:/usr/share/nginx/html
- ${VHOSTS_DIR-./config/nginx/sites-enabled}:/etc/nginx/sites-enabled
- ${PHP_INI-./config/php/php.ini}:/etc/php/7.3/fpm/conf.d/php.ini
- ${LOG_DIR-./logs/nginx}:/var/log/nginx
Your assumption
I know that it is impossible to change the permissions of a file shared via volume
Is only partially correct, there is actually a set of modes — :ro & :rw – you can use when mounting a volume via docker-compose that are described in the documentation:
Standard modes are ro for read-only and rw for read-write (default).
Source: https://docs.docker.com/compose/compose-file/#short-syntax-3
You can also use the :Z and :z modes if your host uses selinux.
If you use selinux you can add the z or Z options to modify the selinux label of the host file or directory being mounted into the container. This affects the file or directory on the host machine itself and can have consequences outside of the scope of Docker.
The z option indicates that the bind mount content is shared among multiple containers.
The Z option indicates that the bind mount content is private and unshared.
Use extreme caution with these options. Bind-mounting a system directory such as /home or /usr with the Z option renders your host machine inoperable and you may need to relabel the host machine files by hand.
Source: https://docs.docker.com/storage/bind-mounts/#configure-the-selinux-label
Here is an example, on my host, here are the permissions of my files:
~ # ls -la ro rw
ro:
total 0
drwxr-xr-x 3 ben staff 96 May 23 23:06 .
drwxr-xr-x 9 ben staff 288 May 23 23:16 ..
-rw-r--r-- 1 ben staff 0 May 23 23:06 file
rw:
total 0
drwxr-xr-x 3 ben staff 96 May 23 23:06 .
drwxr-xr-x 9 ben staff 288 May 23 23:16 ..
-rwxr-xr-x 1 ben staff 0 May 23 23:06 file
Then with this docker-compose.yml
version: '3.8'
services:
test:
image: alpine
volumes:
- ./ro:/root/ro:Z
- ./rw:/root/rw:Z
command: sleep 100000000000
Here is the result on the container
~ # ls -la ro rw
ro:
total 4
drwxr-xr-x 3 root root 96 May 23 21:06 .
drwx------ 1 root root 4096 May 23 21:17 ..
-rw-r--r-- 1 root root 0 May 23 21:06 file
rw:
total 4
drwxr-xr-x 3 root root 96 May 23 21:06 .
drwx------ 1 root root 4096 May 23 21:17 ..
-rwxr-xr-x 1 root root 0 May 23 21:06 file

docker-compose mounted directory does not rename folders

So I have gotten a folder to rename in a directory. Every other folder I put in that mounted directory says:
Warning:
rename(/app/public/CoverImages/S3-Uploaded/subfolder-for-depth/2019-01-15-Uploaded,/app/public/CoverImages/S3-Uploaded/subfolder-for-depth/2019-01-15-Uploaded-AWS):
Permission denied in /app/src/S3CoverImages.php on line 260
My php code is the following:
if(is_dir($dirPath)) {
$success = rename(rtrim($dirPath, " -/"), rtrim($dirPath, " -/") . "-AWS");
My dockerfile code for permissions is as follows:
RUN chown www-data:www-data /usr/local/etc/php-fpm.conf && \
chown -R www-data:www-data /app && \
chmod -R 755 /app && \
chown -R www-data:www-data /var/www && \
chmod +x /usr/local/bin/start.sh; sync
In docker-compose.yml we have it mounting the directory we need:
version: '3.4'
services:
nginx:
image: americanreading/someproject:dev
restart: on-failure:3
ports:
- 8001:8080
php:
image: americanreading/bisapi-php:dev
environment:
#lots of server details here.
volumes:
- /mnt/coverimages/S3-Uploaded/subfolder-for-depth:/app/public/CoverImages/S3-Uploaded/subfolder-for-depth/
Is there something else I need to make sure any folder put into this mounted directory will have php permission to rename the folder? Is there anything evident that I am not doing at the moment?
Almost forgot, here are the current permissions we looked up in bash:
arcit#arcvm13:~/arcbis-dev$ docker-compose run --rm php bash
www-data#9a78e9799427:/app$ ls -la /app/public/CoverImages/S3-Uploaded/subfolder-for-depth/
total 16
drwxrwxrwx 2 root root 4096 Feb 25 14:31 .
drwxr-xr-x 3 root root 4096 Feb 25 14:38 ..
-rwxrwxrwx 1 root root 6148 Feb 25 14:32 .DS_Store
drwxrwxrwx 2 root root 0 Oct 20 2015 2010-02-15
drwxrwxrwx 2 root root 0 Feb 25 14:30 2019-01-15-Uploaded
drwxrwxrwx 2 root root 0 Feb 25 14:31 2019-02-07-Uploaded-AWS
drwxrwxrwx 2 root root 0 Feb 22 16:24 2019-02-18
Here is the ls -la on the subfolder-for-depth itself:
drwx------# 1 toddcoulson staff 16384 Oct 20 2015 2010-02-15
drwx------# 1 toddcoulson staff 16384 Feb 25 09:30 2019-01-15-Uploaded
drwx------ 1 toddcoulson staff 16384 Feb 25 09:31 2019-02-07-Uploaded-AWS
drwx------# 1 toddcoulson staff 16384 Feb 22 11:24 2019-02-18
The volume is created when you run the container, not when you build it. By default the permissions on that directory will be root.
An other issue here is that you are binding your volume to a host path "/mnt/coverimages/S3-Uploaded/subfolder-for-depth".
So that directory is going to have the permissions of your host (for example user 1000).
A solution is to add that user the the www-data group, in your Dockerfile.

vagrant / puphpet : running apache as vagrant user

I am developing symfony inside of a vagrant box. Recently I tried to boost performance (10k latency) and found this article.
It is suggesting to use /dev/shm/ for cache and logs.
This works like a charm, but comes with a problem. Lets say, I create the assets. Then the created files belong to the wrong user, because I am logged in as a vagrant user.
[05:33 ]-[vagrant#machine1]-[/var/www/backend]-[git dippingbird/master]
$ ls -lsah /dev/shm/appname/
total 0
0 drwxrwxr-x 4 vagrant vagrant 80 Mar 21 05:28 ./
0 drwxrwxrwt 3 root root 80 Mar 21 05:28 ../
0 drwxrwxr-x 3 vagrant vagrant 60 Mar 21 05:28 cache/
0 drwxrwxr-x 2 vagrant vagrant 60 Mar 21 05:29 logs/
When I want to visit the page in dev environment, it clearly does not work. It says in the browser:
If I change the permissions to -R 777 it clearly works.
So I figured out, that I want apache run as the vagrant user. But internet documentation is vague. I want to do it via my puphpet config.yaml so it is available for all developers in the team.
Or do I have a braintwist? Why is this permission error not occuring when I am using the dirname(__DIR__).'/var/logs'; for logs / cache.

Categories