I try to build docker image for use it in gitlab-ci
What already written:
FROM ubuntu:latest as ubuntu
FROM php:8.1-cli as php
FROM node:14.15.0-stretch AS node
FROM selenium/standalone-chrome:latest
COPY --from=ubuntu /bin /bin
COPY --from=php /app /app
COPY --from=node /usr/local/lib/node_modules /usr/local/lib/node_modules
COPY --from=node /usr/local/bin/node /usr/local/bin/node
RUN ln -s /usr/local/lib/node_modules/npm/bin/npm-cli.js /usr/local/bin/npm
# EXPOSE 4444
# EXPOSE 7900
LABEL miekrif uzvar-selenium
ENTRYPOINT ["/bin/bash"]
What I wanted to achieve: I have a stage in .gitlab-ci.yml with the following points:
test-dev:
stage: test
image: miekrif/uzavr-selenium:latest
script:
- npm i
- npm run prod
- nohup /opt/bin/start-selenium-standalone.sh &
- npx mocha tests/js/screenshots-* --timeout 50000
- npx playwright test tests/js/pw_*
- php artisan test
What I want to achive: Current job maked for test our project, I couldn't think of another way to start selenium\standalone-chrome so that npx could run tests because it refers to 127.0.0.1
Currently I have following failure:
/usr/bin/sh: /usr/bin/sh: cannot execute binary file
Related
I have created a custom Docker image, pushed it to dockerhub, added it to the bitbucket pipeline. It gets pulled but I cannot execute php. Note even php -v works, no output is visible. What am I doing wrong?
I have built the image in an intel processor. Also building the image on bitbucket and directly from there pushing it to dockerhub did not change anything.
This is the output I get...
The image is incuded in the pipeline as follows:
definitions:
services:
mariadb:
image: mariadb:10.6.4
environment:
MYSQL_DATABASE: 'redacted'
MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
MYSQL_USER: 'redacted'
MYSQL_PASSWORD: 'redacted'
steps:
- step: &TestPHP
name: Test PHP
image:
name: "sensetence/hcwcrm-php80fpm:latest"
username: $DOCKERHUB_USERNAME
password: $DOCKERHUB_TOKEN
deployment: test
services:
- mariadb
script:
- cd api
- ls -lh
- ls -lh /usr/local/bin
- php -v
And this is how I am building the image (left out some software installation parts):
FROM php:8.0.25-fpm-alpine
ARG UID
ARG GID
ENV UID=${UID}
ENV GID=${GID}
RUN mkdir -p /var/www/html
WORKDIR /var/www/html
# MacOS staff group's gid is 20, so is the dialout group in alpine linux. We're not using it, let's just remove it.
RUN delgroup dialout
RUN addgroup -g ${GID} --system symfony
RUN adduser -G symfony --system -D -s /bin/sh -u ${UID} symfony
RUN sed -i "s/user = www-data/user = symfony/g" /usr/local/etc/php-fpm.d/www.conf
RUN sed -i "s/group = www-data/group = symfony/g" /usr/local/etc/php-fpm.d/www.conf
RUN echo "php_admin_flag[log_errors] = on" >> /usr/local/etc/php-fpm.d/www.conf
[... here installation, copying files, setting up supercronic]
Solution found. The Dockerfile also loads a php.ini, in this file are the following two lines to warm up opcache:
opcache.preload=/var/www/html/config/preload.php
opcache.preload_user=symfony
Looks like this path is not defined when the image is executed in the bitbucket? At least when the image is started and FPM wants to start running? I don't know, to be honest. I removed these two lines, everything worked.
For a PHP/Symfony project, I currently setup the Gitlab CI (self-hosted CE) for building the Docker images and running tests and code-style checks.
To run it parallel, I have one build job which is building the Docker image and two jobs, the first is running the phpunit tests, the other one runs the code style checks like phpstan and codesniffer.
The project has some composer dependencies which are installed with the command docker-compose run --entrypoint="composer install -n" php. The project folder is a volume configured in the docker-compose.yml file:
php:
image: 'git.cd.de:5050/sf/sf-software:dev_latest'
depends_on:
- database
environment:
TIMEZONE: Europe/Berlin
XDEBUG_MODE: 'off'
XDEBUG_CONFIG: >-
client_host=host.docker.internal
client_port=9003
idekey=PHPSTORM
PHP_IDE_CONFIG: serverName=sf
volumes:
- './:/var/www/html'
- './docker/php/php.ini:/usr/local/etc/php/php.ini:ro'
This is working on my local machine and also in the CI, it's working - but only, if it's running on "the one" gitlab runner, which is installed on a second virtual machine, while the "second runner" is installed on the same machine on which is also gitlab running. The runner "second runner" fails with the following message:
$ docker-compose run --entrypoint="composer install -n" php
Creating cd-software_php_run ...
Creating cd-software_php_run ... done
Composer could not find a composer.json file in /var/www/html
To initialize a project, please create a composer.json file. See https://getcomposer.org/basic-usage
The composer.json file does not exists in the docker image php.
It makes no difference in the result whether the "second runner" performs the test-job or the check-job. The "second job" always fails with that error.
My .gitlab-ci.yml file:
variables:
DOCKER_DRIVER: overlay
before_script:
- apk add --no-cache docker-compose
- docker info
- docker-compose --version
build_dev:
services:
- docker:dind
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker pull $CI_REGISTRY_IMAGE:dev_latest || true
- docker build --cache-from $CI_REGISTRY_IMAGE:dev_latest --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $CI_REGISTRY_IMAGE:dev_latest .
- docker push $CI_REGISTRY_IMAGE:dev_latest
tests:
services:
- docker:dind
needs:
- build_dev
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker-compose pull
- docker-compose run --entrypoint="composer install -n" php
- docker-compose run --entrypoint="bin/console doctrine:migrations:migrate -n" php
- docker-compose run --entrypoint="bin/console doctrine:schema:validate" php
- docker-compose run --entrypoint="bin/console doctrine:fixtures:load -n" php
- docker-compose run --entrypoint="vendor/bin/simple-phpunit -c phpunit.xml.dist" php
checks:
services:
- docker:dind
needs:
- build_dev
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
- docker-compose pull
- ls -la
- docker-compose run --entrypoint="ls -la" php
- docker-compose run --entrypoint="composer install -n" php
- docker-compose run --entrypoint="composer run check-style" php
- docker-compose run --entrypoint="composer run phpstan" php
I'm new with Github Actions and I try to make some continuous integration with functional tests.
I use Codeception and my workflow run perfectly, but when some tests fail the step is written as success. Github don't stop the action and continue to run the nexts steps.
Here is my workflow yml file :
name: Run codeception tests
on:
push:
branches: [ feature/functional-tests/codeception ]
jobs:
build:
runs-on: ubuntu-latest
steps:
# —— Setup Github Actions 🐙 —————————————————————————————————————————————
- name: Checkout
uses: actions/checkout#v2
# —— Setup PHP Version 7.3 🐘 —————————————————————————————————————————————
- name: Setup PHP environment
uses: shivammathur/setup-php#master
with:
php-version: '7.3'
# —— Setup Docker Environment 🐋 —————————————————————————————————————————————
- name: Build containers
run: docker-compose build
- name: Start all container
run: docker-compose up -d
- name: Execute www container
run: docker exec -t my_container developer
- name: Create parameter file
run: cp app/config/parameters.yml.dist app/config/parameters.yml
# —— Composer 🧙️ —————————————————————————————————————————————————————————
- name: Install dependancies
run: composer install
# —— Check Requirements 👌 —————————————————————————————————————————————
- name: Check PHP version
run: php --version
# —— Setup Database 💾 —————————————————————————————————————————————
- name: Create database
run: docker exec -t mysql_container mysql -P 3306 --protocol=tcp -u root --password=**** -e "CREATE DATABASE functional_tests"
- name: Copy database
run: cat tests/_data/test.sql | docker exec -i mysql_container mysql -u root --password=**** functional_tests
- name: Switch database
run: docker exec -t php /var/www/bin/console app:dev:marketplace:switch functional_tests
- name: Execute migrations
run: docker exec -t php /var/www/bin/console --no-interaction doctrine:migrations:migrate
- name: Populate database
run: docker exec -t my_container php /var/www/bin/console fos:elastica:populate
# —— Generate Assets 🔥 ———————————————————————————————————————————————————————————
- name: Install assets
run: |
docker exec -t my_container php /var/www/bin/console assets:install
docker exec -t my_container php /var/www/bin/console assetic:dump
# —— Tests ✅ ———————————————————————————————————————————————————————————
- name: Run functional tests
run: docker exec -t my_container php codeception:functional
- name: Run Unit Tests
run: php vendor/phpunit/phpunit/phpunit -c app/
And here is the logs of the action step :
Github Action log
Anyone know why the step don't faile and how to throw an error ?
Probably codeception:functional set an exit code = 0 even though an error occured. docker exec passes the exit code of the process through. GitHub Actions fails the step if a command returns with an exit code != 0.
I don't know how to do cache dependency in gitlab-ci -> docker.
My project has 82 dependencies and they get very slow.. (vendor is in gitignore)
Full process:
change local file -> comit and push to repo remote -> run gitlab-ci -> build docker image -> push image to other server -> publish image
My example project:
app -> my files (html, img, php, css, anything)
gitlab-ci.yml
composer.json
composer.lock
Makefile
Dockerfile
Dockerfile:
FROM hub.myserver.test/image:latest
ADD . /var/www
CMD cd /var/www
RUN composer install --no-interaction
RUN echo "#done" >> /etc/sysctl.conf
gitlab-ci:
build:
script:
- make build
only:
- master
Makefile:
all: build
build:
docker build hub.myserver.test/new_image .
How I can caching dependencies (composer.json)? I do not want to download libraries from scratch.
Usually it's not a good idea to run composer install inside your image. I assume you need eventually to run your php app not composer itself, so you can avoid having it in production.
One possible solution is to split app image creation into 2 steps:
Install everything outside image
Copy ready-made files to image
.gillab-ci.yml
stages:
- compose
- build
compose:
stage: compose
image: composer # or you can use your hub.myserver.test/image:latest
script:
- composer install # install packages
artifacts:
paths:
- vendor/ # save them for next job
build:
stage: build
script:
- docker build -t hub.myserver.test/new_image .
- docker push hub.myserver.test/new_image
So in Dockerfile you just copy files from artifacts dir from first stage to image workdir:
# you can build from your own image
FROM php
COPY . /var/www
WORKDIR /var/www
# optional, if you want to replace CMD of base image
CMD [ "php", "./index.php" ]
Another good consideration is that you can test your code before building image with it. Just add test job between compose and build.
Live example # gitlab.com
Been banging my head for hours now. I have just gotten started with Docker, and build systems. I have a PHP Codeception (unit testing framework) codebase as a repo example. Locally, I am able to successfully run the test, but when I check-in my code and it goes to the CI (Jenkins), the client program can be invoked but says no tests found.
*note I know that my build definitions could use some work, my goal is just to get a working build.
Dockerfile
FROM codeception/codeception
Makefile
APP_NAME=codeception
ROOT_DIR=${PWD}
WORK_DIR=app
docker_build:
docker build -t $(APP_NAME) .
run_test:
docker run --rm --name ception -w /$(WORK_DIR) -v $(ROOT_DIR):/$(WORK_DIR) $(APP_NAME) run acceptance
Just for arguments sake, this is how it prints in the build system
docker run --rm --name ception -w /app -v /var/jenkins_home/workspace/codeception:/app codeception run
Build Triggers
make docker_build
make run_test
Local Output
OK (1 test, 0 assertions)
Jenkins Output
*note did not fail, but..
no tests executed!
Jenkins Installation
Host is a Docker machine droplet on digital ocean and I ran a Jenkins container voluming Docker sock so I can invoke Docker on build triggers
ANOTHER QUESTION
Given the above preconditions with the command:
*note the pwd
docker run --rm --name ception -w /app -v /var/jenkins_home/workspace/codeception:/app codeception run; pwd
Jenkins output:
/var/jenkins_home/workspace/codeception
I was expecting it to output /app because my understanding is, it invokes pwd from inside the docker container, so this should have outputed the /app directory. I am confused now
Logs:
Started by GitHub push by edsk3o
Building in workspace /var/jenkins_home/workspace/codeception
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/edsk3o/codeception.git # timeout=10
Fetching upstream changes from https://github.com/edsk3o/codeception.git
> git --version # timeout=10
using GIT_ASKPASS to set credentials
> git fetch --tags --progress https://github.com/edsk3o/codeception.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/master^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 1a7dd08ef1ed9e8e7b3f236c50690b65c65f37e8 (refs/remotes/origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 1a7dd08ef1ed9e8e7b3f236c50690b65c65f37e8
Commit message: "laksjd"
> git rev-list --no-walk 528640c7be393aaa06f94edc80f12234a759efd6 # timeout=10
[codeception] $ /bin/sh -xe /tmp/jenkins9213645655529951332.sh
+ make docker_build
docker build -t codeception .
Sending build context to Docker daemon 3.408MB
Step 1/1 : FROM codeception/codeception
---> 1681da57f253
Successfully built 1681da57f253
Successfully tagged codeception:latest
+ make run_test
docker run --rm --name ception -w /app -v /var/jenkins_home/workspace/codeception:/app codeception run acceptance; pwd
Codeception PHP Testing Framework v2.3.8
Powered by PHPUnit 6.5.6 by Sebastian Bergmann and contributors.
[1mAcceptance Tests (0) [22m---------------------------------------
------------------------------------------------------------
Time: 89 ms, Memory: 10.00MB
[30;43mNo tests executed![0m
/var/jenkins_home/workspace/codeception
Finished: SUCCESS