So I recently discovered docker and vagrant, and I'm starting a new Php project in which I want to use both:
Vagrant in order to have a interchangeable environment that all the developers can use.
Docker for production, but also inside the vagrant machine so the development environment resembles the production one as closely as possible.
The first approach is to have all the definition files together with the source code in the same repository with this layout:
/docker
/machine1-web_server
/Dockerfile
/machine2-db_server
/Dockerfile
/machineX
/Dockerfile
/src
/app
/public
/vendors
/vagrant
/Vagrantfile
So the vagrant machine, on provision, runs all docker "machines" and sets databases and source code properly.
Is this a good approach? I'm still trying to figure out how this will work in terms of deployment to production.
Is this a good approach?
Yes, at least it works for me since a few months now.
The difference is that I also have a docker-compose.yml file.
In my Vagrantfile there is a 1st provisioning section that installs docker, pip and docker-compose:
config.vm.provision "shell", inline: <<-SCRIPT
if ! type docker >/dev/null; then
echo -e "\n\n========= installing docker..."
curl -sL https://get.docker.io/ | sh
echo -e "\n\n========= installing docker bash completion..."
curl -sL https://raw.githubusercontent.com/dotcloud/docker/master/contrib/completion/bash/docker > /etc/bash_completion.d/docker
adduser vagrant docker
fi
if ! type pip >/dev/null; then
echo -e "\n\n========= installing pip..."
curl -sk https://bootstrap.pypa.io/get-pip.py | python
fi
if ! type docker-compose >/dev/null; then
echo -e "\n\n========= installing docker-compose..."
pip install -U docker-compose
echo -e "\n\n========= installing docker-compose command completion..."
curl -sL https://raw.githubusercontent.com/docker/compose/$(docker-compose --version | awk 'NR==1{print $NF}')/contrib/completion/bash/docker-compose > /etc/bash_completion.d/docker-compose
fi
SCRIPT
and finally a provisioning section that fires docker-compose:
config.vm.provision "shell", inline: <<-SCRIPT
cd /vagrant
docker-compose up -d
SCRIPT
There are other ways to build and start docker containers from vagrant, but using docker-compose allows me to externalize any docker specificities out of my Vagrantfile. As a result this Vagrantfile can be reused for other projects without changes ; you would just have to provide a different docker-compose.yml file.
An other thing I do differently is to put the Vagrantfile at the root of your project (and not in a vagrant directory) as it is a place humans and tools (some IDE) expect to find it. PyCharm does, PhpStorm probably does.
I also put my docker-compose.yml file at the root of my projects.
In the end, for developing I just go to my project directory and fire up vagrant which tells docker-compose to (eventually build then) run the docker containers.
I'm still trying to figure out how this will work in terms of deployment to production.
For deploying to production, a common practice is to provide your docker images to the ops team by publishing them on a private docker registry. You can either host such a registry on your own infrastructure or use online services that provides them such as Docker Hub.
Also provide the ops team a docker-compose.yml file that will define how to run the containers and link them. Note that this file should not make use of the build: instruction but rely instead on the image: instruction. Who wants to build/compile stuff while deploying to production?
This Docker blog article can help figuring out how to use docker-compose and docker-swarm to deploy on a cluster.
I recommend to use docker for development too, in order to get full replication of dependencies. Docker Compose is the key tool.
You can use an strategy like this:
docker-compose.yml
db:
image: my_database_image
ports: ...
machinex:
image: my_machine_x_image
web:
build: .
volumes:
- '/path/to/my/php/code:/var/www'
In your Dockerfile you can specify the dependencies to run your PHP code.
Also, i recommend to keep my_database_image and my_machine_x_image projects separated with their Dockerfiles because perfectly can be used with another projects.
If you are using Mac, you are already using a VM called boot2docker
I hope this helps.
Related
Either I miss something, or the whole chain lacks something.
Here's my assumption:
The whole point of containerization in development, is to reduce the cost of environment setup, and create a prepared image with all the required pieces.
So, when I read that Laravel Sail is installing laravel via containerization, I get excited. Thus I install it via their instructions, and everything works.
Then the problem begins. Because:
After a successful installation, I create a git repo, with GitHub's default laravel .gitignore
Then I push the newly installed laravel app into my git repo.
Then I ask a developer to start developing it. Please note that:
He does not have PHP installed
He does not have Composer installed
He clonse the repo, and as per installation guide, runs ./vendor/bin/sail up
But ./vender folder is correctly excluded in .gitignore
Thus his command results in:
bash: ./vendor/bin/sail: No such file or directory
He Googles it of course, and finds out that people suggest to run composer update
He goes to install composer, then before that PHP, then all extensoins of PHP, then ...
Do I miss something here? The whole point of containerization was to not install the required environment locally.
What is the proper way of running a laravel app, that is not installed from https://laravel.build, but is cloned from a git repo, WITHOUT having PHP or Composer installed locally?
Update
I found Bitnami laravel docker and it's exactly what containers should be.
You are right and the other developer doesn't need to have php nor composer installed.
All he/she needs is Docker installed on the local machine.
If you scaffolded the project with what is mentioned in the official Laravel docs under the Getting started section, then you will have a docker-compose.yml file in your project root directory.
For Windows
For Linux
For Mac OS
All the developer has to do after git cloning the repository is to run
docker-compose up --build -d
That's it.
For those struggling with this issue... I've found a command that work perfectly fine.
First of all, you don't need to locally have any PHP or Composer installed, maybe there is a misunderstanding about it, all you need is Docker.
Docker will install everything you need in something I understand is like a sandbox, not locally, for each project.
And for those downloaded projects, from GIT as example, that does not have vendor folder, and obviously cannot execute sail up you can simple execute:
docker run --rm --interactive --tty -v $(pwd):/app composer install
That command will download a composer image for docker, if you do not have one yet. Then, will run a composer install and you are free to execute a ./vendor/bin/sail up if you hadn't configured an alias or just sail up if you already configure an alias.
That's all.
The official documentation lists the following command.
docker run --rm \
-u "$(id -u):$(id -g)" \
-v $(pwd):/var/www/html \
-w /var/www/html \
laravelsail/php81-composer:latest \
composer install --ignore-platform-reqs
If you were to clone a Laravel project and run this command in the project root, it would create a very small container with php and composer installed and run composer in the project root to install all php dependencies. In effect, this installs the Laravel core code into the cloned project. Once the project in set up this way, the user should create a local .env file to match their development evironment.
cp .env.example .env # creates a .env file to be populated for the local environment
With the envronment set up, they can now create the application containers in docker and run the application. Laravel provides the Sail helper for this.
./vendor/bin/sail up -d # runs the docker containers in detached mode
Now it's a matter of setting up the laravel app and running the Laravel app. (I'm assuming the app uses one of the Laravel start kits that rely on Node.js. If you are using a Blade only application, you can skip the "npm" commands.)
sail artisan key:generate # (Best Practice) Generate a new application key on each machine
sail artisan migrate # Scaffold the database structure
sail artisan db:seed # (Optional) Seed the database with data
sail npm install # (Optional) Install front-end dependencies (Inertia, Vue, React, others...)
sail npm run dev # (Optional) Run the front-end framework in development mode
With this, the new developer should be running an exact copy of both the project and the development environment as the original developer.
Your project README may include additional steps to set up some other dependencies, but this is the basic workflow for contributing to a Laravel project.
The only prerequisites for this workflow is to have Docker installed with an Internet connection. This is most easily accomplished on Windows, Mac, and Linux by installing Docker Desktop.
Alternate for Older Projects
If you are working on an older project that doesn't use Laravel Sail, but does have a docker-compose.yml file, you should be able to build and run the necessary containers with the following command.
docker-compose up --build -d
Once you have the containers running, you would need to install the project dependencies directly into the container.
docker ps # find the container ID of your project's container
docker exec -it CONTAINER_ID php artisan key:generate
docker exec -it CONTAINER_ID php artisan migrate
docker exec -it CONTAINER_ID php artisan db:seed
docker exec -it CONTAINER_ID npm install
docker exec -it CONTAINER_ID npm run dev
Of course, Docker Desktop simplifies this process. With a button click you can have a terminal shell open directly in your container eliminating the need for the docker exec command.
I have created a command line application using symfony 3.4 which doesn't need to display any web page.
I generally run the commands like following:
php bin/console MY_COMMAND_NAME
I want to dockerize the application and share it with others, so inside the root directory of my project I created a docker-compose.yml file, which looks like following:
version: "3.3"
services:
web:
image: php:7.3-cli
Then I ran docker-compose up, after that I checked the PHP version by the following command and it showed my the correct version:
docker run php:7.3-cli php -v
However, when I ran docker ps, it didn't show any container running.
My question is how to run the commands inside my project root directory. FYI, I am using Docker Toolbox, on windows 10 Home Edition and my project location is:
C:\Users\{my_user_name}\Desktop\folder_1\folder_2
The docker container need to have a long running process defined in CMD to stay running. php-cli is not that. If you run composer up, you'll see something like this:
$ docker-compose up
Creating network "tempphpdocker_default" with the default driver
Pulling web (php:7.3-cli)...
7.3-cli: Pulling from library/php
b8f262c62ec6: Pull complete
a98660e7def6: Pull complete
4d75689ceb37: Pull complete
639eb0368afa: Pull complete
2cdbfdb779b1: Pull complete
e0b637fa9606: Pull complete
da7333b0ef25: Pull complete
01d65ff46009: Pull complete
673e50bed3b9: Pull complete
bf6c6e34305d: Pull complete
Digest: sha256:1453f5ef0d4d1d424ed8114dd90a775bdec06cc6fb3bbae9521dcb4ca0c8ca90
Status: Downloaded newer image for php:7.3-cli
Creating tempphpdocker_web_1 ...
Creating tempphpdocker_web_1 ... done
Attaching to tempphpdocker_web_1
web_1 | Interactive shell
web_1 |
tempphpdocker_web_1 exited with code 0
The exit code is 0. This means your command in the docker image php:7.3-cli has successfully run and finished.
To properly dockerize your applicaiton, you should override this by writing you own docker file with proper COPY calls that bundle your CLI program into it. Your Dockerfile should probably look something like this:
FROM php:7.3-cli
RUN mkdir -p /opt/workdir/bin
RUN mkdir -p /opt/workdir/vendor
COPY bin/ /opt/workdir/bin
COPY vendor/ /opt/workdir/vendor
WORKDIR /opt/workdir
CMD php ./bin/console COMMAND
You can simply build and run this Dockerfile, or you if you prefer docker-compose, you can define docker-compose.yml in the same folder as the Dockerfile:
version: "3.3"
services:
web:
image: php-custom
build: ./
Please noted that a dockerized application can only access files and folder in the docker image. You should bind volumes of your local file system to the container before it can actually work on your filesystem.
Quick and dirty fix to keep you container running just override the container command in docker-compose.
version: "3.3"
services:
web:
image: php:7.3-cli
command: tail -f /dev/null
when you run docker-compose up it will keep the docker container but it will do not thing, just will give away to run command inside container.
docker exec -it php-cli_web_1 ash
My question is how to run the commands inside my project root
directory.
As mentioned by #David, you need to mount your host project to the container in docker-compose.
For instance
your project is placed on the host /home/myporject, mount the project within docker-compose and it will be available inside the container. then you can update the command of your docker-compose to run the script.
keep in mind
The life of container is the life of docker-compose command
When the execution completed your container will be die after execution. so your container will run until the php:7.3-cli /app/your_script.php this script completed.
version: "3.3"
services:
web:
image: php:7.3-cli
command: php:7.3-cli /app/your_script.php
volumes:
- /home/myporject:/app
I try to run a simple Symfony 4 project on a docker container.
I have tested regular PHP scripts, and they work very well. But, with Symfony project, the execution gets ridiculously slow. For example, a page without any significant content takes 5-6 seconds.
I have attached the screenshots from Symfony's performance profiler.
Do you have any idea what how to reduce this execution time to an acceptable level?
It seems that changing the consistency level greatly increases Symfony performance. (see Docker docs)
Here is my new docker-compose.yml file. Note the ":cached" after the volumne.
version: '3'
services:
web:
image: apache-php7
ports:
- "80:80"
volumes:
- .:/app:cached
tty: true
Note from manual:
For directories mounted with cached, the host’s view of the file
system is authoritative; writes performed by containers are
immediately visible to the host, but there may be a delay before
writes performed on the host are visible within containers.
Since the provided answer is working with macOSX, only, but performance issues exist with Docker for Windows as well the preferred answer didn't help in my case. I was following different approach partially described in answers to similar questions here on SO.
According to Performance Best Practices folders with heavy load such as vendor and var in a Symfony application shouldn't be part of a shared mount. If you require to persist those folders you should use volumes instead.
To prevent interferences with shared volume in /app I was relocating those two folders to separate folder /symfony in container. In Dockerfile folders /symfony/var and /symfony/vendor are created in addition.
The script run on start of container is setting symbolic links from /app/var to /symfony/var and from /app/vendor to /symfony/vendor. These two new folders are then mounted to volumes e.g. in a docker-compose.yml file.
Here is what I was adding to my Dockerfile:
RUN mkdir /app && mkdir /symfony/{var,vendor}
COPY setup-symfony.sh /setup-symfony.sh
VOLUME /symfony/var
VOLUME /symfony/vendor
Here is what I was adding to my startup script right before invoking composer update or any task via bin/console:
[ -e /app/var ] || ln -s /symfony/var /app/var
[ -e /app/vendor ] || ln -s /symfony/vendor /app/vendor
This is what my composition looks like eventually:
version: "3.5"
services:
database:
build:
context: docker/mysql
volumes:
- "dbdata:/var/lib/mysql"
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: 1
application:
depends_on:
- database
build:
context: docker/lamps
ports:
- "8000:8000"
volumes:
- ".:/app:cached"
- "var:/symfony/var"
- "vendor:/symfony/vendor"
environment:
DATABASE_URL: mysql://dbuser:dbuser#database/dbname
volumes:
dbdata:
var:
vendor:
Using this setup Symfony is responding within 500ms rather than taking 4000ms and more.
UPDATE: When using an IDE for developing Symfony-based application like PhpStorm you might need the files in vendor/ for code assist or similar. In my case I was able to take a snapshot of those files and put them into a different folder which is shared with host as well, but isn't actively used by Symfony/PSR, e.g. vendor.dis/. This snapshot is taken manually once per install/upgrade e.g. by entering the running container with a shell like so:
docker exec -it IDofContainer /bin/sh
Then in shell invoke
cp -Lr vendor vendor.dis
Maybe you have to fix the pathnames or make sure to switch into folder containing the your app first.
In my case using PhpStorm the vendor.dis/ was picked up by background indexing and obeyed by code inspection and code assist. Visual Studio code was having issues with the great number of untracked changes with regards to git so I had to explicitly make this snapshot ignored by git, adding its name in .gitignore file.
UPDATE 2020: More recent setups may have issues with accessing folders like /symfony/templates or /symfony/public e.g. on warming up the cache. This is obviously due to using relative folders in auto-loading code now existing in /symfony/vendor due to relocation described above. As an option, you could directly mount extra volumes in /app/var and /app/vendor instead of /symfony/var and /symfony/vendor. Creating deep copies of those folders in /app/var.dis and /app/vendor.dis keeps enabling code assist and inspections in host filesystem.
do not sync the vendor folder
In your docker file, you can prevent the vendor folder to sync with the container. This has the biggest impact on performance because the folder gets very huge:
#DockerFile:
volumes:
- /local/app:/var/www/html/app
- /var/www/html/app/vendor # ignore vendor folder
This will have the effect that you will need to copy the vendor folder manuelly to the container once after the build and when you update your composer dependencies:
docker cp /local/app/vendor <CONTAINER_ID>:/var/www/html/app/
do not sync the cache folder
in your src/Kernel.php:
public function getCacheDir()
{
// for docker performance
if ($this->getEnvironment() === 'test' || $this->getEnvironment() === 'dev') {
return '/tmp/'.$this->environment;
} else {
return $this->getProjectDir().'/var/cache/'.$this->environment;
}
}
sync the app folders in cached mode
use cached mode for volume mounts on development environments: http://docs.docker.oeynet.com/docker-for-mac/osxfs-caching/#delegated
The cached configuration provides all the guarantees of the delegated
configuration, and some additional guarantees around the visibility of
writes performed by containers. As such, cached typically improves the
performance of read-heavy workloads, at the cost of some temporary
inconsistency between the host and the container.
For directories mounted with cached, the host’s view of the file
system is authoritative; writes performed by containers are
immediately visible to the host, but there may be a delay before
writes performed on the host are visible within containers.
This makes sense for dev envrionemtns, because normally you change your code with your IDE on the host not in the container and sync into the container.
#DockerFile:
volumes:
- /local/app:/var/www/html/app:cached
disable Docker debug mode
check if Docker is NOT in debug mode:
docker info
# It Should display: Debug Mode: false
Disable in docker-config:
{
"debug": false,
}
do not use a file cache
this is extra slow in a docker box, use for examle a SQLITE cache: Symfony Sqlite Cache
for Windows 10 users: Use Docker Desktop with WSL 2 support
Use Docker Desktop with WSL 2 support, whichs incredibley boosts performance in general:
https://docs.docker.com/docker-for-windows/wsl/
Prevent syncing the vendor directory with the container:
# docker-compose.yml:
volumes:
- ./app:/var/www
- /var/www/vendor # ignore vendor map
When building in your Dockerfile copy the vendor map to the container location:
# Dockerfile
COPY app/vendor /var/www/vendor
Sebastian Viereck his answer helped me solve this. Loading went from 14000 to 500ms average on Symfony 5.3
The only downside is that you have to rebuild after you add/update something via composer. But thats not all to bad.
One more very important thing for container's performances.
It's essential to check if a Dockerfile contain build of unnecessary layers.
For example,
Bad Practice -> use multiple unnecessary chained RUN
Best Practice -> use && from shell for chianed command as often as possible
e.g. , for example
We might write in our Dockerfile:
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf
&& apt-get update && apt-get install -y --no-install-recommends \
locales apt-utils git \
\
&& echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
&& echo "fr_FR.UTF-8 UTF-8" >> /etc/locale.gen \
&& locale-gen \
Instead of :
RUN echo "ServerName localhost" >> /etc/apache2/apache2.conf
RUN apt-get update && apt-get install -y --no-install-recommends \
locales apt-utils git
RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
&& echo "fr_FR.UTF-8 UTF-8" >> /etc/locale.gen
RUN locale-gen
More layers improve container's slowness... Check your Server Dockerfiles friends !
I hope this comment help someone somewhere !
You can avoid using bind mounts which are extremely slow on Mac or Windows when they contain a big amount of files.
So, instead you can sync files between the host and the container volumes by using Mutagen, it's almost as fast as native with Linux. A benchmark is available here.
Here is a basic configuration of Mutagen:
sync:
defaults:
ignore:
vcs: true
permissions:
defaultFileMode: 644
defaultDirectoryMode: 755
codebase:
alpha: "./app" # dir of your app
beta: "docker://project_container_1/var/www" # targets an absolute path in the container named project_container_1
mode: "two-way-resolved"
This repository shows a full configuration with a simple PHP project (Symfony 5) but it can be used for any type of project in any language.
I would recommend using docker-sync. I have used it myself and it reduced the load time of my Laravel based app.
Developing with docker under OSX/ Windows is a huge pain, since sharing your code into containers will slow down the code-execution about 60 times (depends on the solution). Testing and working with a lot of the alternatives made us pick the best of those for each platform, and combine this in one single tool: docker-sync.
I have a project on gitlab, and I would use gitlab CI for unit testing.
Actually, I have a other repository name "docker" with docker-compose.yml and Dockerfile for two project (because i reproduce the production configuration, the two project are interdependant)
Actually in my dev configuration
in Projects directory:
docker
project_1
project_2
in docker directory:
docker-compose.yml
Dockerfile-project1
Dockerfile-project2
[some config file ask in dockerFile]
docker-compose.yml have relative path as ../project_1 and ../project_2
For set up my configuration, I make :
cd docker
docker-compose up -d project1 (name in docker-compose.yml)
docker exec -ti project1 bash
Question ?
I want know how I can pull the git repository "docker" and launch docker-compose up for the project1 since gitlab CI start ?
Thanks
We've built a gitlab runner with docker-compose support. See its README for setup and configuration.
Basically you just use the same commands like in development, see here for an example with Makefiles or this one with native docker-compose commands.
After years of spaghetti code (I'm Italian, I do really know what spaghetti are) I'm trying to set up a decent php development environment.
This is my battleplan:
install git and docker on my laptop;
create a docker virtual enviroment as much similar as possible to
the production LAMP (shared) server;
use sshfs to mount the docker VE web server root directory on my laptop;
on the laptop, init a git repository inside the mount point;
use my favourite ide (aptana studio) to create a php project in the mount point directory;
test the code pointing a browser to docker VE ip;
set up a bitbucket account to automatically deploy git commits on production server.
What do you think about? Any chance it could work?
Thanks!
I suggest using the official php language docker image:
https://registry.hub.docker.com/_/php/
This enables you to create an image that packages your php code, rather than having to map volumes at run-time.
Example 1 (with Dockerfile)
├── build_and_run.sh
├── Dockerfile
└── src
└── index.php
Dockerfile
FROM php:5.6-apache
COPY src/ /var/www/html/
build_and_run.sh
Script that builds a new container image and launches it:
docker build -t my-php-app .
docker run -it --rm --name my-running-app -p 8080:80 my-php-app
Apache is configured to listen on port 8080
Example 2 (without Dockerfile)
The php image can also be run without a docker file. Just provide a mapping to the source code locally:
docker run -it --rm --name my-apache-php-app -v "$(pwd)/src":/var/www/html -p 8080:80 php:5.6-apache