We are using Atlassian Bamboo to deploy our web applications to testing and production servers. This is a two-step process.
build and test the release
deploy the release to the environment
This run relatively stable, but we are running into some issues with Symfony 2 projects.
Step 1 simply checks out the most recent version from the app from Git, does some tests and other tasks, including composer:install. This last one will execute some scripts (post-install): buildBootstrap, clearCache, installAssets, installRequirementsFile and removeSymfonyStandardFiles.
This step is executed on the build server. Since the parameters.yml file is not present in Git, composer install fails. If we do a composer install --no-scripts, the build succeeds as these scripts are never called.
Step 2 is to ship the files to the production server, install the parameters.yml (which is copied from a predefined location on the target server), do a app/console cache:clear and app/console assets:install. The release appears to be working just fine on the target server, but the buildBootstrap, installRequirementsFile and removeSymfonyStandardFiles scripts or equivalent have not run. What are the consequences of that? Are there any app/console alternatives for them (running app/console doesn't appear to show any)?
Alternatively, are we just doing it wrong? We want to let as much work be done by the build server, as the target servers are often limited in capabilities (eg. shared hosting).
Build
checkout from git
copy parameters for test purposes
do a composer install (not an update)
run phpunit for testing (using PHPUnit as task type)
After test if all is good an Artifact will be created, so add some tasks to remove all libraries:
rm -rf vendor
rm -rf app/cache
The build ends here.
Deploy
Download the artifact
I'm deploying with Capistrano/Capifony, so here I run it.
For your question about composer scripts don't worry if they doesn't runs, you don't need it if vendor list doesn't change. My advise in any case is to have small artifact which doesn't include libraries, and install libs with composer. If you want to do heavy work on deploy server (and not in production) you can configure Capifony with set deploy_via, :copy.
Related
I'm using PHP_CodeSniffer in my GitLab CI/CD pipelines to ensure my code is properly formatted. The job looks like follows:
stages:
- test
- build
- deploy
coding_standard:
stage: test
script:
- curl -OL https://squizlabs.github.io/PHP_CodeSniffer/phpcs.phar
- php phpcs.phar --extensions=php .
That's working as expected. However, the exact version of the tool is not specified here. So if there's suddenly a new major version of PHP_Codesniffer, the CI/CD job might fail, although my PHP code hasn't changed.
Furthermore, I currently have the tool installed globally on my local machine. In that way, I cannot have a specific version of the tool for every PHP project.
Now I'd like to add the tool as Composer dev-dependency (require-dev).
In the CI/CD job I would then call composer install instead of downloading the tool via curl.
The problem: That will download all packages needlessly, instead of just PHP_Codesniffer and its dependencies. Can I prevent that?
You can't do this with composer. You can't even install "only the dev dependencies". It's all the dependencies, all the non-dev dependencies, and that's all.
And it's generally a bad idea to install this kind of dependency as a project dependency, since very easily you can enter in dependency hell for reasons beyond your actual application needs. Development tools should not bring that level of complexity and danger to your deployment strategy.
To get around this, you could use something like the Composer Bin Plugin to isolate these dependencies and yet install them through composer. Then on CI you'd run composer install on this directory only, and run the tool from this location (or symlink it to bin, which is what the plugin does when it's installed, but you wouldn't have it installed in CI if you are not installing all the dependencies anyway).
Why not download any tagged version from Github through https://github.com/squizlabs/PHP_CodeSniffer/releases, like https://github.com/squizlabs/PHP_CodeSniffer/releases/download/3.6.0/phpcs.phar?
Using a PHAR is better than installing such stuff using Composer, as you might install other incompatible dependencies that way (this is not the case with phpcs, but other tools like phpmd install other dependencies from Symfony)
I have a couple of servers that all use the same git repo. They auto deploy when I push updates to the master branch. Some of these are app servers, some are cron servers, etc.
I am using puppeteer for some of my background tasks. Is it possible to only require puppeteer on the cron servers? I know there are devDependencies but that would install PHPUnit, etc, and it just doesn't seem right. Am I overthinking this?
We have some batch script that executes before composer:
if [[ "$SERVER_TYPE" == "dev-server" ]];
then
cp composer.json.dev composer.json
else
cp compsoer.json.prod composer.json
code is may not work, as it's out-of-head written but idea is clear I think:
If your environment variable SERVER_TYPE is dev, then copy dev-specific file, else - copy other one.
I am developing a PHP website. I have a version on my laptop where I develop everything and my web server which runs the site.
I have found that I can use composer to install PHPUnit only on my laptop and not on my web server using the "require-dev" option Using "require-dev" to install packages in composer
However, this comes with some downsides:
From now on I have to call php composer update --no-dev on the webserver, and if I forget --no-dev then its also installed on the web server
I have to use $ ./vendor/bin/phpunit to call phpunit
I have to do install phpunit for each project on my laptop.
Would't it be much better to just install phpunit on Ubuntu sudo apt-get install phpunit? This way I would not have to worry about using the --no-dev option on the server and I could simply call it by $ phpunit. Am I missing anything important here?
Fast answer is:
You can have a version of phppunit you want in your project and another in another. And --no-dev should be used in production anyway, because you don't want to install all the dev dependencies in Production
if you don't want to call ./vendor/bin/phpunit add a script to your composer.json and then run the tests by composer test or anything you create
Explained in the first one. It really makes sense, especially when you work with some legacy code that works only with some particular versions of php/phpunit etc.
I usually install phpunit, and other tools in the 'require-dev' section, but another entirely reasonable option is to download the phpunit.phar file from the website, and check it in with the rest of your code - updating it manually occasionally.
A local (or global) Composer install will allow for better control of exactly which version is available though, and you can see when it, or your other dependencies are out of date with composer outdated.
As for a production deployment, you should be automating it as much as possible, to make sure that exactly the same thing happens every time. With that, it's just another few characters in your deployment script or other mechanism.
I have a Yii 2 web application which uses Composer for its dependencies. How am I supposed to deploy this application at a production server without having to run composer? What I would like to be done is to zip the whole application directory at the development server, copy it to the production server, unzip it and have it deployed there without any action run by composer.
I found the solution. Run:
composer install --prefer-dist --no-dev --optimize-autoloader
at the development system and to then copy the whole directory to the production server.
From the documentation:
--prefer-dist: Reverse of --prefer-source, Composer will install from dist if possible. This can speed up installs substantially on
build servers and other use cases where you typically do not run
updates of the vendors. It is also a way to circumvent problems with
git if you do not have a proper setup.
--no-dev: Skip installing packages listed in require-dev. The autoloader generation skips the autoload-dev rules.
--optimize-autoloader (-o): Convert PSR-0/4 autoloading to classmap to get a faster autoloader. This is recommended especially
for production, but can take a bit of time to run so it is currently
not done by default.
As a PHP developer I find myself working with Composer a lot. In the past it was on personal projects and such so I didn't have much problems with it, but now with Laravel 4 it's on project that require deploying and I'm in kind of a struggle to adapt my workflow.
All my projects are git repositories, thus per convention and because it's still quite buggy, like most developers I put the vendor directory in my .gitignore.
Now the problem is : I also use Git to deploy to the server, and by all logic the vendor directory is not uploaded as it's not tracked by the repository.
So my question is towards people that have worked with Composer and Git for longer than me : what is the best workflow to keep the server in sync ? How to track the vendor folder without really tracking it ?
I tried uploading it every time I update with Composer but some of my vendor folders are quite big and I can't manually upload 30Mb of files every time something updates.
I don't really know, how do you guys work it out ? I tried not ignoring the vendor folder but Git just messes it up, half are recognized as cloned repos and are just ignored anyway, etc.
UPDATE : Note that I'm on a shared host so I don't have access to the server's terminal.
The best way is to run composer install on the server after updating to the latest code. You should also make sure you commit your composer.lock file, which the server will then use to install (you should not run composer update on the server).
Capistrano (or Capifony if you are using Symfony2) is very useful for deployments with composer. You can trigger a deployment remotely and it will run a composer install in isolation so the site remains online until it has been deployed successfully. There are many other benefits such as retaining previous deployments and rolling back, copying old vendors before deployments, compiling assets etc. etc.
I'm working on something like this in the git post-receive hook on the server. This isn't tested and may be buggy, but you should get the idea.
#!/bin/bash
# get the updated composer.json
git checkout master -- composer.json
# only do this stuff if composer.json is different
# you could check this manually, or with git or cmp
cp composer.json tmp/composer.json
# may take a minute, but won't take the site down
(cd tmp; composer install --prefer-dist)
# this doesn't seem to be atomic
git checkout -f
# switch vendors over
# this isn't quite an atomic operation, but is very close
# you could probably do it with symlinks and "mv -Tf" to make it atomic
mv vendor vendor.old
mv tmp/vendor vendor
rm -r tmp vendor.old
Ideally all of the deploy (i.e. in this case the git checkout and the composer install) except one single mv would happen in isolation, outside of www. This doesn't work if you have untracked files (eg CMS uploads) in your working tree and rely on PHP's __FILE__ not resolving symlinks (due to this PHP bug).
This is an old question but in case anybody is looking a solution:
I slightly modify the #dave1010 answer to use git pull instead of git checkout--force
#!/bin/bash
# get only composer files
git fetch
git checkout origin/master -- composer.json
git checkout origin/master -- composer.lock
# make sure tmp is empty
rm -rf tmp
mkdir tmp
# copy the composer files to tmp
cp -r vendor tmp/vendor
cp composer.json tmp/composer.json
cp composer.lock tmp/composer.lock
# may take a minute, but won't take the site down
(cd tmp; composer install --no-scripts --verbose; cd ..)
# switch vendors over
rm -rf vendor_old
mv vendor vendor_old
mv tmp/vendor vendor
# update your code
git pull
# run again composer install. This time will print 'Nothing to install or update'
# but will execute pre/post scripts & generate autoload files
composer install --optimize-autoloader
There is maybe a better solution using capistrano/composer. But I like mine better.
You can use something like jenkins to ftp your files over this way you can direct jenkins to run composer install on the jenkins server and then ftp the files over.
This also allows you to ignore the vendor folder.
It does require a build server to be made and you would need to be able to execute commands vs the build server
The key is your composer.lock file. The composer.lock keeps track of exactly what packages (and versions) you have installed. When you deploy, send your composer.lock file up to the production server as well, and simply do a composer update. All the exact same package versions will be installed. With deployment software like Capistrano or Flightplan you can make the composer update step part of the process so it happens automatically.