I have an app which must work with a specific package in various versions.
I'd like to automatically run local tests against different versions of the package.
How is it done?
I can think of a manual way:
change composer.json
run composer update
run tests
repeat with another version
Update:
As mentioned here, I can just pull another version on top of an existing composer.json:
composer require vendor/package:$(MY_PACKAGE_VERSION)
run tests
pull another version
But one must be careful because you are messing with a working copy of the app. As ob-ivan suggested to clone the app in another place, pull new version and perform tests over there.
Inspired by this question, I have developed a utility just for that purpose.
Introducing DiversiTest: https://github.com/ob-ivan/diversitest
Its primary advantage over running tests in a CI server is that it allows you to test your work in progress before you commit, or even before you code --- for example, if you don't want to make an explicit commit after creating red tests in your TTD process.
How to use
Install by composer require ob-ivan/diversitest
Add a configuration file like follows:
# diversitest.yaml
package_manager: 'composer require $package $version'
test_runner: 'vendor/bin/phpunit'
packages:
illuminate/support:
- 5.4.*
- 5.5.*
And run with:
vendor/bin/diversitest
This copies your working directory to a temporary folder and loops into installing dependencies versions and running tests with commands provided by config.
Note that if you list several packages each with a list of versions, it will go through a cartesian product of all supplied package:version pairs.
The utility is still at development stage, feature requests and bug reports are welcome on GitHub issues page.
Related
composer install will install whenever stated in the composer.lock file, but composer update will update all the dependencies and create a new composer.lock file based on what is required in composer.json.
So many said only run composer update in development. But my question is doing composer update did replaced the old composer.lock file, if your app is going to break it will break, because there might be conflict with the new updated dependencies.
I came across with a situation where I must do composer update, the issue is related to pcntl extension. The only solution is to do composer update PHP pcntl module installation
I don't understand why people are afraid of running composer update on production.
TLDR;
Do not run composer update nor composer install in production. Execute it somewhere else and upload the result to the production server, but not to the same directory where the application is hosted. As a general rule, you shouldn't modify the application that's being served while it's being served. Create a different copy of the application and when it's ready replace it with the closest to instantaneous command you can (e.g. mv or ln -s).
But if you HAVE to run either: always run install and create a fresh installation; and never update. install is more predictable and reliable, with update you are at the mercy of any of the project's dependencies.
Composer works recursively. So even if you have very tight version constraints in your composer.json, by running composer update you would be updating not only your dependencies, but your dependencies' dependencies.
While most of the time this won't introduce breakage, sometimes it will. One dependency down the line may introduce a change of behaviour that may impact your code in a way you may have not tested against.
Also, it's basically using the wrong tool for the job. Composer is a dependency management tool, not a deployment tool. To deploy your code to production you should be using some sort of code deployment tool (even if that "tool" is as simple as an FTP upload and a couple of scripts).
The appropriate flow is:
Do all the require and update calls on your development machine, where you can test the project without risk. This generates a composer.lock, which is a known state for the whole project, with discrete installed versions.
Create a new installable version doing install --no-dev. On this step you also should dump an optimized autoloader, run after-install scripts, etc. I usually separate this in more than one step:
composer install --prefer-dist --no-scripts --no-progress --no-suggest --no-interaction --no-dev:
^^ This for a complete, silent installation of everything, excluding development dependencies.
composer dump-autoload --optimize --no-dev
^^ To dump an optimized autoloader script suitable for production.
composer run-script --no-dev post-install-cmd
^^ This is mostly for Symfony, but if you have any post-install scripts to run (e.g. to copy assets to your "public" directory, warm-up some type of cache, anything like that), this would be a good moment to do it.
The result of the above step should be tested (in what typically is a staging environment), and then pushed to production whole (your client code, the vendor folder, the configuration tailored for prod, etc); using whatever deployment method you prefer.
Note that if you use any "slow" push method (FTP, copy, rsync, etc), you shouldn't write directly to the application filesystem, but create a fresh copy of the application and once the file transfer is ready, use a quick method to replace "production" with the new release. A popular and effective way is use a symlink as "production" root, so you only need to update the symlink once all of the above is done and ready, without impacting a running application (that otherwise could be temporarily broken, by virtue of suddenly containing files that belong to different versions of the application).
My thoughts about this are,
The current working state of the system is very important as I would assume some tests have been run against it.
To do composer update would mean that, libraries that are part of the app would have their updates and which may lead to breakage in the system. Because they are libraries that depends on libraries that depends on libraries.
Finally, I would rather do this if composer-update is needed:
Checkout on a dev environment and composer update,
Ensure the app is thoroughly tested on a dev environment
then install on live/production with composer install
My thoughts here :
You should never use composer update without argument.
composer update reads every package listed on composer.json, and updates it to the latest available version compatible with the specified version constraints.
In a perfect world, all librairies would follow semver correctly, and it shouldn't have any side effects. But technically, that is never always true, and you could download a version incompatible with the previous one, or just a version with uncorrected bugs.
So, updating all your packages at once would probably lead to some issues, unless you have the time to check everything on your website to ensure nothing went wrong.
But of course, you'll have to update specific packages sometimes, so using composer update xxx/xxx is useful, assuming you'll check all your implementations of the package.
When the updated package is fully tested, you can commit your code to staging/production, and then run composer install to ensure you'll have the same exact version of package and dependencies on all your platforms.
Long story short, here's the process I use :
composer require xxx/xxx to install new packages
composer update xxx/xxx to update a specific package
composer install on all environments when the package.lock file has been updated.
Additional thoughts
I stumbled once upon an implementation which would give the exact version of the package in composer.json. The developer explained that this way you could use composer update without damage.
I disagree with this option, since even with the exact versions in composer.json, the dependencies are not fixed, and a composer update could lead to potential bugs in them.
I recently started a project using composer for the first time, and I just deployed it to Q&A (demo), with git I used to just do a git pull and update the Q&A environment, but now with composer update only the dependencies get updated.
My question is, what is the SOP(standard operating procedure) for updating the core project, do I still use git, or is there a way to do it with composer?
Or am I completely doing this wrong, and should be working out of the vendors folder?
The point of Composer is that you don't need to version control the dependencies which means anything that ends up in vendor/.
The project has a composer.json and composer.lock. These are within git's control so it knows the packages and versions to use. However, the vendor/ directory should be ignored, with .gitignore. If you don't already have that set up simply add this line:
/vendor/*
You version control your other files as normal.
So the operating procedure is to use git and normal. Followed by composer update.
The advantage of this setup is that git doesn't have to bother managing potentially thousands of files (inside vendor/) that will never normally change. The only circumstance under which they'd change is if you want to start using a different version of a package, or adding new ones. Well, all of those packages/versions are defined in your composer.json (which git is monitoring for changes). All you need to do is run composer update and it will update everything in your vendor/ directory to the "right" version.
That's one of the advantages of using Composer - all developers can have a "list" of the correct packages/versions, without the need to version control all of the files in them.
Edit as per the comments below:
Note composer update should only be run in development. Use composer install when deploying to QA or production. This will install the exact versions referenced by your composer.lock file.
You keep using git for your project and composer for 3rd party libs.
I have a large web app in PHP (link in my profile), and that's how I've been doing it, and it works good.
When I have new production-ready code and it's ready to be released, I do:
git fetch && git pull
And when I want to update composer, I do:
composer -o update
I don't know if you're familiar with -o flag - it generates static autoloading maps, which makes your project load classes faster. More info here.
Is there an automated way of updating the version number in composer.json and adding the necessary tags before publishing, like the way npm version does?
I mean, if you had a composer.json with the line "version": "2.1.3", and executed:
composer version minor
It would do the following:
Updating the version number in composer.json to 2.2.0
Triggering composer update to update composer.lock
Making a new git commit
Making a new git tag v2.2.0
I know that the composer version command doesn't exist, but is there an equivalent tool?
npm version does stuff that you very likely do not need for Composer:
The version number is not recorded in composer.json if there are other means available - and because you are referring to Git later on, they are available.
Updating dependencies in the lock file is unnecessary. The lock file will be ignored when the project you are dealing with is included somewhere else.
Because nothing has changed in the project, a git commit wouldn't do anything.
All this leaves you with creating a new tag in the Git repository. Putting this into Composer would mean you'd exchange one command with another, without any big benefit besides you won't have to lookup the current version number you are dealing with if you use some of the relative version parameters.
All in all I'd say that simply tagging your new version is enough for Composer. You'd probably need to have some infrastructure in place and configured to make the world aware of the new version:
If your package is open source and on packagist.org, you should have a post-commit hook to notify them as soon as a new version is available. This is a standard option on Github, I don't know about other source code hosts.
Otherwise if you have to feed closed source code, you'd probably start a new update cycle of whatever system is used to create an alternative package information source (be it Satis, locally hosted Packagist, Toran Proxy or Private Packagist)
This however depends on how you set up things.
If for some reason and despite all voices against it you still want to use a tool like the OP asked for, https://www.npmjs.com/package/composer-version works quite well.
composer install will install whenever stated in the composer.lock file, but composer update will update all the dependencies and create a new composer.lock file based on what is required in composer.json.
So many said only run composer update in development. But my question is doing composer update did replaced the old composer.lock file, if your app is going to break it will break, because there might be conflict with the new updated dependencies.
I came across with a situation where I must do composer update, the issue is related to pcntl extension. The only solution is to do composer update PHP pcntl module installation
I don't understand why people are afraid of running composer update on production.
TLDR;
Do not run composer update nor composer install in production. Execute it somewhere else and upload the result to the production server, but not to the same directory where the application is hosted. As a general rule, you shouldn't modify the application that's being served while it's being served. Create a different copy of the application and when it's ready replace it with the closest to instantaneous command you can (e.g. mv or ln -s).
But if you HAVE to run either: always run install and create a fresh installation; and never update. install is more predictable and reliable, with update you are at the mercy of any of the project's dependencies.
Composer works recursively. So even if you have very tight version constraints in your composer.json, by running composer update you would be updating not only your dependencies, but your dependencies' dependencies.
While most of the time this won't introduce breakage, sometimes it will. One dependency down the line may introduce a change of behaviour that may impact your code in a way you may have not tested against.
Also, it's basically using the wrong tool for the job. Composer is a dependency management tool, not a deployment tool. To deploy your code to production you should be using some sort of code deployment tool (even if that "tool" is as simple as an FTP upload and a couple of scripts).
The appropriate flow is:
Do all the require and update calls on your development machine, where you can test the project without risk. This generates a composer.lock, which is a known state for the whole project, with discrete installed versions.
Create a new installable version doing install --no-dev. On this step you also should dump an optimized autoloader, run after-install scripts, etc. I usually separate this in more than one step:
composer install --prefer-dist --no-scripts --no-progress --no-suggest --no-interaction --no-dev:
^^ This for a complete, silent installation of everything, excluding development dependencies.
composer dump-autoload --optimize --no-dev
^^ To dump an optimized autoloader script suitable for production.
composer run-script --no-dev post-install-cmd
^^ This is mostly for Symfony, but if you have any post-install scripts to run (e.g. to copy assets to your "public" directory, warm-up some type of cache, anything like that), this would be a good moment to do it.
The result of the above step should be tested (in what typically is a staging environment), and then pushed to production whole (your client code, the vendor folder, the configuration tailored for prod, etc); using whatever deployment method you prefer.
Note that if you use any "slow" push method (FTP, copy, rsync, etc), you shouldn't write directly to the application filesystem, but create a fresh copy of the application and once the file transfer is ready, use a quick method to replace "production" with the new release. A popular and effective way is use a symlink as "production" root, so you only need to update the symlink once all of the above is done and ready, without impacting a running application (that otherwise could be temporarily broken, by virtue of suddenly containing files that belong to different versions of the application).
My thoughts about this are,
The current working state of the system is very important as I would assume some tests have been run against it.
To do composer update would mean that, libraries that are part of the app would have their updates and which may lead to breakage in the system. Because they are libraries that depends on libraries that depends on libraries.
Finally, I would rather do this if composer-update is needed:
Checkout on a dev environment and composer update,
Ensure the app is thoroughly tested on a dev environment
then install on live/production with composer install
My thoughts here :
You should never use composer update without argument.
composer update reads every package listed on composer.json, and updates it to the latest available version compatible with the specified version constraints.
In a perfect world, all librairies would follow semver correctly, and it shouldn't have any side effects. But technically, that is never always true, and you could download a version incompatible with the previous one, or just a version with uncorrected bugs.
So, updating all your packages at once would probably lead to some issues, unless you have the time to check everything on your website to ensure nothing went wrong.
But of course, you'll have to update specific packages sometimes, so using composer update xxx/xxx is useful, assuming you'll check all your implementations of the package.
When the updated package is fully tested, you can commit your code to staging/production, and then run composer install to ensure you'll have the same exact version of package and dependencies on all your platforms.
Long story short, here's the process I use :
composer require xxx/xxx to install new packages
composer update xxx/xxx to update a specific package
composer install on all environments when the package.lock file has been updated.
Additional thoughts
I stumbled once upon an implementation which would give the exact version of the package in composer.json. The developer explained that this way you could use composer update without damage.
I disagree with this option, since even with the exact versions in composer.json, the dependencies are not fixed, and a composer update could lead to potential bugs in them.
I'm already half way done with a project in Symfony2.
I need to install a couple of new vendor bundles using composer.
I already have everything (minus logs, cache and parameters.yml) in version control (including the vendor folder).
Problem is when using composer update, it deletes the .svn folders in the vendor folders that where updated. So it's basically impossible to commit now (gives me not a working copy error).
Additional information: I'm working locally and committing to a dev server and then once approved an application server. Therefore it has to be perfect (cannot just run php composer install or php composer update on the dev/application server after commit).
I also tried exporting everything and copying and pasting them back into the repo but that also didn't work (index page broke locally).
Regarding to vendor versioning the best way is not version vendors at all.
The only things you need to version are composer.json and composer.lock. This may cause a problem with vendors which doesn't have stable versions or with that for which you need not stable one (eg. master with particular commit).
As a solution you should create your own (private) vendor repository (let's say your own packagist). Composer has a tool for that, which is called Satis.
https://github.com/composer/satis
So my suggestion would be:
Create a private repository with Satis. You place every package you need in satis.json and whenever you need to update a version of vendor, or add new one, you only modify satis.json and rebuild repository.
In your project's composer.json you set your new private repository as the only repository and set option: packagist to false.
Now, every time you run composer install it will use only your private repository, so it's fast and you always sure that every environemnt has the same versions
-
I was in similar situation two years ago.
The hard lesson I learned was never to edit files within vendor. At first I totally rejected using composer and manually cloned everything I needed. Later on, I decided to fork projects I needed to edit and referenced my forks instead.
Composer supports private GitHub repos - you don't need to register it to Packagist in order to work.
You should not keep your vendor directory in your version control. This is how it is done in Symfony Standard Edition and you should follow this. Running composer install command should be a part of your deployment process
Including vendor packages in your codebase is not recommended, so if you need to maintain the same version of the packages you use on your local machine, the best way is to keep composer.lock in the VCS and running only composer install on other environments.
Additionally, if you want the prod deployment to be instant, without depending on the composer process, you could run composer install on the dev server, and once it's validated you can make your prod deployment script copy the vendor folder from the dev env.