I am reading/learning about Composer, the application-level package manager for PHP.
In this blog post written by lead dev Jordi Boggiano, he writes:
Composer on the other hand forces you to declare your project
dependencies in a one-stop location (composer.json at the root). You
just checkout the code, install dependencies, and they will sit in the
project directory, not disturbing anything else on the machine.
Another related feature is the composer.lock file that is generated
when you install or update dependencies. It stores the exact version
of every dependency that was used. If you commit it, anyone checking
out the project will be able to install exactly the same versions as
you did when you last updated that file, avoiding issues because of
minor incompatibilities or regressions in different versions of a
dependency.
If I understand Composer properly, when we're talking about packages downloaded/installed by Composer, we are talking about PHP code packages, ie, programming code written in PHP, and not system-level packages, eg, extensions to the PHP runtime installed on the server. So once these PHP code packages have been downloaded and added to a PHP project, I would have thought those packages become part of the PHP application source code, eg to be checked in to whichever version control system is being used for the project. If another developer comes along and checks out the code, why would they need to then "install the packages", as is stated in the blog post? Wouldn't they get a copy of all code packages when they check out the code from source control? This line in the blog post is confusing me, and making me think I don't understand Composer.
Any clarity on this would be greatly appreciated. Thanks.
The dependencies themselves should not be commited to source control. The composer.json and composer.lock files, on the other hand, should. There's various reasons for this, amongst them:
Every time you update the dependency you would have to commit the changes. That kind of tightly couples your code to the dependency, when it should be exactly the other way around.
The packages themselves are already in their own repository with their own history. Why repeat that in your project's history?
Those repositories can be huge, just muddling the waters around your project. Why carry around all that weight?
Instead, having each developer just run composer install (very important: not composer update) whenever they check out the project is much more efficient. Composer will install the dependencies from composer.lock, making sure everyone running the same commit is on the exact same page. The same goes for deploying.
You can read more about this here.
On the other hand, there might be situations where you have to commit your packages to get around a problem, like for example when you know you won't be able to run composer install on your production server (shared hosting)
Normally packages installed via composer don't get checked in to source control, only the code you write and the composer.json and composer.lock files.
This way the repository for your project does not get bloated with code you did not write and possibly don't really care that much about.
Yes its normal after cloning down your repository a developer will need to run the "composer install" command. The composer.lock file will ensure they get the same modules and versions of them you used when creating your project.
Not including the composer modules in your source control also allow you to easily update to the modules to get bug fixes and new features in new versions of them.
Related
composer install will install whenever stated in the composer.lock file, but composer update will update all the dependencies and create a new composer.lock file based on what is required in composer.json.
So many said only run composer update in development. But my question is doing composer update did replaced the old composer.lock file, if your app is going to break it will break, because there might be conflict with the new updated dependencies.
I came across with a situation where I must do composer update, the issue is related to pcntl extension. The only solution is to do composer update PHP pcntl module installation
I don't understand why people are afraid of running composer update on production.
TLDR;
Do not run composer update nor composer install in production. Execute it somewhere else and upload the result to the production server, but not to the same directory where the application is hosted. As a general rule, you shouldn't modify the application that's being served while it's being served. Create a different copy of the application and when it's ready replace it with the closest to instantaneous command you can (e.g. mv or ln -s).
But if you HAVE to run either: always run install and create a fresh installation; and never update. install is more predictable and reliable, with update you are at the mercy of any of the project's dependencies.
Composer works recursively. So even if you have very tight version constraints in your composer.json, by running composer update you would be updating not only your dependencies, but your dependencies' dependencies.
While most of the time this won't introduce breakage, sometimes it will. One dependency down the line may introduce a change of behaviour that may impact your code in a way you may have not tested against.
Also, it's basically using the wrong tool for the job. Composer is a dependency management tool, not a deployment tool. To deploy your code to production you should be using some sort of code deployment tool (even if that "tool" is as simple as an FTP upload and a couple of scripts).
The appropriate flow is:
Do all the require and update calls on your development machine, where you can test the project without risk. This generates a composer.lock, which is a known state for the whole project, with discrete installed versions.
Create a new installable version doing install --no-dev. On this step you also should dump an optimized autoloader, run after-install scripts, etc. I usually separate this in more than one step:
composer install --prefer-dist --no-scripts --no-progress --no-suggest --no-interaction --no-dev:
^^ This for a complete, silent installation of everything, excluding development dependencies.
composer dump-autoload --optimize --no-dev
^^ To dump an optimized autoloader script suitable for production.
composer run-script --no-dev post-install-cmd
^^ This is mostly for Symfony, but if you have any post-install scripts to run (e.g. to copy assets to your "public" directory, warm-up some type of cache, anything like that), this would be a good moment to do it.
The result of the above step should be tested (in what typically is a staging environment), and then pushed to production whole (your client code, the vendor folder, the configuration tailored for prod, etc); using whatever deployment method you prefer.
Note that if you use any "slow" push method (FTP, copy, rsync, etc), you shouldn't write directly to the application filesystem, but create a fresh copy of the application and once the file transfer is ready, use a quick method to replace "production" with the new release. A popular and effective way is use a symlink as "production" root, so you only need to update the symlink once all of the above is done and ready, without impacting a running application (that otherwise could be temporarily broken, by virtue of suddenly containing files that belong to different versions of the application).
My thoughts about this are,
The current working state of the system is very important as I would assume some tests have been run against it.
To do composer update would mean that, libraries that are part of the app would have their updates and which may lead to breakage in the system. Because they are libraries that depends on libraries that depends on libraries.
Finally, I would rather do this if composer-update is needed:
Checkout on a dev environment and composer update,
Ensure the app is thoroughly tested on a dev environment
then install on live/production with composer install
My thoughts here :
You should never use composer update without argument.
composer update reads every package listed on composer.json, and updates it to the latest available version compatible with the specified version constraints.
In a perfect world, all librairies would follow semver correctly, and it shouldn't have any side effects. But technically, that is never always true, and you could download a version incompatible with the previous one, or just a version with uncorrected bugs.
So, updating all your packages at once would probably lead to some issues, unless you have the time to check everything on your website to ensure nothing went wrong.
But of course, you'll have to update specific packages sometimes, so using composer update xxx/xxx is useful, assuming you'll check all your implementations of the package.
When the updated package is fully tested, you can commit your code to staging/production, and then run composer install to ensure you'll have the same exact version of package and dependencies on all your platforms.
Long story short, here's the process I use :
composer require xxx/xxx to install new packages
composer update xxx/xxx to update a specific package
composer install on all environments when the package.lock file has been updated.
Additional thoughts
I stumbled once upon an implementation which would give the exact version of the package in composer.json. The developer explained that this way you could use composer update without damage.
I disagree with this option, since even with the exact versions in composer.json, the dependencies are not fixed, and a composer update could lead to potential bugs in them.
I followed instructions from an answer of a similar topic(https://stackoverflow.com/a/17531897/4388482). Well, my app is getting deployed on Heroku but it doesn't work good. I'm getting the following warning
Your project only contains an 'index.php', no 'composer.json'.
Using 'index.php' to declare app type as PHP is deprecated and may lead to unexpected behavior.
Do I need to install something maybe?
UPDATE
Project structure was initially this:
I did the following:
Installed PHP 5 and composer.
I renamed package.json to composer.json and removed package-lock.json.
Typed "composer update" command. I got "nothing to install or update" message.
Added vendor to gitignore. Pushed changes to heroku.
I got the following warnings
Your 'composer.lock' is out of date!
Composer vendor dir found in project!
The complaint that Heroku has is regarding this folder.
For the record, the contents of this folder presently are:
bootstrap
fontawesome-free
jquery-easing
jquery
What has happened here is that someone has committed dependencies to your version control, which is not good practice. It will work as is, but it is not very easy to do upgrades, especially since you cannot easily see what versions you currently do have.
There are three ways to go about this.
Decide if these are PHP dependencies, by searching Packagist. There is a Composer dependency for Bootstrap, but you would need to see if the version you are using is available (or whether you can upgrade to one that is available).
Decide if these are JavaScript dependencies, by searching NPM. I wonder if it is worth examining the contents of your package.json in case these are already covered. For what it is worth, I would generally consider these candidates for JavaScript libraries rather than PHP, but do what works for you.
Choose to leave these dependencies committed in the existing vendor folder. It will work, but it is not ideal for the reasons already stated.
In the last two cases, you could probably get away with a composer.json file thus, which you should commit to the repo:
{
"require": {
}
}
You could try a composer install after this, to see if it will generate a .lock file on an empty dependency list. If this does generate, then you should commit this also.
Is there an automated way of updating the version number in composer.json and adding the necessary tags before publishing, like the way npm version does?
I mean, if you had a composer.json with the line "version": "2.1.3", and executed:
composer version minor
It would do the following:
Updating the version number in composer.json to 2.2.0
Triggering composer update to update composer.lock
Making a new git commit
Making a new git tag v2.2.0
I know that the composer version command doesn't exist, but is there an equivalent tool?
npm version does stuff that you very likely do not need for Composer:
The version number is not recorded in composer.json if there are other means available - and because you are referring to Git later on, they are available.
Updating dependencies in the lock file is unnecessary. The lock file will be ignored when the project you are dealing with is included somewhere else.
Because nothing has changed in the project, a git commit wouldn't do anything.
All this leaves you with creating a new tag in the Git repository. Putting this into Composer would mean you'd exchange one command with another, without any big benefit besides you won't have to lookup the current version number you are dealing with if you use some of the relative version parameters.
All in all I'd say that simply tagging your new version is enough for Composer. You'd probably need to have some infrastructure in place and configured to make the world aware of the new version:
If your package is open source and on packagist.org, you should have a post-commit hook to notify them as soon as a new version is available. This is a standard option on Github, I don't know about other source code hosts.
Otherwise if you have to feed closed source code, you'd probably start a new update cycle of whatever system is used to create an alternative package information source (be it Satis, locally hosted Packagist, Toran Proxy or Private Packagist)
This however depends on how you set up things.
If for some reason and despite all voices against it you still want to use a tool like the OP asked for, https://www.npmjs.com/package/composer-version works quite well.
I'm new to Composer (getcomposer.org) and wasn't sure how it works if I install a package locally using Composer and then push my codebase to my production server using Git. Do I have to run Composer again on the production server?
cheers,
J
When you setup your project, you add your dependencies into your composer.json file in your local project directory.
Once you have done this, you will need to run composer update. You can also run composer install, however, without a composer.lock file, composer install actually runs composer update.
Composer update goes out and resolves all the dependencies of all the libraries you are using, downloads them to the /vendor directory, creates an autoloader script and generates the composer.lock file.
For your project what you want to do is version your composer.json AND your composer.lock file.
On your production server, you will always run composer install, which insures that the libraries on your production server are the exact same ones you utilized in your development process.
composer install is also a lot faster as it does not have to do all the dependency management work, and can almost always just pull a specific commit#. It doesn't have to look at version strings. Thus is is usually very fast, once a server has already gone through it once.
In development the only time you should run composer update, is when you introduce a new library OR you have an issue where an underlying library has been changed and you know that you need to have composer go out and re-calculate the dependencies. composer update always recalculates and downloads the latest revisions of any library available even if the version level did not change. This means that there is a potential for something to have become broken, necessitating the potential for as full a set of regression tests as you might have available. In short, something having nothing to do with what you're actually changing could have broken, so you only want to introduce the potential for change when you are forced to.
Of course, if you did introduce a new library, you have no choice but to run composer update.
Once you run composer update, your composer.lock file will be updated (as expected) and the production server will pick this up when you run composer install on it.
As others stated, put the vendors in your gitignore. The point is that these are external libraries that you depend on, but that do not belong in your project, and should not be versioned. In the old days some people utilized git submodules, and it's a big PITA you really want to avoid, not to mention that submodules don't address dependencies of the libraries you included.
It depends how are you working. If you, like getcomposer.org says, are ignoring the "vendor" folder then you need to run it again. If you are versioning the "vendor" folder then you don't need to run it again.
Have in mind that composer will get in charge of managing your dependencies versions, so there is no need to put your dependencies files under versioning. If you put these files under git you will only make your repository bigger.
Read https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies.
For clarification when you ignore the "vendor" folder Git don't track the files under the folder so if you clone the repo it will be like composer never was executed
I've installed composer and added some packages via 'composer install'. It installed them under "my_project\vendor" path but some of the packages were cloned using git, so when I committed "my_project", those cloned packages were ignored.
The problem is that when other developers are cloning "my_project", they are missing the packages that were ignored. Is there a way to automatically add the packages to "my_project" so other developers will fetch them from me?
I think this should be done using submodules, but I don't know how to automatically add every new package from composer as a submodule to my project.
Preface: Jordi - I love composer, keep up the great work, and if I'm mistaken on any of this let me know and I'll both update my workflow and edit the post :D
Unfortunately this isn't the "general recommendation" depending on who you ask, it's by far a developer-only perspective. And the caveats to using the practice prescribed in the composer FAQ have many more considerations than I can cover here. So I'll leave a couple major points for the consideration of others.
By #Seldaek's own admission composer isn't really 100% stable, far better from a year ago, but still a very active project regardless. So relying on composer to implement an identical environment on a dev server vs staging server vs production server wouldn't be a general recommendation from any QA / Deployment group. This is not meant as a slight to Jordi, but rather an expression of the maticulous nature of QA peoples.
From the FAQ, it states when merging vendor libs into your own repository you should:
Limit yourself to installing tagged releases (no dev versions)
However if you use composer to manage your CI or automated deployments, the same constraint would apply - only more so - because deploying a master or dev tag to your production environment could be a very different package than what you tested in staging only a day or even an hour ago.
Even outside of changes introduced in third party libs (which would be solved by using only tagged versions regardless of dev or production deployments) unless you can rely on composer doing the exact same thing every time, you'll risk introducing bugs into production. This is not really a risk-case I would concern myself with, but then again I'm a developer too ;) But issues can result from simple changes like this where unless you maintain the exact same version of composer.phar on all environments, you could really muck up a staging or production server.
The other major issue I have is really related to all of the points listed under this heading:
While it can be tempting to commit it in some environment, it leads to a few problems:
I don't see the consequences as problems, but instead benefits! A large vcs repository isn't that big of a deal in modern high bandwidth environments. And unless you are using a very active vendor lib, your diffs won't be that big either. Even if they were big, git/hg/dvcs systems are all capable of re-using chunks, compressing chunks and keeping all your ducks in a row. But more so, they are an alert to the developer when changes are introduced to those packages, and diff -w is a great summary view of the total changesets, especially if you are on dev/master tags.
Duplication of the history of all your dependencies in your own VCS.
This is worded a little incorrectly, it won't duplicate the entire commit history of the vendor lib, just a single commit (your commit) covering the full delta between now and the last time you ran a composer update resulting in changes. You're probably not updating all of your libs every time you update, even if you don't specify individual packages. And if you did accidentally update a vendor lib, you can easily revert, whereas if you did so on a dev/master tag and it broke your environment, you'd have to figure out what version you were previously using and specify the tag in composer.json, and update again to revert it. git checkout /vendor/3rdpartylib --force just seems easier to me.
Adding dependencies installed via git to a git repo will show them as submodules. This is problematic because they are not real submodules, and you will run into issues.
Ideally, composer would give you a config option. It could automatically delete the .git directory from git pulls, and automatically rm the directory (or temporarily mv it) before updating a lib, when and only when an updated version exists. And doing so would be far more reliable than leaving that manual process up to individual developers. There are an equal number of reasons to have vendor libs integrated into your version control repo so the choice really depends on the details of your situation.
The biggest reason for versioning all of your files is being able to reliably deploy the exact package you tested in development to staging to production, a key purpose of vcs and automated deployments to begin with. Unless you configure your development environment to use specific tags for every package and you version control your composer.phar you should not rely on composer to deploy your software.
You should ideally just add vendor/ to your .gitignore, and then every developer of the project would run composer install to get the vendors on his setup.
You can read the FAQ entry on commiting vendors for more details.