I've been developing a lot of packages for composer recently, as our company is moving to composer for package management, but I'm running into issues with how to handle versions.
I've been using the git workflow, with tagged releases, but this means I can't go back to a previous major or minor release and make a patch release for it. I also have been running into issues with understanding how tagged development/beta/RC releases work with composer.
I've seen other projects use branch aliases, and I think it might solve these issues for me, but after reading the composer docs on them I still don't feel like I understand how to practically use them.
Does anyone with more experience using composer have tips/tricks on how best to version a git package repository?
1. Source Control of Composer project
I normally ignore the vendor directory using .gitignore. Indeed, this is so common that .gitignore file comes with most PHP framework would have already included the vendor directory.
Upon release, some of the steps will be:
checkout or clone from release branch/tag,
update composer ( self update )
composer install ( install all dependencies into the vendor directory )
2. Composer versoining/stability/alias confusion
Yes this is confusing, I think the most confusing part is where versions come from, and it is not easy found in the online docs. Versions come from source control's branches and tags
a Tag name is the exact version name. if you have a tag called 'xx', you can reference it in the package.json file as 'xx'. if your tag follows the semantic naming convention (e.g. 1.0.1 or v1.0.1), you can refer to it using semantic syntax like ~1.0.*.
tags are 'stable' UNLESS their semantic names contains somethings like 'RC1', 'RC2', 'alpha' etc. (e.g. 1.0.1-rc1, 1.0.1-alpha) In those cases they can be referenced by 1.0.1#RC or 1.0.1#alpha.
branches are not 'stable' by default, numbered branch will be treated as development version. e.g. branch 2.0 will be referenced as 2.0.x-dev (note the extra .x ); Non-numbered branch name will be referenced with be prefixed with 'dev-'. i.e. 'master' branch becomes dev-master and 'testing' branch becomes dev-testing.
Branch alias is used, for example, when you want to treat the master branch as 2.0.x#dev. You might want to use the dev-master branch of package ABC but it happens that one of the packages you used in your project depends on the 2.0 branch of package ABC. As you can only use one version of package ABC in your project, you basically ask all other packages to use the dev-master when they want to use the 2.0.x#dev branch
Other issues like minimum-stability and nested dependencies are clear in the online docs and I am not repeating them here.
Related
I'm currently experimenting with the Satis. I would like to be able to get the exact version of my private packages somewhere, so everything that is normally in the composer.lock. I always commit the composer.lock via Git.
But if I understand that correctly, the Satis in its packages.json always only includes the require parts, i.e. the sections from my composer.json and thus of course only version ranges.
Is there a way to configure the Satis so that the composer.locks are also stored or how do I get the exact "snapshot" of my packages?
+++ Example +++
Ok, I try to explain a bit more.
Let's say I have a package my/package. Here I add several files, including a composer.json, in which I define that symfony/console should be installed in a version greater than or equal to 4. Now I do a "composer install" and Symfony is installed in version 4.4. I commit all files, including composer.lock and create a release 1.0.
Now I'm going to the Satis. Here I add my/package and the corresponding repository URL for my/package to satis.json and update it. The Satis checks out my package correctly and in packages.json or more precisely the all*.json my package is listed with version 1.0. So far everything is fine.
But if I now take a look at the metadata that Satis stores for my package in all*.json, I see here practically my specified requirements, i.e. that symfony/console should be installed in a version greater than or equal to 4. So Satis takes a snapshot of the composer.json and apparently ignores the composer.lock. So I have no chance to see that my release 1.0 works with the exact version 4.4 of Symfony, while for example a release 1.1 works with symfony/console 4.5. But this information is interesting for me.
When installing a package, Composer recalculates all dependencies on the fly. This is based on the composer.json of your application and the composer.json files of all dependencies.
A composer.lock should not be part of any package, and it is not taken into account when a package is installed.
So, I've now built a workaround. The whole thing is not quite perfect, since the runtime for large repositories is relatively long, which is why I have to run it as a cron once a day. But it works fine.
I have created a new Satis console command.
This command uses the PackageSelection class to determine all existing packages.
I iterate over the package list and look for the paths and names to the dist files.
I extract the ZIP files in memory and look for the composer.lock. If there is one, I parse it and read the exact version numbers of the dependent packages.
I summarize the information in a separate JSON file and store it in parallel to packages.json under htdocs. From there I can call it up and integrate it into my own application or process it further.
I am reading/learning about Composer, the application-level package manager for PHP.
In this blog post written by lead dev Jordi Boggiano, he writes:
Composer on the other hand forces you to declare your project
dependencies in a one-stop location (composer.json at the root). You
just checkout the code, install dependencies, and they will sit in the
project directory, not disturbing anything else on the machine.
Another related feature is the composer.lock file that is generated
when you install or update dependencies. It stores the exact version
of every dependency that was used. If you commit it, anyone checking
out the project will be able to install exactly the same versions as
you did when you last updated that file, avoiding issues because of
minor incompatibilities or regressions in different versions of a
dependency.
If I understand Composer properly, when we're talking about packages downloaded/installed by Composer, we are talking about PHP code packages, ie, programming code written in PHP, and not system-level packages, eg, extensions to the PHP runtime installed on the server. So once these PHP code packages have been downloaded and added to a PHP project, I would have thought those packages become part of the PHP application source code, eg to be checked in to whichever version control system is being used for the project. If another developer comes along and checks out the code, why would they need to then "install the packages", as is stated in the blog post? Wouldn't they get a copy of all code packages when they check out the code from source control? This line in the blog post is confusing me, and making me think I don't understand Composer.
Any clarity on this would be greatly appreciated. Thanks.
The dependencies themselves should not be commited to source control. The composer.json and composer.lock files, on the other hand, should. There's various reasons for this, amongst them:
Every time you update the dependency you would have to commit the changes. That kind of tightly couples your code to the dependency, when it should be exactly the other way around.
The packages themselves are already in their own repository with their own history. Why repeat that in your project's history?
Those repositories can be huge, just muddling the waters around your project. Why carry around all that weight?
Instead, having each developer just run composer install (very important: not composer update) whenever they check out the project is much more efficient. Composer will install the dependencies from composer.lock, making sure everyone running the same commit is on the exact same page. The same goes for deploying.
You can read more about this here.
On the other hand, there might be situations where you have to commit your packages to get around a problem, like for example when you know you won't be able to run composer install on your production server (shared hosting)
Normally packages installed via composer don't get checked in to source control, only the code you write and the composer.json and composer.lock files.
This way the repository for your project does not get bloated with code you did not write and possibly don't really care that much about.
Yes its normal after cloning down your repository a developer will need to run the "composer install" command. The composer.lock file will ensure they get the same modules and versions of them you used when creating your project.
Not including the composer modules in your source control also allow you to easily update to the modules to get bug fixes and new features in new versions of them.
I want to create my own package manager, and currently reviewing existing solutions.
I'm playing with PHP's Composer now, and it was quite surprising that it has two files:
composer.json for project configuration, and non-pinned dependencies
composer.lock for exact pinned dependencies
I do understand why one needs to pin dependencies, .lock information by itself seems logical to me.
What I do not understand is why project metadata was split into two files.
Can anyone explain, why it was designed this way? Why deps could not be pinned right in the composer.json?
UPD. Turns out, Rust's Cargo has the same two file configuration in place, and has a nice explanation of the meaning of the .lock file: http://doc.crates.io/guide.html#cargotoml-vs-cargolock
During development, you usually want to be able to upgrade to the latest compatible version of dependencies easily. composer.json has the information on what the dependencies are and which versions are compatible. composer.lock lacks the compatibility information, it may say that the package was built against version 2.2.7 of a dependency but information is missing about rules such as that versions >= 2.1 and < 3 of that dependency are compatible while lower versions aren't and the next major version isn't guaranteed to be so play it safe.
When building for testing or release, on the other hand, it's necessary to make sure you build against the exact same set of dependency versions every time. composer.lock allows that by listing out the exact versions used. Even if new versions of dependencies come out, the dependency pinning insures that the build won't change so you won't have to worry about changes in behavior caused by changes in dependency packages.
.lock information is absolutely pinned, typically created by a composer update request based on the json information... but developers don't necessarily want to pin everything to an exact version, and without that .json file they have to upgrade the .lock file manually for every version upgrade of their dependencies.
The .lock also holds dependencies of dependencies, and dependencies of dependencies of dependencies, etc... whereas the .json file only holds immediate dependencies.... and as a developer, you should only need to control your immediate dependencies, and allow those libraries to control their own dependencies via their own .json files
Basically, you should build your application against the json but deploy against the .lock
I have tried several different methods of tagging versions in my Mercurial repo including branch names, bookmarks and tags and satis appends -dev to bookmarks and branches. Can I configure satis to consider the default branch as stable instead of dev? Do I have to make explicit tags for each stable version?
Satis add "dev-" prefix to any branch name, and "-dev" suffix to branch names resembling version numbers like 1.0.x. And this is intentional, because that is how Composer deals with branches that do change it's content when people continue development and commit stuff.
So on one day the branch "dev-master" points to a totally broken version, and one day later it is close to the next possible release.
It is not recommended to use branches to include software into another project. Tag your software, and let Satis create downloadable ZIP packages. If you stick to sematic versioning, you and anyone using your library will be able to update without getting into trouble with incompatible changes.
If you really want to use a branch, you have to either set minimum-stability to "dev", or explicitly include a development branch as a stable version's alias. Note that you have to run Satis every time you commit something to that repository to reflect the changes, and only then you would see the update happening in your other software.
Tagging the software isn't too bad, I'd recommend you try it.
I am having some weird issues with git submodule update for a Plugin dependency with Jenkins # CloudBees.
So I am switching some, if not, all my dependencies from git submodule to Composer.
I came across this repo called composer installers. https://github.com/composer/installers
I was wondering how to use this for both Plugin and Vendor dependencies.
I am not familiar with Composer and even after reading the docs, I am uncertain how to say, place this dependency specifically to Plugin/xxx
I know how to do this with git submodule add.
So anyone knows how I should use Composer or better yet, Composer installers, please advise me.
To make things easier, I want to use 2 actual examples.
https://github.com/milesj/Utility is to be placed inside my app/Plugin/Utility
https://github.com/simkimsia/php-gd-simpleimage is to be placed inside my app/Vendor/SimpleImage
There is a Composer plugin for CakePHP that has a Backery article about it. The code is available on Github:
https://github.com/uzyn/cakephp-composer
It's actively being developed (last commit was yesterday), but in my early use of it (today), it seems to be working as expected.
Packagist has loads of Compose-ready libraries. Some of them are CakePHP related. Some are not.
The two examples you listed aren't in Packagist (yet?). Thankfully, Composer makes it possible to work directly with Git (and other VCS) repos. For the milesj/Utility plugin (which has a composer.json file), you'll need to follow the Repositories guide in the Composer docs to set things up properly.
For the php-gd-simpleimage repo, you'll need to write a composer.json file, then follow the Repositories steps.
One of the most confusing things about Composer is that composer.json is the same system/file-format for both libraries and "projects." Really, they're all the same to Composer. In your "project" repo, though, you're only outlining requirements (usually), not making your application installable via Composer. Regardless of their locations, both composer.json files are for the same thing: tracking and installing dependencies. You can imagine it as a tree with your project (and it's composer.json) at the top, and then a branching dependency tree all the way down.
Happy Composing!