When putting together a PHP project with composer, on installation / deployment, composer would fetch the dependencies usually from their original sources.
This could lead to problems when deploying, when a source (maybe only temporarily) becomes unavailable.
Is there any included mechanism to keep at least the current, stable versions of the dependencies some where to be always able to deploy the current version to other instances?
Right now there is no one click solution for this, but I plan to work on something soon that will give you more reliability.
Broker looks like a tool which could serve as a proxy to keep files, and is now integrated into Satis (see https://github.com/researchgate/broker)
broker is a full repository proxy for composer. It takes a composer file, downloads all
requirements and all dependencies, and then publishes a new repository with all these
packages. Instead of packagist or satis, all packages, including dist and source files will > be served directly by broker.
Note: this project is not actively maintained anymore. Since satis supports a similar
functionality now, you should use satis instead.
Related
I am reading/learning about Composer, the application-level package manager for PHP.
In this blog post written by lead dev Jordi Boggiano, he writes:
Composer on the other hand forces you to declare your project
dependencies in a one-stop location (composer.json at the root). You
just checkout the code, install dependencies, and they will sit in the
project directory, not disturbing anything else on the machine.
Another related feature is the composer.lock file that is generated
when you install or update dependencies. It stores the exact version
of every dependency that was used. If you commit it, anyone checking
out the project will be able to install exactly the same versions as
you did when you last updated that file, avoiding issues because of
minor incompatibilities or regressions in different versions of a
dependency.
If I understand Composer properly, when we're talking about packages downloaded/installed by Composer, we are talking about PHP code packages, ie, programming code written in PHP, and not system-level packages, eg, extensions to the PHP runtime installed on the server. So once these PHP code packages have been downloaded and added to a PHP project, I would have thought those packages become part of the PHP application source code, eg to be checked in to whichever version control system is being used for the project. If another developer comes along and checks out the code, why would they need to then "install the packages", as is stated in the blog post? Wouldn't they get a copy of all code packages when they check out the code from source control? This line in the blog post is confusing me, and making me think I don't understand Composer.
Any clarity on this would be greatly appreciated. Thanks.
The dependencies themselves should not be commited to source control. The composer.json and composer.lock files, on the other hand, should. There's various reasons for this, amongst them:
Every time you update the dependency you would have to commit the changes. That kind of tightly couples your code to the dependency, when it should be exactly the other way around.
The packages themselves are already in their own repository with their own history. Why repeat that in your project's history?
Those repositories can be huge, just muddling the waters around your project. Why carry around all that weight?
Instead, having each developer just run composer install (very important: not composer update) whenever they check out the project is much more efficient. Composer will install the dependencies from composer.lock, making sure everyone running the same commit is on the exact same page. The same goes for deploying.
You can read more about this here.
On the other hand, there might be situations where you have to commit your packages to get around a problem, like for example when you know you won't be able to run composer install on your production server (shared hosting)
Normally packages installed via composer don't get checked in to source control, only the code you write and the composer.json and composer.lock files.
This way the repository for your project does not get bloated with code you did not write and possibly don't really care that much about.
Yes its normal after cloning down your repository a developer will need to run the "composer install" command. The composer.lock file will ensure they get the same modules and versions of them you used when creating your project.
Not including the composer modules in your source control also allow you to easily update to the modules to get bug fixes and new features in new versions of them.
In my company, we build most of our projects using composer, which means a lot of repeated packages (same library with same version) getting downloaded from the internet across my different teams.
I have tried Satis Composer Server,but the problem is the cache is not generated on demand.
I want to implement a central caching service which can help implement runtime caching or on demand cache.
Is it possible to implement?
I've developed a solution for this exact problem, called Velocita:
https://github.com/isaaceindhoven/velocita-proxy
Works together with a Composer plugin. It improves reliability and performance of Composer installs and you can configure which locations you want to mirror.
Satis is still great for local repositories or proactively generated caches based on composer.json contents, but Velocita allows for a more dynamic pull-through cache.
Today, Github faced a short outage and that made me look into your question. I've got multiple webservers and I'd like to setup one of them as a git proxy server. There is no need for a webserver to look download from Github (or Bitbucket, Gitlab, etc.) if it can download the same package from the internal network.
I found this blog post, explaining two (not actively maintained) options:
Gitpod
https://github.com/sitaramc/gitpod
local caching server for git when the actual server is on the other
side of a (possibly slow) WAN link
broker
https://github.com/researchgate/broker
A full proxy for composer repositories
I'm still new to coding and I'm learning everything on my own. This is a silly question for you but after reading a dozen of articles I am still confused.
I have a php based website on a shared host. After reading the various articles on benefits of using repositories and Composer, I decided to give it a try. These are my difficulties so far:
Which version of the operating system of Composer should I download, to enable me to install/update repositories of my cPanel based shared hosting?
If I am to install Windows version, how do I connect to my shared hosting to install/update the repositories?
My apologies for my silly questions, but it would really help.
If you are using shared hosting, you are unlikely to be able to use Composer on the host itself. Furthermore, you are not encouraged to use Composer "on production".
I would recommend you use Composer locally (on the O/S of your local machine), to compose your project and install your dependent packages. Once it's all working and tested with your own code, you upload your entire development directory tree including the resulting vendor library - as one big FTP/SCP upload of "flat files".
Once you get more advanced you could adventure into automated deployment techniques, but I feel for now you would be best to stick to using Composer as a local development tool to manage your codebase.
Update, further details:
Composer is really a tool to help you manage your codebase in development. It's not intended as a "deployment" tool. Previously you used to find a library you liked, download it, unzip it into your codebase somewhere random like "lib/stuff" and then link to it, and commit it into your version control system (VCS). OK, but they a year later you want to update it and you have to download it again, figure out where you saved it and how to overwrite the files, or delete old ones... it gets hard. Also your VCS repository gets full of 3rd-party components - even duplicates of the same one! Composer solved this by bringing order to this long-term dependency management chaos.
The reason you don't want to run Composer "on production" (i.e. your live website), is that during the process of download, update, composition your website will probably be broken. Even if the composer process works, this could be several minutes of broken site. After the update has finished - you now have a completely new set of 3rd party packages: how do you know they are compatible with your codebase?
So therefore you only do composer updates locally, test everything, amend your code to work the shiny new updates, and only then do you decide to upload the whole new site to the server - just as if you'd cobbled it all together manually. The deployment is independent.
Currently nearly the entire packagist.org-based dependency loading relies on GitHub-based repos. But GitHub users have the possibility to delete public repositories, which leads to the question:
What to do if a necessary Composer-loaded dependency does not exist anymore (or gets deleted by vandalism etc.) ? Are there archives somewhere to provide long-term service ?
Afaik packagist.org does not host any data (yet) and GitHub also does not keep public copies of deleted or renamed repositories.
That's where Satis comes into play. With Satis you can create a local copy of either the packages you need from "packagist.org", and also create local downloaded ZIP versions of all the versions found online.
This comes with the added benefit of being hosted in your local network, so it is much faster when accessing it, and you have a local copy available whenever your online connection goes down, or Github experiences issues, or whatever.
These locally created versions are yours to backup and take care alone, and if you install something from them, that location will be persisted in the composer.lock file (it registers the URL any ZIP was downloaded from, which is not the Github API URL, but your local HTTP server hosting the Satis files).
Using Satis you can ensure a bit more that every software you use is accessible when you need it in your local environment. This comes at a little cost of maintaining a list of all the software packages you need, running the Satis update once in a while, having a local HTTP server hosting everything, and adding your Satis repo in every composer.json file you create. Note that this last step makes it impossible to use your software if they does not have access to your Satis hosted files - it's a closed user group solution.
Although it's very unlikely to happen if you use popular third party components, you will have a copy on you development/production space so if it goes down you can create a new repo and upload a copy of that library.
If a third party component is not very popular and you are concerned about its continuity you can fork it just in case
I've installed composer and added some packages via 'composer install'. It installed them under "my_project\vendor" path but some of the packages were cloned using git, so when I committed "my_project", those cloned packages were ignored.
The problem is that when other developers are cloning "my_project", they are missing the packages that were ignored. Is there a way to automatically add the packages to "my_project" so other developers will fetch them from me?
I think this should be done using submodules, but I don't know how to automatically add every new package from composer as a submodule to my project.
Preface: Jordi - I love composer, keep up the great work, and if I'm mistaken on any of this let me know and I'll both update my workflow and edit the post :D
Unfortunately this isn't the "general recommendation" depending on who you ask, it's by far a developer-only perspective. And the caveats to using the practice prescribed in the composer FAQ have many more considerations than I can cover here. So I'll leave a couple major points for the consideration of others.
By #Seldaek's own admission composer isn't really 100% stable, far better from a year ago, but still a very active project regardless. So relying on composer to implement an identical environment on a dev server vs staging server vs production server wouldn't be a general recommendation from any QA / Deployment group. This is not meant as a slight to Jordi, but rather an expression of the maticulous nature of QA peoples.
From the FAQ, it states when merging vendor libs into your own repository you should:
Limit yourself to installing tagged releases (no dev versions)
However if you use composer to manage your CI or automated deployments, the same constraint would apply - only more so - because deploying a master or dev tag to your production environment could be a very different package than what you tested in staging only a day or even an hour ago.
Even outside of changes introduced in third party libs (which would be solved by using only tagged versions regardless of dev or production deployments) unless you can rely on composer doing the exact same thing every time, you'll risk introducing bugs into production. This is not really a risk-case I would concern myself with, but then again I'm a developer too ;) But issues can result from simple changes like this where unless you maintain the exact same version of composer.phar on all environments, you could really muck up a staging or production server.
The other major issue I have is really related to all of the points listed under this heading:
While it can be tempting to commit it in some environment, it leads to a few problems:
I don't see the consequences as problems, but instead benefits! A large vcs repository isn't that big of a deal in modern high bandwidth environments. And unless you are using a very active vendor lib, your diffs won't be that big either. Even if they were big, git/hg/dvcs systems are all capable of re-using chunks, compressing chunks and keeping all your ducks in a row. But more so, they are an alert to the developer when changes are introduced to those packages, and diff -w is a great summary view of the total changesets, especially if you are on dev/master tags.
Duplication of the history of all your dependencies in your own VCS.
This is worded a little incorrectly, it won't duplicate the entire commit history of the vendor lib, just a single commit (your commit) covering the full delta between now and the last time you ran a composer update resulting in changes. You're probably not updating all of your libs every time you update, even if you don't specify individual packages. And if you did accidentally update a vendor lib, you can easily revert, whereas if you did so on a dev/master tag and it broke your environment, you'd have to figure out what version you were previously using and specify the tag in composer.json, and update again to revert it. git checkout /vendor/3rdpartylib --force just seems easier to me.
Adding dependencies installed via git to a git repo will show them as submodules. This is problematic because they are not real submodules, and you will run into issues.
Ideally, composer would give you a config option. It could automatically delete the .git directory from git pulls, and automatically rm the directory (or temporarily mv it) before updating a lib, when and only when an updated version exists. And doing so would be far more reliable than leaving that manual process up to individual developers. There are an equal number of reasons to have vendor libs integrated into your version control repo so the choice really depends on the details of your situation.
The biggest reason for versioning all of your files is being able to reliably deploy the exact package you tested in development to staging to production, a key purpose of vcs and automated deployments to begin with. Unless you configure your development environment to use specific tags for every package and you version control your composer.phar you should not rely on composer to deploy your software.
You should ideally just add vendor/ to your .gitignore, and then every developer of the project would run composer install to get the vendors on his setup.
You can read the FAQ entry on commiting vendors for more details.