I am going to rewrite this question to be more clear. I have the following application structure:
applications/
api/
public/
composer.json
frontend/
public/
composer.json
backend/
public/
composer.json
common/
vendor/
... composer libraries here
How can I make that every single application's composer install gets installed into common/vendor, so that way I can have the most up to date version of the library in wherever is used with just one composer update; while at the same time only load the libraries that are in the composer.json file of each application. So, when I include vendor/autoload.php, only the needed libraries are loaded.
EDIT: Edited the whole question. Please reread
You have to create one bigger meta project that requires the API package, the frontend and the backend. You can define which directory should be used for placing dependencies for this meta project, and should be able to define for the special packages API, frontend and backend that they should go into their respective directories and not the common folder.
Updating that meta package will have to check more dependencies, but it is guaranteed that you either get the newest possible versions that conform to your version constraints (which may NOT install the newest version available if one of your packages requires a lower version). That way you would avoid installing dependencies that are not allowed for one of the projects, and you would be immediately notified if you attempt to install conflicting versions.
Note that I wouldn't recommend this at all. I would write a script, placing it at applications/updatecomposer.sh and add all the commands necessary to update each project individually. You gain all the flexibility that Composer is about, because essentially you want to have the central library installation of PEAR back. This central installation and the resulting inability to update any of the PEAR packages without risking to break something is one of the reasons that PEAR is considered dead.
Or think about any pre-Composer originating framework like Zend Framework 1. Having this installed in a central point that every application is using will effectively prevent you from ever updating it, if you are not prepared to also deal with incompatibilities in ALL your applications at the same time. Just an example: Updating from any ZF 1.11 to ZF 1.12 (the currently maintained up to date version) is a potentially backwards-incompatible change, because at least one abstract class (dealing with REST interfaces) got new abstract methods that have to be implemented.
Related
I am reading/learning about Composer, the application-level package manager for PHP.
In this blog post written by lead dev Jordi Boggiano, he writes:
Composer on the other hand forces you to declare your project
dependencies in a one-stop location (composer.json at the root). You
just checkout the code, install dependencies, and they will sit in the
project directory, not disturbing anything else on the machine.
Another related feature is the composer.lock file that is generated
when you install or update dependencies. It stores the exact version
of every dependency that was used. If you commit it, anyone checking
out the project will be able to install exactly the same versions as
you did when you last updated that file, avoiding issues because of
minor incompatibilities or regressions in different versions of a
dependency.
If I understand Composer properly, when we're talking about packages downloaded/installed by Composer, we are talking about PHP code packages, ie, programming code written in PHP, and not system-level packages, eg, extensions to the PHP runtime installed on the server. So once these PHP code packages have been downloaded and added to a PHP project, I would have thought those packages become part of the PHP application source code, eg to be checked in to whichever version control system is being used for the project. If another developer comes along and checks out the code, why would they need to then "install the packages", as is stated in the blog post? Wouldn't they get a copy of all code packages when they check out the code from source control? This line in the blog post is confusing me, and making me think I don't understand Composer.
Any clarity on this would be greatly appreciated. Thanks.
The dependencies themselves should not be commited to source control. The composer.json and composer.lock files, on the other hand, should. There's various reasons for this, amongst them:
Every time you update the dependency you would have to commit the changes. That kind of tightly couples your code to the dependency, when it should be exactly the other way around.
The packages themselves are already in their own repository with their own history. Why repeat that in your project's history?
Those repositories can be huge, just muddling the waters around your project. Why carry around all that weight?
Instead, having each developer just run composer install (very important: not composer update) whenever they check out the project is much more efficient. Composer will install the dependencies from composer.lock, making sure everyone running the same commit is on the exact same page. The same goes for deploying.
You can read more about this here.
On the other hand, there might be situations where you have to commit your packages to get around a problem, like for example when you know you won't be able to run composer install on your production server (shared hosting)
Normally packages installed via composer don't get checked in to source control, only the code you write and the composer.json and composer.lock files.
This way the repository for your project does not get bloated with code you did not write and possibly don't really care that much about.
Yes its normal after cloning down your repository a developer will need to run the "composer install" command. The composer.lock file will ensure they get the same modules and versions of them you used when creating your project.
Not including the composer modules in your source control also allow you to easily update to the modules to get bug fixes and new features in new versions of them.
I've recently inherited a project build on a particular PHP framework, Asgard. The framework itself is less important.
The code is stored on GitHub and contains the framework files, some of them modified, and a few dependencies. Upon installation, one should run composer install, which will bring the rest of the dependencies.
Most of the custom code is in two folders, located in public_html/modules and in framework/Modules, respectively. However, there are some framework files that have been modified.
The code management/installation procedure seems illogical to me, and I would rather all of the framework code (the specified version of it) would be brought by composer and only the few files be applied on top, as a patch.
Is there some development workflow that would allow me to push to GitHub just the changes, not all of the framework files, and then somehow apply a patch over the framework code when composer install runs?
I'm aware that composer is configured via composer.json and composer.lock, I just don't know how to do the patch.
Also, I don't know how to push just the changes to Git, without including the initial framework files, while retaining full functionality in my working folder.
Any idea would be appreciated.
Thanks.
I want to create my own package manager, and currently reviewing existing solutions.
I'm playing with PHP's Composer now, and it was quite surprising that it has two files:
composer.json for project configuration, and non-pinned dependencies
composer.lock for exact pinned dependencies
I do understand why one needs to pin dependencies, .lock information by itself seems logical to me.
What I do not understand is why project metadata was split into two files.
Can anyone explain, why it was designed this way? Why deps could not be pinned right in the composer.json?
UPD. Turns out, Rust's Cargo has the same two file configuration in place, and has a nice explanation of the meaning of the .lock file: http://doc.crates.io/guide.html#cargotoml-vs-cargolock
During development, you usually want to be able to upgrade to the latest compatible version of dependencies easily. composer.json has the information on what the dependencies are and which versions are compatible. composer.lock lacks the compatibility information, it may say that the package was built against version 2.2.7 of a dependency but information is missing about rules such as that versions >= 2.1 and < 3 of that dependency are compatible while lower versions aren't and the next major version isn't guaranteed to be so play it safe.
When building for testing or release, on the other hand, it's necessary to make sure you build against the exact same set of dependency versions every time. composer.lock allows that by listing out the exact versions used. Even if new versions of dependencies come out, the dependency pinning insures that the build won't change so you won't have to worry about changes in behavior caused by changes in dependency packages.
.lock information is absolutely pinned, typically created by a composer update request based on the json information... but developers don't necessarily want to pin everything to an exact version, and without that .json file they have to upgrade the .lock file manually for every version upgrade of their dependencies.
The .lock also holds dependencies of dependencies, and dependencies of dependencies of dependencies, etc... whereas the .json file only holds immediate dependencies.... and as a developer, you should only need to control your immediate dependencies, and allow those libraries to control their own dependencies via their own .json files
Basically, you should build your application against the json but deploy against the .lock
Introduction
This is quite a lengthy question, the question I asked in the title is probably ambiguous and I may change this to something more suitable.
A similar question has already been asked and answered here although I do not think this entirely answers the question.
Synopsis
I am working with a team of developers on a project. We are using a framework (for argument sake - the framework is irrelevant) except one of the requirements is that we use composer.
These packages are essentially de-coupled from the application and each other, however one package may depend on another package.
These packages have their own git repository, and during development of the application have branch aliases set to dev-master.
Problem #1
In order for the application to work with my packages I need to register them with composer.json, which is fine until I have to commit the existing work of my package development to their repository before I can run composer update.
Problem #2
I have committed the initial package architecture and the composer.json. I run composer update which completes and my package is available to the application. Yet, I continue to develop this package at the same time another developer has already committed a different change to this package - i.e. a bug fix.
I need to update another package in order for my work to continue, yet I can't because doing so would throw a warning similar to:
Loading composer repositories with package information
Updating dependencies (including require-dev)
- Removing hu/admin (dev-master)
The package has modified files:
M composer.json
M src/...
Discard changes [y,n,v,?]?
If I respond with y my changes are blown away and lost forever. If I choose n composer is aborted and my project is broken due to a package not being updated parallel to my changes.
Problem #3
Another problem with this way of developing is that if I am working on my package outside of vendor and I commit my changes I can run composer update but my application is broken due to a fatal error. I fix the missing ; or other small syntax error. I commit this change and run composer update - yet I don't want my git history full of little typo fixes and parse error fixes just because I can't work on my package within the application parallel to other development/developers on the application and other packages or this package.
Problem #4
I discovered a package on GuitHub called franzliedke/studio which seems to part-solve my problem. Once a package has been published due to being complete/functional, this then cannot remain inside the vendor/bin directory alas causing the initial problems to rise once more.
Conclusion
I am wondering the best way to work around this or any best practices in order to work on packages with teams of developers without having to commit everything every time before I run composer update.
laravel did have a workbench feature which was pretty cool. But was removed from version 5.0 and so was that bad practice?
That's what we do in huge projects consisting of several little composer components which are developed at the same time:
Develop your application 'in one piece' like described in the other answer your mentioned by just keeping all components separate inside their own namespaces and directory structure.
/Application
-composer.json (Application json)
-/src
--/Component1
----composer.json (Component json)
--/Component2
----composer.json (Component json)
--/ApplicationFeature
----Class1.php
----Class2.php
The whole application is developed in a single git repository which would eliminate most of your aforementioned problems. Then occasionally split the application repo into single component repositories using git subtree. I wrote a little php cli script which splits the project into smaller components and pushes them to the according component repositories. There are a lot of advantages compared to git submodules. The whole commit history of a component is kept and pushed to the component repository. More information on subtrees here
In case you are interested please let me know, I am happy to share the script which splits/tags and finally pushes the single components by just defining a directory <-> componentName mapping as a json.
There is a main application, let's call it APP.
APP has several dependencies (including open source projects and proprietary libraries).
There are multiple clients that use their own instance of APP (on different domains that I manage). Some of these clients use a slightly adjusted version of APP. I implemented this by creating a specific module (let's call it SM) for each client that I just add to their instance of APP (so that I don't change any of the code from APP).
Currently, I've implemented this as follows:
Develop APP locally, use Composer to update its dependencies (composer update), push APP on central repo
For each regular client, pull APP from central repo and install the Composer dependencies (composer install)
For clients with specific implementation, create a new SM (specific module), that has the following composer.json file:
...
"require": {
"APP": "X.X.X"
}
...
Then apply the same steps as before for this SM (composer update locally, PUSH to central repo, PULL from central repo, composer install).
Everything is fine, except for two issues that I'd like to overcome:
composer.lock from APP will be ignored by SM (since APP is loaded as a library in the vendor/ folder, and composer ignores the composer.lock files of libraries); this is not good at all, as I will not be confident that the specific clients will use the exact same libraries as APP.
Each time I fix a bug or implement a new feature in APP (and this happens frequently - a few times a day), apart from the steps that I perform for the regular clients, I also need to rebuild the SMs (since one of their libraries - APP - was updated to a new version that I need to use). This is an overhead since most of the changes that I perform are inside APP (and not SM). So, if it was the other way (APP having SM as a dependency), it would have been working faster (since I wouldn't need to composer update on each SM).
Are there any known workflows or best practices that cover this scenario in order to mitigate the two issues above or at least to decrease the complexity of the upgrade/deployment process?
Please note that most of the steps above are already automated, so my question is not about the automation part, but the complexity of this architecture
I implemented this by creating a specific module (let's call it SM) for each client that I just add to their instance of APP
For clients with specific implementation, create a new SM (specific module), that has the following composer.json file:
It's an application with a client specific module (next to other dependencies).
The application has the module as dependency (APP having SM as a dependency).
And not: the module pulls the application as it's vendor dependency in.
This will only cause extra steps to take during the development phase (your issue 2).
I would suggest to refactor the application and it's modules until you get the following folder structure:
|-application #< the application has dependencies
|-src
|-tests
|-vendor
|-framework #< maybe your application is framework based
|-libs #< more dependencies
|-... #< other modules
|-sm #< the client specific module
This allows to pull in dependencies, which extend "the application" for client-specific needs.
This overcomes your issue 1, because APP is the main repository and contains the lock file. It's essential to lock the versions, so that all developers are bound to the same versions and also for packaging exactly the same set of versions.
So, if it was the other way (APP having SM as a dependency), it would have been working faster (since I wouldn't need to composer update on each SM).
Yes! The need to rebuild the module, each time you change APP would vanish, if you start to "develop inside APP" with module dependencies.
And for multiple clients, simply use multiple application repos, which have a custom set of requirements. 10 clients, 10 application repos, 10 composer.json files. Run composer install no-dev then pre-package each repo and place zip into downloads. Done.
You can use a "container" or "packaging" project here, where the composer.json of each project would include the app and the specific modules. You might utilize the caret or tilde operator to specify a version range for the app ("vendor/app": "^1.2.3") and then simply update and repackage, after a new version of the application is released. This approach should work with the composer autoloading, because the application will remain inside the vendor folder, too. Only a little wrapper is needed, to set the composer autoloader up and switch over to your application.
Or, if the application is really modular. Just package the main application and provide the client-specific modules as extra downloads. With this approach upgrades will have multiple download steps: upgrade app, upgrade modules. Think of it as "wordpress-style" updates/upgrades.
You might reduce the complexity of the upgrade/deployment process further by dropping the composer install --no-dev part on the client machine
by building "client-specific application archives" on the developer machine.
These are basically the "--no-dev" package of the application with all it's dependencies, including the client-specific module(s) = pre-packaged.
Like, Application-v1.2.3-WithModuleAForClientA-v3.2.1.zip.
On the dev machine: composer install --no-dev --optimize-autoloader + zip.
To install or upgrade simply download to the client, extract, execute the upgrade script. Done.