I have a home-made framework which contains two types of bundles. Application bundles and dependencies. Application bundles are specific to the application (can be a news module, administration, etc.). Dependencies are libraries which I re-use in several projects.
For every new project, I start by adding the app bundles and dependencies. Until now I kept a separate copy of these bundles and copied them manually into my project.
I rarely need to modify dependencies. However with application bundles, I always have to customize the views and controllers, so they are meant to be modified in all the projects they are used.
I want to improve the way I manage these bundles and thought of using Composer. However, it does not comply very well with the application bundles, as you are not supposed to modify the packages. I would need to be able to install packages for once, then ignore them in the next updates (they become part of the application source code).
I could add some override mechanism (copy classes to another directory, which would override the default bundle classes) as Symfony does I think, but I don't like this idea. It just adds unnecessary complexity.
Could use another bundles management system, or even a custom one, but then this means no access to Composer packages, which would be great.
Using Composer for dependencies and another system for the application bundles would be messy..
How would you handle it?
It it should run on unix, use apt, yum, yast ...
Simply try to build your own dep / rpm pakages.
Build foreach application bundle and depencencies bundle.
That add at theapplication bundle the depencencies bundle as depends.
Or do you looking for something like pear?
Related
I am working on very old legacy code, mostly procedural. Trying to improve it. Rewriting applications is impossible right now. The plan is to add a few libraries which would help organize things and improve that way.
I added a Symfony dependency-injection component in order to do that. It would provide the possibility to fetch needed services with its dependency easy.
I watched symfonycast tutorial on how to play with container. And with that knowledge, I managed to write a simple loader to start the container and to use services made by me. It is simple, it guesses FQCN based on file path, and then uses reflection to get dependencies. But I can not figure out how to load vendor classes, because here you can not guess namespace that way. :)
The question is: What exactly Symfony uses to load classes from the vendor folder, does it reads composer.json files to see namespaces, does it uses some composer feature, or something else?
Loading classes is different than instancing services.
The first can in fact use regular composer facilities to discover vendored classes in a legacy project like yours, even if they weren't installed with composer. This uses the standard php autoload mechanism with some added magic.
To include the, let's say lib/ legacy directory in the discoverable files you would add the following to composer.json:
"autoload": {
"classmap": ["lib/"]
}
And then run composer dump-autoload. Note that by including vendor/autoload.php in your legacy files you could even forego the require directives for your dependencies and rely on composer as well. This can be a path for migrating them to composer-managed dependencies, too.
Service instancing requires not only being able to locate the classes themselves, but also their respective dependencies so the container can create the object tree automatically. This usually involves hand-writing service definition files: classes in the vendor/ folder are not automatically registered as services. A bundle (or your own definitions) enables support for an specific library.
Take for instance the Mailer component: you can use it as a standalone library, but for framework integration (which includes service definitions and depen) you'd need to install Mailer bundle as well.
The exception where automatic service registration applies (when using symfony framework, not the standalone dependency injection component) is for files under src/. During container compilation, services.yaml is loaded and the ContainerConfigurator with help from FileLoader, looks for *.php files the directories configured as a resource, creating service definitions for them.
I guess you could do a similar thing for your legacy dependencies in a CompilerPass by using a similar technique or by trying to leverage the composer classmap but, specially if your legacy dependencies do not follow a PSR loading standard, I'd advise against it, since it can pull in tests, example files, etc.
Background
I am developing a framework in PHP. I started by making each component separately so that it can be used independently from the framework.
After creating four libraries A, B, C and D:
A has no dependencies
B and C require A
D requires A, B and C
Now I have some issues when releasing a new version of one library, I may have to change dependencies of others and have to release new versions for them. For example: new version of A means new version of B, C and D.
I looked how other frameworks like Symfony and Laravel solved this issue. I learned that they are using the subtree feature of Git and the replace feature of Composer. It works as follows:
Each component is in a read-only repository with his own composer.json
Each component can require other components but doesn't replace them.
The framework repository uses subtrees to include all components. So no need to require them using composer. But it should require all their dependencies (since this is no longer handled by Compser).
The framework replaces all its components.
I have also noticed that
A component repository contains only the source code (no unit tests !)
Laravel made the component Contracts just to store all the interafces of all component and each component is requiring that.
Questions
Is my explanation of how Laravel and Symfony solved the issue correct ?
Do I really have to remove tests from the components repositories and put them in the framework one ?
If yes, how can someone who wants just to use a single component be sure it passes the tests regardless of the whole framework being passing the wole tests ?
Do I have to make sure that all components dependencies are compatible and require them manually in the framework composer.json ?
What is the point of having a component for Interfaces ? This could not be used standalone anyway !
Is there a better way to solve this problem ?
P.S: here are links to A, B, C and D
Now I have some issues when releasing a new version of one library, I may have to change dependencies of others and have to release new versions for them. For example: new version of A means new version of B, C and D.
You have a multi-repo approach.
Edit of A => new version of A => version bump needed in B, C and D.
I think the most important thing is to get away from using dev-master and versionize your components properly, once they are stablized and ready to be out of dev-phase. Then you might use Composers range operators (caret ^ & tilde ~) to automatically update to the latest released version in a certain major.minor version range. This helps a great deal and takes the tedious manual version updating work out of your hands.
Is my explanation of how Laravel and Symfony solved the issue correct ?
It's not correct. The underlying development concept, publication and consumption of packages work differently, then to what you described.
They use a monolithic repo development style. It's a single repository, which contains code for group of packages. The opposite of the mono-repo is many-repo approach. The alternative are git submodules.
All modules/bundles of a framework and the framework core/kernel are in one repository! For Laravel it's https://github.com/laravel/framework/tree/5.4/src/Illuminate
Each module/bundle folder contains a composer.json and the framework itself contains a composer.json
This allows to "split out" the module folders into standalone read-only repositories. Using a custom git helper, e.g. git subsplit publish like Laravel uses https://github.com/laravel/framework/blob/636020a96a082b80fa87eed07d45c74fa7a4ba70/build/illuminate-split-full.sh or splitsh https://github.com/splitsh/lite, like Symfony uses
The development happens in the main repo.
Finally, from the user/consumer perspective (in the composer.json of your CMS/app whatever), you simply require a module/bundle from the "standalone read-only repository" source. This is many-repo, because your app depends on many repositories.
When you update a dependency using Composer, then Composer replaces your packages with a newer version.
Do I really have to remove tests from the components repositories and put them in the framework one ?
No. You could also leave the tests in the /moduleA/tests folder and adjust your unit test collector.
If yes, how can someone who wants just to use a single component be sure it passes the tests regardless of the whole framework being passing the wole tests ?
Two things. The subject under test is:
(a) the component, which is ideally independently testable and
(b) the framework, which consumes many components and tests functionalities, which rely on functions from multiple components (e.g. a core/kernel). You could also split a kernel out, once it stabilizes and is testable independently. (e.g. your component D)
Do I have to make sure that all components dependencies are compatible and require them manually in the framework composer.json ?
The monorepo developer perspective:
The developer/maintainer of a framework can only release a new version, when all unit-tests of all components and all unit-tests of the framework itself pass, right? Then he can start a subtree split and automatically versionize the new components.
The application developer perspective:
Yes. As the user of components of a monorep you are simply consuming standaloen dependencies (from the read-only repos). That means you have to maintain the versions of the components you require in your composer.json manually or automatically.
What is the point of having a component for Interfaces ? This could not be used standalone anyway !
Good question!
Maybe the developers want to do things differently and "keep things sorted"
Or, they have a bad optimization idea on their minds:
One could argue that interfaces are only development contracts.
When all components are written against interfaces you could simply pull the plug on them, after testing and before doing a production release.
In other words: you could leave the interfaces repository away
and run an interface removal, when you are releasing for production.
Leaving the interfaces repo away would lead to "interfaces X not found" fatal errors. Then you run an "optimizer pass" over the rest of the classes and remove all "implements interfaceX" strings. Less files to include. Less code to parse. Less IO. And now i will probably be killed in the comment section by suggesting this :) And no, Laravel or Symfony are not doing this.
Is there a better way to solve this problem?
I'd suggest to do the following: for a project with <5 components, use multi-repo. If >5 components, go monorepo.
In general there are not so many options to solve this:
git submodules
mono-repo
multi-repo
Each of the approaches has pro's and con's.:
Updating git submodules a.k.a. git version bumping and submodule updating leads to git madness, because it'll be constantly broken. and git madness leads to the dark side. :)
Mono-repo is easy to maintain and easy to publish. It gives you easy maintainace for the developer and multi-repo for the consumer. You can replace/rename/refactor across all modules/components at once.
Many-repo is hard to maintain, when you have a large number of components.
See also:
https://www.tomasvotruba.cz/blog/2017/01/31/how-monolithic-repository-in-open-source-saved-my-laziness/
DrupalCon New Orleans 2016: The Symfony Monolith Repository
Sounds weird, but I thought composer was a tool that one used to install packages in PHP stacks. An efficient and robust way to make sure that php environments are setup correctly.
But I keep coming across forum posts that talk about caching and advising to do things like composer clear-cache as if it was part of the actual running application. Like it's actively doing things in the running app.
Am I missing something?
Composer is a tool for dependency management in PHP. It allows you to declare the libraries your project depends on and it will manage (install/update) them for you(refer link.
It helps us installing/updating various requirements/components for our app such as Laravel framework, Doctrine, lodash etc..
If you have ever written anything in PHP before, you have probably found that it feels like you have to keep re-inventing the wheel anytime you want to do a common task such as User Authentication, Database Management or Request Routing.
If you were to start manually picking the bits you wanted from Laravel then it would become very difficult to manage. Each library might also have dependencies, and so you would end up in a mess, particularly if you required other people to work on your project.
This is where Composer comes in. Composer is a dependency manager for PHP. Composer will manage the dependencies you require on a project by project basis. This means that Composer will pull in all the required libraries, dependencies and manage them all in one place.
Managing your dependencies manually in any programing language is an immense pain. This is often why, in most programming languages these days you may notice that all of them have some implementation of a dependency management system or generally a package manager.
In PHP, we use NPM i.e Node Package Manager in frontend technologies like JavaScript, VueJS. For backend, Composer is the de facto dependency manager.
Laravel is itself a package of packages, hence to develop our projects smoothly among the team members, dependency management becomes a must and composer does its work under the hood, silently but efficiently.
Composer is an application-level package manager for the PHP programming language that provides a standard format for managing dependencies of PHP software and required libraries.
Composer runs through the command line and installs dependencies (e.g. libraries) for an application. It also allows users to install PHP applications that are available on "Packagist" which is its main repository containing available packages. It also provides autoload capabilities for libraries that specify autoload information to ease usage of third-party code.
Composer is used as an integral part of several popular open-source PHP projects, including Laravel.
Thanks for your attention, this is a question of organization, I work with PHP and GIT for version control. I use Netbeans IDE to program, GIT integrated (although I am still a rookie).
Normally, I follow the approach that Symfony2 specifies for organize the project files, but I use my own framework for my projects.
The (main) question is: Any component or code part which has its own version control must be located under the /vendor/directory?
For example:
I have my project files in src\Acme\ProjectX\, also the utility package which use all my projects: src\Acme\Util\, and it is under the version control too (GIT).
and now let's remember the basic skeleton of a project based on Symfony or similar:
/app (application related elements)
/src (source code of the project)
/vendor (third party libraries)
/web (front end controller, the web directory, assets resources etc...)
So, Must be 'Acme\Util' included in the vendor directory? And, is necessary to use composer to declare the dependences?
In addition, the Utility package has a lot of classes but only few are used in projects. Must I remove those are not using by the project.
Summarizing, It will be nice if someone can contribute his knowledge for help me to represent an scenario like this.
I hope I could explained...
Thanks in advance!
Vendor directory
It's a good practice to separate external dependencies and the application code. If you are using Composer you can change it to something else.
Unused classes
Unused classes shouldn't matter if they aren't being loaded. They'll just take a bit of extra disc space.
It might be a good idea to separate the Utility package into multiple packages if you find yourself frequently using only a small part of it.
Dependency managers
It isn't necessary to use a dependency manager, but it sure does help. Having to install, configure and maintain everything manually (especially with many dependencies and sub-dependencies) would be a horror.
Let's consider I have different projects for my company. What is the best practice concerning symfony 2 ?
1. Add new bundle for each project in the same symfony 2 skeleton (there could be several bundles for one project: even shared bundles between differents project)
2. Add a new Symfony 2 skeleton for one project (there could be several bundles for one project)
if way number 1 is acceptable, is there a maximum number of bundles for one symfony 2 skeleton ?
A Bundle is a logical component in your website like a backend or a menue. You should build your bundles global that you can use it in new projects.
I would prefer to make more instances and build bundles that were included in your projects (vendor folder). Then you have single components and can use it in new projects.
The advantage is that you can have different versions in different projects. Perhaps you need another version of a bundle in Project A and Project B. Thats its a bit complicated with one instance.
When need to scale your website its better to have more instances to put it on different servers. When you have only one instance with all projects then you need everytime the complete sources.
You can build your own composer packages to update and deploy over composer.
https://packagist.org/
I think there are some more package builder.