When using composer to get dependencies a huge amount of unnecessary files are download such as documentations, test units, etc. These make the built file large. How to only include production files from composer folder?
Unfortunately, the most package developer don't exploit .gitattributes files and using or not using --prefer-dist is the same for them. I come up with the following gulp task: https://gist.github.com/salarmehr/b62703afb6617f4cacac
This isn't as easy as it looks at first glance.
Composer itself isn't the right place to fiddle with project's published code. So the next stop is .gitattributes.
Symfony tried to do this, and their experience made them revert this decision. Composer had information about using .gitattributes in the documentation, but removed it.
In essence, removing some part of a package from a distinct distribution path is likely to cause more problems than it solves. From my perspective, the CLI switch --prefer-dist and --prefer-source is a selector of either having to clone a huge repository that takes ages or download a ZIP with that exact version - but the results should be equal, i.e. I should not be forced to --prefer-source for ALL my dependencies just because one single package that decided to "optimize for deployment" decided to remove documentation and tests from their ZIP.
Yes, during development I usually look at their code and tests to help me understand what's going on - or what SHOULD go on, and isn't.
Conclusion: Composer is NOT a deployment tool. If you care about the size of your application, it is your task to remove everything you don't need or want, and probably optimize other things as well (minify JS and CSS, optimize images etc.). It should not be Composers' or any package maintainers task to do this optimization for you.
Related
I want to minimize traffic usage with composer packages load, so I need for production loads to only download the source/lib folders without examples/docs/readme.
I haven't found any information in the official composer documentation.
UPD
One of the best solution - add .gitattributes file into your project and add main rows from this gist
This is not really up to composer and instead a responsibility of package maintainers. You can use --prefer-dist to install a distribution instead of checking out the git repository as is usually done during development, but what this distribution contains is managed by the package maintainer.
If the packages you install allow for pull requests & issues, you could ask them to provide a .gitattributes file which is recognized by github when creating an archive or provide a pull request providing one. In this file you can specify which files and folders will be excluded from the archive using export-ignore. In any case since everything is bundled in a zip the bandwidth saved is probably negligible.
So, short answer is the only way to minimize bandwidth usage and allocated disk space for dependencies is using the --prefer-dist option - and omitting dev dependencies with --no-dev - when installing them for production.
I am reading/learning about Composer, the application-level package manager for PHP.
In this blog post written by lead dev Jordi Boggiano, he writes:
Composer on the other hand forces you to declare your project
dependencies in a one-stop location (composer.json at the root). You
just checkout the code, install dependencies, and they will sit in the
project directory, not disturbing anything else on the machine.
Another related feature is the composer.lock file that is generated
when you install or update dependencies. It stores the exact version
of every dependency that was used. If you commit it, anyone checking
out the project will be able to install exactly the same versions as
you did when you last updated that file, avoiding issues because of
minor incompatibilities or regressions in different versions of a
dependency.
If I understand Composer properly, when we're talking about packages downloaded/installed by Composer, we are talking about PHP code packages, ie, programming code written in PHP, and not system-level packages, eg, extensions to the PHP runtime installed on the server. So once these PHP code packages have been downloaded and added to a PHP project, I would have thought those packages become part of the PHP application source code, eg to be checked in to whichever version control system is being used for the project. If another developer comes along and checks out the code, why would they need to then "install the packages", as is stated in the blog post? Wouldn't they get a copy of all code packages when they check out the code from source control? This line in the blog post is confusing me, and making me think I don't understand Composer.
Any clarity on this would be greatly appreciated. Thanks.
The dependencies themselves should not be commited to source control. The composer.json and composer.lock files, on the other hand, should. There's various reasons for this, amongst them:
Every time you update the dependency you would have to commit the changes. That kind of tightly couples your code to the dependency, when it should be exactly the other way around.
The packages themselves are already in their own repository with their own history. Why repeat that in your project's history?
Those repositories can be huge, just muddling the waters around your project. Why carry around all that weight?
Instead, having each developer just run composer install (very important: not composer update) whenever they check out the project is much more efficient. Composer will install the dependencies from composer.lock, making sure everyone running the same commit is on the exact same page. The same goes for deploying.
You can read more about this here.
On the other hand, there might be situations where you have to commit your packages to get around a problem, like for example when you know you won't be able to run composer install on your production server (shared hosting)
Normally packages installed via composer don't get checked in to source control, only the code you write and the composer.json and composer.lock files.
This way the repository for your project does not get bloated with code you did not write and possibly don't really care that much about.
Yes its normal after cloning down your repository a developer will need to run the "composer install" command. The composer.lock file will ensure they get the same modules and versions of them you used when creating your project.
Not including the composer modules in your source control also allow you to easily update to the modules to get bug fixes and new features in new versions of them.
I'm new to Composer (getcomposer.org) and wasn't sure how it works if I install a package locally using Composer and then push my codebase to my production server using Git. Do I have to run Composer again on the production server?
cheers,
J
When you setup your project, you add your dependencies into your composer.json file in your local project directory.
Once you have done this, you will need to run composer update. You can also run composer install, however, without a composer.lock file, composer install actually runs composer update.
Composer update goes out and resolves all the dependencies of all the libraries you are using, downloads them to the /vendor directory, creates an autoloader script and generates the composer.lock file.
For your project what you want to do is version your composer.json AND your composer.lock file.
On your production server, you will always run composer install, which insures that the libraries on your production server are the exact same ones you utilized in your development process.
composer install is also a lot faster as it does not have to do all the dependency management work, and can almost always just pull a specific commit#. It doesn't have to look at version strings. Thus is is usually very fast, once a server has already gone through it once.
In development the only time you should run composer update, is when you introduce a new library OR you have an issue where an underlying library has been changed and you know that you need to have composer go out and re-calculate the dependencies. composer update always recalculates and downloads the latest revisions of any library available even if the version level did not change. This means that there is a potential for something to have become broken, necessitating the potential for as full a set of regression tests as you might have available. In short, something having nothing to do with what you're actually changing could have broken, so you only want to introduce the potential for change when you are forced to.
Of course, if you did introduce a new library, you have no choice but to run composer update.
Once you run composer update, your composer.lock file will be updated (as expected) and the production server will pick this up when you run composer install on it.
As others stated, put the vendors in your gitignore. The point is that these are external libraries that you depend on, but that do not belong in your project, and should not be versioned. In the old days some people utilized git submodules, and it's a big PITA you really want to avoid, not to mention that submodules don't address dependencies of the libraries you included.
It depends how are you working. If you, like getcomposer.org says, are ignoring the "vendor" folder then you need to run it again. If you are versioning the "vendor" folder then you don't need to run it again.
Have in mind that composer will get in charge of managing your dependencies versions, so there is no need to put your dependencies files under versioning. If you put these files under git you will only make your repository bigger.
Read https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies.
For clarification when you ignore the "vendor" folder Git don't track the files under the folder so if you clone the repo it will be like composer never was executed
Wen I use composer to update/install a new project it starts cloning a lot of git repositories. If you have a lot of dependencies, this takes ages. After all I just need the latest version of all the libraries. I'm not going to change anything.
Is there a way to tell composer to download the requested version without cloning? Just fetching the zip version from github would be much faster.
Accoding to https://github.com/composer/composer/issues/840 this is not possible.
I think we need to add an option for this at some
point. What #stof said is not true. The thing is that with dev
packages, the chance of updating them on update is much higher than if
you just follow tags, and updates are much cheaper once you have a git
clone than redownloading the full thing every time. It's a trade-off.
But we should probably add a --prefer-dist flag.
I've installed composer and added some packages via 'composer install'. It installed them under "my_project\vendor" path but some of the packages were cloned using git, so when I committed "my_project", those cloned packages were ignored.
The problem is that when other developers are cloning "my_project", they are missing the packages that were ignored. Is there a way to automatically add the packages to "my_project" so other developers will fetch them from me?
I think this should be done using submodules, but I don't know how to automatically add every new package from composer as a submodule to my project.
Preface: Jordi - I love composer, keep up the great work, and if I'm mistaken on any of this let me know and I'll both update my workflow and edit the post :D
Unfortunately this isn't the "general recommendation" depending on who you ask, it's by far a developer-only perspective. And the caveats to using the practice prescribed in the composer FAQ have many more considerations than I can cover here. So I'll leave a couple major points for the consideration of others.
By #Seldaek's own admission composer isn't really 100% stable, far better from a year ago, but still a very active project regardless. So relying on composer to implement an identical environment on a dev server vs staging server vs production server wouldn't be a general recommendation from any QA / Deployment group. This is not meant as a slight to Jordi, but rather an expression of the maticulous nature of QA peoples.
From the FAQ, it states when merging vendor libs into your own repository you should:
Limit yourself to installing tagged releases (no dev versions)
However if you use composer to manage your CI or automated deployments, the same constraint would apply - only more so - because deploying a master or dev tag to your production environment could be a very different package than what you tested in staging only a day or even an hour ago.
Even outside of changes introduced in third party libs (which would be solved by using only tagged versions regardless of dev or production deployments) unless you can rely on composer doing the exact same thing every time, you'll risk introducing bugs into production. This is not really a risk-case I would concern myself with, but then again I'm a developer too ;) But issues can result from simple changes like this where unless you maintain the exact same version of composer.phar on all environments, you could really muck up a staging or production server.
The other major issue I have is really related to all of the points listed under this heading:
While it can be tempting to commit it in some environment, it leads to a few problems:
I don't see the consequences as problems, but instead benefits! A large vcs repository isn't that big of a deal in modern high bandwidth environments. And unless you are using a very active vendor lib, your diffs won't be that big either. Even if they were big, git/hg/dvcs systems are all capable of re-using chunks, compressing chunks and keeping all your ducks in a row. But more so, they are an alert to the developer when changes are introduced to those packages, and diff -w is a great summary view of the total changesets, especially if you are on dev/master tags.
Duplication of the history of all your dependencies in your own VCS.
This is worded a little incorrectly, it won't duplicate the entire commit history of the vendor lib, just a single commit (your commit) covering the full delta between now and the last time you ran a composer update resulting in changes. You're probably not updating all of your libs every time you update, even if you don't specify individual packages. And if you did accidentally update a vendor lib, you can easily revert, whereas if you did so on a dev/master tag and it broke your environment, you'd have to figure out what version you were previously using and specify the tag in composer.json, and update again to revert it. git checkout /vendor/3rdpartylib --force just seems easier to me.
Adding dependencies installed via git to a git repo will show them as submodules. This is problematic because they are not real submodules, and you will run into issues.
Ideally, composer would give you a config option. It could automatically delete the .git directory from git pulls, and automatically rm the directory (or temporarily mv it) before updating a lib, when and only when an updated version exists. And doing so would be far more reliable than leaving that manual process up to individual developers. There are an equal number of reasons to have vendor libs integrated into your version control repo so the choice really depends on the details of your situation.
The biggest reason for versioning all of your files is being able to reliably deploy the exact package you tested in development to staging to production, a key purpose of vcs and automated deployments to begin with. Unless you configure your development environment to use specific tags for every package and you version control your composer.phar you should not rely on composer to deploy your software.
You should ideally just add vendor/ to your .gitignore, and then every developer of the project would run composer install to get the vendors on his setup.
You can read the FAQ entry on commiting vendors for more details.