I am working on Laravel webapp right now and kept vendor directory out of git (version control) so far and every time for fresh install I used to have composer install command added to automated script and everything was fine.
Now just 2 days back I added added laravelcollective (https://laravelcollective.com/) to my project for helping me with forms and html in blade templates. Now somehow one of the dependency requires me to generate GIT private token to install it and that is pain as it would hurt my automation. I can still hack it by calling the url and scrapping html to read token and stuff like that but I don't like it. And then I thought is it good idea to keep vendor directory out of SVN/GIT? Isn't source code for a product contain all dependencies within itself? I am not talking about stuffing JRE in the installer but when it comes to libraries of a product in native language.
I would like to hear more about it on industry standards or best practices on this.
P.S:
This question is much generic and not just limited to laravel or even php for the matter.
Now somehow one of the dependency requires me to generate GIT private token to install it and that is pain as it would hurt my automation.
You're just running into Github's rate limits for package downloads for anonymous users. No reason you can't automate this. Generate a Github token (you only need to do it once - they get very high rate limits for authenticated requests), then have your automation use that token like so:
composer config -g github-oauth.github.com <oauthtoken>
https://getcomposer.org/doc/articles/troubleshooting.md#api-rate-limit-and-oauth-tokens
Well, for production environment you usually run a build process first in your CI software. If 'composer install' fails during the build – application won't be deployed to production environment, so you are safe.
Yes, most (99%+) people keep 'vendor' folder out of the repo because it's a third-party code, it's not yours. You may not even have rights to host it in your repo.
If you want to be sure that your production version will have all the dependencies in order, the way you had them during CI, and will always release – you could build Docker images and ship them to production. Then, everything comes prepackaged.
Related
I'm still new to coding and I'm learning everything on my own. This is a silly question for you but after reading a dozen of articles I am still confused.
I have a php based website on a shared host. After reading the various articles on benefits of using repositories and Composer, I decided to give it a try. These are my difficulties so far:
Which version of the operating system of Composer should I download, to enable me to install/update repositories of my cPanel based shared hosting?
If I am to install Windows version, how do I connect to my shared hosting to install/update the repositories?
My apologies for my silly questions, but it would really help.
If you are using shared hosting, you are unlikely to be able to use Composer on the host itself. Furthermore, you are not encouraged to use Composer "on production".
I would recommend you use Composer locally (on the O/S of your local machine), to compose your project and install your dependent packages. Once it's all working and tested with your own code, you upload your entire development directory tree including the resulting vendor library - as one big FTP/SCP upload of "flat files".
Once you get more advanced you could adventure into automated deployment techniques, but I feel for now you would be best to stick to using Composer as a local development tool to manage your codebase.
Update, further details:
Composer is really a tool to help you manage your codebase in development. It's not intended as a "deployment" tool. Previously you used to find a library you liked, download it, unzip it into your codebase somewhere random like "lib/stuff" and then link to it, and commit it into your version control system (VCS). OK, but they a year later you want to update it and you have to download it again, figure out where you saved it and how to overwrite the files, or delete old ones... it gets hard. Also your VCS repository gets full of 3rd-party components - even duplicates of the same one! Composer solved this by bringing order to this long-term dependency management chaos.
The reason you don't want to run Composer "on production" (i.e. your live website), is that during the process of download, update, composition your website will probably be broken. Even if the composer process works, this could be several minutes of broken site. After the update has finished - you now have a completely new set of 3rd party packages: how do you know they are compatible with your codebase?
So therefore you only do composer updates locally, test everything, amend your code to work the shiny new updates, and only then do you decide to upload the whole new site to the server - just as if you'd cobbled it all together manually. The deployment is independent.
I'm getting my feet wet with dependencies. I recently wrote a small application in PHP and used Composer to get some dependencies (mainly Slim, twig, hassankhan/config and illuminate/database). So I have the project in my local computer and basically have:
public folder
app folder
vendor folder
composer.phar
composer.json
composer.lock
It's now time to upload to my web server (I work with a GoDaddy hosting account with Linux CPanel). The question is, what should I do? Do I upload the whole thing, with the vendor folder and composer files? Or am I supposed to use Composer in a different way when uploading to my hosting? Never done this before so any guidance will be deeply appreciated.
Thanks!
You are not required to use Composer in any particular way. I recommend using it in the way that works best for you. That will depend on the type of development and release process you use. It sounds like you are using a fairly short and simple process where you develop for a while, decide the code is ready, and upload it to your production server. With this process, you could simply upload everything, vendor directory and all - just as you state. This could lead to unused files/directories not being removed but that is a general problem with this type process anyway (you can work around it by first deleting everything, breaking your site temporarily).
Composer helps you (among other things) "lock" all of the required libraries at some specific version. This is very useful when used with version control, tagged releases, and multiple environments (such as your development environment, a QA/Testing environment, and a Production environment). This helps ensure that each environment uses the exact same versions of the required vendor libraries.
If you were to try to use composer directly on your hosted server (i.e composer update), you would have to make sure that the php cli is available and that all other prerequisites for composer (including composer itself) were available. This is usually hard to count on for generic hosting providers so, you really are better off not trying to execute composer there.
As your project matures and uptime is important, you will want your development and deployment processes to mature as well. That's when you will realize many of the benefits of Composer.
I am new to composer and would like to know how do you guys deploy a project to the production server using composer?
In deploying, would composer also push the dependency packages needed to the server?
Would/ Can composer build the application with minification process?
I think the current best practice is to not run Composer on the target production server. The regular process of deploying a web application usually requires several independent steps, and Composer is only suitable for some of them, regardless of what people make it do additionally.
You mention minification, and I would add the process of pulling in JavaScript dependencies in general. This is no domain for Composer. It has been done in the past to offer Composer packages that contain Jquery, but this requires additional work to put Jquery in the right directory afterwards, adding the need to run post install scripts or add installers that need configuration. I guess the right way to do it would be to use Bower for this.
So the deployment would be at least a three step process.
Use composer to install PHP dependencies.
use bower to install JavaScript dependencies.
Use rsync, SFTP, SCP or FTP(S) to move all the files to the server.
Any optimization steps would be done prior to moving the files onto the server inside the deployment script.
And if anything fails during the collection of dependencies, be it an unsuspected downtime of Github, or your deployment server running out of disk space, you don't end up with a halfway deployed new website version. You can stop the deployment script before syncing if anything is missing or gone wrong.
yes, composer install get all dependencies in your test server, check it, if everything work ok, then sync all the files to your production server, which can save you from the unexpected problem running composer install on your production server.
other way is to sign all the files from your test server after composer install, then use composer install to get all dependencies in your production server, sign the files, now you must check two signs separately generated in test and production server, if it match, congratulations, the deploy is ok.
I'd like to run phpunit on an external server, the feed those results into a Jenkins plugin like the Clover PHP Coverage Report action.
The code base has some library dependancies that the owner of the Jenkins server does not want deployed.
Well with some tweaking I guess it is possible to make use of the monitoring external plugin. You will have to figure something out to get the files on the jenkins server.
An other option is, if the server owner is willing, to create a specific build agent for your needs. That doesn't affect his jobs and you can use your builds like you want.
Yes, library dependencies shouldn't be installed globally on a Jenkins server, but this is where Composer comes into play: You'd essentially install exactly the dependencies your current software needs locally in the workspace of your job.
Nobody should be bothered by this, because you could also include the needed library code there manually - or even worse, you could use the same file and class names and code something entirely different. All of this must not interfere with any other job running on that same server, and it doesn't.
You can't really avoid Composer, because PHPUnit will stop being distributed via PEAR, as well as some Symfony components that are being used. Better go to the Composer project page and learn how to use it. By the way: You can include the needed version of PHPUnit with Composer as well, so you need not rely on a centrally installed version (which is hard to update because there are so many jobs that would need updates then - too much work in one go).
I've installed composer and added some packages via 'composer install'. It installed them under "my_project\vendor" path but some of the packages were cloned using git, so when I committed "my_project", those cloned packages were ignored.
The problem is that when other developers are cloning "my_project", they are missing the packages that were ignored. Is there a way to automatically add the packages to "my_project" so other developers will fetch them from me?
I think this should be done using submodules, but I don't know how to automatically add every new package from composer as a submodule to my project.
Preface: Jordi - I love composer, keep up the great work, and if I'm mistaken on any of this let me know and I'll both update my workflow and edit the post :D
Unfortunately this isn't the "general recommendation" depending on who you ask, it's by far a developer-only perspective. And the caveats to using the practice prescribed in the composer FAQ have many more considerations than I can cover here. So I'll leave a couple major points for the consideration of others.
By #Seldaek's own admission composer isn't really 100% stable, far better from a year ago, but still a very active project regardless. So relying on composer to implement an identical environment on a dev server vs staging server vs production server wouldn't be a general recommendation from any QA / Deployment group. This is not meant as a slight to Jordi, but rather an expression of the maticulous nature of QA peoples.
From the FAQ, it states when merging vendor libs into your own repository you should:
Limit yourself to installing tagged releases (no dev versions)
However if you use composer to manage your CI or automated deployments, the same constraint would apply - only more so - because deploying a master or dev tag to your production environment could be a very different package than what you tested in staging only a day or even an hour ago.
Even outside of changes introduced in third party libs (which would be solved by using only tagged versions regardless of dev or production deployments) unless you can rely on composer doing the exact same thing every time, you'll risk introducing bugs into production. This is not really a risk-case I would concern myself with, but then again I'm a developer too ;) But issues can result from simple changes like this where unless you maintain the exact same version of composer.phar on all environments, you could really muck up a staging or production server.
The other major issue I have is really related to all of the points listed under this heading:
While it can be tempting to commit it in some environment, it leads to a few problems:
I don't see the consequences as problems, but instead benefits! A large vcs repository isn't that big of a deal in modern high bandwidth environments. And unless you are using a very active vendor lib, your diffs won't be that big either. Even if they were big, git/hg/dvcs systems are all capable of re-using chunks, compressing chunks and keeping all your ducks in a row. But more so, they are an alert to the developer when changes are introduced to those packages, and diff -w is a great summary view of the total changesets, especially if you are on dev/master tags.
Duplication of the history of all your dependencies in your own VCS.
This is worded a little incorrectly, it won't duplicate the entire commit history of the vendor lib, just a single commit (your commit) covering the full delta between now and the last time you ran a composer update resulting in changes. You're probably not updating all of your libs every time you update, even if you don't specify individual packages. And if you did accidentally update a vendor lib, you can easily revert, whereas if you did so on a dev/master tag and it broke your environment, you'd have to figure out what version you were previously using and specify the tag in composer.json, and update again to revert it. git checkout /vendor/3rdpartylib --force just seems easier to me.
Adding dependencies installed via git to a git repo will show them as submodules. This is problematic because they are not real submodules, and you will run into issues.
Ideally, composer would give you a config option. It could automatically delete the .git directory from git pulls, and automatically rm the directory (or temporarily mv it) before updating a lib, when and only when an updated version exists. And doing so would be far more reliable than leaving that manual process up to individual developers. There are an equal number of reasons to have vendor libs integrated into your version control repo so the choice really depends on the details of your situation.
The biggest reason for versioning all of your files is being able to reliably deploy the exact package you tested in development to staging to production, a key purpose of vcs and automated deployments to begin with. Unless you configure your development environment to use specific tags for every package and you version control your composer.phar you should not rely on composer to deploy your software.
You should ideally just add vendor/ to your .gitignore, and then every developer of the project would run composer install to get the vendors on his setup.
You can read the FAQ entry on commiting vendors for more details.