I'm working with symfony2 for some time and I don't really get the correct way to work with the vendors.
So here is what I'm doing:
I have the deps and deps.lock files in my git while I ignore the whole vendors folder. Now when I install the application to a new server, I do a php bin/vendors install to pull the vendors onto the server. I get the message that I have to use install --reinstall and do that.
From my understanding, the versions should now be the exact same as on my development machine, as both deps and deps.lock are the same. But it seems that the deps.lock gets (partly) ignored?
There is also a vendors update command, which I read should not be used. BUt I didn't get the idea what it really does.
So I'm a bit confused now as of what command should be used when and what it is supposed to do. Maybe someone can shed some light on this topic! I'm especially interested in the correct way to use the vendors command both local and on the server so the vendors are in the correct version on both systems!
install and update both fetch new code from the git repositories specified in your deps file
install checks for hashes in you deps.lock files for each library. If it finds something, it checkouts the commit corresponding to the hash. If it does not, it checkouts the tag or branch in your deps.lock if one is specified and creates an entry in the deps.lock file
update is useful when you want to update to a new version of symfony (or any library in the deps file). If you one day, you feel like updating, you can read this post I wrote about the update process.
To sum up, I always use update on all machines, and I try to always specify a version for each library, so that the production environment does not get updated to an unstable version unexpectedly.
install --reinstall is the same as install but it also deletes vendor folder content before doing installation.
vendors update updates all your vendors to the latest version or version specified in your deps file and updates your deps.lock file. But you rarely need it, don't know where you read "should not be used".
If you look inside vendors file, you can see this line:
if (is_dir($vendorDir.'/symfony') && !is_dir($vendorDir.'/symfony/.git') && !in_array('--reinstall', $argv))
...Try to run ./bin/vendors install --reinstall...
So you have vendor/symfony folder without .git in it.
have you download the standard edition which already contains the vendors, that's why it displays this message to use "install - reinstall"
I personally made php bin / install-reinstall vendors in the deployement project
Related
I recently started a project using composer for the first time, and I just deployed it to Q&A (demo), with git I used to just do a git pull and update the Q&A environment, but now with composer update only the dependencies get updated.
My question is, what is the SOP(standard operating procedure) for updating the core project, do I still use git, or is there a way to do it with composer?
Or am I completely doing this wrong, and should be working out of the vendors folder?
The point of Composer is that you don't need to version control the dependencies which means anything that ends up in vendor/.
The project has a composer.json and composer.lock. These are within git's control so it knows the packages and versions to use. However, the vendor/ directory should be ignored, with .gitignore. If you don't already have that set up simply add this line:
/vendor/*
You version control your other files as normal.
So the operating procedure is to use git and normal. Followed by composer update.
The advantage of this setup is that git doesn't have to bother managing potentially thousands of files (inside vendor/) that will never normally change. The only circumstance under which they'd change is if you want to start using a different version of a package, or adding new ones. Well, all of those packages/versions are defined in your composer.json (which git is monitoring for changes). All you need to do is run composer update and it will update everything in your vendor/ directory to the "right" version.
That's one of the advantages of using Composer - all developers can have a "list" of the correct packages/versions, without the need to version control all of the files in them.
Edit as per the comments below:
Note composer update should only be run in development. Use composer install when deploying to QA or production. This will install the exact versions referenced by your composer.lock file.
You keep using git for your project and composer for 3rd party libs.
I have a large web app in PHP (link in my profile), and that's how I've been doing it, and it works good.
When I have new production-ready code and it's ready to be released, I do:
git fetch && git pull
And when I want to update composer, I do:
composer -o update
I don't know if you're familiar with -o flag - it generates static autoloading maps, which makes your project load classes faster. More info here.
Is there an automated way of updating the version number in composer.json and adding the necessary tags before publishing, like the way npm version does?
I mean, if you had a composer.json with the line "version": "2.1.3", and executed:
composer version minor
It would do the following:
Updating the version number in composer.json to 2.2.0
Triggering composer update to update composer.lock
Making a new git commit
Making a new git tag v2.2.0
I know that the composer version command doesn't exist, but is there an equivalent tool?
npm version does stuff that you very likely do not need for Composer:
The version number is not recorded in composer.json if there are other means available - and because you are referring to Git later on, they are available.
Updating dependencies in the lock file is unnecessary. The lock file will be ignored when the project you are dealing with is included somewhere else.
Because nothing has changed in the project, a git commit wouldn't do anything.
All this leaves you with creating a new tag in the Git repository. Putting this into Composer would mean you'd exchange one command with another, without any big benefit besides you won't have to lookup the current version number you are dealing with if you use some of the relative version parameters.
All in all I'd say that simply tagging your new version is enough for Composer. You'd probably need to have some infrastructure in place and configured to make the world aware of the new version:
If your package is open source and on packagist.org, you should have a post-commit hook to notify them as soon as a new version is available. This is a standard option on Github, I don't know about other source code hosts.
Otherwise if you have to feed closed source code, you'd probably start a new update cycle of whatever system is used to create an alternative package information source (be it Satis, locally hosted Packagist, Toran Proxy or Private Packagist)
This however depends on how you set up things.
If for some reason and despite all voices against it you still want to use a tool like the OP asked for, https://www.npmjs.com/package/composer-version works quite well.
Bit of a strange set up but have come across a project where Composer has been used in a local environment to get a project started. The original developer did not have ssh access to the production server therefore he used Composer locally and uploaded the 'vendor' directory from his desktop to the server using FTP.
I now need to add the PHPMailer package so have done the following locally on my Mac:
cd Desktop/
composer require phpmailer/phpmailer
This has created the following structure on my desktop:
Desktop/composer.json
Desktop/composer.lock
Desktop/vendor/autoload.php
Desktop/vendor/composer/*
Desktop/vendor/phpmailer/*
Which of these do I need to upload through FTP? I realise vendor/phpmailer/* is the package I want, so will need uploading.
What about the others? I already have an autoloader configured so guessing vendor/autoload.php is not required here?
composer.json I could add the package to what's already there, e.g.
"require": {
*other packages here*
"phpmailer/phpmailer": "^5.2"
}
But I wasn't sure if that's necessary because I'm not going to be using ssh/Composer on the server to run any updates?
The usual workflow would be:
Checkout the current version from the version control.
Add dependencies via command line composer require new/package.
This will download the new package and update the autoloading.
Test the result locally or on a test website environment.
If happy with the result, upload the whole folder to the production server.
There may be several exceptions from this general workflow:
ad 1: If there is no version control, you'd probably better of starting a local git repo right now, and download the current production state into it as the first commit. Not having version control will make things harder, especially going back to known working versions. And because the files on the production server are probably unmanaged, you'd also check in the vendor folder into your newly created version control just to avoid canceling any changes that had been made to these files.
ad 2: Manually editing the composer.json file sometimes is a faster way to get what you want if you know what you are doing, but you'd have to correctly edit the JSON. For me it usually is too much hassle if I already have a command line ready. The command will also select a matching version that fits into the already installed dependencies. Manual editing may lead to version conflicts that you'd have to untangle. Remember to only install dependencies that work with the PHP version in production. You probably should run composer config platform.php X.Y.Z in order to add the production version of PHP into the composer.json file, which prevents Composer from installing dependency versions based on your development PHP. Adding the -g switch will add this setting to your global (user) setting instead, which will affect all composer operations you start, also for other projects.
ad 3: Manual editing will require you to run composer update on the command line, so there's probably no reason to not do composer require instead.
ad 4: How this could be done is entirely dependent on what environment you have to work with.
ad 5: At this stage you have assembled all files necessary to create a working website. Uploading them to production will always result in a working website unless the upload fails somehow. You could also use some "upload first to temporary folder, then move on the server" approach if you fear FTP would be unreliable. Some people take a different approach: They have a git repository on the production server and they simply push the version that should go live onto that remote repo. Some post-push scripts will run composer install then. This automated approach will also work (but not using FTP), but has the higher risk of something failing during deployment, and probably has no easy way back to the previous situation.
So in the end I'd say that uploading the whole folder structure via FTP (well, that protocol is insecure itself, better replace it with FTPS (FTP with SSL), SFTP or SCP) is better compared to running Composer on the production server.
Your specific question regarding which folders to upload: All of them. Especially upload the whole vendor folder. It contains the current autoloader and all dependency packages the software needs. If you worked correctly, you downloaded the existing composer.json and composer.lock file together with everything else and added the new dependency to it. This has changed both these files, added the new package to the vendor folder and the classes to the autoloader.
Don't fiddle with uploading only parts of the vendor folder, or manually editing a component of the autoloading. You will only create surprises for the developer coming after you if you do some aspect incorrectly, and it also takes more time. Composer is a very good tool to manage dependencies - use it!
You could copy the composer.json from the server to your local server, add the requirements and run composer update.
After that you can copy all files (composer.json, composer.lock and vendor folder to your server)...
Or you can copy local vendor/phpmailer into servers vendor folder, search for the entry of phpmailer in vendor/composer/autoload_psr4.php and add it to your servers vendor/composer/autoload_psr4.php.
Using this method also add phpmailers dependencies the same way.
composer depends phpmailer/phpmailer
I'm already half way done with a project in Symfony2.
I need to install a couple of new vendor bundles using composer.
I already have everything (minus logs, cache and parameters.yml) in version control (including the vendor folder).
Problem is when using composer update, it deletes the .svn folders in the vendor folders that where updated. So it's basically impossible to commit now (gives me not a working copy error).
Additional information: I'm working locally and committing to a dev server and then once approved an application server. Therefore it has to be perfect (cannot just run php composer install or php composer update on the dev/application server after commit).
I also tried exporting everything and copying and pasting them back into the repo but that also didn't work (index page broke locally).
Regarding to vendor versioning the best way is not version vendors at all.
The only things you need to version are composer.json and composer.lock. This may cause a problem with vendors which doesn't have stable versions or with that for which you need not stable one (eg. master with particular commit).
As a solution you should create your own (private) vendor repository (let's say your own packagist). Composer has a tool for that, which is called Satis.
https://github.com/composer/satis
So my suggestion would be:
Create a private repository with Satis. You place every package you need in satis.json and whenever you need to update a version of vendor, or add new one, you only modify satis.json and rebuild repository.
In your project's composer.json you set your new private repository as the only repository and set option: packagist to false.
Now, every time you run composer install it will use only your private repository, so it's fast and you always sure that every environemnt has the same versions
-
I was in similar situation two years ago.
The hard lesson I learned was never to edit files within vendor. At first I totally rejected using composer and manually cloned everything I needed. Later on, I decided to fork projects I needed to edit and referenced my forks instead.
Composer supports private GitHub repos - you don't need to register it to Packagist in order to work.
You should not keep your vendor directory in your version control. This is how it is done in Symfony Standard Edition and you should follow this. Running composer install command should be a part of your deployment process
Including vendor packages in your codebase is not recommended, so if you need to maintain the same version of the packages you use on your local machine, the best way is to keep composer.lock in the VCS and running only composer install on other environments.
Additionally, if you want the prod deployment to be instant, without depending on the composer process, you could run composer install on the dev server, and once it's validated you can make your prod deployment script copy the vendor folder from the dev env.
A composer install normally takes a few minutes. And on a production environment it's feels too slow.
Is it possible to make a composer install to a temp directory and then switch it? If that is possible the downtime should be about zero.
Or are there any other way to do a composer install faster?
I created a composer plugin to download packages in parallel.
https://packagist.org/packages/hirak/prestissimo
$ composer global require hirak/prestissimo
Please try it. In my environment, composer install become 10 times faster.
You can sometimes speed up composer install significantly by using the --prefer-dist flag, which just happens to be recommended for production use:
--prefer-dist: Reverse of --prefer-source, composer will install from dist if possible. This can speed up installs substantially on build servers and other use cases where you typically do not run updates of the vendors.
composer install docs here: http://getcomposer.org/doc/03-cli.md#install
Edited To Clarify Sometimes
I say it sometimes speeds up composer install because there are plenty of factors that go into it feeling slow, not the least of which are network performance and the current Github status. A slow install can be really frustrating, but it's not always b/c of Composer.
You are asking two different and unrelated things.
Yes, it is a solution to build the next version of your site in a separate directory then put it in place after you moved the old version out of the way. It is, actually, the best solution.
This is how the deployment scripts I build work:
prepare the next version of the site in a separate directory (let's say /var/www/new); the following list of items and their order is not static, some projects need a different flow:
get the last version of the code from the repo;
remove the files that are not needed on the live site (.gitignore, IDE project files, placeholders etc);
run composer install;
copy/generate the configuration files containing the settings for the live servers (the ones stored in the repository contain dummy values);
change the user and permissions of all files; make some directories writable by the web server;
create symlinks, directories etc; for example, the directory that contains the user uploaded files is somewhere outside the server directory and a symlink to it is created during deploy to make its content available through the web server;
move the live code out of the way; I use mv to move the entire directory (/var/www/html to /var/www/old, for example);
move the prepared new version to the proper place (mv /var/www/new /var/www/html);
move the previous version into the archive (after I remove the content of vendor and other files that do not change or are external).
The advantages:
the downtime is zero (microseconds, probably, between steps #2 and #3);
a failed build doesn't affect the live site (if handled properly);
reverting to the previous version (if needed) can be done easily.
Regarding the other question, the only way I know to speed composer up is to avoid running it using a PHP that loads the xdebug extension. The xdebug extension shouldn't be loaded on the production server anyway.