In my PHP project I use composer.
For some reasons (stupid, but it's not depending on me) I must store all project files in a git repository, including "vendor/" files fetched by composer. I've removed "vendor/" from the .gitignore file, but some modules (like https://github.com/RWOverdijk/AssetManager) are treated as submodules -
and they are not being added to the repository (git add ignores them).
What do I need to do, to force addition?
You can use the following commands:
# remove `.git` folders recursively
find vendor -type d -name '.git' -exec rm -rf {} \;
# Add the vendor folder
git add vendor
Probably it would be better to persuade your boss that composer isn't meant to work like that.
Related
unfortunately I can't use composer, due to low memory on the webspace. That's why I'm trying to push vendor/* to the git repository, to be able to pull the full project, including the dependencies.
In my .gitignore I'm forcing git to unignore the vendor files by !/vendor/*.
It works for most files, but not for /vendor/friendsofsymfony/jsrouting-bundle.
Content of /vendor/friendsofsymfony/jsrouting-bundle/.gitignore:
/phpunit.xml
/composer.lock
/composer.phar
/vendor/
/node_modules/
My repository is hosted at BitBucket. I don't know yet the meaning of the different folder-icon and the hashs next to it. If anyone knows, please comment.
Does anyone know how to force git to handle all the vendor files?
Thanks in advance!
Solutions
As the jsrouting-bundle is a git submodule, I chose this solution:
git submodule add git://github.com/FriendsOfSymfony/FOSJsRoutingBundle.git vendor/friendsofsymfony/jsrouting-bundle
Another way would be to use the deps file (not tested), source:
https://github.com/XKEYGmbH/ifresco-client/tree/master/vendor/friendsofsymfony/jsrouting-bundle/FOS/JsRoutingBundle/Resources/doc
The jsrouting-bundle folder is a Git submodule. A Git submodule is actually a reference to another Git repository. This is why you cannot add changes from it to your original Git repository.
I had to do this in the past, here is what I did:
php composer.phar selfupdate
php composer.phar update
In the vendor directory run sudo find . -type d -name .git | xargs rm -rf
Commit all modifications: git add -A .
With this, your vendor will be commit like the src directory, so no need to run composer install when deploying in your production environment. When wanting to update just repeat the process. But of course it isn't a good practice and you should do this only if you can't run composer on your production server.
I'm trying do to the following:
1) Create a new laravel project using:
sudo composer create-project laravel/laravel /var/www/html/laravel5`
2) Overwrite only the different files from github. I want to keep the following files and folders unchanged:
.env
.env.example
.gitattributes
.gitignore
vendor/
Clone from github is not working because the folder is not empty.
The only workaround that I found is:
create a new project
copy the files and folders that I want to another folder
clone the project from github
copy the files back to the folder
I'm using Xubuntu, Laravel 5.0 and Github.
Thanks in advance.
You should be able to do this via an all-git method rather than copy/paste from another directory. (I'm not sure why your workaround is a bad solution, though.)
The basic plan is:
Create a repository.
Checkout some bogus branch that you won't keep around.
Add the directories you want to keep to the index and commit them.
Force a checkout to the branch you want. It probably will need forcing because of the other things you didn't just commit.
Apply the patch of the recently created commit to your working directory.
And in commands:
cd /path/to/repo
git init
git branch temp-branch
git add .env && git add .env.example && git add .gitattributes && git add .gitignore && git add vendor/
git commit -m "Sustain Local Changes"
git remote add origin https://github.com/owner/repo
git fetch
git checkout -f master # Should detect origin/master. If it doesn't, just set a branch up manually. Replace master if that's not the branch you want.
git merge --no-commit --no-ff temp-branch
Personally, I find this a lot more ugly than your workaround.
I am sure I once read it somewhere but I cannot find it anymore anywhere, DAMN!
So basically what I am trying to do is to specify some exclusion criteria in my composer.json file for a certain library of mine so that, when used as a dependency of a project, the importing project does not get test files, .git folders, READ.md files and all that stuff (totally useless when you only want a library as a dependency and not for development).
So basically I am trying to lighten up my libs when they are downloaded as dependencies. Anyone on that?
Ta
You can add a .gitattributes file to your project root, looking something like this:
/Tests export-ignore
READ.md export-ignore
When someone installs your dependency this files will be excluded from the distribution zip.
There are some prerequisites for your lib to be downloaded as a zip by composer
You need to have a stable tagged version. dev-master will always be cloned by composer.
If the user installs with composer install --prefer-source composer will also clone from your git repo.
In all other cases composer will download the zip and all the files in .gitattributes will be excluded from it.
Hope this helps.
If you're on a unix-like system, you can do this in one cmd, simply cd to vendor dir and :
find . -type d -regextype posix-egrep -regex ".*\/(tests?|docs?|\.git)" -exec rm -rf {} \;
adapt the regex to your needs
find cmd:
http://unixhelp.ed.ac.uk/CGI/man-cgi?find
regards
As a PHP developer I find myself working with Composer a lot. In the past it was on personal projects and such so I didn't have much problems with it, but now with Laravel 4 it's on project that require deploying and I'm in kind of a struggle to adapt my workflow.
All my projects are git repositories, thus per convention and because it's still quite buggy, like most developers I put the vendor directory in my .gitignore.
Now the problem is : I also use Git to deploy to the server, and by all logic the vendor directory is not uploaded as it's not tracked by the repository.
So my question is towards people that have worked with Composer and Git for longer than me : what is the best workflow to keep the server in sync ? How to track the vendor folder without really tracking it ?
I tried uploading it every time I update with Composer but some of my vendor folders are quite big and I can't manually upload 30Mb of files every time something updates.
I don't really know, how do you guys work it out ? I tried not ignoring the vendor folder but Git just messes it up, half are recognized as cloned repos and are just ignored anyway, etc.
UPDATE : Note that I'm on a shared host so I don't have access to the server's terminal.
The best way is to run composer install on the server after updating to the latest code. You should also make sure you commit your composer.lock file, which the server will then use to install (you should not run composer update on the server).
Capistrano (or Capifony if you are using Symfony2) is very useful for deployments with composer. You can trigger a deployment remotely and it will run a composer install in isolation so the site remains online until it has been deployed successfully. There are many other benefits such as retaining previous deployments and rolling back, copying old vendors before deployments, compiling assets etc. etc.
I'm working on something like this in the git post-receive hook on the server. This isn't tested and may be buggy, but you should get the idea.
#!/bin/bash
# get the updated composer.json
git checkout master -- composer.json
# only do this stuff if composer.json is different
# you could check this manually, or with git or cmp
cp composer.json tmp/composer.json
# may take a minute, but won't take the site down
(cd tmp; composer install --prefer-dist)
# this doesn't seem to be atomic
git checkout -f
# switch vendors over
# this isn't quite an atomic operation, but is very close
# you could probably do it with symlinks and "mv -Tf" to make it atomic
mv vendor vendor.old
mv tmp/vendor vendor
rm -r tmp vendor.old
Ideally all of the deploy (i.e. in this case the git checkout and the composer install) except one single mv would happen in isolation, outside of www. This doesn't work if you have untracked files (eg CMS uploads) in your working tree and rely on PHP's __FILE__ not resolving symlinks (due to this PHP bug).
This is an old question but in case anybody is looking a solution:
I slightly modify the #dave1010 answer to use git pull instead of git checkout--force
#!/bin/bash
# get only composer files
git fetch
git checkout origin/master -- composer.json
git checkout origin/master -- composer.lock
# make sure tmp is empty
rm -rf tmp
mkdir tmp
# copy the composer files to tmp
cp -r vendor tmp/vendor
cp composer.json tmp/composer.json
cp composer.lock tmp/composer.lock
# may take a minute, but won't take the site down
(cd tmp; composer install --no-scripts --verbose; cd ..)
# switch vendors over
rm -rf vendor_old
mv vendor vendor_old
mv tmp/vendor vendor
# update your code
git pull
# run again composer install. This time will print 'Nothing to install or update'
# but will execute pre/post scripts & generate autoload files
composer install --optimize-autoloader
There is maybe a better solution using capistrano/composer. But I like mine better.
You can use something like jenkins to ftp your files over this way you can direct jenkins to run composer install on the jenkins server and then ftp the files over.
This also allows you to ignore the vendor folder.
It does require a build server to be made and you would need to be able to execute commands vs the build server
The key is your composer.lock file. The composer.lock keeps track of exactly what packages (and versions) you have installed. When you deploy, send your composer.lock file up to the production server as well, and simply do a composer update. All the exact same package versions will be installed. With deployment software like Capistrano or Flightplan you can make the composer update step part of the process so it happens automatically.
Let's say I have 3 git repositories, each with a lib and tests folder in the root. All 3 repositories are part of what I want to be a single package, however it is important to me to keep the repositories separate.
I am new to git coming from svn, so I have been reading up on submodules and how they differ from svn:externals. In SVN I could have a single
lib/vendor/package
directory, and inside package I could setup 3 externals pointing to each of my 3 repositories lib directory, renaming it appropriately like
lib/vendor/package/a -> repo1/lib
lib/vendor/package/b -> repo2/lib
lib/vendor/package/c -> repo3/lib
but from my understanding this is not possible with git. Am I missing something?
Really I'm hoping this can be solved in one of two ways.
Someone will point out how to create a 4th git repository which has the other 3 as submodules organized as I have mentioned above (where I can have an a, b, and c folder inside the root)
Someone will point out how to set this up using svn:externals in combination with githubs svn support, referencing the lib directory within each git repository (from my understanding this is impossible)
Update:
I had actually tried to follow the submodules tutorial you linked to, but I run into the following problem.
Doing things as shown above, instead of a mapping like
lib/vendor/package/a -> repo1/lib
lib/vendor/package/b -> repo2/lib
lib/vendor/package/c -> repo3/lib
I am left with
lib/vendor/package/a -> repo1
lib/vendor/package/b -> repo2
lib/vendor/package/c -> repo3
this is not ideal since now to access ClassA inside repo1's lib folder, the path is
lib/vendor/package/a/lib/ClassA
when I'm really trying to get (and this is possible with svn:externals)
lib/vendor/package/a/ClassA
since a above is actually repo1/lib, and not the root directory of repo1.
Something like this is important since, with PHP5.3 for example, using the SplClassLoader ( http://gist.github.com/221634 ), it requires a namespace-to-directory mapping like
\Package\a\ClassA -> lib/vendor/package/a/ClassA
this is where my conceptual misunderstanding is, how to setup that 4th git repository to allow my directory mappings like above.
You are right, Git submodules can not directly do exactly what you want. It works in SVN because the root of a repository, branches, and any subdirectory thereof are the same kind of object. In Git, a repository, a branch, and a directory are all distinct kinds of objects (you can not use a directory as a full repository or as a branch).
There are a couple of indirect ways to accomplish what you want though.
Using Submodules and Symlinks
The core of a Git submodule is a clone of another repository in the work tree of the “superproject”*.
Git only clones full repositories. It is not possible to clone just a single subdirectory out of an existing repository†.
* Normal submodules also require a special reference in the superproject's commits/index and (normally) an entry in the superproject's .gitmodules file.
It is possible to have non-tracked clones of other repositories in an unrelated working tree, but such usage does not create a submodule.
† Git 1.7.0 and later has a “sparse checkout” feature, but it would not help to relocate the lib directory the top level of each submodule clone.
You might, however be able to use Git's support for symbolic links to do something that is fairly close:
#
# Make the lib directory of each submodule appear in the superproject as
# lib/vendor/packages/$submod_name
#
# With this structure in each of the submodules (a, b, c):
#
# lib/
# tests/
#
# We end up with this structure in the superproject:
#
# lib/
# vendor/
# packages/
# a (a symlink to ../../../_submodules/a/lib)
# b (a symlink to ../../../_submodules/b/lib)
# c (a symlink to ../../../_submodules/c/lib)
# _submodules
# a/ (a Git submodule)
# lib/
# tests/
# b/ (a Git submodule)
# lib/
# tests/
# c/ (a Git submodule)
# lib/
# tests/
#
add_one() {
dir=lib/vendor/package
dest="$dir/$1"
# use fewer ".."s to put the _submodules closer to the symlinks
s=../../../_submodules/"$1"
git submodule add "$2" "$dir/$s"
ln -s "$s"/lib "$dest"
git add "$dest"
}
cd "$main_repo_toplevel"
mkdir -p lib/vendor/package
add_one a git#githost.example.com:user/package-a.git
add_one b git://public.example.com/work/package-b-dev.git
add_one c ssh://special.example.com/foo.git
Using git subtree
apenwarr's git subtree can split off and merge parts of repositories (i.e. individual subdirectories; it is a wrapper around “subtree merging” with other nice features). The first step would be to extract the history of lib in each of your sub-projects. Then, either directly use the extracted history as a submodule, or use git subtree to do a subtree merge into your main repository. Either way, this would introduce an extra step (re-extracting the lib history) before you could integrate changes from a sub-project into your main repository.
You can have a forth 'lib' main Git repo, with:
lib/vendor/package content
3 submodules (see "true nature of submodules")
But this reference is totally independent from any 'svn:external' property you may have setup in a mirror SVN repo.
So if you have 3 SVN repos already, you can git-svn them, publish them on GitHub, and then create a fourth repo on GitHub in which you will add those 3 Git repos as submodules, following the submodules tutorial (supposing here you have all 4 repos already on GitHub)
$ mkdir -p lib/vendor/package
$ cd lib/vendor/package
$ for package in a b c d; do
$ git submodule add git://github.com/path/to/$package.git $package
$ done
$ cd ..
$ git commit -m "lib with submodules"
$ git push