I have seen Jenkins being used as CI for Docker containers. Is Dokku also a CI platform like Jenkins?
If, what is the difference when I want to do CI with Docker containers for a PHP application?
Are you maybe confusing drone with Dokku? Dokku is a platform for execution of heroku apps drone is a docker based CI. I don't know much about drone but since docker can't be run inside a docker container without some hacking you are better off sticking to a traditional CI like jenkins, bamboo, team city or such.
Continuing from Usman Ismail's answer...
If you look at dokku-alt, the distinction is less clear. In particular dokku-alt allows you to use a Dockerfile for the build rather than buildstep, so it's not specific to Heroku like apps.
Dokku-alt is not in itself a CI system, but out of the box it does verify that the build completes without error before it's deployed, and using git hooks you could connect in your test-suite to run on every git push and block deployment when it fails.
CI typically is a bit more than this. You'd normally have multiple deployments for test, staging and live, and to some extent it also encompasses a set of practices. Dokku-alt gives you some very useful parts of CI, and a fairly clear path to building more of it fairly easily, but it's not a complete CI system in itself.
You might well prefer to keep your main git repository elsewhere, and keep jenkins in the picture for automating transfer to dokku-alt.
Related
It seems the standard way to deploy with Rocketeer is to do a pull deploy, that is, it will do a git clone from the server you are deploying to. What I want to do is push a set of files after having done a build on a CI server to the server being deployed to.
The reason I want to do this is that usually my projects have lots of extra stuff not required for production. I usually like to construct a build folder and run a build script that packages a final product. I want to use Rocketeer to push the results to staging/production servers. It was suggested in this article it can be done: Deploying PHP Applications with Rocketeer and Docker
However, after reading the rocketeer documentation there is nothing that speaks to that strategy and its seems a bit against the grain to try. I'm open to ideas given my problem.
As the author of the article, I owe you a clarification. I mentioned those two types of deployment paradigms in a general sense just to introduce the different concepts. As I am aware of, Rocketeer supports only "pull" deployments. Sorry for the confusion!
For deploying the generated files to your server from a CI, I think the most straightforward way is to usea tool like scp, rsync or just download it from S3 if you're storing your built package in a bucket.
I'm trying to figure out a way to deploy my company intranet PHP apps automatically, using GitLab, but so far, I'm not sure if the options that I've found on the internet will do the job.
Here's the scenario:
VM1: Remote Git server, that uses GitLab to administrate projects repos.
VM2: Development server. Each project has it's own development server.
VM3: Production server. Each project has it's own, as well.
Developers: Each developer uses a vagrant box, based on the project's development server.
What I want to do:
Whenever a developer push it's commits to the development branch, a hook in the remote server must update the development server tree with the last commit on that branch.
The same must occur when the master branch is updated, but it must update the production server tree.
However, since we use Laravel, we must run some extra console commands using Artisan, depending on each case.
We are following Vincent Driessen's Git Branching Model.
What I know so far:
I know that GitLab uses Web Hooks, but I'm not sure if that's going to work in this case, since I need to access a custom URL, doesn't sound very safe, but if it's the only solution, I can write a script to handle just that.
The other possible solution is to use Jenkins but I'm not an expert and I don't know yet if Jenkins is too much for my case, since I'm not using unit tests yet.
Do you guys have implemented a solution that could help in my case? Can anyone suggest an easy and elegant way to do that?
Thanks guys! Have a nice one
We do it the following way:
Developers checkout whatever Git branches, and as many branches as they want/need locally (Debian in VM Ware on Mac)
All branches get pulled to dev server whenever a change is pushed to Git. They are available in feature-x.dev.domain.com, feature-y.dev.domain.com etc., running against dev database.
Release branches are manually checked out on live system for testing, made available on release-x.test.domain.com etc. against the live database (if possible, depending on migrations).
We've semi-automated this using own scripts.
Database changes are made manually, due the sensitivity of their nature. However, we don't fint that a hassle, after getting used to migrations - and just remembering to note the alterations. We find good support by cloning databases locally for each branch that needs changes. An automated schema comparison quickly helps then, if changes to a migration file have been forgotten.
(The second point is the most productive one, making instant test on the dev platform available to everyone as soon as the first commit of a new branch is pushed)
I would suggest to keep things simple and work with git, hooks and remote repositores.
Pulling out heavy guns, like Jenkins or Gitlab for this task could be a bit too much.
I see your request as the following: "git after push and/or git after merge: push to remote repo".
You could setup "bare" remote repositories - one for "dev-stage", one for "production-stage".
Their only purpose is to receive pushes.
Each developer works on his feature-branch, based on the development branch.
When the feature-branch is ready, it is merge back to the main development branch.
Both trigger a "post merge" or "post-receive" hook, which execute a script.
The executed script can do whatever you want.
(Same approach for production: When the dev-branch has enough new features, it is merged to prod branch - triggers merge event - scripts...)
Here you want two things:
You want to push a specific branch to a specific remote repo.
In order to do this, you have to find out the specific branch in your hook script.
That's tricky, but solveable, see: https://stackoverflow.com/a/13057643/1163786 (writing a "git post-receive hook" to deal with a specific branch)
You want to execute additional steps for configuration/setup, like artisan, etc.
You might add these steps directly or as triggers to the hook script.
I think this request is related to internal and external deployment via git.
You might also search for tutorials, like "deployment with git", which might be helpful.
For example: http://ryanflorence.com/deploying-websites-with-a-tiny-git-hook/
http://git-scm.com/book/en/Git-Basics-Working-with-Remotes
http://githooks.com/ & https://www.kernel.org/pub/software/scm/git/docs/githooks.html
If you prefer to keep things straightforward and don't mind using paid third-party options, check out one of these:
http://deploybot.com/
https://www.deployhq.com/
https://envoyer.io/
Alternatively, if you fancy shifting to an integrated solution, I've not used better than Beanstalk.
So - let's say I develop a PHP app which I develop in a vagrant box identical to production envrionment. So - as an end result I would have a *.tar.zip file with a code...
How would one organize a deployment into production environment where there are a lot of application servers? I mean - I'm confused how to push code into production synchronously all at once?
More information:
on server code is stored like this:
project
+current_revision ->link to revisions/v[n]
+revisions
+v1
+v2
+v3
...
+data
So when I have to deploy changes I usually run a deploy script that uploads updated tar onto server with ssh, untars into specific dir under revisions, symlinks it into current_revision and restart php-fpm.... This way I can rollback anytime just by symlinking to an older revision.
with multipe servers what bothers me is that not all boxes will be updated at once, ie. technically some glitches might be possible.
If you're looking for a "ready-to-go" answer, you'll need to provide some more info about your setup. For example, if you plan to use git for VCS, you could write a simple shell script that pulls the latest commit and rsyncs with the server(s). Or if you're building on top of Symfony, capifony is a great tool. If your using AWS, there's a provider plugin written by the author of Vagrant that's super easy to use, and you can specify a regex for which machines to bring up or provision.
If instead you're looking for more of a "roadmap", then the considerations that you'll want to take are:
Make building of identical boxes in the remote and local environments as easy as possible, and try to make sure that your provisioning emphasizes idempotence.
Consider your versioning/release structure; what resources will rarely or never change? Include those in a setup function instead of a deploy function, and don't include them in your sync run.
Separate your development and system administration concerns; i.e. do not just package a vagrant box with a *.tar.gz and tie it through config.vm.box_url. The reason for this is that you'd have to repackage every production server with a new box every time you deploy, instead of just changing files on the server, or adding/removing some packages from the server.
Check out some config management tools like Chef and Puppet; even if you don't end up using them, they'll give you an idea of how sysadmin professionals approach this problem.
Lots of ways. If starting from barebones (no cloud infrastructure), I'm a fan of the SVN branch hook. Have a SVN repo for your code. Set up a post-commit hook on it, which checks if anything in /branch/production/ has been changed.
If it has, let the post-commit hook fire all your automated roll-out procedure - and in this case, an easy way to do so is to let all your servers known* to svn export the branch. As simple as that!
(* that's the hard step)
We're considering using a CI server soon.
From my reading, I've found that Sismo and Hudson were available for PHP project.
Considering that we're actually using GIT and PHPUnit, what are the big difference between Hudson and Sismo that we should know in order to make the best choice for our situation ?
Thanks
The language match is not key in your hunt for the best CI server; it's all the features around:
source control
concurrent build
trigger build
notification
Even for simple project, Jenkins (the new name for Hudson) is easy to use and quick to install. Then it is really easy to scale Jenkins up by adding more nodes (satellite machine that can execute build) when you need to. Also Jenkins has hundreds of plugin for numerous task.
Have a look at Bamboo, Jenkins, TeamCity, and CruiseControl Features to compare some of the features of the big names (you might actually want to consider Bamboo, TeamCity or Cruise Control over Jenkins)
I would lean towards Sismo since it matches the language of the project you are developing (PHP) and can be ran from just a single PHP and config file. Then you don't have to deal with having a java environment just for Hudson.
There is a really good php-integration for Jenkins by the phpunit inventor Sebastian Bergmann. You should really have a look at it.
As far as I see the biggest downside of Sismo is, that is not a "real" CI server, but more a build-and-report-environment, because you need to trigger the builds yourself (or let something trigger it).
I'll preface this by saying that I haven't used sismo.
We use Hudson with applications being built & tested in both Java and PHP. It has a nice plugin system, and getting it up and running on a centOS box took about 15 minutes yesterday. (We had to move it from one box to another).
For PHP Hudson integrates with both PHPUnit and Selenium so we run both unit tests and functional tests against the same codebase. Hudson has a great 'one-click' plugin system that really lets you customize your installation.
One thing we had to get a plugin for was sending an email on every build whether successful or not. Hudson by default will only email when your build goes from good (tests pass) to bad, from bad to good, or repeatedly bad. This means it will not send an email for every build if 2 builds in a row were successful. The email plugin solves this but it was confusing to uncover that.
I am wondering what is your procedure method of a web development using Git?
When you finish coding, do you just overwrite the files on the FTP to the live server?
How does git handle number of version of same project? like v1, v1.5, etc
Let say 2 people working on the project locally at work (same office), how do you work together? Do I have to keep asking them to give me a source ready (save on USB?) for merge?
Can two people work on the same project on the same server? Wouldn't this be easier than question 3?
The idea behind git is that it actually takes care of all that for you.
When you write code you commit your code and you can push it out to the server. Git tracks the changes so its easy to rollback to a previous version.
It tracks the versions of files as they change so you can easily undo any changes that was made in the past, see tags for more details.
NO. You can push your changes to the server and the other person can pull these changes. Some merging will have to occur but its quite easy with git. No need to transfer files from one dev to another. Branching and merging is discussed here.
Yes. Thats the idea.
To better understand the concepts behind a distributed version control system you can read this tutorial by Joel Spolsky. It is about Mercurial, but you will find the concepts very similar and this is probably the best tutorial written about this subject on the web.
This is how I would do it.
Each developer has his own git repository to develop his code. You as merger hold a third repository, and this repository has separate branches for each developer, for your test system and your production site.
Your developers can push their changes to you, or you can pull their changes from them into branches specifically for them. You hold a branch that you control which contains the merged code in a state for testing. You either use git-cherry-pick, or maybe just git-merge to pull their changes into your testing branch were you test things (and possibly make your own changes - or fire bug reports of to the develops and you re-incorporate their changes). When you are happy you will merge off to a "production" branch. This is normally initially derived from the test branch, but with changes necessary for the live system (I always find there is something, even if its just the database name and password).
I normally use a git hook with some code which checks which branch I am on and then uses rsync over ssh to push the code to my production site.
#!/bin/bash
branch=$(git branch | sed -n s/^\*\ //p)
version=$(git describe --tags)
cd "$git rev-parse --show cdup)"
if [ "$branch" == "production" ]; then
echo "?php echo '$version';?>" > web/version.inc
rsync -axq --delete web/ site:public_html/
fi
google "git flow", it shows you a way of managing work and releasing when you want.
For deploying via a branch, see:
Deploy a project using Git push