I have Jenkins set up on a remote server which hosts my PHP project. Jenkins creates a build from a git repository and I would like then to move the files to the servers document root.
However the problem is all the files have the Jenkins user as their owner. So if I moved the workspace to my document root I wouldn't have permission to access them.
I'm wondering what people do to automate this? Maybe execute shell commands after the build is completed? I'm very new to Jenkins so I may be on the completely wrong track.
Short answer would be yes, you need a post-build step that runs chown (change ownership) and/or chmod (change permissions) linux commands. This will allow you to switch owning user (if jenkins user has enough rights for it) and set read/write permissions for files up to 777 (free-for-all).
However, it's more than that. You've stumbled upon deployment phase of project, and it's much more complex than that. Usually that's handled by external tools called via bash (i'm not very familiar with those, but i certainly know capistrano is an industry standard in ruby), and those tools usually (every project may require special scenario) update project files, update dependencies, deploy migrations to database, manage permissions, clean cache, etc., etc.
I want to upgrade my website without downtime. I made researches and didn't find a way to upgrade it without downtime (few seconds are fine). I was thinking a way as follows, but I am not sure whether any good professional way is there. Please help me out on how to improve this.
Add new tables/CFs to the database (database is Cassandra,
we are not supposed to do any changes in existing tables/CF)
Deploy the project in online server in different Directory, so that
users can still use the existing site.
Point the uploaded project in different port and check whether
everything is working properly.
If everything is working change the symlink to the uploaded directory
Please let me know any other good methodology if you have.
I am using SVN in my local server
UPDATING THE SERVER ALMOST INSTANTANEOUSLY
Ok for this one of the best and the easiest method is using Git. So, what you should do is host your code in Git and then whenever you want to update the site, just SSH into the server and do a Git Pull. It's instantaneous and it would update it. Also you can use Git Hooks to further solve your problem as well.
There are a few ways you can go about it, but one of the easier methods would be this.
Let me explain it in detail on how to do it:
The source for your web site should live in a Git repository on the local workstation. I shall describe how I set things up so that I can make changes live by running just "git push online".
The local repository
It doesn't really matter how the local repository is set up, but for the sake of argument, let's suppose you're starting one from scratch.
$ mkdir somesite && cd somesite
$ git init
Initialized empty Git repository in /home/sankalpsingha/somesite/.git/
$ echo 'Test!' > index.html
$ git add index.html
$ git commit -q -m "This is the first push."
The remote repository
I assume that the web site will live on a server to which you have ssh access, and that things are set up so that you can ssh to it without having to type a password (i.e., that your public key is in ~/.ssh/authorized_keys and you are running ssh-agent locally).
On the server, we create a new repository to mirror the local one.
$ mkdir coolsite.git && cd coolsite.git
$ git init --bare
Initialized empty Git repository in /home/sankalpsingha/coolsite.git/
Now lets make and define the hoolks as a post-receive hook that checks out the latest tree into the web server's DocumentRoot (this directory must exist; Git will not create it for you):
$ mkdir /var/www/www.somesite.org
$ cat > hooks/post-receive
#!/bin/sh
GIT_WORK_TREE=/var/www/www.somesite.org git checkout -f
$ chmod +x hooks/post-receive
Back on the workstation, we define a name for the remote mirror, and then mirror to it, creating a new "master" branch there.
$ git remote add online ssh://server.somesite.org/home/sankalpsingha/coolsite.git
$ git push online +master:refs/heads/master
On the server, /var/www/www.somesite.org should now contain a copy of your files, independent of any .git metadata.
The update process
Just run :
$ git push online
This will transfer any new commits to the remote repository, where the post-receive hook will immediately update the DocumentRoot for you.
(This is more convenient than defining your workstation as a remote on the server, and running "git pull" by hand or from a cron job, and it doesn't require your workstation to be accessible by ssh.)
You can use some CI or deployment tools. I'm doing this manually. Here is the way i'm doing it.
I have a workcopy directory from git, and git hook to sync it on push.
Then i sync worckopy directory with site using rsync.
All database changes done with migrations, so 1 command yiic migrate is enough to make new tables or alter existing.
With this methodology there is no actuall downtime, some pages can be unavailable for 5-10 sec max, but all system works on this time.
Point the uploaded project in different port and check whether
everything is working properly. If everything is working change the
symlink to the uploaded directory
It's bad way, for testing you need test server, or test it on local machine with same configs and params as on server. For example Vagrant can make any environment on your local machine for testing.
If you need to pass tests (e.g. unit tests or functional), then watch on CI tools.
We are trying to automatically deploy our web application using Git, GitHub, and PHP on a Cpanel/WHM server.
I've tried, using the information in the article below, to set up a deploy script on our server that GitHub posts to when we push to the repo.
https://gist.github.com/1809044
Unfortunately, it seems that the fact that apache is running scripts as "nobody" is preventing the script from running. We created SSH keys as the account's user, and the git pull command is not running.
Is there any way to successfully pull a git repo from GitHub on a deploy hook and have it update without installing something complex like Jenkins?
Do you have control over how apache runs? I'm dealing with some e-mail/spam issues. I've also been reading that if you run suPHP, your apache will run as the user of that account. Sounds like that might be what you need.
I am currently developing a medium sized web application using PHP and need to use some kind of version control. I also need to have the master copy of the code running in the apache document root so I can test the app while it is in development. Does anyone have any sugestions?
Thanks,
RayQuang
You can't go wrong with Git; everything you need to know is here: http://progit.org/book/
Yeah you should definitely use version control (Git or Subversion).
Here a short explanation how I'm using it in my web projects (I am using SVN):
I have a SVN project which I have checkouted on my local machine and on the webserver
Always when you change something you can commit your current running version
Log into the server (Could also be multiple servers) and do a svn update, so the newest code gets automatically deployed on the machine. The only thing you have to do is restart of the webserver
Note:
Take care what you commit. You've maybe another database configuration file on your local machine than on your server. You can put this into the svn ignore file list (guess git has something similar)
It is also easy possible that multiple persons work on the same project..
Don't commit logfiles
Links:
Git: http://git-scm.com/
Subversion: http://subversion.tigris.org/
I'd recommend Mercurial for its ease of use and that it keeps the working copy uncluttered, all versioning information is kept in just one .hg folder. I'd do it like this:
Set up a Mercurial repository at the server (hg init)
Do a hg clone of that repository to where you want your working copy
Work away!
When you want to test on the server, do a hg commit and hg push to move the changed files to the server
Run hg update on the server, or add
[hooks]
changegroup = hg update >&2
to the .hg/hgrc file (create it if it doesn't exist) on the server to have it automatically update.
For more info, you can also check out: http://hginit.com/
I've heard the phrase "deploying applications" which sounds much better/easier/more reliable than uploading individual changed files to a server, but I don't know where to begin.
I have a Zend Framework application that is under version control (in a Subversion repository). How do I go about "deploying" my application? What should I do if I have an "uploads" directory that I don't want to overwrite?
I host my application through a third party, so I don't know much other than FTP. If any of this involves logging into my server, please explain the process.
Automatic deploy + run of tests to a staging server is known as continuous integration. The idea is that if you check in something that breaks the tests, you would get notified right away. For PHP, you might want to look into Xinc or phpUnderControl
You'd generally not want to automatically deploy to production though. The normal thing to do is to write some scripts that automates the task, but that you still need to manually initiate. You can use frameworks such as Phing or other build-tools for this (A popular choice is Capistrano), but you can also just whisk a few shell-scripts together. Personally I prefer the latter.
The scripts themselves could do different things, depending on your application and setup, but a typical process would be:
ssh to production server. The rest of the commands are run at the production server, through ssh.
run svn export svn://path/to/repository/tags/RELEASE_VERSION /usr/local/application/releases/TIMESTAMP
stop services (Apache, daemons)
run unlink /usr/local/application/current && ln -s /usr/local/application/releases/TIMESTAMP /usr/local/application/current
run ln -s /usr/local/application/var /usr/local/application/releases/TIMESTAMP/var
run /usr/local/application/current/scripts/migrate.php
start services
(Assuming you have your application in /usr/local/application/current)
I wouldn't recommend automatic updating. Just because your unit tests pass doesn't mean your application is 100% working. What if someone checks in a random new feature without any new unit tests, and the feature doesn't work? Your existing unit tests might pass, but the feature could be broken anyway. Your users might see something that's half-done. With automatic deployment from a check-in, you might not notice for a few hours if something made it live that shouldn't have.
Anyhow, it wouldn't be that difficult to get an automatic deployment going if you really wanted. You'd need a post-check-in hook, and really the steps would be:
1) Do an export from the latest check-in
2) Upload export to production server
3) Unpack/config the newly uploaded export
I've always performed the last steps manually. Generally it's as simple as SVN export, zip, upload, unzip, configure, and the last two steps I just alias a couple of bash commands together to perform. Then I swap out the root app directory with the new one, ensuring I keep the old one around as a backup, and it's good to go.
If you're confident in your ability to catch errors before they'd automatically go live, then you could look at automating that procedure. It gives me the jibbly-jibblies though.
At my webdev company we recently started using Webistrano, which is a Web GUI to the popular Capistrano tool.
We wanted an easy to use, fast deployment tool with a centralized interface, accountability (who deployed which version), rollback to previous versions and preferably free. Capistrano is well-known as a deployment tool for Ruby on Rails applications, but not centralized and targeted mainly to Rails apps. Webistrano enhances it with a GUI, accountability, and adds basic support for PHP deployment (use the 'pure file' project type).
Webistrano is itself a Ruby on Rails app, that you install on a development or staging server. You add a project for each of your websites. To each project you add stages, such as Prod and Dev.
Each stage can have different servers to deploy to, and different settings. Write (or modify) a 'recipe', which is a ruby script that tells capistrano what to do. In our case I just used the supplied recipe and added a command to create a symlink to a shared uploads dir, just like you mentioned.
When you click Deploy, Webistrano SSHs into your remote server(s), does an svn checkout of the code, and any other tasks that you require such as database migrations, symlinking or cleanup of previous versions. All this can be tweaked of course, after all, it's simply scripted.
We're very happy with it, but it took me a few days to learn and set up, especially since I wasn't familiar with Ruby and Rails. Still, I can highly recommend it for production use in small and medium companies, since it's proven very reliable, flexible and has saved us many times the initial investment. Not only by speeding up deployments, but also by reducing mistakes/accidents.
This sort of thing is what you would call "Continous Integration". Atlassian Bamboo (cost), Sun Hudson (free) and Cruise Control (free) are all popular options (in order of my preference) and have support to handle PHPUnit output (because PHPUnit support JUnit output).
The deployment stuff can be done with a post build trigger. Like some other people on this thread, I would exercise great caution before doing automated deployments on checkin (and test passing).
check fredistrano, it's a capistrano clone
works great (litle bit confusing installing but after all runs great)
http://code.google.com/p/fredistrano/
To handle uploads, the classic solution is to move the actual directory out of the main webspace, leaving it only for a fresh version to be checked out (as I do in the script below) and then using Apache to 'Alias' it back into place as part of the website.
Alias /uploads /home/user/uploads/
There are less choices to you if you don't have as much control of the server however.
I've got a script I use to deploy a given script to the dev/live sites (they both run on the same server).
#!/bin/sh
REV=2410
REVDIR=$REV.20090602-1027
REPOSITORY=svn+ssh://topbit#svn.example.com/var/svn/website.com/trunk
IMAGES=$REVDIR/php/i
STATIC1=$REVDIR/anothersite.co.uk
svn export --revision $REV $REPOSITORY $REVDIR
mkdir -p $REVDIR/tmp/templates_c
chown -R username: $REVDIR
chmod -R 777 $REVDIR/tmp $REVDIR/php/cache/
chown -R nobody: $REVDIR/tmp $REVDIR/php/cache/ $IMAGES
dos2unix $REVDIR/bin/*sh $REVDIR/bin/*php
chmod 755 $REVDIR/bin/*sh $REVDIR/bin/*php
# chmod -x all the non-directories in images
find $IMAGES -type f -perm -a+x | xargs -r chmod --quiet -x
find $STATIC1 -type f -perm -a+x | xargs -r chmod --quiet -x
ls -l $IMAGES/* | grep -- "-x"
rm dev && ln -s $REVDIR dev
I put the revison number, and date/time which is used for the checked-out directory name. The chmod's in the middle also make sre the permissions on the images are OK as they are also symlinked to our dedicated image server.
The last thing that happens is an old symlink .../website/dev/ is relinked to the newly checked out directory. The Apache config then has a doc-root of .../website/dev/htdocs/
There's also a matching .../website/live/htdocs/ docroot, and again, 'live' is another symlink. This is my other script that will remove the live symlink, and replace it with whatever dev points to.
#!/bin/sh
# remove live, and copy the dir pointed to by dev, to be the live symlink
rm live && cp -d dev live
I'm only pushing a new version of the site every few dats, so you might not want to be using this several times a day (my APC cache wouldn't like more than a few versions of the site around), but for me, I find this to be very much problem-free for my own deployment.
After 3 years, I've learned a bit about deployment best practices. I currently use a tool called Capistrano because it's easy to set up and use, and it nicely handles a lot of defaults.
The basics of an automated deployment process goes like this:
Your code is ready for production, so it is tagged with the version of the release: v1.0.0
Assuming you've already configured your deployment script, you run your script, specifying the tag you just created.
The script SSH's over to your production server which has the following directory structure:
/your-application
/shared/
/logs
/uploads
/releases/
/20120917120000
/20120918120000 <-- latest release of your app
/app
/config
/public
...etc
/current --> symlink to latest release
Your Apache document root should be set to /your-application/current/public
The script creates a new directory in the releases directory with the current datetime. Inside that directory, your code is updated to the tag you specified.
Then the original symlink is removed and a new symlink is created, pointing to the latest release.
Things that need to be kept between releases goes in the shared directory, and symlinks are created to those shared directories.
It depends on your application and how solid the tests are.
Where I work everything gets checked into the repository for review and is then released.
Auto updating out of a repository wouldn't be smart for us, as sometimes we just check in so that other developers can pull a later version and merge there changes in.
To do what you are talking about would need some sort of secondary check in and out to allow for collaboration between developers in the primary check in area. Although I don't know anything about that or if its even possible.
There is also issues with branching and other like features that would need to be handled.