I have to deploy my php/html/css/etc code to multiple servers and i am looking at my options for software that allows easy and secure deployment to multiple servers.
Also helps if it could be tied into my SVN.
Any suggestions?
Capistrano is pretty handy for that. There's a few people using it (1, 2, 3) for deploying PHP code as evidenced by doing a quick search.
Setting up password-less publickey authentication with ssh would allow you to scp your files to any of your servers very quickly (or be automated by a shell script).
Here's a simple tutorial: http://rcsg-gsir.imsb-dsgi.nrc-cnrc.gc.ca/documents/internet/node31.html
If you're running on Redhat or Debian, consider packaging up your code into RPM's or Debs. Then build a yum or dpkg repository and put your packages there. You can then use your system's package management to do upgrades/rollbacks, etc. You can even use puppet to automate the process.
If you want to tie it into subversion, you can create a branch for each new version. Use the commit scripts to build the RPM's when a new branch shows up in a directory.
I'll second Capistrano. It's incredibly powerful and flexible. Our current project uses Capistrano for deploying to different servers as well as multiple servers. We pass two arguments to the cap command:
1) the name of the set of machine specific config options to run and
2) the name of the action to run
ends up looking like this:
cap -f deploy.rb live deploy
or
cap -f deploy.rb dev deploy
Of course the default use case - deploy to lots of machines at once - is a doddle with Capistrano AND you don't need to have Capistrano on the machines you are deploying to. All in all, tasty technology.
I've used Automated Build Studio before for a similar task. It gives you a lot of flexibility in what you can do.
I concur -- set your svn tree up, and use rsync over ssh to copy the tree out to the remote locations. rsync will make it fast and efficient, only copying changes rather than full files.
You want to export your svn tree to some directory, then rsync from there to the remote host's directory tree.
I also forgot to mention that if you use rsync, you can set up rsync to use ssh, so you will only transfer the files that have changed, saving on time and bandwidth.
You can also use kwateeSDCM which is free and allows remote installation via ssh. It also enables you to manage server-specific configuration from a central location and make upgrades seemless.
I had marked a post on how to deploy your websites using Subversion : http://blog.lavablast.com/post/2008/02/I2c-for-one2c-welcome-our-new-revision-control-overlords!.aspx
I found capistrano to be very easy to use once it's setup. The configuration file can be a bit confusing at first for more complicated environments but it soon becomes worthwhile. I deploy to 14 servers on production. I also use multiple environments for deployment to a staging server. One quirk, there's a bug in Ruby that breaks parallel deployment but serially isn't too bad with svn exports.
Capistrano setup is just too complicated. We found that KwateeSDCM was very straightforward to use with a simple web interface and no scripting. We've got our deployment configuration done in no time for Dev and QA configuration on windows and linux servers.
Related
I'm developing web app using CodeIgniter PHP framework. The server I'm working with does not support any type of source control (i.e. Subversion) unless you go to a higher price tier.
I would still like to put the code under some sort of source control. Does it make sense to do the following:
Install git or SVN on my local machine and develop there
Copy changes from my local machine to development directory on the server (using FileZilla, WinSCP, etc.) and test
Copy changes from development directory to production directory on the server
Does that sound reasonable? Are there better alternatives? Thanks!
If you are using svn locally it's a bit dangerous because then you will need to protect also your computer - I think the best way is to work with the commercial sites offer fully supported svn/git - like http://www.beanstalkapp.com/ or http://www.github.com
You could use source control on your local machine (SVN, Git, etc.) and use an open source tool like Capistrano to deploy the code from your local source control repo to your server via SSH. Or if you're limited to FTP, this blog post has a potential solution.
An advantage of using a tool like Capistrano instead of directly mirroring the files on your local machine to the server via FileZilla or WinSCP is that Capistrano will version your deployed files so that, if you end up breaking something and need to roll back quickly to the previously-deployed version, it can be as easy as changing a symlink to the previous deployment directory.
Does that sound reasonable?
Partially, in p.1. But even in this case I'll suggest to have your repository also at some Repository-Hosting (BitBucket, GitHub, Assembla)
For pp. 2-3: your deploys must to be automatic and non-interactive, thus - you'll have to select another tools (for using in post-commit hook of SCM-of-choice)
Somehow better alternative to 2-3 may be:
Use 2 different branches (DEVEL and PROD) as sources of DEVEL and PROD dir
Post-commit hook, which upload only changed in committed revision files to corresponding dir (NCFTP for FTP, SCP with scenario for ssh)
Main development happens in DEVEL branch
PROD have only mergesets from DEVEL
Workflow is SCM-agnostic and scalable to any reasonable amount of branches and developers
I'm just trying to find an easier way to deploy a site I'm working on. I'm working alone with a test a production server and right now deployment means copying a subset of the files and database data onto my computer and uploading it to the prod site. I'm sure there's a simple synchronization tool out there but so far I've had no luck in finding anything.
What I'd really like is an application I can run locally (on windows) or something I could install on my server for let me have a one-click deployment. Any suggestions?
Thanks!
godwin
Edit
I have decided for now to go with GoodSync and Toad. Thanks for the suggestions.
man scp
SCP(1) BSD General Commands Manual SCP(1)
NAME
scp - secure copy (remote file copy program)
SYNOPSIS
scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-S program] [[user#]host1:]file1
[...] [[user#]host2:]file2
DESCRIPTION
scp copies files between hosts on a network. It uses ssh(1) for data transfer, and uses the same authentication and provides the same
security as ssh(1). Unlike rcp(1), scp will ask for passwords or passphrases if they are needed for authentication.
Any file name may contain a host and user specification to indicate that the file is to be copied to/from that host. Copies between two
remote hosts are permitted.
When copying a source file to a target file which already exists, scp will replace the contents of the target file (keeping the inode).
If the target file does not yet exist, an empty file with the target file name is created, then filled with the source file contents. No
attempt is made at "near-atomic" transfer using temporary files.
The options are as follows:
-1 Forces scp to use protocol 1.
-2 Forces scp to use protocol 2.
...
I use GoodSync http://www.goodsync.com/ for this sort of thing. It's really good. Runs on windows, can sync between any combination of local files (S)FTP, windows, linux network shares etc.
Then create a scheduled task/cronjob to run an export of the database into the syncronised folder and have one do an import at the other end. Obviously this process is one way.
http://www.phing.info/docs/guide/stable/
PHing is an automated build system made for PHP. Works with GIT, SVN, PHPUnit, etc...
You basically set up XML files that give PHing instructions on what to do. Allows you to run test suites along with build creation, build multiple varied versions at a time, copy files as well as db, and a bunch of other cool features.
Also, it's open source and platform independent.
What are you using for source control? Some tools like Git and SVN have ready-made methods for this sort of thing. See here for a quick Git solution.
I would second the advice about Git/SVN, but would put in a strong plug for Git via GitHub. Use GitHub as your "central" Git repository. Your local Git repository will push to GitHub, and your production server will pull from GitHub.
There is some overhead to learning Git/GitHub, but really, in the situation you've described (a single engineer and two servers), Git isn't any more complicated then SVN (or CVS or anything else).
We use an FTP Synchronizer, which seems to work pretty well. I don't know offhand of any good free ones.
Example: http://www.ftpsynchronizer.com/
Depends on what type of server you are running, but you could run SVN (Subversion). There is a plugin for Eclipse, Aptana, and Zend Studio if you use that to develop.
Essentially you could have a development repository that sits on the server. You would pull your code down to your local environment and commit it back after changes. Then you can setup another repository that is your live data or production thats linked back to your Development repository.
When you want to update the live data, you just update it so if any trouble happens you can roll back that code without having to roll back your development code. Once you get good at all that you can start branching and tagging your projects.
I personally use both SVN and Git, but I prefer Git because it works so much better. Though if you are using Windows, the command line tools just aren't the same as linux.
I have a question related to phing. Do I use it on my local development machine to deploy an application or do I call it via ssh from production machine? I'm not sure where to do the steps from Eran Galperin (What is your preferred PHP deployment strategy). I'm a beginner with deployment scripts. So be forgiving :)
Marco
You'll usually have two deploy scripts - One that you run locally, which ssh's in to the remote server and then kicks off the remote script.
Your local script might do an svn export, zip the contents and push them (scp) to the remote server first. The remote script would do stuff like migrating, restarting services etc.
The basic strategy is that you create a build (well, basically a copy of your web application that you can put online) on your local machine, and then deploy it on the server.
You run it on your local machine and the PHPing will connect to production server and deploy the website. :)
Phing can be applied in many ways.
If you don't have directives on how to deploy, the best way is usually first to generate a "test build" from your local copy, check out if it is working as intended and then generate the productive build.
In some cases where you have to follow certain rules in your development enviroment you can easily adapt Phing to meet those demands.
I developed a phing deploy script and I've used it in both of the ways you've described. As long as the phing library and dependencies are installed locally or on the remote machine you can do it in both ways, and it's actually a benefit to be able to run it from any of your environments.
Let me elaborate on my experience, but keep in mind that every person lives within the microcosm of the needs and requirements of their own projects/studios so it's possible that what works for me, may not work for other people.
I think deployment is usually so important for a business that it needs to happen in the exact same way every time. When there's a multitude of people doing the deployments, then it's easy for bad habits or procedures to diverge and that's when mistakes can happen. Automated deployments with phng are handy because if everyone is using the same procedure, then there's much lower potential for mistakes. Phing makes it easy to follow strict procedures because all the developers have to do is to run a simple command like "phing deploy" or "phing build" or what have you.
Now going back to your original question, it's also helpful for these procedures to be adhered to on local development machines as well as on staging, testing, production servers because just like people, machines can have small ideosyncracies that differ from machine to machine. With phing, you can develop a script that will have the same results on any of your enterprise machines (depending on the strength and intent of your code but phing lives in a format that makes it easy to meet that ideal). So if you run your phing build from your development box, ideally it has the same effect if you run it from your testing box or your production server. If you can run the deployment the same way on any machine, you can feel confident that any problems that arise after deployment were not a result of your deploy procedures (as they will not differ from when you deployed to your testing or local dev box).
It's really nice to be able to deploy my code to my local machine with the exact same procedure that I use to deploy it to production.
As far as what is the best to do, I say, why not all of the above? If you can run it locally, then you can run it remotely. If the deploy procedures are strong, then the result will be the same.
You can do it both ways. You can run the tool locally as you develop to run tests and just generally ensure that your code is in good condition. The server automation (if you choose to do that) can also run tests, generate documentation, and deploy changes to test or production setups.
Doing it locally reduces the chance that the process will get stalled on the server setup, while the CI works on a regular basis to ensure nothing has been broken via mistake.
I have worked within a web development company where we had our local machines, a staging server and a a number of production servers. We worked on macs in perl and used svn to commit to stage, and perl scripts to load to production servers. Now I am working on my own project and would like to find good practices for web development when using shared web hosting and not working from a unix based environment (with all the magic I could do with perl / bash scripting / cron jobs etc)
So my question is given my conditions, which are:
I am using a single standard shared web hosting from an external provider (with ssh access)
I am working with at least one other person and intended to use SVN for source control
I am developing php/mysql under Windows (but using linux is a possibility)
What setup do you suggest for testing, deployment, migration of code/data? I have a xampp server installed on my local machine, but was unsure which methods use to migrate data etc under windows.
I have some PHP personnal-projects on shared-hosting ; here are a couple of thoughts, from what I'm doing on one of those (the one that is the most active, and needs some at least semi-automated synchronization way) :
A few words about my setup :
Some time ago, I had everything on SVN ; now, I'm using bazaar ; but the idea is exactly the same (except, with bazaar, I have local history and all that)
I have an ssh access to the production server, like you do
I work on Linux exclusivly (so, what I do might not be as easy with windows)
Now, How I work :
Everything that has te be on the production server (source-code, images, ...) is commited to SVN/bazarr/whatever
I work locally, with Apache/PHP/MySQL (I use a dump of the production DB that I import locally once in a while)
I am the only one working on that project ; it would probably be OK for a small team of 2/3 developpers, but not more.
What I did before :
I had some PHP script that checked the SVN server for modification between "last revision pushed to production" and HEAD
I'm guessing this homemade PHP script looks like the Perl script you are currently usng ^^
That script built a list of directories/files to upload to production
And uploaded those via FTP access
This was not very satisfying (there were bugs in my script, I suppose ; I never took time to correct those) ; and forced me to remember the revision number of the time I last pushed to production (well, it was automatically stored in a file by the script, so not that hard ^^ )
What I do now :
When switching to bazaar, I didn't want to rewrite that script, which didn't work very well anyway
I have dropped the script totally
As I have ssh access to the production server, I use rsync to synchronise from my development machine to the production server, when what I have locally is considered stable/production-ready.
A couple of notes about that way of doing things :
I don't have a staging server : my local setup is close enough to the production's one
Not having a staging server is OK for a simple project with one or two developpers
If I had a staging server, I'd probably go with :
do an "svn update" on it when you want to stage
when it is OK, launch the rsync command from the staging server (which will ba at the latest "stable" revision, so OK to be pushed to production)
With a bigger project, with more developpers, I would probably not go with that kind of setup ; but I find it quite OK for a (not too big) personnal project.
The only thing "special" here, which might be "linux-oriented" is using rsync ; a quick search seems to indicate there is a rsync executable that can be installed on windows : http://www.itefix.no/i2/node/10650
I've never tried it, though.
As a sidenote, here's what my rsync command looks like :
rsync --checksum \
--ignore-times \
--human-readable \
--progress \
--itemize-changes \
--archive \
--recursive \
--update \
--verbose \
--executability \
--delay-updates \
--compress --skip-compress=gz/zip/z/rpm/deb/iso/bz2/t[gb]z/7z/mp[34]/mov/avi/ogg/jpg/jpeg/png/gif \
--exclude-from=/SOME_LOCAL_PATH/ignore-rsync.txt \
/LOCAL_PATH/ \
USER#HOST:/REMOTE_PATH/
I'm using private/public keys mecanism, so rsync doesn't ask for a password, btw.
And, of course, I generally use the same command in "dry-run" mode first, to see what is going to be synchorised, with the option "--dry-run"
And the ignore-rsync.txt contains a list of files that I don't want to be pushed to production :
.svn
cache/cbfeed/*
cache/cbtpl/*
cache/dcstaticcache/*
cache/delicious.cache.html
cache/versions/*
Here, I just prevent cache directories to be pushed to production -- seems logical to not send those, as production data is not the same as development data.
(I'm just noticing there's still the ".svn" in this file... I could remove it, as I don't use SVN anymore for that project ^^ )
Hope this helps a bit...
Regarding SVN, I would suggest you go with a dedicated SVN host like beanstalk or use the same server machine to run an SVN server so both developers can work off it.
In the latter case, your deployment script would simply move the bits to a staging web folder (accessible via beta.mysite.com) and then another deployment script could move that to the live web directory. Deploying directly to the live site is obviously not a good idea.
If you decide to go with a dedicated host or want to deploy from your machine to the server, use rsync. This is also my current setup. RSync does differential syncs (over SSH) so it's fast and it was built for just this sort of stuff.
As you grow you can start using build tools with unit tests and whatnot. This leaves only the data sync issue.
I only sync data from remote -> local and use a DOS batch file that does this over SSH using mysqldump. Cygwin is useful for Windows machines but you can skip it. The SQL import script also runs a one line query to update some cells such as hostname and web root for local deployment.
Once you have this setup, you can focus on just writing code and remote deployment or local sync and deployement becomes a one click process.
One option is to use a dedicated framework for the task. Capistrano fits very well with scripting languages such as php. It's based on Ruby, but if you do a search, you should be able to find instructions on how to use it for deploying php applications.
There's another post on SO relating to .NET -- not us. Pure PHP. Trying to find the best way/process to deploy stable version of our PHP app. I've seen an article on Capistrano, but am curious what else is out there. Aside from the obvious reasons, I'm also looking to add some scripting so that the SVN rev number gets added in there as well.
Much thanks.
I've used a home-grown script for quite some time. It will (based on an application configuration file):
Run svn export on the repository based on a tag.
Package the export into a tar or zip file, which includes the tag in the name.
Use scp to copy the package to the appropriate server (QA or release).
Connect to the server with ssh to install the package and run post-install scripts.
The application configuration file is part of the project. It can tell the script (at step 2) to strip paths and otherwise process specified files. It also specifies server names and how to handle externals.
I've recently migrated the script to support Git as well as Subversion. I'm also probably going to migrate it to PHP since we're now running in a mixed (Linux and Windows) set up, with Linux now in the minority.
I have plans to automatically call the script with post-commit hooks, but haven't had the need to implement that just yet.
Coincidentally, I was just reading about an Apache Ant/gnu make like build tool called Phing. What I like about it is the ability to write custom extensions in PHP!
I don't know if it works for deploying an app live, but phpUnderControl is a continuous integration suite (which I'm just now starting to look into). If it doesn't support doing deployments natively, it can probably be extended to do them.
Chris Hartjes has a nice view on this: Deployment is not a 4 letter word
We're using Webistrano, which is a web frontend for Capistrano, to deploy a few dozen projects. It's built as a Ruby on Rails app, and provides a nice, centralized and consistent user interface for Capistrano deployments.
Instead of having cap recipes in every project, and running command-line tools, Webistrano stores the recipes in its database, and allows you to attach the recipes to multiple projects and stages. This reduces duplication of scripts.
Also nice is that all deployment logs are stored so there's an auditing trail. Who deployed which revision to the live server, that sort of thing.
As you requested, the Revision number is stored in the deployed project as well.
All in all, we're very pleased with it.