I want to upgrade my website without downtime. I made researches and didn't find a way to upgrade it without downtime (few seconds are fine). I was thinking a way as follows, but I am not sure whether any good professional way is there. Please help me out on how to improve this.
Add new tables/CFs to the database (database is Cassandra,
we are not supposed to do any changes in existing tables/CF)
Deploy the project in online server in different Directory, so that
users can still use the existing site.
Point the uploaded project in different port and check whether
everything is working properly.
If everything is working change the symlink to the uploaded directory
Please let me know any other good methodology if you have.
I am using SVN in my local server
UPDATING THE SERVER ALMOST INSTANTANEOUSLY
Ok for this one of the best and the easiest method is using Git. So, what you should do is host your code in Git and then whenever you want to update the site, just SSH into the server and do a Git Pull. It's instantaneous and it would update it. Also you can use Git Hooks to further solve your problem as well.
There are a few ways you can go about it, but one of the easier methods would be this.
Let me explain it in detail on how to do it:
The source for your web site should live in a Git repository on the local workstation. I shall describe how I set things up so that I can make changes live by running just "git push online".
The local repository
It doesn't really matter how the local repository is set up, but for the sake of argument, let's suppose you're starting one from scratch.
$ mkdir somesite && cd somesite
$ git init
Initialized empty Git repository in /home/sankalpsingha/somesite/.git/
$ echo 'Test!' > index.html
$ git add index.html
$ git commit -q -m "This is the first push."
The remote repository
I assume that the web site will live on a server to which you have ssh access, and that things are set up so that you can ssh to it without having to type a password (i.e., that your public key is in ~/.ssh/authorized_keys and you are running ssh-agent locally).
On the server, we create a new repository to mirror the local one.
$ mkdir coolsite.git && cd coolsite.git
$ git init --bare
Initialized empty Git repository in /home/sankalpsingha/coolsite.git/
Now lets make and define the hoolks as a post-receive hook that checks out the latest tree into the web server's DocumentRoot (this directory must exist; Git will not create it for you):
$ mkdir /var/www/www.somesite.org
$ cat > hooks/post-receive
#!/bin/sh
GIT_WORK_TREE=/var/www/www.somesite.org git checkout -f
$ chmod +x hooks/post-receive
Back on the workstation, we define a name for the remote mirror, and then mirror to it, creating a new "master" branch there.
$ git remote add online ssh://server.somesite.org/home/sankalpsingha/coolsite.git
$ git push online +master:refs/heads/master
On the server, /var/www/www.somesite.org should now contain a copy of your files, independent of any .git metadata.
The update process
Just run :
$ git push online
This will transfer any new commits to the remote repository, where the post-receive hook will immediately update the DocumentRoot for you.
(This is more convenient than defining your workstation as a remote on the server, and running "git pull" by hand or from a cron job, and it doesn't require your workstation to be accessible by ssh.)
You can use some CI or deployment tools. I'm doing this manually. Here is the way i'm doing it.
I have a workcopy directory from git, and git hook to sync it on push.
Then i sync worckopy directory with site using rsync.
All database changes done with migrations, so 1 command yiic migrate is enough to make new tables or alter existing.
With this methodology there is no actuall downtime, some pages can be unavailable for 5-10 sec max, but all system works on this time.
Point the uploaded project in different port and check whether
everything is working properly. If everything is working change the
symlink to the uploaded directory
It's bad way, for testing you need test server, or test it on local machine with same configs and params as on server. For example Vagrant can make any environment on your local machine for testing.
If you need to pass tests (e.g. unit tests or functional), then watch on CI tools.
Related
Env: Linux
PHP apps runs as "www-data"
PHP files in /var/www/html/app owned by "ubuntu". Source files are pulled from git repository. /var/www/html/app is the local git repository (origin: bitbucket)
Issue: Our Developers and Devops would like to pull the latest sources (frequently), and would like to initiate this over the web (rather than putty -> and running the git pull command).
However, since the PHP files run as "www-data" it cannot run a git pull (as the files are owned by "ubuntu").
I am not comfortable with both alternatives:
Running Apache server as "ubuntu", due to obvious security issue.
The git repository files to be "www-data", as it makes it very inconvenient for developers logging into the server and editing the files directly.
What is the best practice for handling this situation? I am sure this must be a common issue for many setups.
Right now, we have a mechanism where the Devops triggers the git pull request from the web (where a PHP job - running as "www-data" creates a temp file). And a Cron job, running as "ubuntu", reads the temp file trigger and then issues the "git pull" command. There is a time lag, between the trigger and the actual git pull, which is a minor irritant now. I am in the process of setting up docker containers, and have the requirement to update the repo, running on multiple containers within the same host. I wanted to use this opportunity to solve this problem, in a better way, and looking for advise regarding this.
We use Rocketeer and groups to deploy. Rocketeer deploys with the user set to the deployment user (ubuntu in your case) and read/write permission for it, and the www-data group with read/execute permission. Then, as a last step, it modifies the permissions on the web-writable folders so that php can write to them.
Rocketeer executes over ssh, so can be triggered from anywhere, as long as it can connect to the server (public keys help). You might be able to setup your continuous integration/automated deployment to trigger a deploy automatically when a branch is updated/tests pass.
In any case, something where the files are owned by one user that can modify them and the web group can read the files should solve the main issue.
If you are planning on using docker, the simplest way would be to generate a new docker image for each build that you can distribute to your hosts. The docker build process would simply pull the latest changes on creation and never update itself. If a new version needs to be deployed, a new immutable image with the latest code is created and distributed.
I am a php developer. I have installed Git on my system(ubuntu 12.04). Now i want to make my system as server and my colleague's systems as clients. ie, a local sharing. Is it possible? we are using eclipse as the editor. We have installed the EGIT plugin in our system. I am very new in Git.Please help me.
Run git daemon to serve your local git repository:
$ git daemon --export-all --base-path=$(pwd)
.. assuming pwd path is at your git repository..
Then others can either clone your repository or just pull from it if they already have the same repository cloned from a common origin:
e.g.
git remote add yourname git://your-ip-address/repo-name
git pull yourname master
First, you need a static local IP address on the server (or a local DNS hostname.) Let's suppose you have a static local ip of 192.168.1.23
Next your colleagues need to have permission to access /home/user/Project. The simplest way is to do this (on the server):
adduser gituser
chown -R gituser /home/user/Project
Now everything in your repo is owned by gituser. Give the password for gituser to your colleagues.
Next, on the client(s), run
git clone gituser#192.168.1.23:/home/user/Project
This should create a copy of your repo on the client.
As for EGit, I recommend leaving that aside until you are more experienced with using Git on the command line. Egit may be easier to use once you understand it, but it is NOT easier to learn.
I am creating my first website that I am taking live soon. It is a big application that I hope goes somewhere eventually but I will be working on it while it is actually up on the web. I want to do it the right way and set up everything efficiently. It is very difficult to actually gain knowledge on how to go about doing this from a book or even the web.
Firstly, I've been doing a lot of research about SVN and although I am currently the only one working on the site I would like the ability to add more people to the project eventually. I know that you have to create a repository for the SVN where your project will actually reside in, and that there are some places that offer it for free like Codesion. I have a good understanding of how it works.
I would like a dev. subdomain for the current website so that I can test and work on everything that I am developing on and that SVN would commit to. How is it that your SVN repository communicates and uploads files to that subdomain? Is this even how you would go about doing this? How would I go about managing all the different databases for each domain?
When I am done updating several changes to my new rollout do I just manually upload all the files to the live site? I am very noob on the process for all this stuff and it is difficult to find information on it. What is the best practice for rolling out new updates? Are there any other different mechanisms or applications that I would need to know about for maintaining my website as it grows?
Please, any help in the right direction would be greatly appreciated.
It depends on how much control you have over your Web server.
Most (IMO) people push their files up to the server through something like FTP/SFTP. Many SVN hosting services offer this as a 'deployment' service. You should absolutely use this option if you don't have shell access or anything of that nature.
Personally, I prefer to have finer levels of control, and let the Web server do the pulling. Using a system with a full SSH access, you can just have the server do a checkout of the code:
svn co http://server/project/trunk --username foo --password bar
If you want ease of use (mixed with potential control levels), you can also create a PHP administration class to do this for you.
<?php
echo `svn co http://server/project/trunk --username foo --password bar --non-interactive`;
And often enough, I build a bridged solution. You can create post-commit hooks and set things up such that the SVN server reach out to a specified URL with the comment. The client server then checks the comment and performs actions based on the comment (it might ignore anything without the tag 'AUTODEPLOY' for instance).
There is also GIT (which you should be able to do similar things with), but I haven't had blow up my current processes (and hundreds of SVN repos) in order to go there yet.
That is a lot of concepts to cover all in one question. First - I host my repo directly on my webserver - but I can cover what happens if its on a remote server as well.
When you setup an SVN repo you should have a post-commit(.tmpl) file - rename that to post-commit - this is what will run with every commit.
Inside of that:
#!/bin/sh
# POST-COMMIT HOOK
# [1] REPOS-PATH (the path to this repository)
# [2] REV (the number of the revision just committed)
REPO="$1"
REV="$2"
if ( svnlook log -r $REV $REPO | grep "*DEPLOY-DEV*" )
then
rm -rf ~/tmp/path/to/repo
svn export -r $REV "file://$REPO" ~/tmp/path/to/repo
rsync -az ~/tmp/path/to/repo/trunk/ ~/path/to/webroot
fi
if ( svnlook log -r $REV $REPO | grep "*DEPLOY-LIVE*" )
then
rm -rf ~/tmp/path/to/repo
svn export -r $REV "file://$REPO" ~/tmp/path/to/repo
rsync -avz -e "ssh -i /path/to/.ssh/rsync" ~/tmp/path/to/repo/trunk/ root#website.com:/path/to/webroot
ssh -i /path/to/.ssh/rsync root#website.com chown -R apache:apache /path/to/webroot/
fi
Now to go through the important parts - include *DEPLOY-DEV* or *DEPLOY-LIVE* in your commit to deploy your site to the path you specify.
The DEPLOY-DEV part of the example works nicely when its on the same server - you just need to apply the correct paths.
If the server lives somewhere else it may require some ssh keys and other trickery that is a little deeper than I'll get into - but that should give you the direction you need to set it up if you need.
If you have ssh access to your webserver checkout two working copies on the server (in dev and production document root).
That way you can easily deploy any changes.
You can use a stable branch you merge your changes into for production.
Database connection you define in a config ini (or something similar) file that is not under version control. That way you can have different values for dev and prod.
If you don't have ssh access and you want to do it "efficiently" look for a different server with ssh access.
So I am working on a really large code base, more than 3000 files, more than 1 million lines of code and more than 500+ tables.
Though that is not really the issue. The issue here is, when a new feature is required, I work on it locally on my machine and when the time comes to update/patch our live production:
I ssh to our prod server
I navigate to the directory, and open the file to patch
I copy and paste??? OMG
Anyway, here is my take, please suggest if you guys have alternatives or more comfortable of doing this
First, we migrate to GIT. (we're in SVN)
Everytime we make release, we branch out in our git repo, and then clone a new copy in our prod server (right now we do a branch in svn, and do svn export, then copy it to target dir
when patching the server with new feature, I can simply go to the target repo/release, and do a git pull?? or should i go with a git patch?
This is how i envision a more simpler life.
Would you guys come up anything much easier than this?
I think you are on the right track. I did something similar.
I have two branches.
Master -> holds latest in dev
Production -> holds latest in production
When I need to make a change to prod, I branch production branch, make my changes and the merge back in to production branch. This gives me the option to work on multiple features at the same time.
Then I log on to the box and do git pull.
Of course the bigger problem here is that with PHP there is no such thing as package like there is java (WAR packages). :( So I am sure this is a pain.
I wish I could help you more but I can't think of anything else to make your life easier.
install svn on your prod server, checkout a specific branch, lets say production,
( cd /var/www && svn co svn://domain/path/to/branch )
then once you are ready to push your code just login to prod and run a script that will svn up that specific folder and maybe do some owner updates on the files
( cd /var/www && svn up )
( cd /var/www && chown -R www-data:www-data )
( cd /var/www && svn info )
if you want to do it remotely w/o login to the server you can do remote commands from ssh
(if the script above is on your home and called updateRepo.sh)
ssh user#server updateRepo.sh
if you have your ssh key pair setup you wont even need to do your password
Note: above will output to bash so you might want to redirect all of this 2>&1
is this helpful?
It may be time to consider using a continuous integration tool like Hudson Jenkins.
If not, at the very least consider having different branches or repositories for development and production code as per #Amir Raminfar's suggestion.
I am currently developing a medium sized web application using PHP and need to use some kind of version control. I also need to have the master copy of the code running in the apache document root so I can test the app while it is in development. Does anyone have any sugestions?
Thanks,
RayQuang
You can't go wrong with Git; everything you need to know is here: http://progit.org/book/
Yeah you should definitely use version control (Git or Subversion).
Here a short explanation how I'm using it in my web projects (I am using SVN):
I have a SVN project which I have checkouted on my local machine and on the webserver
Always when you change something you can commit your current running version
Log into the server (Could also be multiple servers) and do a svn update, so the newest code gets automatically deployed on the machine. The only thing you have to do is restart of the webserver
Note:
Take care what you commit. You've maybe another database configuration file on your local machine than on your server. You can put this into the svn ignore file list (guess git has something similar)
It is also easy possible that multiple persons work on the same project..
Don't commit logfiles
Links:
Git: http://git-scm.com/
Subversion: http://subversion.tigris.org/
I'd recommend Mercurial for its ease of use and that it keeps the working copy uncluttered, all versioning information is kept in just one .hg folder. I'd do it like this:
Set up a Mercurial repository at the server (hg init)
Do a hg clone of that repository to where you want your working copy
Work away!
When you want to test on the server, do a hg commit and hg push to move the changed files to the server
Run hg update on the server, or add
[hooks]
changegroup = hg update >&2
to the .hg/hgrc file (create it if it doesn't exist) on the server to have it automatically update.
For more info, you can also check out: http://hginit.com/