What is the best process for updating a live website?
I see that a lot of websites (e.g. StackOverflow) have warnings that there will be downtime for maintenance in advance. How is that usually coded in? Do they have a config value which determines whether to display such a message in the website header?
Also, what do you do if your localhost differs from the production server, and you need to make sure that everything works the same after you transfer? In my case, I set up development.mydomain.com (.htaccess authentication required), which has its own database and is basically my final staging area before uploading everything to the live production site. Is this a good approach to staging?
Lastly, is a simple SFTP upload the way to go? I've read a bit about some more complex methods like using server-side hooks in Git.. Not sure how this works exactly or whether it's the approach I should be taking.
Thanks very much for the enlightenment..
babonk
This is (approximately) how it's done on Google App Engine:
Each time you deploy an application, it is associated with a subdomain according to it's version:
version-1-0.example.com
version-1-1.example.com
while example.com is associated with one of the versions.
When you have new version of server-side software, you deploy it to version-2-0.example.com, and when you are sure to put it live, you associate example.com with it.
I don't know the details, because Google App Engine does that for me, I just set the current version.
Also, when SO or other big site has downtime, that is more probable to be a hardware issue, rather than software.
That will really depend on your website and the platform/technology for your website. For simple website, you just update the files with FTP or if the server is locally accessible, you just copy your new files over. If you website is hosted by some cloud service, then you have to follow whatever steps they offer to you to do it because a cloud based hosting service usually won’t let you to access the files directly. For complicated website that has a backend DB, it is not uncommon that whenever you update code, you have to update your database as well. In order to make sure both are updated at the same time, you will have to take you website down. To minimize the downtime, you will probably want to have a well tested update script to do the actual work. That way you can take down the site, run the script and fire it up again.
With PHP (and Apache, I assume), it's a lot easier than some other setups (having to restart processes, for example). Ideally, you'd have a system that knows to transfer just the files that have changed (i.e. rsync).
I use Springloops (http://www.springloops.com/v2/) to host my git repository and automatically deploy over [S/]FTP. Unless you have thousands of files, the deploy feels almost instantaneous.
If you really wanted to, you could have an .htaccess file (or equivalent) to redirect to a "under maintenance" page for the duration of the deploy. Unless you're averaging at least a few requests per second (or it's otherwise mission critical), you may not even need this step (don't prematurely optimize!).
If it were me, I'd have a an .htacess file that holds redirection instructions, and set it to only redirect during your maintenance hours. When you don't have an upcoming deploy, rename the file to ".htaccess.bak" or something. Then, in your PHP script:
<?php if (file_exists('/path/to/.htaccess')) : ?>
<h1 class="maintenance">Our site will be down for maintenance...</h1>
<?php endif; ?>
Then, to get REALLY fancy, setup a Springloops pre-deploy hook to make sure your maintenance redirect is setup, and a post-deploy hook to change it back on success.
Just some thoughts.
-Landon
Related
I'm developing a simple website(in a VPS) using php.
My question is: how can I send the changes I do using a secure protocol?
At the moment I use scp but I need a software that check if the file is changed, if yes sends it to the server.
Advice?
use inotify+rsync or lsyncd http://code.google.com/p/lsyncd/
check out this blog post for the inotify method http://www.cyberciti.biz/faq/linux-inotify-examples-to-replicate-directories/
If you have complete access to your VPS, then you can mount your remote home directory, via ssh and webdav, locally as a normal filesystem. Afterwards you can edit on your computer locally. Any changes should be automatically uploaded.
Have a look at this relevant article
FTP(S): Set up an FTP server on the target, together with an SSL cert (a personal cert is good enough).
SSH: You can pass files around through SSH itself, but I found it most unreliable (imho).
Shares: You could share a folder (as the other people suggested) with your local machine. This would be the perfect solution if it wasn't hard at diagnosing disconnection issues. I've been through this and it is far from a pleasant experience.
Dropbox: Set up a dropbox server on your VPS and a dropbox client on your local PC. Using dropbox (or similar methods like skydrive, icloud..), also has some added advantages, eg, versioning, recovery, conflict detection...
All in all, I think the FTPS way is the most reliable and popular one I've ever used. Via SSH, it caused several issues after using it from about 2 days. The disk sharing method caused me several issues mostly because it requires that you are somewhat knowledgeable with the protocol etc.
The dropbox way, although a bit time-consuming to set up, is still a good solution. However, I would only use it for larger clients, not the average pet hobby website :)
Again, this is just from experience.
You could also use Subversion to do this, although the setup may take a little while.
You basically create a Subversion repository and commit all of your code. Then on your web server, make the root directory a checkout of that repository.
Create a script that runs a svn update command, and set it as a post-commit hook. Essentially, the script would be executed every time you would do a commit from your workstation.
So in short, you would...
Update your code locally
Commit via Subversion (use Tortoise SVN, it's easy and simple to use)
Automatically, the commit would trigger the post-commit hook on the server and run the svn update command to synchronize the code in your root directory.
You would of course have to hide all of the .svn folders to the public using .htaccess files (providing you're using Apache).
I know this would work, as I have done it in the past.
The beauty of this option is that you also get a two in one. Source versionning and easy updates. If you make a mistake, you can also easily revert them back.
I mantain a custom PHP application (build for me) that is hosted in a web server. Sometimes I add new features or repair bugs, and after test in local I upload the changes to the web server. It's not a critical application (is a game), but the most of the time there are some people connected.
The steps that I make to upgrade the application:
Access via FTP (Filezilla)
Upload a .htaccess file that redirects all the people (except my IP) to a mantain.html file
Check that access is denied for other IP except mine.
Backup old code
Upload new code
Go to PhPMyAdmin
Backup DB
Execute scripts for the DB
Test that all works fine (if not -> revert the backups)
remove .htaccess file
I usually spend an average of 30 minutes doing these steps, and I'm wondering if there is any way to optimize, automatize or doing something to spend less time. Also I know that if I can automatize some steps there are less prone to have errors.
Several other answers suggest PHP-specific deployment tools, but being as I'm not very familiar with PHP, I'll offer some general tips. These suggestions may be redundant by some of the other tools already suggested, though.
First off, don't upload a new .htaccess file every time--just have two of them on your server. Perhaps call them .htaccess-permanent, and .htaccess-maintenence. Then create a symlink to the one that ought to be active. Then once you've tested that access is properly denied once, you don't have to do this manual testing phase every single time you do an upgrade.
I'd also write a shell script to do most everything for me. My new work flow would look like this:
Upload new code to server in a directory called new/
Log in to the server via shell, and execute the upgrade script
Test the new site
Run upgrade-finalize
The end.
Now for the interesting part, the upgrade script will do this:
It will delete the .htaccess symlink, and re-create it, pointing to .htaccess-maintenence.
It will copy the current code in current/ to backup/
It will back up the DB, using the exact same commands that PHPMyAdmin uses
It will move the contents of new/ (which you just uploaded) to current/
It will execute the scripts for the DB
And the upgrade-finalize script will simply:
Delete the .htaccess symlink, and re-create it, pointing to .htaccess-permanent once again
The only possibly tricky part here will be getting the exact commands that PHPMyAdmin uses to back up your database, but it's probably a simple mysqldump command, and you can probably get that info from PHPMyAdmin or some logs, or something. Sorry, I don't know more about PHPMyAdmin to help in this specific area.
Look into a deployment tool like Capistrano that allows you to automate those steps.
I usually spend an average of 30 minutes doing these steps, and I'm wondering if there is any way to optimize, automatize or doing something to spend less time.
There are many ways. For starters, steps one through eight can be done in a single shell script. You could checkout Phing, an automated deployment system. Also, you might want to delve in continuous integration for even more control over how and when the software can be deployed.
Doing this manually is, like you say, asking for trouble.
For starters, you could upload your files into a new webroot and when done, switch over the DocumentRoot in apache, leaving it available during the copy process. For any shared files you could use a symlink to a common folder (eg, uploaded images etc)
You could probably take the backup during operation as well if you don't care about consistency in the database. For migrations that doesn't "break" the functionality, you could also migrate it and test it on your new webroot with another hostname if consistency isn't a problem.
The best option is always to use multiple webservers so that you can take one offline for testing while the other one is operational, but you will still have problem with consistency, however I assume that is not an option since you don't mention it.
I'm using Codeigniter if this makes it easier. I'm wondering if a website is live with populated database and users accessing, and I have a new idea to implement into the website, how should I do it? Do you work directly on the live site?
Or do you copy the database and the files to a local server (MAMP/WAMP/XAMMP) and work on it there then if it works update the live site with the changes. For this second method, is there anyway to check which are the files that have been changed and only upload those? What if it works on local sever, but after updating the live site it does not work?
Codeigniter configuration also has the option of default database and other database. I wonder how these can be used for testing?
Don't work directly on the live site. Instead, have a development environment (using, say, vmware or virtualbox on your machine) and clone the live environment. Get you code in version control (I'll say it again: GET YOUR CODE IN VERSION CONTROL), do your development on the development machine, against a dev branch in version control. After you're done testing and happy with the changes, commit them to a 'deployments' or 'live' branch, and deploy on the live site from there. Be sure to do a backup of the database before you roll out the new code.
Edit: use symlinks to stage your new code base on the live site. If it doesn't work, just switch it back to the old directory. Saves you a lot of greif!
Read up on version control (svn, git, et al.).
Never work on a live site, preferably on another server (to prevent while(1){..} crashes etc.), but on the same server at least on another documentroot/domain, preferably with limited access to your IP only.
Normally I only copy the table-definitions (mysqldump -t is nice for that) and have another database altogether, if you need the latest & greatest data, you could replicate your main database to a test-database, which also gives you the advantage of a cheap backup if you haven't got one already.
I usually set a switch in Apache/Vhost configuration (SetEnv DEV=1), so that in code I can use if(getenv('DEV')==1) to check whether I can just dump variables on error conditions, and which limit the possibility of accidentaly committing/uploading code with a 'development switch' still on by accident.
The typical answer to this question is going to be do your work in the test environment, not the production environment. And I agree that that is often the best way to handle changes. If you have the luxury of a test environment, then take full advantage of it. After all, that's what it's there for--to test.
However, that doesn't mean that working in the production environment is completely off-limits. Your decision should be based on a few factors:
Is the operation of your website critical to your business needs?
If so, do all your work in a test environment and deploy it to your live environment when you've fully tested your changes.
Are the changes you're about to make going to have a large impact on the rest of the website?
For example, are you about to change the Database schema? Are you about to change the way users log in or out of your website? If so, do your work in the test environment. If you're changing the behavior of a page that doesn't have any effect elsewhere, you could get away with making the change in the production environment.
How long will your changes take to implement?
If you can't guarantee that your changes won't take longer than 15-20 minutes, do your work in a test environment.
I'm planning an application that allow users to create a specific type of website. I wanted to link account names to 'account.myapp.com'. Going to 'account.myapp.com' will serve up a website. I haven't a clue on how to map this. I'll be using Code Igniter as my development tool.
I would like to give users the ability to add a registered domain name for their website, rather than use the standard sub domain. Any tips/methods on this?
Also, what are some pitfalls and problems I should be looking for when developing something with this design? My biggest concern is backing myself in a corner with bad database design, and creating a nightmare of an app to maintain or update.
Thanks for your time!
Your plan for having a single app that serves all the sites is going to be quite an undertaking and a lot of work. That is not to say it isn't possible (plenty of enterprise CMS's, including Sharepoint, allow you to run 'virtual sites' etc. from a single install).
You are going to need to undertake a lot of planning and design, specifically on the security front, to make sure the individual sites operate in isolation. I assume that each site will have its own account(s) - you are going to have to do a lot of work to make sure users can't accidentally (or malicously) start editing another site.
And you are right to consider maintanence - if you have all the sites running under a single application, and therefore a single database, that database is going to get big and messy, very quickly. It also then becomes a single point of failure.
A better plan would be to develop a self-contained solution (for a single website) - this can then run from it's own directory, with it's own database with it's own set of accounts. It would be significantly smaller (in terms of both code and database) and therefore probably perform a lot better. Day-to-Day maintanence would be easier (restore a website from backup), but software updates (to add a new feature) would be a bit trickier, though as it's PHP it's just file uploads and SQL patches, therefore you can automate this with ease.
In terms of domains: if you went with the invidual app (one per website) approach, you can use Apache's Dynamic Virtual Hosts feature, which effectively maps a URL to the filesystem, (so website.mydomain.com could be translated to automatically be severed from /home/vhosts/com/mydomain/website): thus deploying a new website would be a matter of simply copying the files into the correct directory, creating the database, and updating a config file, all of which could be automated with ease.
If users want to use their own URL's then they have to firstly update their DNS to point to your server and secondly you would need to configure an Apache vhost for that domain, which would probably involve a restart of apache and thus affect all other users.
This is very easy to do in codeigniter and can be complete done using routes.php, and a pre-controller hook.
THAT BEING SAID... Don't do this. It's generally not a good idea. Even 37Signals, who made this sort of account management famous is recanting, and moving towards centralized accounts. Check http://37signals.com/accounts
If I'm understanding your question correctly, you need to setup wildcard DNS, and use Apache mod_rewrite to internally redirect (example) myaccount.myapp.com to myapp.com/?account=myaccount. You app logic can take it from there.
I just Googled, "wildcard dns mod_rewrite account" (without quotes) and found some examples with instructions, such as:
http://www.reconn.us/content/view/46/67/
This is a valid and desirable way to structure certain web apps IMO. I'm not aware of serious drawbacks.
I don't really know of a great (automated/scalable) way to allow the users to specify their own individual domain names but you might be able to do it if you had them modify their domain's DNS to point to your web server, then added a ServerAlias directive to your "myapp" Apache configuration. You're still left with the problem of your myapp runtime instance understanding that requests coming through a customer's domain are specific to a customer account. So (example) customeraccount.com really equates to myapp.com/?account=customeraccount. Some more mod_rewrite rules could probably take care of this, but it's not automated (perhaps it could be though with an include file or such).
Sorry, you said you were using CodeIgniter ... substitute in then myapp.com/account/myaccount where I wrote myapp.com/?account=myaccount.
First, you have to setup your Apache (or whatever webserver you're using) to give every subdomain the same DNS settings (wildcard match). Then, you have to use CodeIgniter's routing features to parse the subdomain from the request URL, set that as param (or whatever that's called in CodeIgniter) and have some fun with it in your controller.
About pitfalls and problems: that depends on what you want to do. ;)
I currently work in a web shop with almost no formal processes and a million PHP websites, including tricky stuff like custom CMS and shopping cart code.
We are trying to improve things. I am pushing for CVS/SVN.
My question is, what is the best practice for sandboxing website work? We are on the LAMP stack. Some of our sites have hardcoded (or user-inputted links) to the current domain, so setting up a different domain like preview.mysite.com will break links pointing to www.mysite.com. If we start applying regression tests, perhaps the domains should be uniform for testing? That can always be done with a local host entry.
So, considering we have lots of sites, it would be nice to have one process for always doing previewing in a proper sandbox. Wondering how this would integrate with an SVN/CVS cycle.
I am just looking for industry best practices because we are trying to get there. If that means cloning a site to an extra server, so be it.
So yes, you should have a second STAGE server. What I do is put my code into CVS on my dev box, and do regular commits as i go along. When I am ready to push a version to the "STAGE" server, I go through the files I want to STAGE and tag them STAGE:
cvs tag -F STAGE
Then I go to the STAGE server and do an update with the STAGE flag to get the STAGE version of files:
cvs up -r STAGE
This also sets the sticky tag to be "STAGE" on those files, so in the future, I can just leave the STAGE tag off when I do updates on my stage server:
cvs up
finally, when I've tested my code on the STAGE server, I roll it to the production server using rsync...
We have several developers working together so keeping a stable STAGE version up can get tricky. In that case, if I just have small changes to one or two files, I will just scp them individually over to the production server..
Finally, to ensure I know what is out on my production servers, after I send a file or files off to the production server, I tag all the files on my stage server as RELEASE, and also as RELEASE20090713 or whatever the current date is.. That way I have moving snapshots though time that I can get if needed. Note though, this doesn't update the sticky tag, so my regular old
cvs up
on the stage server still gets me the latest STAGE files.
Now in your case, as far as the hardcoded URL's goes... You already know.. bad bad bad... so fix them as you go... But you may be able to use apache URL rewriting to rewrite URL's on STAGE to talk to a custom TCP port.
If you have a smart network device like a cisco router, you can set it up to do PAT (port address translation) for your IP's. Port 80 can forward to your regular production webserver, and port 8080 can forward to your STAGE server (its port 80).. Then all you do is have apache do URL rewriting on your STAGE server and append 8080 to all the hostnames it sees. Now all your posts and links will go to the correct STAGE servers, and your apache configs can be exactly the same also.
Regarding the hard coded (or user-inputted) domain names: you could add the domains to your hosts file. That should solve your problems during development and preview. Your browser will retrieve the IP for www.mysite.com and find 127.0.0.1 or the IP of the preview site in the hosts file. The tricky part is that just by looking at the URL in the browser, you cannot determine whether you are looking at the production site or not. (The ShowIP addon for Firefox can help you here.)
Regarding CVS/SVN: I would really advice you to go for SVN. It's not harder to use than CSV but has some advantages (e.g. renaming is possible). For more information see e.g. this question.
As for the previewing in a sandbox, this is what we do: we do most of our development on trunk (or on a branch, but the rest of the process is almost the same). Once we are ready to show it to the customer, we create a tag. This tag is used to update the preview server. If the customer isn't satisfied, we develop some more on trunk (or the branch), create a new tag, update preview with the tag, etc. Once the customer is happy we use the exact same tag running on preview to also update the production server. This way we can be sure that the preview and production server have the same code base.