Like many projects, we deploy to many environments, QA, UA, Developer trunks, etc..
What is the best way to store sensitive configuration parameters in SVN? Or, should you not and just maintain a smaller unversioned file with credentials in it on the server?
Mainly, we do not want to expose production credentials to every developer.
I'd rather provide configuration examples than real config files. In my project there is setup.default.php file in root directory that every user need to copy as setup.php and amend to match local environment. Additionally, to prevent checking in back customised setup files there is a rule for it in .svnignore.
$ echo 'setup.php' > .svnignore
$ svn propset svn:ignore -F .svnignore .
This is a problem I have run into as well. I think the answer is to check in a Template (such as you have with setup.php.default) and then use an automated tool such as Phing to make the push to development. If you use recognizable tokens in the setup.php file then Phing will be able to replace these tokens with individual server values. Also, an easy one step push live will be a helpful process to have.
I would not store configuration information in the repository at all. That way you don't have to worry about SVN trying to update the config when you update your source.
I would agree with adam. If its not something that benefits everybody who works on the project, it shouldn't be under version control. If somebody checks out a copy of your code, will your personal project files help them? Probably not. It would most likely just clutter things up.
Related
I've just gotten my production site up and running. I have a lot more work to do, and I'm realizing the need now for a development server before pushing changing live onto the production site (with users) - obviously...
This thread (and a lot more on Stack) describe me:
Best/Better/Optimal way to setup a Staging/Development server
Anyhow... Reading these threads is outright confusing at times, with all of the thrown around terminology, and my smaller CentOS/Apache knowledge.
My goal is to:
Make some changes to files as needed.
Test the changes on the same server, to ensure settings are identical.
If all is ok, I can now save a version of this somewhere, perhaps locally is enough for now (Bazaar seems like a possibility?)
Finally, replace all of the changed files via SSH or SFTP or something...
My worries are:
Uploading changes while users are in the system.
How to upload the files that have changed, only.
I'd love somebody to either link to a great guide for what I'm thinking about (that leaves nothing to imagination I'd hope) - or some kind of suggestion/etc... I'm running in circles trying out different SVN's and progarms to manager them, etc...
I'm the only one developing, and just want a repeatable, trust-worthy solution that can work for me without making my life too miserable trying to get it set up (and keep it set up).
Thanks much.
If you have the ability to create a staging subdomain on the production server, here is how I would (and do) handle it:
Develop on your development machine, storing your code in a VCS. I use subversion, but you may find another you prefer. After making changes, you check in your code.
On your production server, you create a subdomain in an Apache VirtualHost which is identical to, but isolated from your production VirtualHost. Checkout your code from the VCS to the staging subdomain area. After making changes, you then run an update from your VCS, which pulls only changed files down. Staging and production share the same data set, or you may have a separate database for each.
The reason for using a subdomain instead of just a different directory is that it enables you to use the same DocumentRoot for both staging and production. It's also easy to identify where you are if you use something like staging.example.com.
When you're certain everything works as it's supposed to you can run a VCS update on the production side to bring the code up to date.
It is important to be sure that you have instructed Apache to forbid access to the VCS metadata directories (.svn, .git, whatever).
Addendum
To forbid access to .svn directories use a rewrite rule like:
RewriteEngine on
RewriteRule .*\.svn/.* - [F]
This will send a 403 on them. You could also redirect them to the homepage to make it less obvious they're even present.
In terms of worry #1, remember that even StackOverflow periodically goes down for maintenance when people are using it. Just provide a good mechanism for you to switch the site to maintenance mode (and back out) and you'll be fine.
Thank you everyone for the tips/hints/etc...
I've finally found the perfect solution for my needs, SpringLoops... http://www.springloops.com/v2/
It manages my SVN (which I use Tortoise SVN with) - and will actually deploy the changes onto my two sites: staging and production.
I highly recommend it, it's working wonderfully so far.
Thanks!
You need a version control system, like Subversion, Git or Mercurial.
When you change files, you commit those changes to a repository. This keeps track of all the changes you've made and lets you undo them if it turns out you did something stupid. It also lets you restore files you accidentally delete or lose.
On top of that, it makes deploying updates as simple as typing 'git update' or 'svn update' on the production server. Only the files that changed get transferred and put in place.
You will get this advice no matter how many times the question is re-asked. You need version control.
Recently I've read this article: http://www.smashingmagazine.com/2009/09/25/svn-strikes-back-a-serious-vulnerability-found/
Developers of many popular sites like apache.org, php.net (http://ru2.php.net/.svn/entries), classmates.com and russian Yandex use SVN, but do not follow the recommendations given by SVN (to use command export).
So, what are the reasons for not using svn export instead of updating the public copy like all they do?
Some people, not including myself, think that to deploy onto production you should just issue an svn up. If you do an export it loses the meta data about the versioning so you can t do that, you have to use another mechanism for tracking which version is where. It is an easy solution, but I think it can make for lazy packaging and also for "fixing in production" as if you do this you can also check back in from production...
From my perspective what I do is lock off/block access to any .svn files on the server (either Apache2 or IIS) this way the hidden folders are not accessible externally, and it allows for version tracking for sites that we use which do not require compiling before rollout
Languages like:
PHP
ASP (not .NET)
PLAIN HTML
COLDFUSION
PDF / IMAGE versioning (if needed, in my case we needed it for updated PDF docs for customers).
So certainly you can use SVN for web development, but you do need to be cautions as you expose your .svn folders to the world if you are not cautious. Otherwise it is a tool you could use to make your job easier and more efficient.
With that said, we simply run an SVN UPDATE on our production to update changed files, and with limited developers working on one piece of code at a time (like I said in my case) we don't get mixups with wrong things getting deployed. PLUS to be safe, always do a SVN CHECK FOR MODIFICATIONS to see what is going to be updated, and hey, if you do make a mistake, roll it back.
With svn export files can never get deleted, only added and modified. This could be issue sometimes.
When the entire website is open-source and available for downloading over a public resource (like PHP's). Protecting the .svn directories so other's can't get the source code is probably not worth the effort over simply doing a svn up.
I am developing (solo web developer) a rather large web based system which needs to run at various different locations. Unfortunately, due to some clients having dialup, we have had to do this and not have a central server for them all. Each client is part of our VPN, and those on dialup/ISDN get dialed on demand from our Cisco router. All clients are accessable within a matter of seconds.
I was wondering what the best way to release an update to all these clients at once would be. Automation would be great as their are 23+ locations to deploy the system to, each of which is used on a very regular basis. Because of this, when deploying, I need to display a 'updating' page so that the clients don't try access the system while the update is partially complete.
Any thoughts on what would be the best solution
EDIT: Found FileSyncTask which allows me to rsync with Phing. Going to use that.
There's also a case here for maintaining a "master" code repository (in SVN, CVS or maybe GIT). This isn't your standard "keep editions of your code in the repo and allow roll backs"... this repo holds your current production code (only). Once an update is ready you check the working updated code into the master repo. All of your servers check the repo on a scheduled bases to see if it's changed, downloading new code if a change is found. That check process could even include turning on the maintenance.php file (that symcbean suggested) before starting the repo download and removing the file once the download is complete.
At the company I work for, we work with huge web-based systems which are both Java and PHP. For all systems we have our development environments and production environments.
This company has over 200 developers, so I guess you can imagine the size of the products we develop.
What we have done is use ANT and RPM build archives for creating deployment packages. This is done quite easily. I haven't done this myself, but might be worth for you to look into.
Because we use Linux systems we can easily deploy RPM packages, the setup scripts within a RPM package can make sure everything gets to the correct place. Also you get a more proper version handling and release process.
Hope this helped you.
Br,
Paul
There's 2 parts to this, lets deal with the simple one first:
I need to display a 'updating' page
If you need to disable the entire site while maintaining transactional integrity, and publishing a message to the users from the server being updated, then the only practical way to do this is via an auto-prepend - this needs to be configured in advance (note - I believe this can be done using a .htaccess file without having to restart the webserver for a new PHP config):
<?php
if (file_exists($_SERVER['DOCUMENT_ROOT'] . '/maintenance.php')) {
include_once($_SERVER['DOCUMENT_ROOT'] . '/maintenance.php');
exit;
}
Then just drop maintenance.php into your webroot and that file will be displayed instead of the expected file. Note that it should probably include a session_start() and auto-refresh to ensure the session is not expired. You might want to extend the above to allow a grace period where POSTs will still be processed e.g. by adding a second php file.
In terms of deploying to remote sites, I'd recommend using rsync over ssh for copying content files - which should be invoked via a controlling script which:
Applies the lock file(s) as shown above
runs rsync to replicate files
runs any database deployment script
removes the lock file(s)
If each site has a different set up then I'd recommend either managing the site specific stuff via a hierarchy of include paths, or even maintaining a comlpete image of each site locally.
C.
I'm starting to use SVN repositories for all of our websites and wanted to know what the best practise was regarding website config files.
Should they be stored in the repository? The problem is the configuration of the websites need to be different for the working copies than that of the live sites. If I edit the config file for a working copy so that I can test on my machine when I commit back to the repository the config file will be updated there too and could then potentially get uploaded to the live site.
What do people generally do with config files, is there a way to tell SVN to skip config files when performing commits etc?
Generally, it's best to put config files into version control if they store significant information.
If you're talking ASP.NET sites here, I'd definitely place the config file in SVN. You can play a few tricks in ASP.NET config files using inheritInChildApplications and allowOverride (see How to: Lock ASP.NET Configuration Settings) which may allow you to force a local debug version to use different settings from the final production version despite using the same config file: just mount the website as a sub-directory in the local debug IIS and lock a few sections you wish to override. And of course, you could just include two config keys for particularly tricky bits and check in code which to load.
In general, it's good practice to make deploying anything from SVN a process involving as few manual steps as possible. That makes it more likely you'll do it correctly under time-pressure, and it makes disaster recovery easier to boot (say, when your datacenter springs a leak and you want to install the web site on some temporary box until you've got those backups sorted). Ideally, an svn checkout or export with at most a compile should suffice to get the web site up and running. I include even binary dll-dependencies directly in svn (stuff like javascript compressors and whatnot) so it'll run without requiring a bunch of custom library installs on the server, and compile on a dev machine with just msbuild.
For PHP, the principle is the same. However, you'll need different tricks. For instance, you might write the config file such that it checks some global system environment variable, and then overrides selected settings if it's a dev-machine. For instance, I've a setup similar to this where I check the IP address; all dev-machines are in a particular IP-block; unless a machine is in that IP block, it's considered a production machine (which doesn't enable various tracing etc. options). You could also check the host name, or simply any old environment variable which all developers agree to set on their development machines.
I think its best to keep config files in SVN. Regarding settings for staging/production environment, what we do is to have seperate config files for each environment, and then swap them out as part of our build process (using Ant and MSBuild). I.e. we can trigger a "production build", which will copy the production web.config file.
I would have the file in version control, absolutely, as it's generally pivotal to the function of the site. To stop it getting loaded to live, you could look at a build script (e.g. Web Deployment Project), which would switch out your development versions of the configuration with a 'live' version.
Sorry, ignore the link, just saw your comment about this being a PHP site - principle is the same however.
If the variables between your dev/live environments are limited to just connecionstrings and appsettings, then you can split your web.config into seperate files, and have a different file loaded in for each environment. That way, you can check everything into SVN and just update the filename reference in your webconfig depending on which environment you are deploying to.
http://kartones.net/blogs/kartones/archive/2009/09/29/asp-net-split-appsettings-and-connectionstrings-to-separate-files.aspx
Edit: Just seen you're talking about PHP.
In general the best-practice is to store all the custom configuration files under version control. You may want to keep a separate config file for the production and development versions.
If possible, try to extract all the config sections that depend on the deployment environment (connection strings, paths, etc) into separate files. Then link to them from the main (common) config file, so that it will be just a matter of updating the reference when you change your environment from development to production.
I currently ftp all my files to my website when i do an update (over a slowish adsl connection)
And I want to make things easier, so I just recently started using a hosted svn service, and i thought i could speed things up a bit by doing an svn export of my website directly onto my webserver
i have tried that a few times and it seems to work ok, however it does fetch the entire site everytime which is a bit slow for a 1 file update
so my questions are
is it possible to do an export and only get the changes since the last export (how will this handle deleted files ?)
OR will it be easier to do an svn checkout and svn update it all the time instead of svn export and just hide the .svn folders using apache htaccess
is this a good idea, or is there a better way to publish my website
i am trying to achieve the 1 click deploy type ideal
maybe there are some gotcha's i haven't thought of that someone else has run into
debian/apache/php
I would do an svn checkout, and have done so successfully on a live site for a number of years. You should add mod_rewrite rules to 404 the .svn directories (and files) though.
This is what I'm doing on my host:
For every project I have a structure that looks more less like this:
~/projects/myproj
~/public_html/myproj
First dir is a checkout from SVN, while second one is just svn export.
I have a small bash script
#!/bin/bash
SOURCE="$HOME/projects/"
TARGET="$HOME/public_html/"
for x in `ls $SOURCE`
do
if [ -d $SOURCE$x ]; then
svn update $SOURCE$x
svn export --force $SOURCE$x $TARGET$x
fi
done
Export is done from working copy so it's very fast.
It might not be exactly the answer you are looking for, but, if you have an SSH access to your webserver (it depends on your hosting service ; some "low cost" don't give such kind of access), you can use rsync to "synchronise" the remote site with what you have on your disk.
In the past, I was using something like the idea you are describing (fetching the svn log between last revision pushed to production and HEAD, analysing every lines, and in the end calculating what to send to the server) ; but it was not really a great process ; I now use rsync, and like it way better.
(Here too, you will have to exclude .svn directories, btw)
You can just accept having .svn directories in your website (generally not a problem esp. if you configure it not to permit access to these) - this is the easy option. Alternatively, do what RaYell does, and have two copies of your website on the webserver. One normal checkout outside of the web-directory, and one in your web-directory. When you update, simply export the svn (just a copy with .svn dirs deleted) into the web directory (and you should make sure to first delete old files if you wish to avoid files that have been removed from SVN from remaining on your website).
I do something like this, using robocopy set to mirror the svn checkout while excluding .svn directories, and get both the export and the old-file deletion in one step, thus minimizing downtime if the copy takes long. I'm sure this is easy on unix too, if that's your hosting environment. For example, you can use a local rsync: http://blog.gilluminate.com/2006/12/12/yes-you-can-rsync-between-two-local-directories/
Old topic but since this is what came up in Google during my research I thought I would add to this. I recommend doing an export to the site instead of checking out to it. I like to keep the repositories and the sites separate. I also don't recommend exporting the entire repo to the site each time, specially if only a few file change at a time. What you can do instead is do a diff on the repo to see what's changed from a release to another and only export those files. More info at: http://www.joeyrivera.com/2011/automate-svn-export-to-site-w-bash-script/