I'm using Codeigniter if this makes it easier. I'm wondering if a website is live with populated database and users accessing, and I have a new idea to implement into the website, how should I do it? Do you work directly on the live site?
Or do you copy the database and the files to a local server (MAMP/WAMP/XAMMP) and work on it there then if it works update the live site with the changes. For this second method, is there anyway to check which are the files that have been changed and only upload those? What if it works on local sever, but after updating the live site it does not work?
Codeigniter configuration also has the option of default database and other database. I wonder how these can be used for testing?
Don't work directly on the live site. Instead, have a development environment (using, say, vmware or virtualbox on your machine) and clone the live environment. Get you code in version control (I'll say it again: GET YOUR CODE IN VERSION CONTROL), do your development on the development machine, against a dev branch in version control. After you're done testing and happy with the changes, commit them to a 'deployments' or 'live' branch, and deploy on the live site from there. Be sure to do a backup of the database before you roll out the new code.
Edit: use symlinks to stage your new code base on the live site. If it doesn't work, just switch it back to the old directory. Saves you a lot of greif!
Read up on version control (svn, git, et al.).
Never work on a live site, preferably on another server (to prevent while(1){..} crashes etc.), but on the same server at least on another documentroot/domain, preferably with limited access to your IP only.
Normally I only copy the table-definitions (mysqldump -t is nice for that) and have another database altogether, if you need the latest & greatest data, you could replicate your main database to a test-database, which also gives you the advantage of a cheap backup if you haven't got one already.
I usually set a switch in Apache/Vhost configuration (SetEnv DEV=1), so that in code I can use if(getenv('DEV')==1) to check whether I can just dump variables on error conditions, and which limit the possibility of accidentaly committing/uploading code with a 'development switch' still on by accident.
The typical answer to this question is going to be do your work in the test environment, not the production environment. And I agree that that is often the best way to handle changes. If you have the luxury of a test environment, then take full advantage of it. After all, that's what it's there for--to test.
However, that doesn't mean that working in the production environment is completely off-limits. Your decision should be based on a few factors:
Is the operation of your website critical to your business needs?
If so, do all your work in a test environment and deploy it to your live environment when you've fully tested your changes.
Are the changes you're about to make going to have a large impact on the rest of the website?
For example, are you about to change the Database schema? Are you about to change the way users log in or out of your website? If so, do your work in the test environment. If you're changing the behavior of a page that doesn't have any effect elsewhere, you could get away with making the change in the production environment.
How long will your changes take to implement?
If you can't guarantee that your changes won't take longer than 15-20 minutes, do your work in a test environment.
Related
I know that this was already asked, but I want to share my situation also to focus on alternative solutions and tips. We develop PHP websites, and we test everything on a remote semi-production server, to see how our web applications work in the real world.
Usually, we mount remote SFTP on our computers, and read/write directly on the server (then we have a nightly backup from the test server to another), so we haven't to save-and-upload every file we change.
Now we are considering to switch to GitHub to track file changes. I already installed and configured GitHub on my Mac, and a webhook that automatically pull from our GIT repository to out test web server. So, when I commit and sync to GitHub, I can refresh the page(s) I changed in my browser to see it in action.
The problem is that web scripting is often made of many micro-changes, sometimes you have to continuously hit "save" and switch to the browser to check if it works correctly... so having to commit (and specify a summary!) everytime is unacceptable.
How we can solve this problem with the workflow?
Thank you in advance.
Commit only good code. Test with sftp if you have to, but only commit good code.
Losing a benefit like git bisect is a pretty serious side-effect of what you're proposing among others.
Simply put: you shouldn't do that.
Source code management systems are not for testing and transferring your code under development to testing servers. You should either work with SFTP as before or set up rsync. These are the tools for transfer.
You should use GitHub to store only working, complete changes. After testing your "micro-changes" and getting to a working snapshot you should commit and describe everything you modified since the last commit.
Anything else would be just hacking the tools you try to use. Doing what you propose would just mean loosing (nearly) all benefits that git gives you and would be much less convenient than using plain sftp/rsync.
I'm developing a simple website(in a VPS) using php.
My question is: how can I send the changes I do using a secure protocol?
At the moment I use scp but I need a software that check if the file is changed, if yes sends it to the server.
Advice?
use inotify+rsync or lsyncd http://code.google.com/p/lsyncd/
check out this blog post for the inotify method http://www.cyberciti.biz/faq/linux-inotify-examples-to-replicate-directories/
If you have complete access to your VPS, then you can mount your remote home directory, via ssh and webdav, locally as a normal filesystem. Afterwards you can edit on your computer locally. Any changes should be automatically uploaded.
Have a look at this relevant article
FTP(S): Set up an FTP server on the target, together with an SSL cert (a personal cert is good enough).
SSH: You can pass files around through SSH itself, but I found it most unreliable (imho).
Shares: You could share a folder (as the other people suggested) with your local machine. This would be the perfect solution if it wasn't hard at diagnosing disconnection issues. I've been through this and it is far from a pleasant experience.
Dropbox: Set up a dropbox server on your VPS and a dropbox client on your local PC. Using dropbox (or similar methods like skydrive, icloud..), also has some added advantages, eg, versioning, recovery, conflict detection...
All in all, I think the FTPS way is the most reliable and popular one I've ever used. Via SSH, it caused several issues after using it from about 2 days. The disk sharing method caused me several issues mostly because it requires that you are somewhat knowledgeable with the protocol etc.
The dropbox way, although a bit time-consuming to set up, is still a good solution. However, I would only use it for larger clients, not the average pet hobby website :)
Again, this is just from experience.
You could also use Subversion to do this, although the setup may take a little while.
You basically create a Subversion repository and commit all of your code. Then on your web server, make the root directory a checkout of that repository.
Create a script that runs a svn update command, and set it as a post-commit hook. Essentially, the script would be executed every time you would do a commit from your workstation.
So in short, you would...
Update your code locally
Commit via Subversion (use Tortoise SVN, it's easy and simple to use)
Automatically, the commit would trigger the post-commit hook on the server and run the svn update command to synchronize the code in your root directory.
You would of course have to hide all of the .svn folders to the public using .htaccess files (providing you're using Apache).
I know this would work, as I have done it in the past.
The beauty of this option is that you also get a two in one. Source versionning and easy updates. If you make a mistake, you can also easily revert them back.
What is the best process for updating a live website?
I see that a lot of websites (e.g. StackOverflow) have warnings that there will be downtime for maintenance in advance. How is that usually coded in? Do they have a config value which determines whether to display such a message in the website header?
Also, what do you do if your localhost differs from the production server, and you need to make sure that everything works the same after you transfer? In my case, I set up development.mydomain.com (.htaccess authentication required), which has its own database and is basically my final staging area before uploading everything to the live production site. Is this a good approach to staging?
Lastly, is a simple SFTP upload the way to go? I've read a bit about some more complex methods like using server-side hooks in Git.. Not sure how this works exactly or whether it's the approach I should be taking.
Thanks very much for the enlightenment..
babonk
This is (approximately) how it's done on Google App Engine:
Each time you deploy an application, it is associated with a subdomain according to it's version:
version-1-0.example.com
version-1-1.example.com
while example.com is associated with one of the versions.
When you have new version of server-side software, you deploy it to version-2-0.example.com, and when you are sure to put it live, you associate example.com with it.
I don't know the details, because Google App Engine does that for me, I just set the current version.
Also, when SO or other big site has downtime, that is more probable to be a hardware issue, rather than software.
That will really depend on your website and the platform/technology for your website. For simple website, you just update the files with FTP or if the server is locally accessible, you just copy your new files over. If you website is hosted by some cloud service, then you have to follow whatever steps they offer to you to do it because a cloud based hosting service usually won’t let you to access the files directly. For complicated website that has a backend DB, it is not uncommon that whenever you update code, you have to update your database as well. In order to make sure both are updated at the same time, you will have to take you website down. To minimize the downtime, you will probably want to have a well tested update script to do the actual work. That way you can take down the site, run the script and fire it up again.
With PHP (and Apache, I assume), it's a lot easier than some other setups (having to restart processes, for example). Ideally, you'd have a system that knows to transfer just the files that have changed (i.e. rsync).
I use Springloops (http://www.springloops.com/v2/) to host my git repository and automatically deploy over [S/]FTP. Unless you have thousands of files, the deploy feels almost instantaneous.
If you really wanted to, you could have an .htaccess file (or equivalent) to redirect to a "under maintenance" page for the duration of the deploy. Unless you're averaging at least a few requests per second (or it's otherwise mission critical), you may not even need this step (don't prematurely optimize!).
If it were me, I'd have a an .htacess file that holds redirection instructions, and set it to only redirect during your maintenance hours. When you don't have an upcoming deploy, rename the file to ".htaccess.bak" or something. Then, in your PHP script:
<?php if (file_exists('/path/to/.htaccess')) : ?>
<h1 class="maintenance">Our site will be down for maintenance...</h1>
<?php endif; ?>
Then, to get REALLY fancy, setup a Springloops pre-deploy hook to make sure your maintenance redirect is setup, and a post-deploy hook to change it back on success.
Just some thoughts.
-Landon
Am new to web development. I am curious as to how people do it.
I am writing some php code that uses a mysql DB. I have the password hardcoded in the code as of now. This code can be checked out by all devs and so every one has access to the password. Seems very very wrong to me. On top of that I can think of some complications. I am listing the issues in bullet point form -
Password hard coded in code is wrong. I don't want all devs to have access to it as all of them can check out the code.
How to differentiate between production and development servers/credentials? I have the same file containing both prod and dev DB credentials. What is the best way to handle this?
I want to prevent against lazy/drunk times so that devs do not delete/drop tables etc. I can obviously have different access to different devs. So is that the solution to all of this?
Potential solution: Do not have the password in code. Ask devs to add the password themselves and make sure its never checked in.
Problem with solution: Tedious process of deployment. Have to add the password for production/QA deployment manually and make sure its able to connect to the DB everytime before deployment. Sounds too painful and error prone. What do people usually do?
Also on the same note (kind of linked to the above question)
If you have 4 devs in the team how do you set up the dev environment? Do all of them use the same DB? If not how do you create the tables and populate the tables with test data? Do you have to write code to populate the test data?
Thanks a lot for any input.
Put the password in a separate PHP file, containing all your app settings, and include it at the top of the page. This file can then be kept out of Version Control, and replaced for each deployment.
Make sure that you keep the config.php file (or whatever you choose to name it) out of your root directory, also, so that it can't be accidentally served up to any users of your app. Also, as a further precaution, make sure that you give it the .php extension, so that if it somehow does still get served up, it should be parsed by PHP first, and any useful information (hopefully) removed - a common practice would be to name it with a .conf.php or .inc.php extension for this reason.
As for the Dev Environment, we use a single database shared by all the devs. It was originally created from live client data, cloned into our database, with certain information redacted / replaced for privacy reasons. The same database is used in our development build as well as our localhost builds.
In that situation you describe, you could write a deployment script that "fills" the password in the correct spot in the source code automatically. Then your production passwords only reside in your production environment deployment scripts. You can have developers manually add it to their own local environments.
Also, you could have a configuration file with all this settings and have your app load them from it, or a even a separate php file as someone else suggested. Either configuration/php file should not be in source control and each developer can do its own, and you can have the correct one in production.
This is often solved by having both a development and production version of a config file. The production version contains connection information for the development database ( servername, database name, username, password). This file can be viewed edited by all developers.
The production version contains connection information for the production server, and is unreadable by untrusted developers. When code is deployed to the production site, do not deploy the development version of the configuration file. The production server's version will then stay intact.
You can consider removing the configuration file from version control altogether. Using this scheme, each developer will maintain his own version or can access a development version from a standard location.
I currently work in a web shop with almost no formal processes and a million PHP websites, including tricky stuff like custom CMS and shopping cart code.
We are trying to improve things. I am pushing for CVS/SVN.
My question is, what is the best practice for sandboxing website work? We are on the LAMP stack. Some of our sites have hardcoded (or user-inputted links) to the current domain, so setting up a different domain like preview.mysite.com will break links pointing to www.mysite.com. If we start applying regression tests, perhaps the domains should be uniform for testing? That can always be done with a local host entry.
So, considering we have lots of sites, it would be nice to have one process for always doing previewing in a proper sandbox. Wondering how this would integrate with an SVN/CVS cycle.
I am just looking for industry best practices because we are trying to get there. If that means cloning a site to an extra server, so be it.
So yes, you should have a second STAGE server. What I do is put my code into CVS on my dev box, and do regular commits as i go along. When I am ready to push a version to the "STAGE" server, I go through the files I want to STAGE and tag them STAGE:
cvs tag -F STAGE
Then I go to the STAGE server and do an update with the STAGE flag to get the STAGE version of files:
cvs up -r STAGE
This also sets the sticky tag to be "STAGE" on those files, so in the future, I can just leave the STAGE tag off when I do updates on my stage server:
cvs up
finally, when I've tested my code on the STAGE server, I roll it to the production server using rsync...
We have several developers working together so keeping a stable STAGE version up can get tricky. In that case, if I just have small changes to one or two files, I will just scp them individually over to the production server..
Finally, to ensure I know what is out on my production servers, after I send a file or files off to the production server, I tag all the files on my stage server as RELEASE, and also as RELEASE20090713 or whatever the current date is.. That way I have moving snapshots though time that I can get if needed. Note though, this doesn't update the sticky tag, so my regular old
cvs up
on the stage server still gets me the latest STAGE files.
Now in your case, as far as the hardcoded URL's goes... You already know.. bad bad bad... so fix them as you go... But you may be able to use apache URL rewriting to rewrite URL's on STAGE to talk to a custom TCP port.
If you have a smart network device like a cisco router, you can set it up to do PAT (port address translation) for your IP's. Port 80 can forward to your regular production webserver, and port 8080 can forward to your STAGE server (its port 80).. Then all you do is have apache do URL rewriting on your STAGE server and append 8080 to all the hostnames it sees. Now all your posts and links will go to the correct STAGE servers, and your apache configs can be exactly the same also.
Regarding the hard coded (or user-inputted) domain names: you could add the domains to your hosts file. That should solve your problems during development and preview. Your browser will retrieve the IP for www.mysite.com and find 127.0.0.1 or the IP of the preview site in the hosts file. The tricky part is that just by looking at the URL in the browser, you cannot determine whether you are looking at the production site or not. (The ShowIP addon for Firefox can help you here.)
Regarding CVS/SVN: I would really advice you to go for SVN. It's not harder to use than CSV but has some advantages (e.g. renaming is possible). For more information see e.g. this question.
As for the previewing in a sandbox, this is what we do: we do most of our development on trunk (or on a branch, but the rest of the process is almost the same). Once we are ready to show it to the customer, we create a tag. This tag is used to update the preview server. If the customer isn't satisfied, we develop some more on trunk (or the branch), create a new tag, update preview with the tag, etc. Once the customer is happy we use the exact same tag running on preview to also update the production server. This way we can be sure that the preview and production server have the same code base.