I am a lone developer most of my time, working on a number of big, mainly PHP-based projects. I want to professionalize and automate how changes to the code base are handled, and create a Continuous Integration process that makes the transition to work in a team possible without having to make fundamental changes.
What I am doing right now is, I have a local test environment for every project; I use SVN for each project; changes are tested locally, and then transferred to the on-line version, usually via FTP. API documentation is generated manually from the source code; Unit tests are something I am getting into slowly, and it's not yet part of my daily routine.
The "build cycle" I am envisioning would do the following:
A changeset gets checked into SVN after having been tested locally.
I start the build process. The SVN HEAD revision gets checked out, modified if necessary, and made ready for upload.
API Documentation gets generated automatically - if I haven't set it up in detail yet, using a default template, scanning the whole code base.
The new revision is deployed to the remote location via FTP (Including some directory renaming, chmodding, importing databases, and the likes.) This is something I already like phing for very much, but I'm open for alternatives of course.
Unit tests residing in a predefined location are run. I am informed about their failure or success using E-Mail, RSS or (preferably) HTML output that I can grab and put into a web page.
(optionally) a end-user "changelog" text file in a pre-defined location gets updated with a pre-defined part of the commit message ("It is now possible to filter for both "foo" and "bar" at the same time). This message is not necessarily identical with the SVN commit message, which probably contains much more internal information.
Stuff like code metrics, code style checking and so on are not my primary focus right now, but on the long run, they certainly will. Solutions that bring this out-of-the-box are very kindly looked upon.
I am looking for
Feedback and experiences from people who are or were in a similar situation, and have successfully implemented a solution for this
Especially, good step-by-step tutorials and walkthroughs on how to set this up
Solutions that provide as much automation as possible, for example by creating a skeleton API, test cases and so on for each new project.
and also
Product recommendations. What I know so far is phing/ant for building, and phpUnderControl or Hudson for the reporting part. I like them all as far as I can see, but I have of course no detailed experience with them.
I am swamped with work, so I have a strong inclination towards simple solutions. On the other hand, if a feature is missing, I'll cry about it being too limited. :) Point-and-click solutions are welcome, too. I am also to commercial product recommendations that can work with PHP projects.
My setup
I am working on Windows locally (7, to be exact) and most client projects are run on a LAMP stack, often on shared hosting (= no remote SSH).
I am looking for solutions that I can run in my own environment. I am ready to set up a Linux VM for this, no problem. Hosted solutions are interesting for me only if they provide all of the aspects described, or are flexible enough to interact with the other parts of the process.
Bounty
I am accepting the answer that I feel will give me the most mileage. There is a lot of excellent input here, I wish I could accept more than one answer. Thanks everyone!
I've been through buildbot, CruiseControl.net, CruiseControl and Hudson. All though I really liked CruiseControl*, it was just too much of a hassle with really complex dependency cases. buildbot is not easy to set up, but it's got a nice aura (I just like python, that's all). But hudson won over the former three because:
It's just easy to set up
It's easy to customize
It looks good and got nice overview functionality
It got point-and-click updates, for itself and all installed plugins. This is a really nice feature, that I appreciate more and more
Caveat: I only ever used linux as base for the above mentioned build servers (CC.net ran on mono), but they should all - according to the docs - run cross-platform.
Setting up a hudson server
Prerequisites:
Java (1.5 will serve you just fine)
Read access to the subversion server (I have a separate account for the hudson user)
From here, it's just:
java -jar hudson.war
This will run a small server instance right off your console, and you should be able to browse the installation at your http://localhost:8080, if you don't have anything else running on that port in advance (you can specify another port by passing the --httpPort=ANOTHER_HTTP_PORT option to the above command) and everything went well in the 'installation' process.
If you go to the available plugins directory (http://localhost:8080/pluginManager/available), you'll find plugins for supporting your above mentioned tasks (subversion support is installed per default).
If that has whet you appetite, you should install a java application server, such as tomcat or jetty. Installation instructions are available for all major application servers
Update: Kohsuke Kawaguchi has constructed a windows service installer for hudson
Setting up a project in hudson
The links in the following walk-through assumes a running instance of hudson located at http://localhost:8080
Select new Job (http://localhost:8080/view/All/newJob) from the menu on the left
Give the job a name and tick Build a free-style software project on the list
Pressing 'ok' will take you to the configuration page of the job. All the options have a little question mark besides them. Pressing this will bring up a help text regarding the option.
Under the option group 'Source Code Management' you would be using Subversion. Hudson accepts both url access as well as local module access
Under the option group 'Build Triggers', you would use 'Poll SCM'. The syntax used here is that of cron, so polling the subversion repository every 5 minutes would be */5 * * * *
The process of building the project is specified under the option group 'Build'. If you already have an ant build file with all the targets you need, you're in luck. Just choose 'Invoke ant' and write the name of the target. The option group supports maven and shell commands as well out of the box, but there is also a plugin available for phing.
Tick off additional build actions in 'Post Build Actions', such as e-mail notifications or archiving of build artefacts.
For setting up processes for which hudson have no plugins, you can either call them directly through a shell script from within the build setup, or you could write you own plugin
Pitfalls:
If you have it produce build artefacts, remember to have hudson clean up after itself in regular intervals.
If you have more than 20 projects set up, consider not displaying their build status as the default main page on hudson
Good luck!
The term you are looking for is "continous integration."
Here is an example of someone who uses GIT + phpundercontrol: http://maff.ailoo.net/2009/09/continuous-integration-phpundercontrol-git/
CruiseControl (which is a CI server), can use Hosted SVN/GIT as a source. So you can even use it with GitHub or Beanstalk or something else.
Then you can integrate that with the following kind of software:
PHPUnit
php-codesniffer
phpdocumentor
PHP Gcov
PHPXref
Yasca
etc.
You could also try this hosted CI: http://www.php-ci.net/hosting/create-project
Keep in mind though, that those tools need custom support if you integrate them yourself.
Have you also thought about project management and patch management?
You can use Redmine for project management. It has integrated continuous integration support, but only as client side (not as CI server).
Try using a hosted SVN/GIT/etc. solution, because they will cover your backups and keep their servers running, so you can focus on development.
For a tutorial on how to setup Hudson, see: http://toptopic.wordpress.com/2009/02/26/php-and-hudson/
I use Atlassian's Bamboo continous integration server for my main PHP project (along with their other products such as fisheye (repository browsing), jira (issue tracker) and clover (code coverage)).
It supports SVN and now supports Git and it has a great user interface. It is available for linux, windows and mac and can run standalone on its own tomcat server which is great for people (like me) who does not like to take days to setup their tools). Although it may look expensive, being a lone developer myself I purchased the starter kit license for 10$ (10$ by software). This is great for small teams and it is worth the look.
PHPTesting PHPCI This is nice, continuous integration server built in php.
Plus, its free and open source. :)
it has number of plugins..
PHPCI includes integration plugins for:
Atoum
Behat
Campfire
Codeception
Composer
Email
Grunt
IRC
PHP
Lint
MySQL
PDepend
PostgreSQL
PHP Code Sniffer
PHP Copy/Paste Detector
PHP Spec
PHP Unit
Shell Commands
Tar / Zip
I am mostly a sys admin but sometimes I code PHP as well. As a side project I created some scripts that will make it simple and painless to set up a full blown PHP CI environment using Jenkins. It also runs a sample project for you so you can see how each build step is configured.
If you want to try it out all you need is a Debian/Ubuntu box and shell access.
http://yauh.de/articles/379/setting-up-a-ci-environment-for-php-projects-using-jenkins-ci
Update To add some content to my answer:
You can simply set up a Jenkins CI for PHP using Ansible. Since v1.4 it supports roles which you can download from their galaxy.ansibleworks.com community site and it will do the heavy lifting for you. It is called jenkins-php.
I would suggest using Jenkins http://jenkins-ci.org/ it's free and it's open source.
It's pretty straight forward to setup, works on multiple platforms and integrates well with other continuous integration tools like SonarQube (+ SQUALE) to measure technical debt and Thucydides for testing automation.
I would highly suggest using GIT or GIT Hub for version control instead of SVN. From my point of view it's just a better version control system that will help you scale your development efforts later.
Since you're working mostly with PHP project there are some other tools you can use.
PHPUnit - For unit testing
PHP CodeSniffer - Check for coding standards
PHP Depend - Shows your PHP code dependencies
XDEBUG - For performance testing
All of these tools and be triggered with a Jenkins job and helps with the quality and performance of your code.
Good luck and Enjoy!
I do not use many of the products, or even types of products that you use, but I will give you my experience.
I run a TEST environment in parrallel with my PROD environment. I have no local testing per se. If it is too hard to get soemthing up into a real TEST environment, then I fix my build process. I don't see the point in testing locally, as the environments are different. UPDATE: The only thing I do locally is run "php -l" before I upload anything. Stops the stupid mistakes.
The build process works with whatever is in the current workspace, which includes uncommitted code. This is not everyone's cup of tea, but I am going to TEST very often. Everything gets committed before going to PROD.
Part of my build process (similar to yours) creates two META files. One contains the last (typically) 100 changes and also gives me the current changelist number. The shows me what changes are installed. The other contains the CLIENTSPEC (in Perforce terms) which shows me exactly what branches were used in this build. Together these give me reproducible builds.
I do not build straight to the target environment, but to a staging area on the server. I use SSH so this makes sense. This gives me a few advantages. Most importantly it avoids dying half way through a large upload. It also gives me a place to store META files, and all the build files are automatically archived (so I can go straight back to any build). The script also logs the update (so there is an entry in the log stream and I can see pre- and post-) and kicks all daemons (I use daemontools so "svc -t"). All of these are better off on the target machine.
One other issue is DB changes. I keep a master script of the DB schema, which I update every time the schema changes. Each of the changes also go into a changes.sql script, which is uploaded with the build to the staging area. The script is run as part of the install script.
I've recently begun the same kind of process, and am using Beanstalk for svn hosting.
There are two nifty features in the paid accounts (start at $15pm i think):
deployment allows the user to create ftp targets for staging and production servers, which can be deployed at the click of a button (inc specifying a revision and branch)
webhooks allow the user to set up a url that is called on each commit/deploy, passing across things like revision number, description and user. This could be used to update docs, run unit tests and update changelogs.
I'm sure there are other hosted or self-hosting svn servers with these two features, but beanstalk is the one i have experience of and it's working very, very well
There's also an API, which I imagine could be used to integrate deployment further in to your process.
Consider fazend.com, a free hosted CI platform, which automates configuration and installation procedures. You don't need to setup version control, bug tracking, CI server, test environment, etc. Everything is done on-demand.
Related
I've been trying to learn more about PHP and more server-side code in general, and I hit a point where I realized that if I want to upgrade my website, I would need to take it down and copy the code while the users were online.
Furthermore, since I haven't launched my website yet, and unindexed by Google, it rarely gets any visits, so I'm free to upload and test out whatever I want using PHPStorm + FTP. But I realize that once I finish my project, and have users, I wouldn't want to change things while they were using them.
How do people write code and debug, and setup before deployment to verify that their website would function fine? Also, how would you copy over the code? Both from a large website (i.e. apple.com, cnn.com - websites that need to be on 24/7) vs. smaller blogs/websites.
You're asking about Continuous Integration / Continuous Deployment ('CI/CD').
Basically you'll want to make use of a testing server to test your code on before the public get to see it. You deploy out your code to this server and test it thoroughly, confirming that everything is ready to go out to production. With more in-depth setups this testing can be automated with tools like Selenium or TestComplete.
In theory, you should be deploying a 'release cut' of a GitFlow workflow out to your testing environment. This ensures that all planned changes have been finalised and are ready to ultimately be shipped out to the public without any other changes getting mixed up in the release.
When you're finally ready to ship out to production, you'll want to plan the release, and meet with any associated parties. I'd recommend a proper release checklist, where you confirm that all of the additional features are working as expected, and that you're not accidentally removing any existing functionality (regression testing).
Note that a deployment out to production should have absolutely no differences to a deployment out to a test environment; the only thing that should be changing on the production environment is the server in which the environment is hosted, along with database configurations. This ensures that you're not accidentally testing / using any data that your customers might also interact with.
The production deployment itself should (essentially) just be a 'copy-paste' of the files to disk, and is often done with a Continuous Integration tool like Jenkins or TeamCity. Assuming you only have a small website, this should be an almost instantaneous procedure, and shouldn't even require any downtime on your production environment. If the process is likely to take a while longer (such as with a complex deployment), you may want to implement a maintenance page. This will inform your users that you're working on the website, and let them know when they can expect functionality to resume.
I'm a developer for an e-commerce website, when we were smaller (50-75 users online at a time) I would just use FTP eg, FileZilla to update the files I had changed with my latest edit, this would mean people would be served my changes and the website would not go down at all. For small edits, I would also just ssh in and manually edit the file if it needed to be done quickly.
Now we have grown much larger and have other developers (100-200 users at a time) to be fair I should be doing it this way before but I deploy directly from PHP storm/version control. I have working branches and once my work is ready to be deployed ill put it into the master branch.
Once the master branch has work pushed into it I have a script that checks for changes and clones the repo. (this happens automaticaly.)
There is a lot to cover here, and some of it is opinion based (which I will try to avoid).
How do people write code and debug, and setup before deployment to verify that their website would function fine
There are many ways to create a local development site, but most modern approaches involve containers. Docker is the most popular, which allows you to create replicas of a web server, database, etc. and run them locally on your machine. You can also use virtual machines to create a replica of your production server, or simply install the needed packages (i.e. php, apache, mysql) on your local machine.
Larger companies will typically have a QA (Quality Assurance) department who test all changes before they are deployed. As others have mentioned, some of this can be automated with testing software. When there is a QA team, you have a staging server where changes are first deployed. Once the changes pass QA and any automated tests (i.e. PHPUnit) they are deployed to production.
Also, how would you copy over the code?
This can get very opinionated, so I will just give you different options and not argue for or against any of them:
Use a service like rsync to transfer the files. With this approach, you typically have a staging server where you checkout your code from version control (or manually copy it if you aren't using version control), run any build scripts such as composer, gulp, etc., and then sync the files to production. In a larger setup, you wrap all of this into a script to automate it.
If you are using version control (let's use Git as an example), you can also pull the code down to your production server (as apposed to pushing it up to production), by running git pull on your production server. This will get all the latest changes, then you run any build scripts such as composer, gulp, etc.
There are also tools for continue integration (i.e. Jenkins) that facilitate a lot of the work mentioned above. You can go as far as setting up hooks on GitHub, so your code is automatically built and deployed when you merge into master.
On a small or personal project, it might be simpler to just use FTP and manually transfer the files.
In the past, I have been developing in a very amateurish fashion, meaning I had a local machine where I developed and tested code and a production machine to which I copied the code when I was done. Recently I modified this slightly to where I developed locally, checked the code into SVN and then updated the production machine through SVN.
Now I would like to start a new project and improve my workflow. Ideally I had the following in mind:
Have one or more local dev environments
Develop and test on local machine(s)
Use SVN (or Git) as code repository
Use a build tool to set up new environments (either dev, staging or production) and deploy code
Since I am not very familiar with this process, I am looking for suggestions on how to best set this idea up and the tools to use, especially when it comes to the build tools. I was looking into Ant and Phing (possibly make), but I am so new to this that I would really like to get some guidance. Are there any good tutorials or books about PHP deployment, especially for beginners? What I am especially interested in are the following topics:
Deployment to different types of servers with different settings (e.g. dev uses different db, db passwords, PHP error reporting than production or staging).
Deployment that automatically pulls code from SVN.
Deployment that temporarily sets a "Maintenance" page for production environment.
Once I mastered the above, maybe even do some testing in the build process.
I know my question might sound quite confused... I admit, I am new to this and might be a little off the target in what I really need. That's why any help is greatly appreciated.
I would suggest making your testing deployment strategy a production-ready install-script -- since you're going to need one of those anyway eventually.
A few tips that may seem obvious to some, but are worth pointing out:
Your config file saved in your VCS should be a template, and should be named differently from the file that will eventually contain the actual settings. E.g. config-dist.php or config-sample.conf or sample/config-mysql.php or something along those lines. Otherwise you will end up accidentally checking in a server-specific configuration file over your template.
For PHP deployment, anticipate that some users will not be able to run server-side scripts through any mechanism other than the web server itself. A PHP-based installer is almost non-negotiable.
You should include a consumer-friendly update mechanism, and for that, wordpress is a great example of a project to emulate. A PHP script can (a) download the latest build, (b) use the ftp functions to update your application's files, and (c) execute an update script which makes the appropriate changes to the database, etc.
For heaven's sake don't do like [redacted] and make your users download and install separate patches for each point release. Have them download the latest (final) release which contains all the updates to date, and applies the correct ALTER TABLE functions in sequence.
Whether the files are deployed via SVN or through FTP, the install/update mechanism should be the same: get the latest files, run the update script. The updater uses the version listed in the PHP script and the version listed in the DB, and uses that knowledge to apply the appropriate DB patches in order. As for how to generate those patches, there are other questions here that you can refer to for more info.
As for the "Maintenance" page, just use the version trick mentioned above to trigger it (compare the version in the DB against the version in the PHP code). It's also useful to be able to mark a site as "down" to the public but make it visible to admins (like Joomla does), which you can trigger through database or filesystem flags.
As for automatically pulling code from SVN, I'd say you're better off with either a cron script or with commit triggers than working that into your application, since it wouldn't be relevant to end users.
This isn't exactly part of your question, but it's relevant:
If you go into distributing code intended for a wide audience, I would advise you to go with building and distributing OpenSSL-signed PHAR packages. You can distribute them over HTTP without a problem, and because they're OpenSSL-signed, you're also mitigating the risk of man-in-the-middle attacks and protecting end-users/customers/clients from someone injecting code if you want to setup an automatic or one-click update.
There's a set of tools I've contributed to in the past that work great for this, but you'll either need PHP 5.3, or you'll need PHP 5.2 with PHAR installed via PECL. https://github.com/koto/phar-util
As far as testing goes, PHPUnit is the de facto standard.
If you are interested in using Git then you should check out this build system from CodeMeme. From what you described it sounds like it would be a good fit. You can add it to any project as a submodule and with the included code you can tailor a build script that will deploy to different multiple servers in multiple environments. It uses Git to build the code for deployment but unfortunately SVN is not supported.
https://github.com/CodeMeme/Phingistrano
I would like to have some input on how a professional development setup with the following requirements might look like.
several PHP-developers (say PHP)
each developer belongs to one group
each group has one team-leader who delegates tasks
each developer works on one Windows 7 machine
and developes either with NetBeans or Eclipse
each developer 'owns' one virtual test-server where he can run the code
the VCS in use is SVN
there is a staging server where the product is ultimately tested before it gets released/deployed
I gave some specific technology to not be too abstract and b/c I also would be interested in concrete suggestions for plug-ins etc.
There are several questions coming to my mind in that setup.
1) So every developer will work on
personal branch.
2) This branch is checked out in a working copy.
Now ... this working copy is edited locally on the PC with the dev's IDE and executed/tested on the server.
What would be in that case the best/usual way to do that? I mean - how do you get your edited code on the server without causing too much overhead?
Would the dev have the code on his local disk at all? Or would it be better to have the IDE write on the remote virtual server through a tunnel or via a specific protocol?
3) Every day a dev will commit his work into his personal branch which resides in a central repository.
Is there a best practice on where the repository is supposed to be located? A seperate server?
4) Then after a dev finished his task either s/he or the team-leader merges the new code into the respective main-branch or trunk.
The most confusing part is about what I wrote between 2) and 3). Because so far I only worked with a local server. For example a VM with a server running a code which is located in a shared folder so I will be able to edit it directly. I'm not sure how to bridge the gap efficiently when the server is now actually remote. Efficiently means not having to upload manually via FTP for example.
Also external sources or book recommendations are welcome.
edit
My question/s is/are aiming at a quasi-standard / best-practice. I think this is pretty much a standard development scenario so there must be a 'usual' solution.
edit 2
Okay ... so lets try with a picture:
V is the virtual test-server for one or more developers D. C and C' are the two code-versions. They should be kept as identical as possible.
Two solutions come to my mind:
1 : Edit C, then upload it to C', then execute C', then commit C.
2 : No C existant. Just C' which is edited through some tunnel technology and executed and committed.
My guts tell me both solutions are semi-optimal. So what would be "professional" / most efficient / fastest / most convenient / most friction-less / least error-prone / best practice / industry standard?
Any questions?
Maybe its not of great help but GIT sounds like a perfect fit to your problems, i recommend to take a look to the GIT features. And if you got time check Linus Torvalds him self talking ablout GIT. http://www.youtube.com/watch?v=4XpnKHJAok8
The standard procedure as you describe is more or less the same. I also you this approach for my team. It can also be called staged application development.
Here is how I am doing it, I use a remote SVN host (ex: assembla.com, unfuddle.com) to store all my codes. My team members store the information there on these remote svn servers. You can also buy an VPS and setup SVN there and user the same approach.
Best practices is to test locally and commit and commit as many times as you can but every commit must solve a problem or include a significant segment that adds any new feature.
Once the commit is done by everyone the lead developer then can login to the staging server via SSH using tools like PuTTY. First time the lead developer has to checkout the code into the folder where the codes are to be located. During this phase file conflict may arise if multiple developers edits same segment of a file. The lead developer should then resolve the code first and then proceed with the checkout. Once checked out, there onwards the lead developer will only need to do a svn update on the staging server to make the code up to date.
Basic idea is to get the code working on local setup then commit and update the staging for testing the application on a simulated scenario and then commit it to the live site.
There are a lot of if's and but's here which will need me to write a chapter on :) but in short this is the zest.
Tools (you can use to work under this setup):
- Tortoise SVN Manager
- PuTTy
- NetBeans
hope it helps :)
I don't like working with personal branches. I worked with ClearCase for almost 15 years and even though ClearCase probably handles personal branching better than most, it was still a big pain. Even worse, personal branches encourages people to not commit their work until the last minute -- usually a day or two before a major release.
For that reason, and to force developers to stay on track with each other, I highly recommend everyone working together on a single branch (or on the trunk) as much as possible. I keep telling developers to take small bites when they make changes.
What you sound like you need is a way to automate the deployment. That is, I make changes on my local machine, and with a single command, I make sure that the server has a duplicate copy of the code. You also want the deployment to be efficient. If you change a single 2 kilobyte file of a 2 gigabyte, 10,000 file deployment, you only want to copy over that one file, not 10,000 gigabytes. For that, I would recommend you write a deployment script in Ant.
Your developers can modify files, then deploy those files via an Ant script. The developers don't have to remember what files they had updated because Ant will automatically handle that. In fact, Ant can even modify files to make sure they contain the right environment information as they get copied over. And, of course, Ant can rearrange the files if the setup on the server is different from the setup in the source repository. And both Netbeans and Eclipse can execute Ant scripts right in the IDE.
So:
Have your developers modify code on their local machine.
Run an Ant script to make sure the server and the local machine are in sync.
Test on the server.
Then, check in their changes once they're happy with the results on the server.
Someone mentioned a Continuous Build System like Jenkins. That actually would be a good idea anyway even though it doesn't solve this particular issue. Jenkins could have its own server and database. Then when you commit your code, Jenkins would update the server and run automated tests. Jenkins can then create a report. It all gets displayed on Jenkin's webpage. Plus, you can archive your deployments on Jenkins, so if you tell someone to test "Build #20", they can simply pull it off of Jenkins where its easy to find.
I'm sure everyone has different ways of doing things but here are my thoughts.
"Best Practice" is probably "Continous Integration" ie each developer doesn't have their own branch but checks in to a common development branch. This forces them to handle conflicts and coordinate with each other early and often to avoid the lead developer from managing a huge train wreck merge later down the road. Take a look at cruisecontrol if you really want to go that route.
The best way is if they have a local apache web server and full php stack. You can use the Zend_Server community edition to get up and running on windows fast. Most standard php code will run just fine on both Windows and Linux, but if you are doing lots of file manipulation or cron job or cli stuff, or need memecache, etc you'll run into incompatabilities. If thats the case and the Linux only stuff is going to bite you use VMWARE or VirtualBox to run local linux instances and install the IDE inside those and just make sure they have goobs of RAM to deal with it.
Each developer needs to run a syncronize inside of Eclipse, basically an svn update, deal with any conflicts with the other developers right then and there, do local testing and commit their changes.
I setup a post_commit hook on the svn server that calls and /autobuild.php on my web server. autobuild.php runs svn update and gets the latest code changes as well as does any chown or chmod file permissions stuff and resets any server specific config files config.php. Its a little tricky to get it setup so that the apache user can run svn update, but once you do your beta/testing server always has the latest committed code. CruseControl, and several others can also help you do this sort of thing and add unit testing, etc
Now your Lead Developer still has a job to do merging the Development Branch into the Production One, testing on the dev server, and reviewing the commits of the others and deciding how and when to push out a release, but your not putting the burden on him of resolving every conflict and merging every change.
Your developers are not ftping files or ssh remoting into servers, they just work locally in their IDE and interact with each other through svn (and email, phone, chat, etc) updating to get the new code and commiting as they finish things.
I don't see any good coming out of having a seperate branch for each developer using SVN. Merging those branches might work in Git but with SVN your lead developer will be hating life very quickly with that type of setup.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm beginning a new project in PHP and I'd love to get some feedback from other developers on their preferred strategy for PHP deployment. I'd love to automate things a bit so that once changes are committed they can be quickly migrated to a development or production server.
I have experience with deployments using Capistrano with Ruby as well as some basic shell scripting.
Before I dive head first on my own it would be great to hear how others have approached this in their projects.
Further information
Currently developers work on local installations of the site and commit changes to a subversion repository. Initial deployments are made by exporting a tagged release from svn and uploading that to the server.
Additional changes are typically made piecemeal by manually uploading changed files.
For PHP, SVN with Phing build scripts are the way to go. Phing is similar to ANT but is written in PHP, which makes it much easier for PHP developers to modify for their needs.
Our deployment routine is as follows:
Everyone develops on the same local server at work, every developer has a checkout on his machine back home as well.
Commits trigger a post-commit hook which updates a staging server.
Tests are ran on staging server, if they pass - continue.
Phing build script is ran:
Takes down production server, switching the domain to an "Under construction" page
Runs SVN update on production checkout
Runs schema deltas script
Runs tests
If tests fail - run rollback script
If tests pass, server routes back to production checkout
There's also phpUnderControl, which is a Continuous Integration server. I didn't find it very useful for web projects to be honest.
I'm currently deploying PHP using Git. A simple git push production is all that's needed to update my production server with the latest copy from Git. It's easy and fast because Git's smart enough to only send the diffs and not the whole project over again. It also helps keep a redundant copy of the repository on the web server in case of hardware failure on my end (though I also push to GitHub to be safe).
We use Webistrano, a web frontend for Capistrano, and are very happy with it.
Webistrano allows multi-stage, multi-environment deployments from SVN, GIT and others. It has built-in rollback support, support for separate server roles such as web, db, app, etc., and deploys in parallel. It allows you to override config parameters on multiple levels, such as per stage, and logs the results of every deploy, optionally mailing it.
Even though Capistrano and Webistrano are Ruby applications, the syntax of the deployment 'recipes' is easy and powerful enough to understand for any PHP programmer. Originally Capistrano was built for Ruby on Rails projects, but easily accommodates PHP projects.
Once configured it is even easy enough to be used by non-programmers, such as testers deploying a staging version.
To provide the fastest deploy possible we installed the fast_remote_cache method, which updates a svn working-copy cache on the remote server, and then hardlinks the result.
I use Apache Ant to deploy to different targets (dev, QA and live). Ant is designed to work for Java deployment, but it provides a pretty useful general case solution for deploying arbitrary files.
The syntax of the build.xml file is pretty easy to learn - you define different targets and their dependencies which run when you call the ant program on the command line.
For example, I have targets for dev, QA and live, each of which depends on the cvsbuild target which checks out the latest head revision from our CVS server, copies the appropriate files to the build directory (using the fileset tag), and then rsyncs the build directory to the appropriate server. There are a few quirks to learn, and the learning curve is not totally flat, but I've been doing it this way for years with no trouble so I'd recommend it for your situation, though I'm curious what other answers I'll see on this thread.
I do stuff manually using Git. One repository for development, which gets git push --mirror'ed to a public repo, and the live server is a third repo pulled from that. This part I suppose is the same as your own setup.
The big difference is that I use branches for nearly every change I'm working on (I've got about 5 right now), and tend to flip back and forth between them. The master branch doesn't get changed directly except for merging other branches.
I run the live server direct from the master branch, and when I'm finished with another branch and ready to merge it, flip the server to that branch for a while. If it breaks, putting it back to master takes seconds. If it works, it gets merged into master and the live code gets updated. I suppose an analogy of this in SVN would be having two working copies and pointing to the live one via a symlink.
I know Phing has been mentioned a few times now, but I've had great luck with phpUnderControl. For us we
Check out individual copies of branches to local machines
Branches are tested and then merged into Trunk
Commits to Trunk are automatically built by phpUnderControl, runs tests and builds all documentation, applies database deltas
Trunk gets run through quality testing and then merged into our Stable branch
Again, phpUnderControl automatically builds Stable, runs tests, and generates documenation and updates database
When we're ready to push to production we run a rsync script that backs up Production, updates the database, and then pushes the files up. The rsync command is invoked by hand so that we make sure someone is watching the promotion.
an alternative to home-made deployment scripts is to deploy to a platform-as-a-service which abstracts away a lot of that work for you. A PaaS will typically offer its own code deployment tool, as well as scaling, fault-tolerance (eg. not going down when hardware fails), and usually a great toolkit for monitoring, log checking etc. There's also the benefit of deploying to a known good configuration which will be kept up-to-date over time (one less headache for you).
The PaaS I would recommend is dotCloud, in addition to PHP (see their PHP quickstart) it can also deploy MySQL, MongoDB and a whole bunch of additional services. It also has nice goodies like zero-downtime deployment, instant rollback, full support for SSL and websocket, etc. And there's a free tier which is always nice :)
Of course I'm slightly biased since I work there! Other options worth checking out in addition to dotCloud are Pagodabox and Orchestra (now part of Engine Yard).
Hope this helps!
Solomon
That you automatically and blindly take changes from a repository to production servers sounds dangerous. What if your committed code contains a regression bug, so your production application gets glitchy?
But, if you want a Continuous Integration system for PHP, I guess Phing is the best choice for PHP. I haven't tested it myself, though, as I do stuff the manual way of e.g. scp.
I am way late to the party, but I thought I would share our methods. We use Phing with Phingistrano, which provides Capistrano-like functionality to Phing via pre-built build files. It is very cool, but only works if you use Git at the moment.
I have a working copy of an SVN release branch on the server. Updating the site (when there aren't schema changes) is as easy as issuing an SVN update command. I don't even have to take the site offline.
Phing is probably your best bet, if you can stand the pain of xml configuration files. The Symfony framework has its own port of rake (pake), which works quite well, but is rather tightly coupled to the rest of Symfony (Though you could probably separate them).
Another option is to use Capistrano. Obviously it doesn't integrate as well with PHP, as it does with Ruby, but you can still use it for a lot of stuff.
Lastly, you can always write shell scripts. So far, that's what I have done.
http://controltier.org/wiki/Main_Page
we are going to use it for multi-server deployments & maintenance.
One year late but...
In my case, deployment is not automatic. I find it dangerous to deploy code and run database-migration scripts automatically.
Instead, subversion hooks are used to deploy only to testing/staging server. Code is deployed to production at the end of an iteration, after having run tests and made sure things will work. For the deployment itself, I use a custom-made Makefile that uses rsync for transferring files. The Makefile may also run the migration scripts on the remote server, pause/resume web and database servers.
At my work myself and my team have developed a Phing oriented replacement for capistrano's deploy and we've also incorporated some of the goodies available in phing like PHPUnit testing, phpcs and PHPDocumentor. We've made it a git repo that can be added to a project as a submodule in git and it works very well. I've attached it to a handful of projects and it's modular enough that it's easy to make it work with any project on any of our several environments (staging, testing, production, etc...).
With the phing build scripts you can run them from the command line manually, and I've also had success automating the build/deploy routines with Hudson and now Jenkins ci.
I can't post any links now because the repo isn't public yet, but I've been told we're going to open source it sometimes soon, so please feel free to contact me if you're interested or if you have any questions on automating your deployment with phing and git.
I guess SVN deploy way is not very good. Because:
You need to open the SVN access for the whole world
have many .svn in the production web servers
I think Phing to produce a branch + combine all the js/css + replace stage config + ssh upload to all www servers is better way.
ssh to 10 www server and svn up is also trouble.
I would like to setup two environments for my new website written in PHP. One - to develop new versions and test them. And second production where my actual stable version of website will be available.
Website in PHP will consist of many PHP and other files (JS, images, and so on). So I think how to prepare this environment in best way to make it easy to do source control, fast copy website from development environment to production environment and to make development version available for people on the web so they will be able to see actual work progress and suggest changes or report bugs.
Could you please give me some advice where to go from this starting point? Are there books about this (from practical point of view?) or do you have experience or tips what to watch out and what is important to make this process easy and good for me and other people involved in developing new project?
For starters use the following three:
SVN - this will give you source control and allow you to track changes. You may want to get GUIs on top of this (Tortoise is a popular one) to ease the learning curve.
RSYNC - this will allow you to streamline your syncing between local and remote site with a single command. RSYNC uses a diff engine to sync which means that incremental syncs happen in a matter of seconds. During intense programming, I will sometimes sync 4-5 times in one hour pushing out little changes real fast just because I can so easily.
MySQLDump - This will allow you to import/export data from your production site. I usually do this once a week to get production data on my local servers which not only gives me a local backup but also lets me toy around with production data on a local test environment.
Those three alone will save you a lot of time in the long run and allow you to scale. Later on you can look into an automated build tools, unit testing frameworks, xml documentation framework and the like to build some serious products.
I work with a setup like this, so I can give you some tips on how to do this. I've been doing this for a while now, working out the kinks here and there, and feel like this is a setup I can honestly say is pretty darn productive.
Small note: I work on OSX, so the specific applications used might be a bit different from you if you're a linux/windows user.
I run a production 'server' on my Mac, using MAMP (www.mamp.info) to easily supply me with an Apache server with PHP and MySQL. You could use a similar tool such as XAMPP or install everything manually, it's really up to you.
Then I have my live servers, where my websites and customer websites are hosted. For each new website project (let's take abc.com as an example) I create a subdomain called staging.abc.com, on which I do my testing. It's always a good thing to test things on the exact same hard- and software before actually going live.
I use Subversion (or in short, SVN) for my versioning needs, with the added bonus that I can easily add 'hooks' to automatically update my online production server whenever I send my newly updated version to the SVN server. SVN also allows you to easily work with more than one person on the same project. For more information on SVN and how to use it, I suggest the great (and free) online book found here: http://svnbook.red-bean.com/
So in short: I work locally with MAMP providing me with a local 'working' server. After that, I test online on a staging.abc.com location to see if everything works well, and to possibly allow others to see the project (in case you want your client to see what's going on, for example), and after that I actually publish the project by putting it on the actual domain.
There are many more things that can be done to optimize your workflow, but this should get you started.
Hope this helps!
-Dave
I prefer to have development occur on the developers local box if possible. If other developers are involved, you probably want to setup your version control such that both the database schema, javascript, css, and the php code can be checked out and setup on a developer's personal box pretty easily (assuming they have the correct LAMP/WAMP setup)
I've also seen it where people maintain a test website on a server where active development occurs. I would avoid this for active development, but use this for black-box testing of the latest checked in code (the latest build).
Once your test website checks out, then its a matter of exporting the code from your version control to the location the live website is. With svn, you can really just do an update of the live code with svn update specifying a revision or tag that indicates the current live version.
I would further recommend keeping some settings, like db access/username/pass, in a separate included file that is not version controlled. Keep this elsewhere, let developers plug in the access rights to their local database on their PC. On your server, plug in everything you need to access the database there. This should be really trivial code (defining a few variables) so not having it version controlled shouldn't be a big deal. If you like, you could version control a templated version, but I wouldn't put the real database info into version control.
Here is a pretty good starting place if you want to use MAMP or WAMP to develop locally and then push that to github and then update your live site from github:
http://www.mybringback.com/bringers/14509/git-local-and-shared-server-development-environment-with-ssh-setup/
Hope that helps.