I have a application in php. I use a lots of javascript, css, fonts and other thing.
Everything is versionned with git, and I update my servers with git. My server are on the master branch, and I push everything to it to update the code on my server.
So, now I would like to upgrade my code and start using gulp with it to automatically compile javascript, use sass and generate fonts.
I do not wish to install gulp on my server in case something goes wrong, my website will be unoperationnal. So I have to be able to create a build version with the compiled files of javascript, css... But don't want to always commit thoses files since it will cause conflict in my team everytime we change version.
What would be the best workflow to create a build version with all the compiled files and not cause any conflict in git?
Thank you
This is my setup:
Notice the task deploy? Well, deploy with the parameter --release, releases to a different location that just plain deploy.
It should be as simple as deploying to different locations. In your case to different hosted folders from which you can access two different web apps. One that works, and one in development.
To your statement:
I do not wish to install gulp on my server in case something goes
wrong,
You don't need gulp on your server at all whatsoever. I have gulp on my build Teamcity server, but that's about it.
Anyhow, that's what works for me. I hope it helps.
Related
In my project the deployable version needs to have a copy of each of the external libs, a different config file and install and setup files, for security concerns, the main project is set to refuse to run if they are present. Thus the upstream copies of the other projects need to be committed to repo. How can I work on code running on localhost where the file layout and sometimes file contents from dev and testing are different to what I need to commit?
Background
I am working on a project on hosted on github and my main IDE is netbeans which has imperfect git support (good enough for >99% of my needs). The project is in PHP and uses several other projects as libraries.
As Netbeans does not have the best support for sub-repos I have chosen to keep each additional project in a separate project. This is fine as the central project looks at the config data for where to find these outside libs.
Half an answer
My instinct is to suppose that there will need to be some "build stage" prior to committing to the github repo but how on earth do I go about setting all that up?
I could write some sort of homebrew thing but then when I pull other people's contributions I would need to reverse the process unless we had a branch for builds and a branch for working copies which seems needlessly complex and could leave the dev(s) config data on public display (not to mention updates being a mess).
I have seen that others have wrestled with somewhat similar problems to no conclusion (at time of asking) (How to push and pull from github without sharing sensitive information? Smudge & clean?) so I am looking for anything that might help me come up with a solution
my main IDE is netbeans which has imperfect git support
Most devs just use the command line. I switch to the NetBeans conflict resolver occasionally, which is very good, but for normal stuff the console is usually faster.
My instinct is to suppose that there will need to be some "build stage" prior to committing to the github repo
... unless we had a branch for builds and a branch for working copies
No, there is only ever one repository. It is better to think of your repo as your code history, rather than your deployment state. Branches should just be for features or large changes, which merge into your mainline/master.
There are a good deal of options available to you when deploying. The first is Composer, which Mark points out: when deploying you issue an install or update command, which fetches the dependencies that satisfy your library requirements recursively. You can use Bower to do the same thing for your JavaScript dependencies.
Some deployment strategies prefer to build locally and then scp/rsync to a remote server. Composer and Bower are still probably a good idea, but you write a build script (using Ant or Phing, for example) to create a build copy in a local temporary folder, and then send it to the server. It is common here also to push it to a new release folder on the server, and then swap a symlink or Apache config file when it's ready to go live.
the deployable version needs to have a copy of each of the external libs, a different config file and install and setup files, for security concerns
Assuming this is a web project, have you tried adding your sensitive environment data to your Apache configuration file? This can be trivially read in PHP, and of course PHP does not care that this information is different according to whether you are developing, testing, demoing a branch or operating live.
Further reading: an excellent PHP deployment book, free of charge, that suggests Phing and Capistrano.
I'd like to run phpunit on an external server, the feed those results into a Jenkins plugin like the Clover PHP Coverage Report action.
The code base has some library dependancies that the owner of the Jenkins server does not want deployed.
Well with some tweaking I guess it is possible to make use of the monitoring external plugin. You will have to figure something out to get the files on the jenkins server.
An other option is, if the server owner is willing, to create a specific build agent for your needs. That doesn't affect his jobs and you can use your builds like you want.
Yes, library dependencies shouldn't be installed globally on a Jenkins server, but this is where Composer comes into play: You'd essentially install exactly the dependencies your current software needs locally in the workspace of your job.
Nobody should be bothered by this, because you could also include the needed library code there manually - or even worse, you could use the same file and class names and code something entirely different. All of this must not interfere with any other job running on that same server, and it doesn't.
You can't really avoid Composer, because PHPUnit will stop being distributed via PEAR, as well as some Symfony components that are being used. Better go to the Composer project page and learn how to use it. By the way: You can include the needed version of PHPUnit with Composer as well, so you need not rely on a centrally installed version (which is hard to update because there are so many jobs that would need updates then - too much work in one go).
Currently using LAMP stack for my web app. My dev and prod are in the same cloud instance. Now I am getting a new instance and would like to move the dev/test environment to the new instance, separating it from the prod environment.
It used to be a simple Phing script that would do a SVN export into the prod directory (pointed to by my vhost.conf). How do I make a good build process now with the environments separated?
Thinking of transferring the SVN repository to the dev server and then doing a ssh+svn push (is this possible with Phing?)
What's the best/common practice for this type of setup?
More Info:
I'm currently using CodeIgniter for MVC framework, Phing for automated builds for localhost deployment. The web app is also supported by a few CRON scripts written in Java.
Update:
Ended up using Phing + Jenkins. Working well so far!
We use Phing for doing deployments similar to what you have described. We also use Symfony framework for our projects (which is not so much important for this but Symfony supports the concept of different environments so it's a plus).
However we still need to produce different configuration files for database, front controllers etc.
So we ended up having a folder with build.properties that define configuration for different environments (and in our case also for different clients we ship our product to). This folder is linked to the file structure using svn externals (again not necessary).
The Phing build.xml file then accept a property file as a parameter on the command line, takes the values from it and produces all necessary configuration files, controllers and other environment specific files.
We store the configuration in template files and then use copy/filter feature in Phing to replace the placeholders in the templates with the specific values.
The whole task of configuring the given environment can then be as simple as something like this:
phing configure-environment -DpropertyFile=./build_properties/build.properties.prod
In your build file you check if the propertyFile property that specifies the properties file is defined and load the file using <property file="./build_properties/build.properties.prod" override="true" />. Then you just do any magic with the values as you need.
You can still use your svn checkout/update and put all the resulting configuration files into svn ignore (you will have them generated by phing). We actually use additional steps in Phing. Those steps in the end produce a Linux shell installation self-deploy package. This is produced automatically in Jenkins. We then send the package to our clients or the support team can grab the package from Jenkins and they can do the whole deployment just by executing it (we still prefer manual deployments to production servers) or Jenkins can deploy it automatically (for example to test servers).
I'll be happy to write more info if needed.
I recommend using Capistrano (looks like they haven't updated the docs since they moved the site) and railsless-deploy for doing deployment. Eventually, you are probably going to need to add more app boxes and run other tasks as part of your deployment so choosing a framework that will support this can save you a lot of time in the future. I have used capistrano for two PHP deployments (one small and one large) and although its not perfect, it works well. It also handles all of the code checkout / update, moving symlinks into place, and rolling back if something goes wrong.
Once you have capistrano configured, all you have to do is something like:
cap dev deploy
cap prod deploy
Another option that I have explored for doing this is fabric. Although I haven't used it, if I had to deploy a complex app again, I would consider it. The interface is simple and straightforward.
A third option you might take a look at thought its still in the early stages of development is gantry (pardon the self promoting). This is something I have been working on out of frustration with using capistrano to deploy a PHP application in an environment with a lot of moving pieces. Capistrano is great and works well for non PHP application deployments, but you still have to some poking around in the code to understand what is happening and tweak it to suit your needs. This is also why I suggest giving fabric a good look.
I use a similar config now. Lamp + SVN + codeigniter + prd and dev servers.
I run the svn repos on dev. I checkout the repos into the root folder of the dev domain. Then use a post-commit hook to update the root folder everytime any developer commits.
When we are happy and have fully tested the code I ssh into the prd server and rsync the dev root to the prd root.
Heres my solution for the different configs. Outside the root folder I have a config.ini file. I parse the file in my codeigniter constants.php script. This means that the prd and dev server can have separate settings without them ever being in the repos.
If you want help with post-commit, rsync and ini code let me know.
I want to use GIT as version control for the PHP/MySQL web app I'm developing. I'm going with the server style setup since a couple of developers will also be working on it. I have the sandbox/dev server running (Ubuntu 10.10 server, LAMP package, CodeIgniter 2.01 framework) and its web root is /data/www/webapp.
How do I correctly set up GIT with my environment? Essentially how do I get GIT's "push to" or "I'm up to date and ready to test" directory the same as the web root (so I can just hit refresh in the browser and see the changes results)?
I'm still learning how GIT works so please excuse any incorrect use of terms. Also please no comments about other VCS systems - I'm sticking with GIT for sure.
Thanks in advance
One possible solution is to use a hook in your repo that will react to either new changes or a tag being updated that will export the new content into the appropriate location for your webserver.
The use of hooks makes things very flexible. There are hook points for quite a few actions in a repository and they can do just about anything (if you write it).
I'm trying to use Phing to automate :
running tests
running DB migrations on each Developer machine [using dbdeply]
deployment to production when needed
I think it does make sense to add a build folder in my project and put all my build configuration files and db deltas in that folder. and commit all that into the SVN repository. so every developer will get the updated build files when he check-out from svn. and be able to run the build to update his DB with the new changes.
on the production server:
I was planning to add another build file there to get the latest Tagged version in svn and perform CSS & JS compression.
I was planning to implement continues integration using PHPUnderControl too, so I can keep track of the result of each build and get notified whenever the build fails.
so, do you think this all does make sense, or you do have other better suggestions?
What you say makes sense : it's pretty close to what I often use (sometimes with ant, sometimes with phing, and sometimes with some shell-scripts).
In the build directory, I would have something like this :
build/
testing/
development/
staging/
production/
common/
With one build.xml file in each sub-directory -- all including another build.xml file, located in the common directory, the idea being to put as much "common" code as possible in the "common" build.xml file, and to have per-environment specific files, that contain as few xml code as possible.
This can be done with the import task that exists in the last version of phing (not sure it's in the stable version -- I'm using an SVN checkout of phing, to have this one, for the project I'm currently working on).
One thing, though : you say you want to deploy to production from the production server ; I would rather, instead :
on a "development" server :
export from SVN
compress JS/CSS and all that
create a tar.gz archive
upload that archive to the production server
on the "production" server :
un-compress that uploaded archive
change a couple of symlink to use the new version of the sources (see the answer I gave here, for more informations)
update what has to be done in the DB
The idea being to :
do as few things as possible on the production server
just in case something goes wrong
and in case, one day, your production server doesn't have access to the SVN server
to have a physical archive that can be deployed on several production servers
Oh, and, as a sidenote : you have to write some kind of documentation of "how to deploy to production", step by step !
This will be really useful the day you are in holiday and someone else has to deploy to production because of an urgent bug-fix ;-)