I have a PHP/Mysql Desktop server. I also installed PHP/Mysql to my laptop and one of my friends for developing a project. Is there anyway that we can synchronise all the changes (i.e. to the php files as well as database changes whether structural or data wise) we do on our laptops to desktop PC ?
For sure you want to start using Git, possibly with a free private repository on somewhere like www.bitbucket.org. Within your versioning, you can backup a version of your SQL inside of the phpMySQL admin tool, and clone the entire repository every time you want to move it.
There are a few choices
https://github.com/axkibe/lsyncd#readme
will sync real time and uses inotify
and if you want to go real nerd
http://www.drbd.org/
High availability clustering
The best way would to be purchase a VPS from a company like Digital Ocean for $5.00 a month. You can then setup a MySQL database that you can both easily connect to. In addition, you will be able to use it to test and deploy (smaller) projects to the server.
Forgot to address the other part of your question. You will be able to setup git/SVN/CVS on this server as well. There are good tutorials online for this. You can both access and commit your changes to this repository. You could also use a website like GitHub for this version control.
Probably the easiest way (if you just want the files synced as is on each change) would be to use something like Google Drive or a similar service. Though if you want revision history and more advanced tools, you'd probably need some sort of version control repository (git, svn, etc.).
However, I definitely recommend version control if that works for you. And if you go with git, I highly recommend bitbucket as they have free closed source hosting for projects of up to 10 (last time I checked) people.
Along with a version control system (such as SVN, Git, or Mercurial) you can't go wrong with using a tool such as FreeFileSync (free as in freeware) for a quick and easy way to push the changes you make to a project to a centralized location. With your project included in a VCS repository the VCS you choose to use will give you control over how you want your project to be synchronized.
This has helped me a lot over the years and has even made the process faster.
Good day to you all,
I am currently developing a project on Laravel. So far I have always developed online, directly editing my files on the webserver throuh FTP (using PSPad or similar simple editing tools).
What I want to do now (and what i believe most people actually do) is setup a (W)LAMP stack on my local machine and program locally. However it is a little bit unclear to me how to keep my local code (including databases) in sync with the live website. How do you folks do that? I know there's probably lots of ways and tools to do that, but what would be your advice for a best practice? Any advice would be very welcome :)
What many companies do is build offline, then push their edits up to a server using git.
Im no expert on the software so ill describe what you do in a basic form:
My advice would be to create an online repo (repository) to store your project while you edit/update.
There are several git project management systems such as github or bitbucket. I personally use bitbucket
What git does, is when you have built or added what you need offline on local (w)lamp, you then git push them up to your repo or server. The changed files then get merged with the existing on the repo or the server. If you'd like the most recent version of your project you'd simply just git pull them down.
Read the full documentation here to see the wide range of options available when using git
We have a settings array within our platform available as $res::Config.
At runtime, a variable is changed from 'dev' to 'live' after checking the HTTP Host, obviously depending on the IP address.
Within our framework bootstrapping, depending on the value of $res::Config->$env, or the environment set previously as either dev or live, the settings for the database connection are set. You store these settings in the Config array as db_live or db_dev.
However you do it, use an environmental variable to figure out whether you want live or dev, and set up and array of settings accordingly.
We also have sandbox and staging for intermittent development stages.
As for version control, use git or subversion.
Edit: It's also possible that within our vhost file, we setup an environmental variable as either live or dev, and our application reads from this accordingly. I'd suggest this approach :)
There are a number of ways of doing this. But this is a deceptively HUGE question you've asked.
Here is some good practice advice - go and research these items, then have a look at my approach.
Typically you use a precess called version control which allows you to create "versions" or snapshots of your system.
The commonly used "SVN" software is good, but the new (not really any more) kid on the block is GIT, and I personally recommend that.
You can use this system to push the codebase live in a controlled fashion. While the files/upload feature is essentially similar to FTP, it allows you to dump a specific version of your site live.
In environments where there are multiple developers, this is ideal - you can compare/test and work around each other, and version control tends to stop errors between devs.
So - advice part 1: Look up and understand version control, then use it to release CODE to the live environment.
Part 2: I use database dumps and farm them back to my machine to work with.
If the live database needs updating, I can work locally and simply export, then re-import on the live system.
For example: on a recent Moodle project I worked on, to refresh the whole database took seconds... I could push a patch and database update in a few minutes.
However: you should think about maintenance and scheduling... if the site is live and has ongoing data changes then you need to be careful with this. Consider adding a maintenance page.
Advice 2: go research SQL dump/export and importing.
I personally use phpmyadmin to dump and re-import, as it's very convenient.
Advice 3: Working locally then pushing live is MUCH BETTER PRACTICE. You're starting down a much safer and better road than you're on!
Hope that helps... but bear in mind - this is a big subject, so you'll need to research a fair bit.
My team has been working on a web application based on php, msql for backend and html,css,jquery for front end. We have been working for quite sometime now without version control. The project has become quite large and complicated now and feel that it is time to use some sort of version control ( vc ).
We have been reading quite a lot about vc and have found several techniques that people have suggested. although it might be a perfect method for them. it does not effectively apply to our case.
our application is split up into various modules and we have outsourced work to a few freelancers. freelancers work independently over front end or back end depending on their skill. The work has been happening over a Private Network so far and we would like to shift to an online system.
Now the problem is that we cannot distribute the source code for the entire project to all developers. developers are only allowed to work on some common libraries and their respective modules.
Hence we cannot allow the entire project to be downloaded onto each developers Local machine. Thus we need to find a way for all developers to be working on the same branch OR on the trunk. to be able to access only certain sections of the code and on saves/commits be able to check how their changes have effected instantly.
Is there a way the above task can be achieved ? for a web application ? using a version control like subversion?
To summarize, the features we are trying to implement are as follows.
Instant effects on save/commit ( When each developer saves/commits they should be able to test the effects like normal through the browser instantly )
Limited access ( Each developer can access only a specific part of the project and not the whole project. )
Online repository / Online copy - ( we are working on a VPN connection, and would like to have the website work on an online copy. so when a commit is made, the developer can view his changes online instead of the VPN )
after a lot of searching online we were able to find the below possibilities. But not sure if it is the right way to go.
For instant effects --> Entire system is checked out in a webroot folder ( eg. wamp/www/projectName ), a bat file is called to update the current system in the webroot, everytime a commit is made, by using the post-commit hook.
Limited access --> All development for all developers happen in the trunk OR a Development branch, developers can access their respective sections, lock it if needed and commit changes when they are ready to see its effects ( note still in dev branch ).
tags are created whenever a stable release is created. these Tagged copies are never modified.
Is such a configuration achievable using subversion or should we look at other open-source version control tools ?
Trying to prevent devs from accessing the entire source tree seems a bit misguided to me. Is it company politics? Don't you trust your developers?
In any case..
The easiest way to achieve what you want would probably be to put each module into its own repository (svn, git, whatever). Then you can selectively choose who has access to which repo.
Post-commit hook, which unconditionally "do site" is in a common not-so-good idea: developer can commit semi-finished code and even don't think about testing it
Make your Working Copy public, when site became public - not-so-good idea from the POV of security and IP. For SVN 1.7 to move WC-root outside web-root is much better idea
If modules are splitted to different directories, you can create repository per module and "super-repository", which, with svn:externals, combine repositories in Project
Path-access inside tree can be easy controlled (in http-repo) by authz_svn_module
Enable creating private personal short-time branches for developers ("shelves"), it helps to avoid (with 1) huge messed up commits
Firstly, I'd recommend reading the "Continuous Delivery" book (website here). It provides many examples for how to set up this kind of thing.
Secondly, yes, SVN allows you to assign permissions on a folder level as well as repo level (but read the "do you really want to do this" section).
Thirdly, making sure developers have up to date copies is something you should instill by discipline - "before starting work, and before committing, run svn update"). Post-commit hooks have a nasty habit of breaking stuff, and you're not preventing commits which break the build.
Fourthly, I'd consider setting up a continuous integration server (also described in the Continuous Delivery book). This makes sure you have a clean, working build whenever you put together the work of your developers.
I apologize if this is obvious or easy, I have looked at a good number of git/github tutorials and read other articles, but I want to make sure what I'm doing is right.
I want to incorporate VC (for obvious reasons) into my development team and process.
Current development process (using Dreamweaver):
* Receive a ticket (or work order)
* Download file on Development server
* Make changes to the file
* Upload file back to development server
* Changes tested/verified
* Send to production server
I'm trying to figure out how to make our new development process with using Git.
I am switching over to PHPStorm (which is an actual PHP IDE with direct integration with Git).
Would it be something like
Receive a ticket (or work order)
Checkout/Update/Download file(s)
Change Files
Upload file (which I assume is also the current working directory...?)
At the end of the day, do a commit
Have build script send data to testing server (nightly build)
Or would it be better to do something like
Receive a ticket (or work order)
Checkout/Update/Download file(s)
Change Files
Upload file/commit
Have build script send data to testing server (nightly build)
Or is there another way? Having a bit of trouble understanding what would be the optimal flow?
Any help would be greatly appreciated.
Edit
I'm trying see if it is best to have a version of the server locally (every developer), and if so, how does that work if you have 7 or so branches?
If not, how do you deal with 7 or so branches with them on the web? Do you FTP files up or use Git Hooks to make them auto update?
Update 07/26/2012
After working successfully with Git for quite a while now I've been following this branching model with great success:
A Successful Git Branching Model
The answer to the above was yes -- should definitely have a local version of the server.
Assuming you have a live server and a development server I would do something along these lines.
Before even starting with a development cycle I would at least have two branches:
Master - the development server runs on this branch
Stable - the live server runs on this branch.
So if a developer gets a ticket or a work order he/she will perform the following actions:
git pull origin master
git branch featureBranch (named as the ticket id or as a good description for the work order)
git checkout featureBranch
Make changes which will accomplish the desired change. Commit as often as is necessary. Do this because you will create valuable history. For instance you can try an approach to a problem and if it doesn't work, abandon it. If a day later you see the light and want to re-apply the solution, it is in your history!
When the feature is fully developed and tested locally, checkout master.
git merge featureBranch
git push origin master
Test the pushed changes on your development server. This is the moment to run every test you can think of.
If all is working out, merge the feature or fix into the stable branch. Now the change is live for your customers.
Getting the code on the server
The updating of servers shouldn't be a problem. Basically I would set them up as users just like you're developers are. At my company we've setup the servers as read-only users. Basically that means the servers can never push anything but can always pull. Setting this up isn't trivial though, so you could just as well build a simple webinterface which simply only allows a git pull. If you can keep your developers from doing stuff on live implementations you're safe :)
[EDIT]
In response to the last questions asked in the comments of this reaction:
I don't know if I understand your question correctly, but basically (simplified a bit) this is how I would do this, were I in you shoes.
The testing machine (or the webroot which acts as testing implementation) has it source code based in a git repository with the master branch checked out. While creating this repository you could even remove all other references to all other branches so you'll be sure no can checkout a wrong branch in this repository. So basically the testing machine has a Git repository with only a master branch which is checked out.
For the live servers I would do exactly the same, but this time with the stable branch checked out. Developer should have a local repository cloned in which all branches exist. And a local implementation of the software you guys build. This software gets its source from a the local git repository. In other words: from the currently checked out branch in this repository.
Actual coding
When a new feature is wanted, a local feature branch can be made based on the current master. When the branch is checked out the changes can be made and checked locally by the developer (since the software is now running on the source of the feature branch).
If everything seems to be in order, the changes get merged from feature branch to master and pushed to your "git machine". "your github" so to speak. Testing can now pull the changes in so every test necessary can be done by QA. If they decide everything is ok, the developer can merge the changes from master to stable and push again.
All thats left now is pulling form your live machines.
From my experience, one of the bigger problems we come across during our webdevelopment process is keeping different setups updated and secure across different servers.
My company has it's own CMS which is currently installed across 100+ servers. At the moment, we use a hack-ish FTP-based approach, combined with upgrade scripts at specific locations to upgrade all of our CMS setups. Efficiently managing these setups becomes increasingly difficult and risky when there are several custom modules involved.
What is the best way to keep multiple setups of a web application secure and up-to-date?
How do you do it?
Are there any specific tips regarding modularity in applications, in order to maintain flexibility towards our clients, but still being able to efficiently manage multiple "branches" of an application?
Some contextual information: we mainly develop on the LAMP-stack. One of the main factors that helps us sell our CMS is that we can plugin pretty much anything our client wants. This can very from 10 to to 10.000 lines of custom code.
A lot of custom work consists of very small pieces of code; managing all these small pieces of code in Subversion seems quite tedious and inefficient to me (since we deliver around 2 websites every week, this would result in a lot of branches).
If there is something I am overlooking, I'd love to hear it from you.
Thanks in advance.
Roundup: first of all, thanks for all of your answers. All of these are really helpful.
I will most likely use a SVN-based approach, which makes benlumley's solution closest to what I will use. Since the answer to this question might differ in other usecases, I will accept the answer with the most votes at the end of the run.
Please examine the answers and vote for the ones that you think have the most added value.
I think using a version control system and "branching" the part of the codes that you have to modify could turn out to be the best approach in terms of robustness and efficiency.
A distributed version system could be best suited to your needs, since it would allow you to update your "core" features seamlessly on different "branches" while keeping some changes local if need be.
Edit: I'm pretty sure that keeping all that up to date with a distributed version system would be far less tedious than what you seem to expect : you can keep the changes you are sure you're never going to need elsewhere local, and the distributed aspect means each of your deployed application is actually independent from the others and only the fix you mean to propagate will propagate.
If customizing your application involves changing many little pieces of code, this may be a sign that your application's design is flawed. Your application should have a set of stable core code, extensibility points for custom libraries to plug into, the ability to change appearance using templates, and the ability to change behavior and install plugins using configuration files. In this way, you don't need a separate SVN branch for every client. Rather, keep the core code and extension plugin libraries in source control as normal. In another repository, create a folder for each client and keep all their templates and configuration files there.
For now, creating SVN branches may be the only solution that helps you keep your sanity. In your current state, it's almost inevitable that you'll make a mistake and mess up a client's site. At least with branches you are guaranteed to have a stable code base for each client. The only gotcha with SVN branches is if you move or rename a file in a branch, it's impossible to merge that change back down to the trunk (you'd have to do it manually).
Good luck!
EDIT: For an example of a well-designed application using all the principles I outlined above, see Magento E-Commerce. Magento is the most powerful, extensible and easy to customize web application I've worked with so far.
I may be wrong, but it seems to me what Aron is after is not version control. Versioning is great, and I'm sure they're using it already, but for managing updates on hundreds of customized installations, you need something else.
I'm thinking something along the lines of a purpose-built package system. You'll want every version of a module to keep track of its individual dependencies and 'guaranteed compatibilities', and use this information to automatically update only the 'safe' modules.
E.g. let's say you've built a new version 3 of your 'Wiki' module. You want to propagate the new version to all the servers running your application, but you've made changes to one of the interfaces within the Wiki module since version 2. Now, for all default installations, that is no problem, but it would break installations with custom extensions on top of the old interface. A well-planned package system would take care of this.
To address the security question, you should look into using digital signatures on your patches. There are lots of good libraries available for public-key-based signatures, so just go with whatever seems to be the standard for your chosen platform.
Not sure whether someone's said this, there are a lot of long responses here, and I've not read them all.
I think a better approach to your version control would be to have your CMS sat on its own in its own repository and each project in its own. (or, all of these could be subfolders within one repo i guess)
You can then use its trunk (or a specific branch/tag if you prefer) as an svn:external in each project that requires it. This way, any updates you make to the CMS can be committed back to its repository, and will be pulled into other projects as and when they are svn updated (or the external is svn:switch 'ed).
As part of making this easier, you will need to make sure the CMS and the custom functionality sit in different folders, so that svn externals works properly.
IE:
project
project/cms <-- cms here, via svn external
project/lib <-- custom bits here
project/www <-- folder to point apache/iis at
(you could have cms and lib under the www folder if needed)
This will let you branch/tag each project as you wish. You can also switch the svn:external location on a per branch/tag basis.
In terms of getting changes live, I'd suggest that you immediately get rid of ftp and use rsync or svn checkout/exports. Both work well, the choice is up to you.
I've got most experience with the rsync route, rsyncing an svn export to the server. If you go down this route, write some shell scripts, and you can create a test shell script to show you the files it will upload without uploading them as well, using the -n flag. I generally use a pair of scripts for each environment - one a test, and one to actually do it.
Shared key authentication so you don't need a password to send uploads up may also be useful, depending on how secure the server to be given the access is.
You could also maintain another shell script for doing bulk upgrades, which simply calls the relevant shell script for each project you want to upgrade.
Have you looked at Drupal? No, not to deploy and replace what you have, but to see how they handle customizations and site-specific modules?
Basically, there's a "sites" folder which has a directory for every site you're hosting. Within each folder is a separate settings.php which allows you to specify a different database. Finally, you can (optionally) have "themes" and "modules" folders within sites.
This allows you to do site-specific customizations of particular modules and limit certain modules to those sites. As a result, you end up with a site that the vast majority of everything is perfectly identical and only the differences get duplicated. Combine that with the way it handles upgrades and updates and you might have a viable model.
Build into the code a self-updating process.
It will check for updates and run them when/where/how you have configured it for the client.
You will have to create some sort of a list of modules (custom or not) that need to be tested with the new build prior to roll-out. When deploying an update you will have to ensure these are tested and integrated correctly. Hopefully your design can handle this.
Updates are ideally a few key steps.
a) Backup so you can back out. You should be able to back out
the entire update at any time. So,
that means creating a local archive
of the application and database
first.
b) Update Monitoring Process - Have the CMS system phone home to look for a new build.
c) Schedule Update on availability - Chances are you don't want the update to run the second it is available. This means you will have to create a cron/agent of some kind to do the system update automatically in the middle of the night. You can also consider client requirements to update on weekends, or on specific days. You can also stagger rolling out your updates so you don't update 1000 clients in 1 day and get tech support hell. Staggered roll-out of some kind might be beneficial for you.
d) Add maintenance mode to update the site -- Kick the site into maintenance mode.
e) SVN checkout or downloadable packages -- ideally you can deploy via svn checkout, and if not, setup your server to deliver svn generated packages into an archive that can be deployed on client sites.
f) Deploy DB Scripts - Backup the databases, update them, populate them
g) Update site code - All this work for one step.
h) Run some tests on it. If your code has self-tests built in, it would be ideal.
Here's what I do...
Client-specific include path
Shared, common code is in shared/current_version/lib/
Site specific code is in clients/foo.com/lib
The include path is set to include from the clients/foo.com/lib, and then share/lib
The whole thing is in a version control system
This ensures that the code uses shared files wherever possible, but if I need to override a particular class or file for some reason, I can write a client specific version in their folder.
Alias common files
My virtual host configuration will contain a line like
Alias /common <path>/shared/current_version/public_html/common
Which allows common UI elements, icons, etc to be shared across projects
Tag the common code with each site release
After each site release, I tag the common code by creating a branch to effectively freeze that point in time. This allows me to deploy /shared/version_xyz/ to the live server. Then I can have a virtual host use a particular version of the common files, or leave it pointing at the current_version if I want it to pick up the latest updates.
Have you looked at tools such as Puppet (for system administration incl. app deployment) or Capistrano (deployment of apps in RubyOnRails but not limited to these)?
One option would be to set up a read-only version control system (Subversion). You could integrate access to the repository into your CMS and invoke the updates through a menu, or automatically if you do not want the user to have a choice about an update (could be critical). Using a version control system would also allow you to keep different branches easily
As people have already mentioned that using version control (I prefer Subversion due to functionality) and branching would be the best option. Another open source software available on sourceforge called cruisecontrol. Its amazing, you configure cruisecontrol with subversion in sach a way that any code modification or new code added in serversion, Cruise control will know automatically and will do build for you. It will save your hell of time.
I have done the same way in my company. we have four projects and have to deploy that project on different servers. I have setup cruiseconrol in such a way that any modification in code base triggers automatic build. and another script will deploy that build on the server. your are good to go.
If you use a LAMP stack I would definitely turn the solutions files into a package of your distribution and use it for propagate changes. I recommend for that matter Redhat/Fedora because of RPM and it's what I have experience on. Anyway you can use any Debian based distribution too.
Sometime ago I made a LAMP solution for managing an ISP hosting servers. They had multiple servers to take care of web hosting and I needed a way to deploy the changes of my manager, because every machine was self-contained and had a online manager. I made a RPM package containing the solution files (php mostly) and some deploying scripts that runned with the RPM.
For automated updating we had our own RPM repository set on every server in yum.conf. I set an crontab job to update the servers daily with the latest RPMs from that trusted repository.
Trustiness can be achieve too because you can use trust settings in the RPM packages, like signing them with your public key file and accepting only signed packages.
Hm could it be an idea to add configuration files? You wrote that a lot of small script are doing something. Now if you'd build them into the sources and steered them with configuration files shouldn't that "ease" that?
On the other hand having branches for every customer looks like an exponential growth to me. And how would you "know" which areas you've done something and do not forget to "make" changes in all other branches also. That looks quite ugly to me.
It seems a combination of revision controls, configuration options and/or deployment receipts seems to be a "good" idea.....
With that many variations on your core software, I think you really need a version control system to stay on top of pushing updates from the trunk to the individual client sites.
So if you think Subversion would be tedious, you've got a good sense for what the pain points will be... Personally, I wouldn't recommend Subversion for this, since it's not really that good at managing & tracking branches. Although benlumley's suggestion to use externals for your core software is a good one, this breaks down if you need to tweak the core code for your client sites.
Look into Git for version control, it's built for branching, and it's fast.
Check out Capistrano for managing your deployments. It's a ruby script, often used with Rails, but it can be used for all sorts of file management on remote servers, even non-ruby sites. It can get the content to the remote end through various stragegies including ftp, scp, rsync, as well as automatically checking out the latest version from your repository. The nice features it provides include callback hooks for every step of the deploy process (e.g. so you can copy your site-specific configuration files which might not be in version control), and a release log system--done through symlinks--so you can quickly roll back to a previous release in case of trouble.
I'd recommend a config file with the list of branches and their hosted location, then run through that with a script that checks out each branch in turn and uploads the latest changes. This could be cron'd to do nightly updates automatically.