Related
I'm working on a symfony 2 web site, it is a product our company sells (every client gets his own installation).
Each of our client gets a slightly different version of the site: sometimes the differences are small (different strings, slightly different templates), and sometimes they are bigger (different bundles activated, different database schema, different security.yml file etc.)
Currently each version sits on its own git branch. This works for now as we only started selling and don't have many clients(=branches) yet.
But I want to move to a better solution.
Any ideas?
Thanks,
A
I disagree. If it's a single application, then branches per client or release, ie configurations that need separate histories (change tracking), are precisely the correct practice. Clients use/share the same app, "core", whatever, but possibly you want to control when changes in one client's features/bugs are merged into the other configurations, ie branches. Each client will have to be "upgraded" by merging from the development branch to client's branch, separately, manually. You want to pay this overhead to gain stability and isolation.
As for the specific differences you mentioned:
Different strings: client specific configs, constants, deployment stuff, should probably be in a per-client/deployment module (include file, whatever), that you can simply ignore when merging, so that the core app code will have no client/deployment specific differences, ever.
Slightly different templates: meh, this smells bad; if you can use symbolic constants or partials or whatever, and move the differences to a config.php, then see the above solution. Otherwise, don't template diffs suggest it's not the same app anymore?
Different bundles activated: if you can put the include statements in the config.php, no problem.
Different database schema: nasty business. Can this app be made schema-independent? Can you do polymorphism somehow? Nu, I don't know, haven't touched PHP in twenty years, I only do CoffeeScript and NoSQL now. ;o)
Different security.yml: same as with config.php, not a problem, just don't merge that file, or even stop tracking it (remove from repo) altogether.
HTH.
;o)
I don't think that a branch is a good solution as its not intended for this kind of use case. As i see it, branches should be used for a staging or features/fixes.
As i see it, you have few options (in case you are not familiar with those options, you should read more about them in the links i attached):
Git submodule. "A submodule allows you to keep another Git repository in a subdirectory of your repository. The other repository has its own history, which does not interfere with the history of the current repository. This can be used to have external dependencies such as third party libraries for example." http://git-scm.com/docs/git-submodule
Composer. This is what i think would best suite you. Use a regular dependencies manager (composer for php, npm for nodejs etc..). I will give you full control over the version been used + easy deployment options. https://getcomposer.org/
Another option would to do it "old school".. In case they are all in the same server, just keep the "Core" stuff in the root folder and include it for each project. It's no a recommended way to go but it might be enough and won't require much changes from you
I'm new to Laravel and now figuring out how to use Laravel for this project i'm working on. I have some ideas about approaching it, and like to know how the experts will do the job.
The case
I'm working on a project which has it's CMS, a informative site and about 35 specialized websites. The CMS, informative site and specialized sites are different on content and design. But all the 35 specialized websites are pretty much the same. The design and layout is identical, but the sites contain different content. Currently all sites have it's own codebase, which makes maintaining it very hard.
I'm now looking for a solution that will use just one codebase. CMS and sites share some functionality and both will have it's own functionality.
I'm just not sure about how to do this.
The sites run on a dedicated webserver that has DirectAdmin installed on it. Within DirectAdmin it's possible to create a domain pointer. This way I can create a site which shows the correct content by checking the http_host variable. Also, if someday a specialized site needs different functionality than the other, I can also check the http_host. I'm not sure if the has any consequents for SEO and whether it's a neat solution.
I've read about bundles and thinking of using them for the informative- and specialized sites. The "main" applications directory will then host the CMS. site.com/ will then be the cms, site.com/info will show a bundle which hosts the informative site and site.com/special will show a bundle for the specialized sites.
When use a domain pointer, some domain will show the site of another domain. But I don't think this will work with a bundle when it's located at site.com/(bundle).
I already have a database that distinct content by http_host. I would now like to know how to structure the code (and how to point 35 domains to the right pieces of code).
I hope the experts over here can tell me some of there best practices on how to do this job.
Thanks in advance.
I would recommend you to keep 3 projects (one for each kind of site).
You could combine the main site and the informative site they way you proposed (as a bundle), but the only reason to do so is to share something between them, which is probably not the case since they have different design and content. If it's only a matter of having it on "/info", this is achievable by a well designed Virtual Host configuration (can't give too much direction on that tho).
Although, I would recommend you to keep the informative site on http://info.site.com instead. It feels "right" to me to have different websites on different domains (or subdomains). Some people even advocate this is the correct thing to do, but who knows...
If you have libraries that you may reuse between all projects, I would recommend putting those on a bundle and include that bundle on all projects.
Now, about the 35 specialized sites, you should point all URL's to the same application through Virtual Host's ServerAlias.
On that application, you should defined Environments which let you configure some aspects of it based on the URL. For instance, you can define a different database for each website, create separate log files, languages, anything that is configurable on /application/config. You can also create your own configurations either.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 12 months ago.
Improve this question
I work for a company that does sites for the pharma industry and we often need to get legal approval before we push any changes live. So, I'd like to migrate a lot of our work to a CMS environment, specifically wordpress but we need the ability to have a staging environment. Is it possible to instead of publishing a page have it publish to a staging environment that someone can browse as with a link as a site. So basically have 2 sites, one staging one live?
Generally speaking other "answers" here are correct, there are alternatives to WordPress that have better built-in support for staging environments and build migrations. However the suggested alternatives aren't exactly equal substitutes to the WordPress platform, so I think it's best to answer the question at hand instead.
WordPress does not natively support hosting the same site from two different hosts. The core relies on absolute URLs stored inside the database and are used in just about every aspect of the core logic. This results in a number of superfluous bugs like the 500 or so related to SSL access because they try to dynamically alter all http:// schemes to https:// on the fly.
As a result when you host on dev.example.com and migrate to staging.example.com and again to www.example.com you have to do very careful search & replace manipulations on the database export each time you switch hosts. And this causes additional problems when you find out that many popular WordPress plugins serialize the URL into values in the database. So when you search & replace dev.example.com with staging.example.com the serialized data which contained the character length of the original value no longer deserializes with the new longer format. Some core contributors believe the solution to this later problem is to only ever setup staging sites with the same number of characters as the production account...
In a similar vein they also suggest swapping host mappings and only ever using the production.com URL on all hosting environments. Depending on your particular use-case requirements this is probably not a valid solution if you need to provide access to off-site clients, tech-illiterate users (versus tech-literate users of course.)
But WordPress itself has a number of great features otherwise and is a very adaptive and powerful rapid development platform. As a result you can extend the core framework to do much of what you need from it. When I was presented with this situation, I had to develop a solution that was viable for all circumstances. Traditionally this problem is solved with root-relative URLs, they work in cross-hosting environments, and they don't suffer from scheme changes, port changes or subdomain swapping practices that are common with staging migrations.
With this plugin: http://wordpress.org/extend/plugins/root-relative-urls/
(biased? yes, I wrote this plugin.) you get root-relative URLs where it's important and dynamic hosts where root-relative URLs don't work (like rss feeds.) All that remains from migrating the site to different hosts is to move the wp-config.php file outside of the www root (one level up is supported natively by WordPress.) so you can maintain different copies on different servers. Or alternatively you can use basic if-statements to distinguish hosts by server name and define key WordPress constants based on the server. In the end your content, code & data will transition seamlessly.
As a note of concern, the referenced plugins require setting write access to the wp-config.php file, a very bad practice from a security perspective for production or publicly accessible servers. Perhaps you can comfortably implement this in a restricted staging environment but then you'd need to disable and remove the plugin in production transitions.
Long story short, yes you can host WordPress in multiple host environments. The long-touted solutions are very case-specific and option-restricted because of the core architecture. But the framework is flexible enough to overcome the core deficit. This core design decision will probably change at some point in the future given the amount of effort the core developers continually spend on overcoming the cascading issues. But there are also devout defenders of the absolute URL religion that will keep the practice in place for the time being. Maybe a different platform that supports server migrations natively (pick just about any of them because most do) would be a better option for you now.
It's possible: Take a look at this Github Gist to see an example on how to switch environments with your wp-config.php file. Furthermore, take a look at wordpress.stackexchange to see some other Qs about this that give you a more in-depth look at the stuff you should consider.
Greg,
An even better CMS with staging environment would be Silverstripe (silverstripe.org). This cms allows for you to browse an entire staging site.
I think that you can try to use some plugins.
For example (fast search on official wordpress plugin repo) wp-deploy or Dev and Staging Environment Plugin (maybe outdated).
Or as alternative you can try to use different wp-config.php files - one for production and one for dev environment and switch them by checking requested url.
If you need to use WP, and need to publish only one or several pages from staging to live site, why not implement kind of tag on pages that needed to be published (seen by live site visitors)? Simple tune up your templates to display tagged or not pages and you're done! Then you can use only one site and maintain both public-available and not-approved pages on the it.
You can as well maintain local copy of your site (stage) and have some script to upload it (at whole) to your hosting - easy to automate task. In such a case, you may want to consider render the whole site into HTML pages and upload (simple rsync) this HTMLs to hosting - the live site will be hard to break as no dynamic script will be there!
But maybe you really should not choose the WP? There are a lot of CMS that support write-approve-publish scheme.
You can create a WordPress Staging environment with just two click with the help of this plugin: https://de.wordpress.org/plugins/wp-staging/
Disclosure: I am the author of this plugin. So ask me anything about it.
Researching various open source, web-based document management/version control systems. I've checked google/questions here, etc...
I'm looking for a lightweight web-based (apache) document mgmt/version control app that runs on top of SVN.
I need to have the ability to:
have multiple users checkin/checkout
have a workflow (when userA checks the file in, and finishes the app passes it to the next person, etc...
the app needs to allow me to have a structure where the files can be moved as a group. the files will be changed on a monthly basis
app needs to have a access/premission control system. some people can see certain files, and perform certain actions on the files
I imagine that I'm going to have 40-50 people dealing with the different files. I imagine that I'm going to have 2000-3000 files that have to be massaged.
I'd prefer that the app be php based if possible, as opposed to a straight java app.
Thanks
I suggest you take a look at Subversion:
WikiPedia: http://en.wikipedia.org/wiki/Subversion_(software)
Mainpage: http://subversion.apache.org.
I honestly don't know if it fits all requirements that you've posted, as I've only used it on projects involving small teams, but you should be able to sort that out by reading the WikiPedia page linked above. That said, it definitely meets the web-dev version control.
A vanilla installation of SVN isn't going to do this; it only does regular version control (commits, branches, etc.). There are plenty of web based svn interfaces, but all they do is give you a way to work with regular svn over the web. The document management features and workflow you require are going to come from some other document management system; I don't know of any that use svn as a back end.
I believe Mercurial provides a web-based interface as well.
For a simplistic, but working version control system made entirely in PHP, built on top of ASVCS, take a look at IntraVersion (http://webscripts.softpedia.com/script/Development-Scripts-js/IntraVersion-39805.html) Not much, but for those stuck like me with no server to host private projects on at the moment, it's a helper.
Have a look at knowledge tree. PHP, open source, with support/hosting available. I've used it in the past. It sucks less than other solutions.
From my experience, one of the bigger problems we come across during our webdevelopment process is keeping different setups updated and secure across different servers.
My company has it's own CMS which is currently installed across 100+ servers. At the moment, we use a hack-ish FTP-based approach, combined with upgrade scripts at specific locations to upgrade all of our CMS setups. Efficiently managing these setups becomes increasingly difficult and risky when there are several custom modules involved.
What is the best way to keep multiple setups of a web application secure and up-to-date?
How do you do it?
Are there any specific tips regarding modularity in applications, in order to maintain flexibility towards our clients, but still being able to efficiently manage multiple "branches" of an application?
Some contextual information: we mainly develop on the LAMP-stack. One of the main factors that helps us sell our CMS is that we can plugin pretty much anything our client wants. This can very from 10 to to 10.000 lines of custom code.
A lot of custom work consists of very small pieces of code; managing all these small pieces of code in Subversion seems quite tedious and inefficient to me (since we deliver around 2 websites every week, this would result in a lot of branches).
If there is something I am overlooking, I'd love to hear it from you.
Thanks in advance.
Roundup: first of all, thanks for all of your answers. All of these are really helpful.
I will most likely use a SVN-based approach, which makes benlumley's solution closest to what I will use. Since the answer to this question might differ in other usecases, I will accept the answer with the most votes at the end of the run.
Please examine the answers and vote for the ones that you think have the most added value.
I think using a version control system and "branching" the part of the codes that you have to modify could turn out to be the best approach in terms of robustness and efficiency.
A distributed version system could be best suited to your needs, since it would allow you to update your "core" features seamlessly on different "branches" while keeping some changes local if need be.
Edit: I'm pretty sure that keeping all that up to date with a distributed version system would be far less tedious than what you seem to expect : you can keep the changes you are sure you're never going to need elsewhere local, and the distributed aspect means each of your deployed application is actually independent from the others and only the fix you mean to propagate will propagate.
If customizing your application involves changing many little pieces of code, this may be a sign that your application's design is flawed. Your application should have a set of stable core code, extensibility points for custom libraries to plug into, the ability to change appearance using templates, and the ability to change behavior and install plugins using configuration files. In this way, you don't need a separate SVN branch for every client. Rather, keep the core code and extension plugin libraries in source control as normal. In another repository, create a folder for each client and keep all their templates and configuration files there.
For now, creating SVN branches may be the only solution that helps you keep your sanity. In your current state, it's almost inevitable that you'll make a mistake and mess up a client's site. At least with branches you are guaranteed to have a stable code base for each client. The only gotcha with SVN branches is if you move or rename a file in a branch, it's impossible to merge that change back down to the trunk (you'd have to do it manually).
Good luck!
EDIT: For an example of a well-designed application using all the principles I outlined above, see Magento E-Commerce. Magento is the most powerful, extensible and easy to customize web application I've worked with so far.
I may be wrong, but it seems to me what Aron is after is not version control. Versioning is great, and I'm sure they're using it already, but for managing updates on hundreds of customized installations, you need something else.
I'm thinking something along the lines of a purpose-built package system. You'll want every version of a module to keep track of its individual dependencies and 'guaranteed compatibilities', and use this information to automatically update only the 'safe' modules.
E.g. let's say you've built a new version 3 of your 'Wiki' module. You want to propagate the new version to all the servers running your application, but you've made changes to one of the interfaces within the Wiki module since version 2. Now, for all default installations, that is no problem, but it would break installations with custom extensions on top of the old interface. A well-planned package system would take care of this.
To address the security question, you should look into using digital signatures on your patches. There are lots of good libraries available for public-key-based signatures, so just go with whatever seems to be the standard for your chosen platform.
Not sure whether someone's said this, there are a lot of long responses here, and I've not read them all.
I think a better approach to your version control would be to have your CMS sat on its own in its own repository and each project in its own. (or, all of these could be subfolders within one repo i guess)
You can then use its trunk (or a specific branch/tag if you prefer) as an svn:external in each project that requires it. This way, any updates you make to the CMS can be committed back to its repository, and will be pulled into other projects as and when they are svn updated (or the external is svn:switch 'ed).
As part of making this easier, you will need to make sure the CMS and the custom functionality sit in different folders, so that svn externals works properly.
IE:
project
project/cms <-- cms here, via svn external
project/lib <-- custom bits here
project/www <-- folder to point apache/iis at
(you could have cms and lib under the www folder if needed)
This will let you branch/tag each project as you wish. You can also switch the svn:external location on a per branch/tag basis.
In terms of getting changes live, I'd suggest that you immediately get rid of ftp and use rsync or svn checkout/exports. Both work well, the choice is up to you.
I've got most experience with the rsync route, rsyncing an svn export to the server. If you go down this route, write some shell scripts, and you can create a test shell script to show you the files it will upload without uploading them as well, using the -n flag. I generally use a pair of scripts for each environment - one a test, and one to actually do it.
Shared key authentication so you don't need a password to send uploads up may also be useful, depending on how secure the server to be given the access is.
You could also maintain another shell script for doing bulk upgrades, which simply calls the relevant shell script for each project you want to upgrade.
Have you looked at Drupal? No, not to deploy and replace what you have, but to see how they handle customizations and site-specific modules?
Basically, there's a "sites" folder which has a directory for every site you're hosting. Within each folder is a separate settings.php which allows you to specify a different database. Finally, you can (optionally) have "themes" and "modules" folders within sites.
This allows you to do site-specific customizations of particular modules and limit certain modules to those sites. As a result, you end up with a site that the vast majority of everything is perfectly identical and only the differences get duplicated. Combine that with the way it handles upgrades and updates and you might have a viable model.
Build into the code a self-updating process.
It will check for updates and run them when/where/how you have configured it for the client.
You will have to create some sort of a list of modules (custom or not) that need to be tested with the new build prior to roll-out. When deploying an update you will have to ensure these are tested and integrated correctly. Hopefully your design can handle this.
Updates are ideally a few key steps.
a) Backup so you can back out. You should be able to back out
the entire update at any time. So,
that means creating a local archive
of the application and database
first.
b) Update Monitoring Process - Have the CMS system phone home to look for a new build.
c) Schedule Update on availability - Chances are you don't want the update to run the second it is available. This means you will have to create a cron/agent of some kind to do the system update automatically in the middle of the night. You can also consider client requirements to update on weekends, or on specific days. You can also stagger rolling out your updates so you don't update 1000 clients in 1 day and get tech support hell. Staggered roll-out of some kind might be beneficial for you.
d) Add maintenance mode to update the site -- Kick the site into maintenance mode.
e) SVN checkout or downloadable packages -- ideally you can deploy via svn checkout, and if not, setup your server to deliver svn generated packages into an archive that can be deployed on client sites.
f) Deploy DB Scripts - Backup the databases, update them, populate them
g) Update site code - All this work for one step.
h) Run some tests on it. If your code has self-tests built in, it would be ideal.
Here's what I do...
Client-specific include path
Shared, common code is in shared/current_version/lib/
Site specific code is in clients/foo.com/lib
The include path is set to include from the clients/foo.com/lib, and then share/lib
The whole thing is in a version control system
This ensures that the code uses shared files wherever possible, but if I need to override a particular class or file for some reason, I can write a client specific version in their folder.
Alias common files
My virtual host configuration will contain a line like
Alias /common <path>/shared/current_version/public_html/common
Which allows common UI elements, icons, etc to be shared across projects
Tag the common code with each site release
After each site release, I tag the common code by creating a branch to effectively freeze that point in time. This allows me to deploy /shared/version_xyz/ to the live server. Then I can have a virtual host use a particular version of the common files, or leave it pointing at the current_version if I want it to pick up the latest updates.
Have you looked at tools such as Puppet (for system administration incl. app deployment) or Capistrano (deployment of apps in RubyOnRails but not limited to these)?
One option would be to set up a read-only version control system (Subversion). You could integrate access to the repository into your CMS and invoke the updates through a menu, or automatically if you do not want the user to have a choice about an update (could be critical). Using a version control system would also allow you to keep different branches easily
As people have already mentioned that using version control (I prefer Subversion due to functionality) and branching would be the best option. Another open source software available on sourceforge called cruisecontrol. Its amazing, you configure cruisecontrol with subversion in sach a way that any code modification or new code added in serversion, Cruise control will know automatically and will do build for you. It will save your hell of time.
I have done the same way in my company. we have four projects and have to deploy that project on different servers. I have setup cruiseconrol in such a way that any modification in code base triggers automatic build. and another script will deploy that build on the server. your are good to go.
If you use a LAMP stack I would definitely turn the solutions files into a package of your distribution and use it for propagate changes. I recommend for that matter Redhat/Fedora because of RPM and it's what I have experience on. Anyway you can use any Debian based distribution too.
Sometime ago I made a LAMP solution for managing an ISP hosting servers. They had multiple servers to take care of web hosting and I needed a way to deploy the changes of my manager, because every machine was self-contained and had a online manager. I made a RPM package containing the solution files (php mostly) and some deploying scripts that runned with the RPM.
For automated updating we had our own RPM repository set on every server in yum.conf. I set an crontab job to update the servers daily with the latest RPMs from that trusted repository.
Trustiness can be achieve too because you can use trust settings in the RPM packages, like signing them with your public key file and accepting only signed packages.
Hm could it be an idea to add configuration files? You wrote that a lot of small script are doing something. Now if you'd build them into the sources and steered them with configuration files shouldn't that "ease" that?
On the other hand having branches for every customer looks like an exponential growth to me. And how would you "know" which areas you've done something and do not forget to "make" changes in all other branches also. That looks quite ugly to me.
It seems a combination of revision controls, configuration options and/or deployment receipts seems to be a "good" idea.....
With that many variations on your core software, I think you really need a version control system to stay on top of pushing updates from the trunk to the individual client sites.
So if you think Subversion would be tedious, you've got a good sense for what the pain points will be... Personally, I wouldn't recommend Subversion for this, since it's not really that good at managing & tracking branches. Although benlumley's suggestion to use externals for your core software is a good one, this breaks down if you need to tweak the core code for your client sites.
Look into Git for version control, it's built for branching, and it's fast.
Check out Capistrano for managing your deployments. It's a ruby script, often used with Rails, but it can be used for all sorts of file management on remote servers, even non-ruby sites. It can get the content to the remote end through various stragegies including ftp, scp, rsync, as well as automatically checking out the latest version from your repository. The nice features it provides include callback hooks for every step of the deploy process (e.g. so you can copy your site-specific configuration files which might not be in version control), and a release log system--done through symlinks--so you can quickly roll back to a previous release in case of trouble.
I'd recommend a config file with the list of branches and their hosted location, then run through that with a script that checks out each branch in turn and uploads the latest changes. This could be cron'd to do nightly updates automatically.