I'm looking for the best way (or easiest way) to manage multiple instances of a PHP web application, that share the same code base.
Let me break it down for you:
Our domain is hosting multiple instances of the application, each with their own settings files and database.
http://mydomain.com
|
|------/customer1/
|
|------/customer2/
|
|------/customer3/ + custom features
Let's say that customer 1 & 2 purchased the application (that we host for them), and they have the base model of that application. (ie. not customized)
However, customer 3 wants feature X or Y, so we code that feature for him and add that to the application.
But whenever there is an update to the code base (ie. a security fix in the core classes of the framework) all three customers should get an update of the base code!
What would be the best way of managing this sort of setup? Manually uploading all files using FTP is a pain, and it's not possible to merge code.
Using Git is perhaps a solution, but how would I go around and do it? Create separate repositories per customer? What if we grow to over one-hundred customers?
Any insight are welcome, including why we should or should not use such a setup. (but remember that we'll be the ones hosting the application for our customers)
I remember doing this years ago so you will have to take into account i'm now a little rusty at this.
I built a standalone framework, which combined all includes into ONE .php file. Any frameworks that used that, would do a PULL request and if the md5 of their framework matched the framework on the central server then no update was needed. Otherwise it would download the new framework over https and replace it's own copy. This created an automatic update system that was PULLED to all other apps that used it.
A major problem to this is, if you cause say a syntax error and you upload that to the central server, it will get pulled to all others and break them! You will be best to use a cron job to make the pull request that does NOT use the framework so the broken framework won't break it from doing a pull request to FIX the syntax error in the framework. This at least adds the ability to automatically fix itself as well once you fix the syntax error on the central server. However, having a staging server to test each update really is very important in this case.
That is only the basics of course as if you have say images that the framework uses they will also need to get pulled over, as well as any SQL updates and so forth.
You must regorisly test this before uploading to the central server in order to prevent mass errors! Not ideal! Unit testing, staging server, small and simple updates but more often (large updates have more potential to go wrong, and more to undo if it does go wrong) will all help mitigate the risk.
You will also have to structure the framework VERY VERY VERY VERY VERY well from the beginning to make it as flexible as possible when planning on having many different sites use it. If you design it wrong in the beginning it may be next to impossible to redesign further down the road. For example it may be wise to use PDO for database access, allowing all the applications the ability to use different databases while your classes etc will still no know how to interact with the database (regardless of if it's mysql or oracle), though, i would advise at least sticking to one if you can.
Design wise, you are best to look at other language frameworks and see how they do what they do. You must stick to good design principles, use design patterns only where applicable, and take note of MVC!
Further Reading...
http://en.wikipedia.org/wiki/Software_design_pattern
http://www.ipipan.gda.pl/~marek/objects/TOA/oobasics/oobasics.html
http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller
http://www.phpframeworks.com/
This is no easy task, so be warned.
You mixed two separate different tasks in one question
Development and support of diverged code
Deploy of code from (any) SCM to live systems
Answer on 1-st question (for any modern) SCM is branching and merge branches (each customer have own branch, into which you merge needed parts from you single development-branch, or /better/ with "branch-per-task" you merge task-branch in all needed targets, avoiding cherry-picking)
Answer on 2-nd question is "Build-tools", which can interact with your SCM (you have to write more details later for more detailed answer)
Make your custom features modular. Use a similar architecture to WordPress/Joomla which have plugins or extensions. This allows your customers to easily have separate feature sets but all share the same base code.
Related
I have a multilingual project,
the project is fully compatible with HE/EN languages.
In one case it's going to be uploaded as a subfolder in a big israeli project,
in another it's going to be used as a full site on it's own .com domain in english.
I currently have an SVN setup for the israeli project,
I haven't yet created an SVN for the .com, is this the right way to go, keeping 2 depo's for this ? I will then have to make sure that the code written on the israeli project is always copied manually to the .com folder,
Can this be managed more efficiently ?
Yes, this can definitely be managed more efficiently. This statement right here is the main problem:
I will then have to make sure that the code written on the israeli project is always copied manually to the .com folder.
That process is not only inefficient, it's highly error-prone. And it absolutely doesn't scale at all. Basically you're doubling your development/testing/deployment/etc. efforts. If you ever need to deploy another instance of the application, you'll have tripled efforts. And so on.
What you'll want to do is isolate the components that are different between the instances of the application. Is it just the language content? Is it the whole UI? If it's the whole UI then you'll want that UI to be as thin and light-weight as possible. (Well, you want that anyway.) Then all you'd have to double is the UI work. The rest of the logic (business logic, data access, services, etc.) can be constant across all instances of the application.
If it's just text content, then maybe you can re-factor that text to be pulled from the database. Every label, every article, every menu item, etc. There are CMS systems which do this pretty well, though I have no specific recommendations. But the overall idea is to have the UI be an empty wrapper populated by data specific to that application instance. Then each instance would control its "language" (its content in general) by simply managing that data.
You definitely don't want multiple code repositories for this. After all, if you have two repositories of code, which one is the correct one? If a change is made in one repository but not the other, which one is right? Your code should have a single "source of truth." Multiple sources of truth means no truth.
I am building a website with individual user customization in mind. So for instance, client #1 may say i want this feature to be designed for my account and then user #2 may say i want this added to my account. I was thinking of implementing a php if code where upon authentication those features will be loaded each time they login. But i have hundreds of thousands of customizations that i will be doing, to add each php code for each client will be terrible, is there a way i can do it more easily and with less stress?
Added from comments to particular answer:
"Basically i am running the technical side of a business intelligence company. We offer various reports and information along with a dozen analytic tools. So one user might say i want all that you offer plus a custom feature where a particular area of the business is being analyzed and then upload to their account. As you can imagine there can be hundreds of such requests for a lot of accounts."
Your best bet is to either design yourself a framework, or find an existing one that supports this level of customization. Either way, it's not a small job, and launching code for each user leaves you prone to security issues and bugs arising from out-of-date methods as time goes on.
Ask yourself what kind of customizations you're doing, and if they can be abstracted away from the code level by (for example) a templating system.
Designs can fit in certain generic requirements. So until and unless you have some generic requirements, you can not do the design. For example, if requirement is for different look and feel for each client, you might go for dynamic theming. Similarly, for positional changes for different users, you can think of user profiling. For language, also you can keep the user language as part of the profile. Study different approaches for user profile management and then take a step ahead.
As others have said, its impossible to give a specific answer without knowing a lot more about what you mean by "Customization". The only sensible answer is the one you've already provided - i.e. to implement as much as possible of the variable behaviour in data.
This in itself may not be a trivial exercise - on one application I worked on, I designed a heuristic artificial intelligence engine (in PHP) which was hugely successful - but this was a fairly complex exercise in software engineering, and one I'd be hesitant to recommend to anyone who needs to ask the question.
Assuming that pushing all the application logic into data is not practical, then there are some other approaches you might consider, e.g. splitting the customizations into seperate php files (or templates if its just a display thing):
<?php
session_start();
/* do authentication checks...*/
$app=$_SESSION['user_profile'] . '/' . dirname($_SERVER['SCRIPT_NAME'] . '/inc.php');
if (! include_once($app)) {
include_once('default/'.dirname($_SERVER['SCRIPT_NAME'].'/inc.php');
}
...
Although it would be posible to store php code in a database and eval it at runtime - I'd not recommend this as it opens the door to code injection attacks. OTOH, it may prove easier to implement the customizations in a different language and call that from PHP e.g. prolog.
C.
Sorry for the confusing title....
We are developing an application to be used by multiple companies. For the most part, the application is the same, your standard sort of database manipulation pages (search pages, edit pages, etc.) customized for the data that it is designed for.
However, each company has a slightly different process, and we will be dealing directly with each company so we'd like to use some sort of system that would allow us to tweak pages depending on which company is viewing the page. For example, one company might want a couple extra fields on a data input page, or another company might want to view a different piece of data on a search results screen, and so on.
I understand this is all hypothetical and I wish I had a concrete example to give you, but honestly the companies haven't even given us very good examples. We just want to be ready.
So my basic question is, what is the most flexible way to allow for these tweaks and customizations on a per-company basis? Obviously, the most flexible but least programmer-friendly way would be to make a complete copy of the app for each company. This obviously isn't an option because we'd need to manage updating code on all the sites, trying to keep them all running and tested and having issues resulting from the customized code.
What are your thoughts on Smarty being a solution to this? Perhaps if we have a master set of templates, but then each company can have a different subfolder with any replacement template files... Of course we'd still need to update a bunch of different template files whenever we change one of them, but it would be a little more localized anyway.
Is there a better way? Some sort of differencing template engine maybe, so that we can still edit the original files and the changes will adapt on top of the originals (kind of like a patch)? Or perhaps we should use the object-oriented features of PHP5 and then use polymorphism? What is your best suggestion, and especially if you've had experience with this sort of thing, what are the options and which have you used and why?
I think the template method pattern will help you out a lot. It's really a great pattern for factoring stuff that is mostly the same but differs in a few places. I'm actually working out a template method hierarchy for my own project right now.
I would suggest you try to create the application either using an mvc framework or using your own implementation of mvc.
In this manner you could create models that could be reused (and also views) for other companies.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I know ASP.NET C# very well. I make a lot of stuff like surveys and custom web applications.
I'm having a lot of trouble wrapping my head around Drupal and figuring out how to do 'special' programmatic things, like searching a list of clients, or creating a web application.
1 - I've been reading books and drupal API but I'm still having a lot of trouble getting started with anything at all. Where does custom code go in respect to pages, modules, snippets...? How do I replicate form X that I did in ASP.NET into a new page in drupal? Etc. It is super hard to wrap my head around Drupal, it seems so large and easy and extensible yet also like it is difficult for me to code. What can you recommend for an ASP.NET application programmer to learn to work with/in Drupal?
2 - Sometimes we write complete ASP.NET applications for clients, and these applications are not indended for clients to go editing them. For example, we will have our entire website in drupal, but inside of that website we will have a place for users to use a special search for database information. We don't want simple website editors and content providers trying to edit this specially coded page(s). How do I handle not wanting clients to edit anything in a Drupal application? Is it handled via page/user permissions? Is the best thing to develop and host an application completely outside of drupal? Is the drupal application it's own drupal instance inside of a drupal website (nested drupals?) ?
3 - Lastly, how do I handle databases? I understand Drupal and PHP are set to work best with MySQL. We have MS SQL databases that are used by multiple applications and would like to use them for new applications as we start using Drupal. For example, we have a Staff directory that feeds ASP.NET reporting application, ASP.NET Staff Listing application, and want to make a new Drupal PHP application that also uses (and perhaps inserts/updates more information into) that database. What is the best/easiest way to handle MS SQL databases and/or MySQL databases being used and updated by multiple Drupal and ASP.NET applications? Would it be easier to have all the applications use the MS SQL databases, or maybe to replicate the databases on MySQL for the Drupal/PHP apps and somehow sync the two databases?
I hope it is alright if I ask multiple related questions in a single post.
I'm afraid that I don't have any experience with Drupal so I can't offer any insights into your first or second questions, but on the third question:
What is the best/easiest way to handle
MS SQL databases and/or MySQL
databases being used and updated by
multiple Drupal and ASP.NET
applications?
If you are going to perform CRUD operations from both your ASP.NET applications and your PHP/Drupal applications, then I would recommend against trying to sync data back and forth between MySQL and MSSQL since this will cause latency problems plus you'll then have to deal with the differences between the two database management systems.
Instead, I would recommend that you look at using stored procedures to control all database accesses. This way you can ensure that all of your database CRUD operations conform to same rules and logic.
Using MSSQL in both ASP.NET and PHP shouldn't be a problem, especially not with the very excellent PDO library that provides a standard interface for accessing a number of popular database management systems. It should allow you to quickly and easily connect with and begin using your MSSQL databases within your PHP applications.
It is some good questions you ask. I'll try to answer them to the best of my ability. I haven't any experience with c# of APS.NET, so I don't know exactly where you are coming from. I myself learned Drupal as an inexperienced python/django developer. So in some ways I can understand some of the troubles you are going through trying to learn Drupal. Some of these things will simply require time/experience/experimenting before it will go away.
To really understand Drupal and the Drupal API you will first need a good understanding of PHP. Sometimes Drupal do some complex PHP things, so if you don't understand the syntax or the PHP functions etc, you can easy get lost.
Where do code live?
Drupal is built as a moduler system. Drupal itself is a set of modules, some of which are required for Drupal to work properly. If you want make some custom code, that is you want to do something, that you can't use a module for. There are about 5000 modules developed for Drupal. Often even when being a skilled programmer, the best choice is to find a module that will do what you need or can get you where close. So what you do, it to create a module of your own.
Make a folder with the name of the module
Create a modulename.info inside it which holds some info about the module formatted in a special way. Drupal will need it to fx display the module on the module list page where you activate modules.
Lastly create a modulename.module file where the core of your code for the module will be.
Another thing that is important to understand with Drupal, is that it uses a hook based system. Hooks are like events that fire once certain things happen that you want to hook into and either alter the flow, or do some things of your own. Fx you could record every time a specific form was displayed, or you could alter the form, adding/removing fields.
Forms
Drupal has a FAIP or Form API, that is uses to generate form, this is something that has it's own page in the documentation. The idea is that you create an associative PHP array which holds information about each element of the form, and Drupal will use that to create the form.
Books
There are a lot of good books for Drupal that you can learn from. The book I myself have learned the most from is Pro Drupal Development
First of all, Drupal has a very fine grained role based permission system, that will allow you to setup exactly what your clients are allowed to do. You can create different roles, like moderator, content creator, admin, sysadmin ect, and give different permissions to each role. This is pretty easy and is setup within the Drupal AI. However, you will need to know the permissions as some are super permissions that will give access to a lot of things. Now for integrating your applications, that is something that you probably want to write a custom module for. I don't know exactly how you want to do this. But I think the best result would be gotten by letting Drupal create the pages, forms etc from your application and just send data back and forth. That way the theme = layout of the site would be consistent. That way users wouldn't get the feeling that they left the site, but this simply was yet another feature the site offered.
Drupal is not just set to handle MYSQL best, You will actually need either MYSQL or PSQL as your drupal database backend. The reason is the way Drupal handles queries, that allow you to write non specific queries that will work, no matter on which of the two types you use, or if your tables has a prefix. So for all of the Drupal internals you will need one of the two. PHP can connect to MSSQL and run queries against that database, so you could without much problems write custom code that run queries to your other application's databases and either fetch or update data. Depending on the data, you might want to create a table in your database that you can read and write to, and then sync the databases when needed. It depends a bit on your use case. I have done the latter, in the case where I didn't need to write to the database, but only needed to fetch some product information form a legacy database that was still being used by other systems.
First of all, I think what's important to understand Drupal's limitations, there are things it's not really made for. It's sort of a web application framework but if you're doing a lot of custom work with your own custom datamodelling and stuff, Drupal might not be the most flexible or easily implementable solution to your kinds of problems. A more general framework like for instance Zend Framework might be more suited.
An important lesson in learning Drupal is: don't hack core or other modules. This will make upgrading core or modules very time-consuming. Instead "do like Drupal" and override using hooks. In theming you can also override with hooks, but also in other ways. Overriding is powerful so understand that concept well.
I'd say, pick a way of learning that is your style, screencasts, IRC, hacking, user meetups IRL, books, articles or any combination and look for material on Drupal. Just start mucking around and get a feeling and understanding of how Drupal works. Understanding the jargon is an important part, so what are: nodes, blocks, regions, hooks, modules, themes.
Drupal has a very flexible permission system, and it's probably the best choice to use that instead of making your own. You can write your own modules that add to or alter the behavior of Drupal. So if you want to write a new application that works with Drupal you can write a module performing the functionality the application has to have and make use of all the facilities Drupal offers, including users, the permission system, etc, etc. I don't really understand the last part of your second question "Is the drupal application it's own drupal instance inside of a drupal website (nested drupals?) ?". Drupal is a collection of PHP files on a server that, together with a slew of tables in a database serves request made to it. Multiple Drupal websites (so different PHP files) can reference the same database or parts of it (for example the user table). There's not really an "instance" of a Drupal site, as there's not really an "application". I could of course be too unfamiliar with these terms but I don't think PHP works with "instances" of "applications".
That's a hard problem, afaik Drupal doesn't work with MS SQL, so any connections in that direction you might have to make yourself. I'd also ask around the Drupal forums or on IRC.
Good luck!
rlb,
I've done quite a bit of Drupal and have strong ASP.NET experience. You really need to read up to get your mindset straight. They use layers in an MVC-like fashion that is very foreign to WebForms & ASP.NET MVC developers ... and quite frankly at first seems odd.
Here's a list of things I did to get really, really good quickly:
Get a host like at hostgator for $8 which will let you install Drupal
Get the book Pro Drupal Development for Beginners ... excellent and really covers a lot of areas http://www.amazon.com/Pro-Drupal-Development-Second-Beginning/dp/1430209895/ref=sr_1_2?ie=UTF8&s=books&qid=1257545528&sr=8-2
Get on the Drupal forums
Be ready to contribute. Drupal has a lot of features, but in the end the best way is to be someone who contributes code back
There are a number of IDEs to consider, but to start a text editor should be fine.
Good luck.
Some good answers here, so I'll just fill in some brief items.
1 - Learning Drupal: Pro Drupal Development is the best book for this. Getting into the issue queues and interacting with developers is a way to get familiar with specifics. Your question about forms relates to how Drupal handles forms. The FAPI is pretty robust in Drupal, and protects you from security gaffes. Also api dot Drupal dot org is where the code is documented, though there are docs elsewhere. Google is your friend. (On the Drupal site itself, use the native search to get faceted results.)
2 - This is a user permissions issue. You can limit creation and editing permissions based on content type. For access control (read), you will want to use one of the many modules for access control. It really depends upon your use case.
3 - Drupal has the most community support for MySQL, but people run Drupal on MS SQL as well. In Drupal 7, you will likely see much more support for non-MySQL as this release introduces database abstraction.
Probably the worst thing to do is hack around the margins. It really pays to learn the ins and outs of the Drupal basics -- administration and coding -- so that you can truly leverage the power Drupal brings to the table.
And if you're hunting for modules, http://drupalmodules.com is your friend. ;)
I just inherited a 70k line PHP codebase that I now need to add enhancements onto. I've seen worse, at least this codebase uses an MVC architecture and is object oriented. However, there is no templating system and many classes are deprecated - only being called once. I think my method might be the following:
Find all the files on the live server that have not been touched in 48 hours and make them candidates for deletion (luckily there is a live server).
Implement a template system (Smarty) and try to find duplicate code in the templates.
Alot of the methods have copied and pasted code ... I don't know how much I want to mess with it.
My questions are: Are there steps that I should take or you would take? What is your method for dealing with this? Are there tools to help find duplicate PHP code?
Find all the files on the live server that have not been touched in 48 hours and make them candidates for deletion (luckily there is a live server)
By "touched" I'm assuming you'll stat the file to see if it's been accessed by any part of the system. I'd go a month and a half on this rather than 48 hours. In older PHP code bases you'll often find there's a bunch of code lying around that gets called via a local cron job once a week or once a month, or a third party is calling it remotely as a pseudo-service on a regular basis. By waiting 6 weeks be more likely to catch any and all files that are being called.
Implement a template system (Smarty) and try to find duplicate code in the templates.
Why? Serious question, is there a reason to implement a template system? (non-PHP savvy designers, developers who get you into trouble by including too much logic in the Views, or you're the one creating templates, and you know you work much faster in smarty than in PHP). If not, avoid it and just use PHP.
Also, how realistic is it to implement a pure smarty template system? I'd give favorable odds that old PHP systems like this are going to have a ton of "business logic" mixed in with their views that can't be implemented in pure smarty, and if you allowed mixed PHP/Smarty your developers will use PHP everytime.
Alot of the methods have copied and pasted code ... I don't know how much I want to mess with it.
I don't know of any code analysis tools that will do this out of the box, but it sould be possible to whip something up with the tokenizer functions.
What You Should Really Do
I don't want to dissuade you or demoralize you, but why do you want to cleanup this code? Right now it's doing what's is supposed to do. Stupidly, but it's doing it. Every re-factoring project is going to put current, undocumented, possibly business critical functionality at risk and at the end of that work you have an application that's doing the exact same thing. It's 70k lines of what sounds like shoddy code that only you care about fixing, no mater what other people are telling you their priorities are. If their priority was clean code, their code would already be clean. One person can't change a culture. Unless there's a straight forward business case for that code to be cleaned (open sourcing the project as a business strategy?), that legacy code isn't going anywhere.
Here's a different set of priorties to consider with legacy PHP applications
Is there a singleton database object or pair of objects that allows developers to easily setup seperate connections for read (slave) and write (master). Lot of legacy PHP applications will instantiate multiple connections to the same database in a single page call, which is a performance nightmare.
Is there a straight forward way for developers to avoid SQL injection? Give this to them for new code (parameterized SQL), and consider fixing legacy SQL to use this new method, but also consider security steps you can take on the network level.
Get a test framework of some kind wrapped around all the legacy code and treat it as a black-box. Use those tests to create a centralized API developers can use in place of the myriad function calls and copy/paste code they've been using.
Develop a centralized system for configuration values, most legacy PHP code is some awful combination of defines and class constants, which means any config changes mean a code push, which means potential DOOM.
Develop a lint that's hooked into the source control system to enforce code sanity for all new code, not just for style, but to make sure that business logic stays out of the view, that the SQL is being contructed in a safe way, that those old copy/paste libraries aren't being used, etc.
Develop a sane, trackable build and/or push system and stop people from hackin on code live in production
I don't know of any specific tools, but I have worked on re-factoring some fairly large PHP projects.
I would recommend a templating system, either Smarty or a strict PHP system that is clearly explained to anybody working on the project.
Take discrete, manageable sections and re-factor on a regular basis (e.g., this week, I'm going to re-write this). Don't bite off more than you can chew and don't plan to do a full rewrite.
Also, I do regular code searches (I use Eclipse and search through the files in my project) on suspect functions and files. Some people are too scared to make big changes, but I would rather err on the bold side rather than accept messy and poorly organized code. Just be prepared to test, test, test!
You need to identify a solid reason for refactoring. Removing duplicate code is not really a very good one; it needs to be coupled with a real desired improvement, such as reducing memory footprint (useful if the webservers are struggling).
Once you have that in mind, now you can start refactoring. And make sure you have a version-control repository, too. Just don't check in broken code.
Don't be too hasty about single-use classes. A lot of small PHP frameworks work like that. Often they could be abstracted better, though. Also, A lot of PHP code also doesn't understand data layer abstraction with the result that there is SQL code littered through the business logic or even the display code. This problem is often coupled with no custom database handler, which is a problem if you suddenly have to teach it about replication, or caching. This is the same abstraction problem from the other direction.
One very practical step: once you start abstracting repeated code away, you'll find reasons to have multiple files open. If you're using a shell and a Unix editor, then screen will help you immensely.