This isn't about a specific problem I'm having, but I have to clear something up as I have alot at stake with this project.
I'm creating a web application that will deployed to some BIG companies in the UK and if we get this wrong it could cost us hugely!
I need to keep each companies data VERY secure, so what I was wondering is would it be a good idea to create a new database for each organisation, this way their data is totally seperate and if one database is compromised the data for all the organisations wouldn't be effected - this would give us time to react to the security problem before all the data is compromised. My Questions are:
What are the short term and long term benefits of this (if any)
What are the short term and long term drawbacks of this (if any)
Is this good practice?
Would it even solve a security issue in the way I anticipate?
Thanks in advance
I'm assuming this is a question about building a "multi-tenanted architecture" for a "software as a service" style application.
The first issue is that separating your databases may or may not be a good idea - but it's not the first question to ask. Someone who can get access to your databases at the level where this matters has already penetrated your application in ways that are hugely damaging - they can almost certainly execute arbitrary commands on your database server. This means that you're not just dealing with damage to one account, but to your entire infrastructure. It's a "lights out" moment, and you have to shut down the whole system whilst you recover.
If they haven't established a shell on your database server, it would mean there's an application-layer security issue - SQL injection, or some way of escalating privileges in your authentication scheme. Again, both are "lights out" moments.
So, make sure all that stuff is totally covered. Include security testing in your development lifecycle; consider using automated penetration testing tools as part of your continuous integration system. Make sure the infrastructure guys harden the whole environment, and consider having a 3rd party security audit when you get close to a release candidate. Consider a code review process focussed on security issues, and agree coding standards with specific security considerations. Tell all your developers about cross site scripting, SQL injection and other application-level vulnerabilities.
Once you've done all that, you've locked the doors and bolted the windows; your database strategy is the equivalent of how you keep the jewelry safe.
Separate databases offer some additional security - but only if you have a corresponding strategy for user management. In most web applications, there are only 2 types of user: "admin" and "web app". "Admin" can create/modify databases (creating databases, tables, views etc.), and can usually also modify data. "Web app" should have only data modification rights, but no rights to modify databases objects.
For splitting the databases to make sense, you must ensure that:
an attacker who can get access to your web application's file system cannot get access to valid user names and passwords, or if they can, only for one client.
an attacker can never get access to the "admin" credentials
However, there are other reasons (beyond security) where it makes sense to split up your databases. It reduces the risk of human error, it allows you to scale your system at a more granular level, and it allows you to offer different levels of hosting ("gold" users get their own server, "silver" their own database, "bronze" take their chances).
The big issue you have to solve to make this happen is deployments - how will you deploy new versions of the code, with changes to the database? This in turn may complicate the testing process.
Separate databases, almost certainly. As #Anigel says, perhaps even separate servers, although the management and cost of that is more complex, so will depend on your exact requirements.
One benefits is that at some point in the future, perhaps one client's data gets bigger, so it's easier to split to a new server, or perhaps they get an upgrade of your platform but another doesn't.
Backup and restore is easier if you can dump/load whole databases, rather than picking out all the tables with names beginning with clientX_.
Performance measurement will be easier on separate databases.
Just a few quick ideas to get started.
#Lee Price, Maintaining the separate database for every company will be great thing.
Advantages:
It will keep your database secure
In long time when the size of the tables are limited to a single company that will give you a drastic performance. The operations will be fast.
Easy to manipulate the data company wise.
Help out in terms of support
Disadvantages:
Need to Keep track of the schema change for every company
Maintain Separate databases
Maintain separate backups
Occupy more space as compare to a single database
But i will personally suggest you to go with the separate database for each company.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
I have a question for people who have experience working with Oracle and PHP. Please this is not to open a meaningless debate. I just want to make sure everything is considered correctly in my system.
I am working on a project in which there are thousands of users, divided into groups and sub groups. Each group has its different access rights and each subgroup has its own privileges.
I need to have your opinion about these two approaches:
Implementing access rights and privileges in PHP with one big
application user(oracle account),(I am clueless as to the advantages
and disadvantages of this approach).
Implementing access rights and privileges in Oracle database(each
user would be an Oracle account) and use the virtual private
database, caching, secure roles.... from a performance stand point
this is the best approach. Security! well I know it is good but I am
afraid I am missing good things not implementing it in PHP.
I did some research on the net but in vain(I scratched my head a lot). I am new to PHP but I have good knowledge about Oracle.
Any suggestions, Ideas?
As you say you're going to have 1000s of users, i assume your software is going to be used in a big company, which probably means there's not one IT department, but several of them - one providing managed basic hardware (OS level but no applications), another managing databases, and a third one putting it all together (hardware+os, database, application) and providing the managed application as a service for the end user. My decision might be heavily influenced by working for such a department for over 10 years now.
I've seen one application that used the "one database user per OS user" approach (VPM by Dassault Systems, an attachment to the Catia V4 CAD system - and it's a horror to maintain. You want to have your permissions in ONE place, not several. Possibly you have a nice GUI to change them, but when your user calls you saying "i can't do X", and your GUI says he should be able to do X, it's simply too tedious to search everywhere. We've had cases where views didn't have the access roles they should have, views were wrongly defined, users had permissions on some tables but not all of them, all that stuff.
Additionally, our database department has - at the moment - about 600 "databases" that are used by diffent departments. So they are running about 20 real "databases" on several clusters, and they have implemented quite a rigid scheme of database names and corresponding user names. Each database has a base name XXX, with XXX_A the user name DDL statements, and XXX_U for DML. Applications may ONLY use XXX_U, which also means applications may not do DDL. This allows the database admins, in case of load issues on the cluster, to move an entire schema, including all users, roles and tables, to a different instance on a different cluster easily, without knowing too much about the individual databases. Of course, our VPM database doesn't fit into that schema, so we had to argue with the DB people a lot - and our monthly charge by the DB department is much higher than normal, because they have much more trouble administrating it.
Bottom line: Don't expect you can do whatever you want within your database. In a large company, with a department managing the databases, you will have restrictions what your application is allowed to do and what it isn't.
Also, your management might, at one time, decide they want to move to a different database system like DB2 for political reasons. This has much less to do with technical advantages than with who's invited whom to golf. You might, at one time, be asked if your system could be moved to a different database, by people you don't want to say "no" to. I wouldn't want to be dependent on too specific oracle features in this case.
Also keep in mind that requirements change over time, and there might be new, more granular, requirements in a while. This strongly favours doing the permission stuff in software, because it's much easier to add another column to a permission table that specifies something new, instead of having to implement something new in a database permission structure that just isn't meant to handle this kind of thing.
If you were developing a native application that runs on individual users' PCs, using only one oracle account might be a big security hole. But as you're using PHP, it's only the server that's communicating with the DB, so noone can extract login information from userspace anyways.
In my opinion, create an api for permission management first. Do not use oracle users, groups and roles for that; instead, manage your permissions in some SQL tables. Create an api (a collection of check_if_user_X_may_do_Y functions), possibly in pl/sql if you feel more comfortable there, better in PHP if you want to be portable. Build your application on this API. It might be more dev work at the start, but will result (imho) in much less administration work later.
Although Guntram makes some very salient points, he has missed what I consider to be fairly essential considerations:
1) you describe a 3 tier model of authorization which the Oracle permissions model does not accomodate (although it is possible to represent this as a 2-tier model but at the cost of creating work and complexity).
2) using the user supplied credentials to authenticate against the database pretty much precludes the use of persistent database connections - which is rather important when your connection setup time is as expensive as it is with Oracle.
By all means store the representation of the groups/sub-groups/users/permissions in the database but not as Oracle users. But use a single (or small number) of oracle accounts for the access from PHP to the DBMS.
If you want to implement strong security, then you should also store mappings between sessions ids and users (not necessarily the session data itself) in the database, allow no direct access from the designated Oracle accounts to the data - only via stored procedures - and pass the session id as an authentication token to the stored procedure and validate it there (but note that this will be rather expensive if you are using trivial ORM).
I´m new on php/mysql, and i´m codding a simple CMS. But in this case i will host multiple companies (each company with their multiple users), that pays a fee to use the system.
So... My question is about how to organize the Data Base... Talking about security, management and performance, i just want to know the opinion of ou guys of wich of these cases is the best:
Host all companies on a single DB and they get a company id to match with the users.
Each company have a separated DB that holds the users in there (and dont need the companies id anymore).
I would start the development following the first situation... But than i thought if i have some hacker attack / sql injection, every client would be harmed. Having separated DBs, the damage will get only one client. So maybe the 2nd situation could be better in terms of security. But could not say the same about management and performance.
So, based on your experience, any help or tip would be great!
Thanks in advance, and sorry about my poor english.
I would go for seperate DBs. But not only for hacking.
Scalability:
Lets say you have a server that handles 10 websites, but 1 of those websites in growing fast in requests, content, etc. Your server is having a hard time to host all of them.
With seperate DB's it is a piece of cake to spread over multiple servers. With a single one you would have to upgrade you current DB or cluster it, but that is sometimes not possible with the hosting company or very expensive.
Performance:
You they are all on 1 DB and data of multiple users is in 1 table, locks might slow down other users.
Large tables, mean large indices, large lookups, etc. So splitting to diffrent DB's would actualy speed that up.
You would have to deal with extra memory and CPU overhead per DB but they normaly do not have an amazingly large impact.
And yes, management for multiple DBs is more work, but having proper update scripts and keeping a good eye on the versions of the DB schema will reduce your management concerns a lot.
Update: also see this article.
http://msdn.microsoft.com/en-us/library/aa479086.aspx
Separate DBs has many advantages including performance, security, scalability, mobility, etc. There is more risk less reward trying to pack everything into 1 database especially when you are talking about separate companies data.
You haven't provided any details, but generally speaking, I would opt for separate databases.
Using an autonomous database for every client allows a finer degree of control, as it would be possible to manage/backup/trash/etc. them individually, without affecting the others. It would also require less grooming, as data is easier to be distinguished, and one database cannot break the others.
Not to mention it would make the development process easier -- note that separate databases mean that you don't have to always verify the "owner" of the rows.
If you plan to have this database hosted in a cloud environment such as Azure databases where resources are (relatively) cheap, clients are running the same code base, the database schema is the same (obviously), and there is the possibility of sharing some data between the companies then a multi-tenant database may be the way to go. For anything else you, you will probably be creating a lot of extra work going with a multi-tenant database.
Keep in mind that if you go the separate databases route, trying to migrate to a multi-tenant cloud solution later on is a HUGE task. I only mention this because all I've been hearing for the past few years around the IT water coolers is "Cloud! Cloud! Cloud!".
I'm part of a team currently developing a dedicated SaaS service for a specific crowd of organizations.
I have knowledge in PHP and mySQL, so I am developing it on these platforms. Will be deploying this on the cloud and releasing in multiple countries.
Ive come to the point of separating organizations/main-users in the database and wanted to see what you guys think.
When the SaaS manages invoices and many other sensitive information what would be the best process of distributing it on the mySQL server? Ive thought of the below options:
1) having all information in a single database in single tables and separated by a organization identifying row. - does seem secure and may be slow when there are a few thousand users and 10,000 rows?
2) having a single database but separating tables with a user id eg. '1000_invoices' - again may be faster but not as secure.
3) have separate databases created on each organization signup and a specific user used to access the database and the database name is stored in the sessions/cookie? per organizations users.
Anyway i was wondering what you guys think will be the best option? and if not the above then what do you recommend? and why? also anything regarding security will be greatly appreciated. Have not worked with large multi-organization applications before.
thanks in advance!
I've developed numerous SaaS applications in the past and we've found the "single application deployment, single datatabase" setup as used by large "public" SaaS services (like KashFlow, possibly Salesforce?) didn't make much sense. Here's why:
Number 1: Client companies with confidential information are going to want assurances their data is more "secure" and it's easier to make these promises when their data is partitioned beyond the application tier.
Different clients sometimes want their software application customized to them, such as their own extra database fields, a different login screen visual design, or different (or custom) system "modules" - having different application instances makes this possible
It also makes scaling easier, at least when beginning. It's easier to load-balance by provisioning a single client's application to its own server separate from the others, whereas with a single application it means you need to spend longer developing it to make it scalable.
Database primary keys are easier to work with. Your customers might start asking questions such as why their "CustomerID" values increment by 500 each time instead of by 1.
If you've got customers located around the world, it's easier to provision a deployment in another country (with its own local server and DB server) rather than deploying a giant application abroad or forcing users to use intercontinential (i.e. slow+laggy) connections to your servers located thousands of miles away.
There are downsides, such as the extra administrative burden of managing hundreds, possibly thousands of databases, in addition to application software deployments, but the vast, vast simplifications it makes to the program code make it worthwhile. Besides, automating provisioning and deployment is easy with a bunch of shell-scripts.
I'm going to try to make this as brief as possible while covering all points - I work as a PHP/MySQL developer currently. I have a mobile app idea with a friend and we're going to start developing it.
I'm not saying it's going to be fantastic, but if it catches on, we're going to have a LOT of data.
For example, we'd have "clients," for lack of a better term, who would have anywhere from 100-250,000 "products" listed. Assuming the best, we could have hundreds of clients.
The client would edit data through a web interface, the mobile interface would just make calls to the web server and return JSON (probably).
I'm a lowly cms-developing kinda guy, so I'm not sure how to handle this. My question is more or less about performance; the most I've ever seen in a MySQL table was 340k, and it was already sort of slow (granted it wasn't the best server either).
I just can't fathom a table with 40 million rows (and potential to continually grow) running well.
My plan was to have a "core" database that held the name of the "real" database, so the user would come in and try to access a client's data, it would go to the core database and figure out which database to get the information from.
I'm not concerned with data separation or data security (it's not private information)
Yes, it's possible and my company does it. I'm certainly not going to say it's smart, though. We have a SAAS marketing automation system. Some client's databases have 1 million+ records. We deal with a second "common" database that has a "fulfillment" table tracking emails, letters, phone calls, etc with over 4 million records, plus numerous other very large shared tables. With proper indexing, optimizing, maintaining a separate DB-only server, and possibly clustering (which we don't yet have to do) you can handle a LOT of data......in many cases, those who think it can only handle a few hundred thousand records work on a competing product for a living. If you still doubt whether it's valid, consider that per MySQL's clustering metrics, an 8 server cluster can handle 2.5million updates PER SECOND. Not too shabby at all.....
The problem with using two databases is juggling multiple connections. Is it tough? No, not really. You create different objects and reference your connection classes based on which database you want. In our case, we hit the main database's company class to deduce the client db name and then build the second connection based on that. But, when you're juggling those connections back and forth you can run into errors that require extra debugging. It's not just "Is my query valid?" but "Am I actually getting the correct database connection?" In our case, a dropped session can cause all sorts of PDO errors to fire because the system no longer can keep track of which client database to access. Plus, from a maintainability standpoint, it's a scary process trying to push table structure updates to 100 different live database. Yes, it can be automated. But one slip up and you've knocked a LOT of people down and made a ton of extra work for yourself. Now, calculate the extra development and testing required to juggle connections and push updates....that will be your measure of whether it's worthwhile.
My recommendation? Find a host that allows you to put two machines on the same local network. We chose Linode, but who you use is irrelevant. Start out with your dedicated database server, plan ahead to do clustering when it's necessary. Keep all your content in one DB, index and optimize religiously. Finally, find a REALLY good DB guy and treat him well. With that much data, a great DBA would be a must.
I am writing a PHP application in ZF. Customers will use it to sell their products to final customers. Customers will host their application on my server or they could use their own. Most of them will host this application on my server.
I could design one database for all customers at once, so every customer will use the same database, but of course products etc. will be assigned to particular customer. Trivial.
I could use separate database for every customer, so the database structure will be simpler. I will then probably use separate subdomains and maybe even file location, but that is just a detail.
Which solution will have better performance and how big will be the difference? Which one would you choose?
I would use a separate database for each customer. It makes backup and scaling easier. If you ever get a large customer that needs some custom changes to the schema, you can do it easily.
If one customer needs you to restore their data, with a single database it is trivial. On a shared db, much harder.
And that if large customer ever gets a lot of traffic, you can easily put them on another server with minimal changes.
If one site gets compromised, you don't have all of teh data for everyone in one place, the damage is mitigated to just the site that was hacked.
I'd definitely recommend going with 1 db per customer if possible.
Personally, I would go with multiple databases - i.e. a database for each client.
As I understand it all your clients will be using just an instance of your application so these instances should have their own databases.
If you go with a single database, you are creating a great potential security risk. One client compromising the login details to the db server would automatically compromise data of all your clients.
Also a single security vulnerability (a SQL injection attack) could destroy data of all clients (with multiple dbs you could still have time to fix the security hole and release a patch before all other sites are attacked).
You don't want to have an army of 1000000 mad clients instead of just 1 angry client.
Multiple databases also give you a greater possibility of load balancing (you can have the dbs spread across more servers).
Performance wise you're basically start with a 'sharding' approach. Because of this, the sharding performance strategy will be piece of cake.
The downside is that you could argue you're losing some (undefined) bit of overhead in the duplication.
One pitfall is that you might not notice performance issues in major components as quickly. This is because they are so scattered, so they might not be visible on your radar. Load testing is the way to get ahead of this.
To some extent this is a question of personal opinion. There are pros and cons of both models.
Personally, and because of the "they could use their own" comment, I would go with a seperate database per customer. This gives you
The ability to move customer data around when necessary. For example moving a single customer onto a different servers/setups depending on things like load.
If something goes wrong you only impact one customer and not everybody.
You can spread DB load across multiple DB servers if necessary.
If a customer comes to you with a specific requirement you can more easily cater for this without impact other customers.
From a performance perspective, to be honest I don't think there is any real performace gain in either model. That said this does of course depend on the structure of your DB and the hardware it runs on.
Don't choose multiple databases solution, if your needs can be fulfilled with one database. Because multiple databases will lead to big burden in long run, and your system will become highly complicated and unmanageable as you grow.
Using proper relationship you can go long way
A Client model can have many Products // why multiple databases?
Performance can achieved in either ways, just going multiple dbs will NOT benefit in that direction