I am building a SaaS application that has a set of requirements. One of these requirements is that the application has to be multi-tenant with a seperate schema for every tenant. But there has to be a single instance (admin interface) that needs to access all those schemas for data validation.
Example:
We have x customers who will access their application through customerx.our-company.com. Every customer gets its own schema with multiple tables. These tables contain users, invoices, contracts, timesheets,... The customer has to be able to view/modify these records.
We as a company providing the service have to be able to get all the timesheets from all those customers to be able to verify them. Once verified these records will be updated. 'verified' => true
My first idea was to store a tenant_id in a customers table in the admin database to be able to change the database connection in eloquent and merge the records before returning them. This is doable, but I have some questions about performance. What if we grow to 1000+ customers. getting all those records could take a long time.
Dumping all the records in a single schema without providing a separate schema per customer cannot be done because of GDPR. The data of the customer has to be separated.
What would be the best approach for this design wise? Are there some good articles about this subject?
What I began to think while writing this question: It is possible to leverage c++, I can create a custom PHP extension that takes multiple tenant database info to get the records via async tasks std::async, merge those records and return them. std::async creates multiple threads in a pool and I can wait until it is finished. The only big deal will be relations that need to be returned.
Updating those records is a simpler task as we know where it is stored (customer_id) so we can change the DB connection on the fly while updating.
EDIT: I created a simple c++ program that creates async MySQL connections. I've tested this with 10 schemas at once retrieving 1000 records from each. This looks promising, will keep this updated.
Related
I am running a Symfony 2.8 web app which allows the user to store and manage several different data entities
Over time the database grew bigger and bigger where a lot of data belongs to inactive/abandoned accounts which are no longer used.
Of course it quite easy to identify these accounts and the corresponding data can simply be deleted. However, I can never be sure if the users may return to his account and re-starts using his data. Thus I would like to implement a two step process:
If the user has not access his data for 3 month the data is moved from the production database to a cold storage. This would be just another database which holds the necessary tables to save the user data.
If the user has not accessed his account for 12 month the account and the data will be deleted.
How to setup such a cold storage?
According to the Symfony docs it should be no problem to work with two different EntityManagers to connect to two different databases.
But how to manage and setup the cold storage database?
How do I tell Symfony/Doctrine to create the tables for EntityA and EntityB entities (the data) in the production database as well as in the cold storage database but the table for EntityC only in the production db (e.g. the user settings)?
How to move the entities from one database/EntityManger to the other? Is there a more effective solution to do this instead of just loading the entity from one EntityManger and persisting it in the other?
Is there any best practice to solve this?
I have a web application where companies can register their company and use a set of features. However, lets say company 1 and company 2 has registered. They are still accessing the same website. Now each of these companies are 100% independent of each other when it comes to sharing information etc. The only thing they might share, is the users/employees.
Now my question is really, what is the best practice if each of these companies are to insert, select, update and deleted about 10K rows a day, each.
It can be everything from project handling, hourlists etc. All of which are split into different tables.
Would it be best practice to have independent databases, or use the same database for all the companies, and identify them by company_id?
Also keeping in mind the web application has to easily adapt to more than 10+ companies.
You could go one of two ways:
Add a companyId column to your tables,
Create a separate database for each company.
Option 1:
This option is the most dynamic one. You can keep the data separated by adding the correct companyId identifier to the where clause of your query.
This method is good when:
You expect a large number of customers,
You expect your number of customers to increase and decrease on a regular basis,
You do not need to share your database access with your customers (they only access it through your API/GUI).
Option 2:
This option gives a better separation of data. You keep each custommers data in their own dedicated instance of the database schema. This option allows you to offload the access-control burden to the database server, instead of having to enforce it in your application logic (which is more error prone).
However, there are some downsides: whenever a new customer shows up, you need to create a new database instance for them, which implies having a user with create database and grant privileges, something not every system administrator would be overly happy about.
The other issue is that whenever something changes in the database structure, you need to apply the chance to each instance of the database.
The good thing about this option is that you can give backup copies of your database to your customers, give them direct access to the database server, if needs be, or, in a more limited form, you could give them a copy of the database structure, without the need to filter out the customerId columns (as would be the case with option 1 above).
In summary:
There is no silver bullet, it all depends on your use-case. Option 1 is more flexible, Options 2 offers a better separation of data and easier access management.
[1]Keep separate database as there is more DML operations with your database.
[2]Keep very good database maintenance plan for Statistics management, Index maintenance and Backup/Recovery,otherwise you will have performance issue or more down time in case of database crash.
I am currently working on a large application that will be used by many people each having a large amount of data. I thought to manage these multiple users through a single database but I am asked to create a separate database for each new user that is registered. Now what I am wondering is : is it a good idea to do so i.e. having the app create a separate database for each new user that gets registered and manage it's data through it? What will be the performance issues, if any?
Separate DB for each new registered user might be bad idea. You can do it like this;
Put 100 users in each separate db. 1-100 => DB1, 101-200 => DB2, n, n+100 => DBn
You can keep a table for which id interval will be connect to which db. By doing this, you can lower db load. 100 users for each db is just an example. You need to use such a structure for a system that has lots of users.
My answer is create a separate database for each new user is a wonderfool idea.you must set appropiate indexes over the table and you will get good performance
I am currently helping my company to redesign the CRM.
They plan to extend the CRM to read data from another database which is used by another software.
All database are using mssql technology.
Actually, our company has three brands and each brand's data is stored in different mssql database.
Brand1:
Customer
Invoice
Payment
Brand2:
Customer
Invoice
Payment
Brand3:
Customer
Invoice
Payment
In my new database schema design. A Customer will have several invoices and each invoice receives several payments.
It is ideal to have all data to store in my newly designed db because I can extract the MOST UPDATED payments like this:
SELECT [Everything I want]
FROM Customer
INNER JOIN Invoice WHERE Invoice.customer_id=Customer.id
INNER JOIN Payment WHERE Payment.invoice_id=Invoice.id
But now I need to MAKE CONNECTION TO THREE database, getting data from them, and COMBINE the result to generate data structure like this:
{
customers:[
{
customer_name: cato,
invoices:[
{
invoice_id: 1,
payments: [bla,bla,bla]
}
]
}
]
}
Now, my company thought of using trigger but it is hard to maintain. Is there any better options that can do the job?
yes, sql has a great solution for this,
actually what you need is sql replication,
it gives you the ability to 'copy' tables from remote db to local, first you copy the table to your local and then you set the replication and the remote db will forward all his insert, update etc.. commands your your DB, so in fact, your local replicated table will be always syncronized with the remote one
take a look at it and follow tutorials to do the replication
I've been working on a web app for a few months now. It's a PHP and MySQL driven database app which relates objects between each other.
I'd like to add functionality so that someone could register to use it and set up a monthly subscription. When they log in the app would simply populate with data from their own database.
I've done some scouring online but I'm struggling to find a starting point for adding this sort of feature.
The app relies on half a dozen tables within a database. So I'm also not sure if creating an individual database per user is practical.
Creating a db per user is very rarely the way to go - it's complicated and has very few benefits with lots of drawbacks (eg maintaining multiple simultaneous Db connections as most libraries only connect to a single Db). What you really need to do is create a user table and then tag various records with a UserId.
When a user registers, create a new User record, then create their new entries in other tables all referencing that Id. (Actually, only the "root" of each relational object needs a UserId eg If you have Order and OrderItems only the Order table needs a UserId)
Without more detail, it's impossible to help further.
Incidentally, the only time you should consider db-per-user is if each user requires unique tables (in which case your app would have to be very complex to actually use the data) or if there are security concerns re: storing data together (more likely with multiple clients than multiple users).