I'm building an application (using the zend framework) where we will have multiple clients who login and use the application, and each of these clients will be storing lots of data about their users (I'm using MySQL btw).
Basically I'm wondering 2 things:
Is having multiple databases, one for each client (ie. ipd_client_CLIENTNAME) with identical tables a stupid idea? Or will it actually be more responsive (than putting everything in one database) if we have 50 clients with 20,000 users' data in 'their' database?
I've already managed to build the same system but all on one database, my model classes simply grab the name of the client logged in (in the model classes' init() method) and then dynamically change the name of the table they grab data from, is this going to be just fine?
If the "every client on the system gets their own database" makes sense, how exactly would I dynamically change what database my db models connect to (based on which client is logged in)?
In case this made no sense, here's an example of what the databases would look like in the two different scenarios (given 2 clients registered on the system):
Multiple Databases:
ClientA has a database called "ipd_clients_ClientA" with tables "users", "lists".
ClientB has a database called "ipd_clients_ClientB" with tables "users","lists".
Single Database:
ClientA has tables called "users_ClientA", "lists_ClientA".
ClientB has tables called "users_ClientB", "lists_ClientB".
Hopefully that makes sense.
Any help would be GREATLY appreciated, thanks!
i would use Solution 2, with multiple Tables for your clients in combination mit MySQL Master / Slave Replication.
MySQL Master Slave
If you want to use different Databases, you can use native functions of Zend Framework:
Tutorial multipe dbs
And than select the one for the client by some kind of prefix.
Although you accepted that answer, I beg to differ. I think having multiple database would be much easier to manage.
The way that tutorial uses multiple databases is far more complicated than it needs to be. The way I do it is by simply defining the client name in a root file (index.php?) and then my bootstrap uses this constant to load in the correct ini file.
If you start with the separation, you can then go on to using the same principles for caching data in separate structures far more easily.
Using separate databases would make it much easier to manage individual clients.
While ZF does have support for multiple database connections, I would not use them in this particular case.
It makes more sense to me to store a client_id (or site_id) in each of your data tables and use that to filter. Duplication makes it very hard to make application-wide changes. Whenever I see dynamically named tables like that, I usually think it's poor design.
For caching, just prefix any cache keys with the client identifier.
Related
I'm staring to build a system for working with native languages, tags and such data in Yii Framework.
I already choose MongoDB for storing my data as I think it feets nicelly and will get better performance with less costs (the database will have huge amounts of data).
My question regards user authentication, payments, etc... This are sensitive bits of information and areas where I think the data is relational.
So:
1. Would you use two different db systems? Should I need them or I'm I complicating this?
2. If you recommend the two db approach how would I achieve that in Yii?
Thanks for your time!
PS: I do not intend this question to be another endless discussion between the relational vs non-relational folks. Having said that I think that my data feets mongo but if you have something to say about that go ahead ;)
You might be interested in this presentation on OpenSky's infrastructure, where MongoDB is used alongside MySQL. Mongo was utilized mainly for CMS-type data where a flexible schema was useful, and they relied upon MySQL for transactions (e.g. customer orders, payments). If you end up using the Doctrine library, you'll find that the ORM (for SQL databases) and MongoDB ODM share a similar API, which should make the experimentation process easier.
I wouldn't shy away from using MongoDB to store user data, though, as that's often a record that can benefit from embedded document storage (e.g. storing multiple billing/shipping addresses within a single user document). If anything, Mongo should be flexible enough to enable you to develop your application without worrying about schema changes due to evolving product requirements. As those requirements become more clear, you'll be able to make a decision based on the app's performance needs and types of database queries you end up needing.
There is no harm in using multiple databases (if you really need), many big websites are using multiple databases so go a head and start your project.
At my work we have two separate sites that are very closely related. One of them is a ASP/MSSQL site and the other is a PHP/Postgres site.
I want to create a REST API that everything from now on is built on top of. I would like it to be tied to both DBs so that it can be a a single point of retrieving and setting data.
I was thinking of using a DBA like Doctine to keep from writing queries in two different syntax. In the same system is it possible to tie parts of Doctrine to the MSSQL and other parts to Postgres?
If so, how? Any other thoughts on design are welcomed.
Within your application framework, you need to configure two separate entity managers, each of which will connect to a different database. More on entity managers at http://docs.doctrine-project.org/projects/doctrine-orm/en/latest/reference/configuration.html
The core architectural pattern is that your models are plain PHP objects, and the entity manager (Data Mapper) will read the mapping configuration to know how to map the models to a database.
If you're writing a REST based API it shouldn't really matter what the DB backend is. For example, if you write your API in a combination of Django and tastypie, you could simply swap out a settings.py config to work with both Postgres and MySQL... or even MongoDB if you so chose
The point is, a REST API is a generic solution that can be used by a multitude of languages, you should chose a framework that allows you the same flexibility in DB backends to implement.
I'm looking into developing a multi-tenant SaaS application, and I found several sites that describe a solid way to separate the data using tenantIDs and updateable views. e.g. This blog post
It all hinges on the ability to have your user accounts authenticated from a master users table and then having their respective database connections use those user-specific credentials. This way, the views can pull the userid and map it to the tenantID to display that user's view. However, most PHP frameworks tend to be very static when it comes to database connections (stored in text config files). They appear to be at odds.
Does anyone know:
a) how to make CodeIgniter handle this gracefully?
b) a different PHP framework that might?
At a horrendously basic level you can do this:
http://philsturgeon.co.uk/blog/2009/06/How-to-Multi-site-CodeIgniter-Set-up
Expand it as required, or move the logic into MY_Controller for more flexibility.
There is a topic talking about this on the Code Igniter forums.
http://codeigniter.com/forums/viewthread/165227/#846845
It looks like you set up your users DB as your main database in the config file, then you generate a config array for a new connection for a user based upon information in that users DB. So, I guess you'd need to at least store the DB name in the users database.
Not sure how well this works though, as I haven't had an occasion to try it out yet.
Sorry if that's not quite what you were looking for, but it should give you an idea of a Code Igniter approach.
Zend Framework.
http://framework.zend.com/manual/en/learning.multiuser.intro.html
I have a Drupal Multisite set up with hundreds of sites. I want to make some tables shared (like banners, and roles) so I don't have to update hundreds of sites when changing a banner (for example).
I know this can be done using these strings in settings.php:
$db_url = 'mysql://user:pwd#localhost/example_db';
$db_prefix = array(
'default' => '',
'users' => 'subsite2_',
'sessions' => 'subsite2_',
'authmap' => 'subsite2_',
);
But ... what if I have a multiple database setup as well? I have one database that holds all shared tables .. So in that database, I have the Banners table - that I want to have all the other sites to use.
The reason for a multiple database setup is because the whole multisite setup consists of hundreds of sites (and will be thousands by the end of this year) .. so every database only holds about 20-40 sites.. I suspect a lot of these tables can be shared..
hope someone can help. thanks!
Marco
There is a walkthrough to setup something like this. It aims at sharing only the user data, but the principle can be extended to more or less all tables. Note that it uses a 'trick' on the table prefixing logic by prepending not only (or not at all) a table name prefix, but also/only a database selector using the dot notation, e.g. someDatabase.someTable. So I'm not sure how this would translate to a PostgreSQL backend. Also, it means that you're still restricted to a single database server, as there is (AFAIK) no mechanism to use two different database connections in a single Drupal instance.
As an alternative, you might try to find a solution from the database engine side of things by 'mapping' your shared tables into the different database instances via some kind of replication/mirroring feature. That way you could use the standard Drupal multisite setup, doing the sharing of tables 'behind the back' of Drupal. Not sure how/if this would work and what consequences this would have on locking and other concurrency issues, though.
These two modules which sound like they might help (quotes are from their project pages):
Data:
Data module helps you model, manage and query related sets of tables. It offers an administration interface and a low level API for manipulating tables and accessing their contents. Data module provides Views integration for displaying table data and Drupal search integration for searching table content.
Table Wizard:
The Table Wizard facilitates dealing with database tables:
It allows surfacing any table in the Drupal default database through Views 2.
Relationships between the tables it manages can be defined, so views combining data in the tables can be constructed.
It performs analysis of the tables it manages, reporting on empty fields, data ranges, ranges of string lengths, etc.
It provides an API for other modules to views-enable their tables.
It provides an API for importing data into tables in the Drupal default database (automatically doing the views integration above).
It is bundled with an implementation of this API, for importing comma- and tab-delimited files.
If you are using mysql 5+ mysql views is a good way of sharing data across multiple sites in drupal. Not only it allows to have shared content but individual sites can have there own content.
devbee tutorial on mysql views in drupal. contains the detailed tutorial on how to implement it using taxomony.
I'm writing an application that that I'm going to provide as a service and also as a standalone application.
It's written in Zend Framework and uses MySQL.
When providing it as a service I want users to register on my site and have subdomains like customer1.mysite.com, customer2.mysite.com.
I want to have everything in one database, not creating new database for each user.
But now I wonder how to do it better.
I came up with two solutions:
1. Have user id in each table and just add it to WHERE clause on each database request.
2. Recreate tables with unique prefix like 'customer1_tablename', 'customer2_tablename'.
Which approach is better? Pros and cons?
Is there another way to separate users on the same database?
Leonti
I would stick to keeping all the tables together, otherwise there's barely any point to using a single database. It also means that you could feasibly allow some sort of cross-site interaction down the track. Just make sure you put indexes on the differentiating field (customer_number or whatever), and you should be ok.
If the tables are getting really large and slow, look at table partitioning.
It depends on what you intend to do with the data. If the clients don't share data, segmenting by customer might be better; also, you may get better performance.
On the other hand, having many tables with an identical structure can be a nightmare when you want to alter the structure.
I'd recommend using separate databases for each user. This makes your application easier to code for, and makes MySQL maintenance (migration of single account, account removal and so on.)
The only exception to this rule would be if you need to access data across accounts or share data.
This is called a multi-tenant application and lots of people run them; see
multi tenant tag
For some other peoples' questions