My question is very similar to this question but a bit more specific.
My application has multiple companies and multiple users per company. It makes the most sense to me (at this point) for each company to have a "private" set of tables. This makes security extremely simple as I don't have to worry about JOIN-ing up my structure tree to be sure I only get data for the specific company. I can also extend the mysqli database extension and have it put a prefix on the table names in the query so that I never have to worry about security while writing my queries.
One other major advantage that I can see is that if one of the companies needs a customization, I can modify their specific tables and not have to take into account everyone else. The way that my app is designed it is extremely modular and implementing custom code is very simple.
There are some disadvantages that I can see but so far it seems that the above advantages would out-weigh them. The above proposed system does sort of grate on my (possibly) hyper-normalized database schema preferences up to this point. Another obvious disadvantage is implementing schema alterations but I can script them and be safe enough. One point that I'm not sure about is performance. If I have MySQL working with so many tables, will I make bottlenecks for myself?
I look forward to your thoughts!
Your proposal sounds reasonable to me. I would suggest that instead of prefixing your tables with the company name, you store the tables for each company in a separate schema. That way you can have tables with the same name, reducing your problems in the code, and have each set of tables protected by a different username and password in a convenient manner. Backups and replication would then all be distinguishable at need.
Lookup tables could be stored in yet another schema to which all users have access.
Related
I need to create an application with an editable database structure. Where you can add/delete/modify tables and fields, views and structure of the database. All in production and real time.
The purpose is that the application can be adapted to the needs of the company. Allowing you to store the information that is needed, where it is needed.
I use laravel 5 and MySQL, but my question is not about my software. My questions are:
Is there a methodology, or a set of steps to follow, to achieve this functionality?
And if it exists, is there any package to apply it to laravel?
Entity-attribute-value model allows to have a DB with something like a "dynamic schema" and be able to run indexed queries on its tables (though "tables" become different from what you would have if you used the normal approach). With it you can add and remove fields and have the values indexed (unlike in a document-oriented NoSQL DB). Downsides: a lot of joins, performance might suffer; however, I've seen pretty large systems get away with it. Don't know if and how it can be applied in Laravel context, but googling gives at least some results.
You need to use Business Intelligence and reporting tools, which fulfil all your needs. They run with any db, no matter which fields you add or remove they will adjust themselves. One of the best example is:-
https://github.com/getredash/redash
lets say i have this mysql db, and all the tables in the db are related to one another, primary keys, foreign keys, etc all are set. Now is it possible to predict, just from the database design, what the queries will be used for the application? Since the database does dictate the application capabilities, then therefore from the design, we can predict what queries that will be used in the application, right?
If it is possible, is there a strategy or automated way to generate the possible queries?
I have written a book on the subject of analyzing data using SQL and Excel, and have spent many years working with databases.
Yes, from a database structure, you can figure out how tables are going to be joined together. You are not going to figure out the harder -- and generally more business relevant -- things that users need. Here are some examples:
You can have a database where the primary table is telephone calls, with the associated information. From this database, you may need to know the maximum number of active calls at one time. Or you may need to know how many different people someone calls in a month.
You can have a database of subscriber records. You may need to figure out the probability that someone will stop after a given amount of time.
You can have a database of products and purchases. You may need to figure out the most common combinations of three products that occur together.
You can have a database of credit card purchases. You may need to figure out who spends more than $200 in a restaurant more than 50 miles from their billing address.
The point is. A database does not represent "application capabilities". A database represents entities and relationships between them, presumably in the real world. There is hubris to think that you can look at a database and know what the business questions are.
Instead, the purpose of a database is to support data, which in turn, supports applications. The needs of applications will change over time. The beauty of databases, as opposed to many other data storage technologies, is that the technology scales as the data increases, supports changes to the structure, and allows new entities and relationships to be added into the system, without completely rewriting it.
Over time, and with experience, you might develop intuition on what's important. Even if you do, you will be constantly surprised at the varied needs of your users.
I am sincerely not trying to be smart here but answer is - yes and no.
Yes, because 3NF design usually outlines business rules behind it pretty well, so you can to a degree tell what is the business logic behind it, you can create an object or graph model from it and get a good idea
what kinds of questions can be asked from based on connections/relations and accessible properties.
No, because combinatorially you might have a untractable number of combinations of questions from a graph. Hence, you can't really tell what question one might ask in reasonable, non-exponential amount of time.
In general, if design is good and tables are meaningfully named you can get a pretty good idea what is going on.
Theoretically it's possible but due to the combinatorial explosion of N rows by X columns by Z tables by W possible functions by Q possible values on each column/row this is an amazingly large number.
The issue here is that you need to take into account the data too. Some queries only make sense when there is particular data and other don't. So you are essentially considering massively large hypercube.
I work with Multidimensional databases (denormalised cubes) and this is essentially denormalised databases. Have a read aup on OLAP theory and you'll see why.
So in short no as it's practically impossible.
Now is it possible to predict, just from the database design, what the queries will be used for the application?
You can, at least in principle, predict which queries can be answered efficiently. Which queries will the applications actually try to execute is another matter.
In an ideal world, database model would take into account all the querying needs of all the applications, now and in the future. We don't live in that world yet ;)
If it is possible, is there a strategy or automated way to generate the possible queries?
No, that requires human understanding of what the model actually means. Unfortunately, there is no good way to teach a tool to have that level of understanding.
A good model will immediately make sense to a person experienced in database modeling and the domain being modeled. Such person will typically be able to predict a fair portion of queries actually being used, but rarely all of them, so the documentation beside the database model itself is desirable. And of course, not all models are good...
I'm staring to build a system for working with native languages, tags and such data in Yii Framework.
I already choose MongoDB for storing my data as I think it feets nicelly and will get better performance with less costs (the database will have huge amounts of data).
My question regards user authentication, payments, etc... This are sensitive bits of information and areas where I think the data is relational.
So:
1. Would you use two different db systems? Should I need them or I'm I complicating this?
2. If you recommend the two db approach how would I achieve that in Yii?
Thanks for your time!
PS: I do not intend this question to be another endless discussion between the relational vs non-relational folks. Having said that I think that my data feets mongo but if you have something to say about that go ahead ;)
You might be interested in this presentation on OpenSky's infrastructure, where MongoDB is used alongside MySQL. Mongo was utilized mainly for CMS-type data where a flexible schema was useful, and they relied upon MySQL for transactions (e.g. customer orders, payments). If you end up using the Doctrine library, you'll find that the ORM (for SQL databases) and MongoDB ODM share a similar API, which should make the experimentation process easier.
I wouldn't shy away from using MongoDB to store user data, though, as that's often a record that can benefit from embedded document storage (e.g. storing multiple billing/shipping addresses within a single user document). If anything, Mongo should be flexible enough to enable you to develop your application without worrying about schema changes due to evolving product requirements. As those requirements become more clear, you'll be able to make a decision based on the app's performance needs and types of database queries you end up needing.
There is no harm in using multiple databases (if you really need), many big websites are using multiple databases so go a head and start your project.
I'm looking for the best possible way of sharing model data between two MVC (I'm using Symfony) driven web sites.
Background information
We have two web sites A and B. The same software is used for both sites, but there are different customers and data. Customers are allowed to release content. Now we're going to introduce a new payment option with the advantage that the user's content is released on both web sites automatically.
Implementation ?!
I have three ideas for the implementation:
Using the same database for both applications. Then I would have to extend some tables by one column which indicates the appropriate target web site (A/B).
I think that this would be bad design. A lot of code has to be rewritten in order to exclude records from query result sets, which does not belong to the respective web site.
Using two databases.
In my opinion, this would decrease performance significantly and would be very hard to implement. Data has always to be requested twice. Also, in future there may be web sites C,D,E...
Synchronizing two databases via web-service.
Some data would be stored twice. Therefore, all operations on such a piece of data has to be performed twice (create, read, update, destroy).
Now I'm stuck, because each solution has serious disadvantages.
Do you have any ideas? If not, which one do you think is the best of mine?
I think your first option is the best. You're going to reduce duplicate data as much as possible and you should have the best performance. You will have to add an extra check to exclude the records not belonging to each particular website but all solutions will require work.
I'm writing an application that that I'm going to provide as a service and also as a standalone application.
It's written in Zend Framework and uses MySQL.
When providing it as a service I want users to register on my site and have subdomains like customer1.mysite.com, customer2.mysite.com.
I want to have everything in one database, not creating new database for each user.
But now I wonder how to do it better.
I came up with two solutions:
1. Have user id in each table and just add it to WHERE clause on each database request.
2. Recreate tables with unique prefix like 'customer1_tablename', 'customer2_tablename'.
Which approach is better? Pros and cons?
Is there another way to separate users on the same database?
Leonti
I would stick to keeping all the tables together, otherwise there's barely any point to using a single database. It also means that you could feasibly allow some sort of cross-site interaction down the track. Just make sure you put indexes on the differentiating field (customer_number or whatever), and you should be ok.
If the tables are getting really large and slow, look at table partitioning.
It depends on what you intend to do with the data. If the clients don't share data, segmenting by customer might be better; also, you may get better performance.
On the other hand, having many tables with an identical structure can be a nightmare when you want to alter the structure.
I'd recommend using separate databases for each user. This makes your application easier to code for, and makes MySQL maintenance (migration of single account, account removal and so on.)
The only exception to this rule would be if you need to access data across accounts or share data.
This is called a multi-tenant application and lots of people run them; see
multi tenant tag
For some other peoples' questions