I'm thinking about how to represent a complex structure in a SQL Server database.
Consider an application that needs to store details of a family of objects, which share some attributes, but have many others not common. For example, a commercial insurance package may include liability, motor, property and indemnity cover within the same policy record.
It is trivial to implement this in C#, etc, as you can create a Policy with a collection of Sections, where Section is inherited as required for the various types of cover. However, relational databases don't seem to allow this easily.
I can see that there are two main choices:
Create a Policy table, then a Sections table, with all the fields required, for all possible variations, most of which would be null.
Create a Policy table and numerous Section tables, one for each kind of cover.
Both of these alternatives seem unsatisfactory, especially as it is necessary to write queries across all Sections, which would involve numerous joins, or numerous null-checks.
What is the best practice for this scenario?
#Bill Karwin describes three inheritance models in his SQL Antipatterns book, when proposing solutions to the SQL Entity-Attribute-Value antipattern. This is a brief overview:
Single Table Inheritance (aka Table Per Hierarchy Inheritance):
Using a single table as in your first option is probably the simplest design. As you mentioned, many attributes that are subtype-specific will have to be given a NULL value on rows where these attributes do not apply. With this model, you would have one policies table, which would look something like this:
+------+---------------------+----------+----------------+------------------+
| id | date_issued | type | vehicle_reg_no | property_address |
+------+---------------------+----------+----------------+------------------+
| 1 | 2010-08-20 12:00:00 | MOTOR | 01-A-04004 | NULL |
| 2 | 2010-08-20 13:00:00 | MOTOR | 02-B-01010 | NULL |
| 3 | 2010-08-20 14:00:00 | PROPERTY | NULL | Oxford Street |
| 4 | 2010-08-20 15:00:00 | MOTOR | 03-C-02020 | NULL |
+------+---------------------+----------+----------------+------------------+
\------ COMMON FIELDS -------/ \----- SUBTYPE SPECIFIC FIELDS -----/
Keeping the design simple is a plus, but the main problems with this approach are the following:
When it comes to adding new subtypes, you would have to alter the table to accommodate the attributes that describe these new objects. This can quickly become problematic when you have many subtypes, or if you plan to add subtypes on a regular basis.
The database will not be able to enforce which attributes apply and which don't, since there is no metadata to define which attributes belong to which subtypes.
You also cannot enforce NOT NULL on attributes of a subtype that should be mandatory. You would have to handle this in your application, which in general is not ideal.
Concrete Table Inheritance:
Another approach to tackle inheritance is to create a new table for each subtype, repeating all the common attributes in each table. For example:
--// Table: policies_motor
+------+---------------------+----------------+
| id | date_issued | vehicle_reg_no |
+------+---------------------+----------------+
| 1 | 2010-08-20 12:00:00 | 01-A-04004 |
| 2 | 2010-08-20 13:00:00 | 02-B-01010 |
| 3 | 2010-08-20 15:00:00 | 03-C-02020 |
+------+---------------------+----------------+
--// Table: policies_property
+------+---------------------+------------------+
| id | date_issued | property_address |
+------+---------------------+------------------+
| 1 | 2010-08-20 14:00:00 | Oxford Street |
+------+---------------------+------------------+
This design will basically solve the problems identified for the single table method:
Mandatory attributes can now be enforced with NOT NULL.
Adding a new subtype requires adding a new table instead of adding columns to an existing one.
There is also no risk that an inappropriate attribute is set for a particular subtype, such as the vehicle_reg_no field for a property policy.
There is no need for the type attribute as in the single table method. The type is now defined by the metadata: the table name.
However this model also comes with a few disadvantages:
The common attributes are mixed with the subtype specific attributes, and there is no easy way to identify them. The database will not know either.
When defining the tables, you would have to repeat the common attributes for each subtype table. That's definitely not DRY.
Searching for all the policies regardless of the subtype becomes difficult, and would require a bunch of UNIONs.
This is how you would have to query all the policies regardless of the type:
SELECT date_issued, other_common_fields, 'MOTOR' AS type
FROM policies_motor
UNION ALL
SELECT date_issued, other_common_fields, 'PROPERTY' AS type
FROM policies_property;
Note how adding new subtypes would require the above query to be modified with an additional UNION ALL for each subtype. This can easily lead to bugs in your application if this operation is forgotten.
Class Table Inheritance (aka Table Per Type Inheritance):
This is the solution that #David mentions in the other answer. You create a single table for your base class, which includes all the common attributes. Then you would create specific tables for each subtype, whose primary key also serves as a foreign key to the base table. Example:
CREATE TABLE policies (
policy_id int,
date_issued datetime,
-- // other common attributes ...
);
CREATE TABLE policy_motor (
policy_id int,
vehicle_reg_no varchar(20),
-- // other attributes specific to motor insurance ...
FOREIGN KEY (policy_id) REFERENCES policies (policy_id)
);
CREATE TABLE policy_property (
policy_id int,
property_address varchar(20),
-- // other attributes specific to property insurance ...
FOREIGN KEY (policy_id) REFERENCES policies (policy_id)
);
This solution solves the problems identified in the other two designs:
Mandatory attributes can be enforced with NOT NULL.
Adding a new subtype requires adding a new table instead of adding columns to an existing one.
No risk that an inappropriate attribute is set for a particular subtype.
No need for the type attribute.
Now the common attributes are not mixed with the subtype specific attributes anymore.
We can stay DRY, finally. There is no need to repeat the common attributes for each subtype table when creating the tables.
Managing an auto incrementing id for the policies becomes easier, because this can be handled by the base table, instead of each subtype table generating them independently.
Searching for all the policies regardless of the subtype now becomes very easy: No UNIONs needed - just a SELECT * FROM policies.
I consider the class table approach as the most suitable in most situations.
The names of these three models come from Martin Fowler's book Patterns of Enterprise Application Architecture.
The 3rd option is to create a "Policy" table, then a "SectionsMain" table that stores all of the fields that are in common across the types of sections. Then create other tables for each type of section that only contain the fields that are not in common.
Deciding which is best depends mostly on how many fields you have and how you want to write your SQL. They would all work. If you have just a few fields then I would probably go with #1. With "lots" of fields I would lean towards #2 or #3.
In addition at the Daniel Vassallo solution, if you use SQL Server 2016+, there is another solution that I used in some cases without considerable lost of performances.
You can create just a table with only the common field and add a single column with the JSON string that contains all the subtype specific fields.
I have tested this design for manage inheritance and I am very happy for the flexibility that I can use in the relative application.
With the information provided, I'd model the database to have the following:
POLICIES
POLICY_ID (primary key)
LIABILITIES
LIABILITY_ID (primary key)
POLICY_ID (foreign key)
PROPERTIES
PROPERTY_ID (primary key)
POLICY_ID (foreign key)
...and so on, because I'd expect there to be different attributes associated with each section of the policy. Otherwise, there could be a single SECTIONS table and in addition to the policy_id, there'd be a section_type_code...
Either way, this would allow you to support optional sections per policy...
I don't understand what you find unsatisfactory about this approach - this is how you store data while maintaining referential integrity and not duplicating data. The term is "normalized"...
Because SQL is SET based, it's rather alien to procedural/OO programming concepts & requires code to transition from one realm to the other. ORMs are often considered, but they don't work well in high volume, complex systems.
The another way to do it, is using the INHERITS component. For example:
CREATE TABLE person (
id int ,
name varchar(20),
CONSTRAINT pessoa_pkey PRIMARY KEY (id)
);
CREATE TABLE natural_person (
social_security_number varchar(11),
CONSTRAINT pessoaf_pkey PRIMARY KEY (id)
) INHERITS (person);
CREATE TABLE juridical_person (
tin_number varchar(14),
CONSTRAINT pessoaj_pkey PRIMARY KEY (id)
) INHERITS (person);
Thus it's possible to define a inheritance between tables.
Alternatively, consider using a document databases (such as MongoDB) which natively support rich data structures and nesting.
I lean towards method #1 (a unified Section table), for the sake of efficiently retrieving entire policies with all their sections (which I assume your system will be doing a lot).
Further, I don't know what version of SQL Server you're using, but in 2008+ Sparse Columns help optimize performance in situations where many of the values in a column will be NULL.
Ultimately, you'll have to decide just how "similar" the policy sections are. Unless they differ substantially, I think a more-normalized solution might be more trouble than it's worth... but only you can make that call. :)
I'm working with mySQL and Laravel.
I have two tables: users and groups. Each has an id field and a name field.
I want users to be able to belong to many groups and vice versa; so I'm setting up a many to many relationship with a junction table: group_user.
My first thought is that it only needs two fields: user_id and group_id, both Foreign Keys.
But something tells me I might need/want an id for this table too. I've seen it both ways in code examples across the internet and I just want to know what I would be missing by not having it.
I just can't think of how I would actually make use of it; since the table is just to relate two other tables.
The artificial primary key can be very useful sometimes in your code. For example when you would like to delete a record you can use the same pattern as in the case of other tables.
You have to pass only one id instead of the parts of the compound key.
So I'm a visual designer type guy who has learned a respectable amount of PHP and a little SQL.
I am putting together a personal multimedia portfolio site. I'm using CI and loving it. The problem is I don't know squat about DB design and I keep rewriting (and breaking) my tables. Here is what I need.
I have a table to store the projects:
I want to do fulltext searcheson titles and descriptions so I think this needs to be MyISAM
PROJECTS
id
name (admin-only human readable)
title (headline for visitors to read)
description
date (the date the project was finished)
posted (timestamp when the project was posted)
Then I need tags:
I think I've figured this out. from researching.
TAGS
tag_id
tag_name
PROJECT_TAGS
project_id (foreign key PROJECTS TABLE)
tag_id (foreign key TAGS TABLE)
Here is the problem I have FOUR media types; Photo Albums, Flash Apps, Print Pieces, and Website Designs. no project can be of two types because (with one exception) they all require different logic to be displayed in the view. I am not sure whether to put the media type in the project table and join directly to the types table or use an intermediate table to define the relationships like the tags. I also thinking about parent-types/sub-types i.e.; Blogs, Projects - Flash, Projects - Web. I would really appreciate some direction.
Also maybe some help on how to efficiently query for the projects with the given solution.
The first think to address is your database engine, MyISAM. The database engine is how MySQL stores the data. For more information regarding MyISAM you can view: http://dev.mysql.com/doc/refman/5.0/en/myisam-storage-engine.html. If you want to have referential integrity (which is recommended), you want your database engine to be InnoDB (http://dev.mysql.com/doc/refman/5.0/en/innodb-storage-engine.html). InnoDB allows you to create foreign keys and enforce that foreign key relationship (I found out the hard way the MyISAM does not). MyISAM is the default engine for MySQL databases. If you are using phpMyAdmin (which is a highly recommended tool for MySQL and PHP development), you can easily change the engine type of the database (See: http://www.electrictoolbox.com/mysql-change-table-storage-engine/).
With that said, searches or queries can be done in both MyISAM and InnoDB database engines. You can also index the columns to make search queries (SELECT statements) faster, but the trade off will be that INSERT statements will take longer. If you database is not huge (i.e. millions of records), you shouldn't see a noticeable difference though.
In terms of your design, there are several things to address. The first thing to understand is an entity relationship diagram or an ERD. This is a diagram of your tables and their corresponding relationships.
There are several types of relationships that can exist: a one-to-one relationship, a one-to-many relationship, a many-to-many relationship, and a hierarchical or recursive relationship . A many-to-many relationship is the most complicated and cannot be produced directly within the database and must be resolved with an intermittent table (I will explain further with an example).
A one-to-one relationship is straightforward. An example of this is if you have an employee table with a list of all employees and a salary table with a list of all salaries. One employee can only have one salary and one salary can only belong to one employee.
With that being said, another element to add to the mix is cardinality. Cardinality refers to whether or not the relationship may exist or must exist. In the previous example of an employee, there has to be a relationship between the salary and the employee (or else the employee may not be paid). This the relationship is read as, an employee must have one and only one salary and a salary may or may not have one and only one employee (as a salary can exist without belonging to an employee).
The phrases "one and only one" refers to it being a one-to-one relationship. The phrases "must" and "may or may not" referring to a relationship requiring to exist or not being required. This translates into the design as my foreign key of salary id in the employee table cannot be null and in the salary table there is no foreign key referencing the employee.
EMPLOYEE
id PRIMARY KEY
name VARCHAR(100)
salary_id NOT NULL UNIQUE
SALARY
id PRIMARY KEY
amount INTEGER NOT NULL
The one-to-many relationship is defined as the potential of having more than one. For example, relating to your portfolio, a client may have one or more projects. Thus the foreign key field in the projects table client_id cannot be unique as it may be repeated.
The many-to-many relationship is defined where more than one can both ways. For example, as you have correctly shown, projects may have one or more tags and tags may assigned to one or more projects. Thus, you need the PROJECT_TAGS table to resolve that many-to-many.
In regards to addressing your question directly, you will want to create a separate media type table and if any potential exists whatsoever where a project is can be associated to multiple types, you would want to have an intermittent table and could add a field to the project_media_type table called primary_type which would allow you to distinguish the project type as primarily that media type although it could fall under other categories if you were to filter by category.
This brings me to recursive relationships. Because you have the potential to have a recursive relationship or media_types you will want to add a field called parent_id. You would add a foreign key index to parent_id referencing the id of the media_type table. It must allow nulls as all of your top level parent media_types will have a null value for parent_id. Thus to select all parent media_types you could use:
SELECT * FROM media_type WHERE parent_id IS NULL
Then, to get the children you loop through each of the parents and could use the following query:
SELECT * FROM media_type WHERE parent_id = {$media_type_row->id}
This would need to be in a recursive function so you loop until there are no more children. An example of this using PHP related to hierarchical categories can be viewed at recursive function category database.
I hope this helps and know it's a lot but essentially, I tried to highlight a whole semester of database design and modeling. If you need any more information, I can attach an example ERD as well.
Another posibble idea is to add columns to projects table that would satisfy all media types needs and then while editting data you will use only certain columns needed for given media type.
That would be more database efficient (less joins).
If your media types are not very different in columns you need I would choose that aproach.
If they differ a lot, I would choose #cosmicsafari recommendation.
Why don't you take whats common to all and put that in a table & have the specific stuff in tables themelves, that way you can search through all the titles & descriptions in one.
Basic Table
- ID int
- Name varchar()
- Title varchar()
etc
Blogs
-ID int (just an auto_increment key)
-basicID int (this matches the id of the item in the basic table)
etc
Have one for each media type. That way you can do a search on all the descriptions & titles at the one time and load the appropriate data when the person clicked through the link from a search page. (I assume thats the sort of functionality you mean when you say you want to be able to let people search.)
I'm working with CakePHP 2.x and I was wondering if there is any built-in functionality for working with HABTM join tables. I don't really see anything in the documentation for directly changing the attributes of a HABTM relationship. This is what I mean:
I have a join table that contains flags for a relationship between two models that can be demonstrated as such:
(int) id | (int) user_id | (int) video_id | (bool) hasRated | (bool) hasWatched | (bool) hasDownloaded
The first column is just the regular table id. The next two (user_id and video_id) are the typical HABTM foreign keys to the user table and video table. The following 3 flags are indicators of the relationship between the two models (has this video been watched? has it been rated? etc.)
I am currently building my own queries and this is my boot-strap setup. My end goal is to add attributes between the relationship of two models. Is this the correct way to approach this, or is there a more Cakey way to do it?
For maintain a HABTM you can maintain join table. The cake convention for join table for HABTM is:
This new join table's name needs to include the names of both models
involved, in alphabetical order, and separated with an underscore ( _
). The contents of the table should be two fields, each foreign keys
(which should be integers) pointing to both of the primary keys of the
involved models. To avoid any issues - don't define a combined primary
key for these two fields, if your application requires it you can
define a unique index. If you plan to add any extra information to
this table, it's a good idea to add an additional primary key field
(by convention 'id') to make acting on the table as easy as any other
model.
So its seems just fine for you. But you must follow the naming convention.
For more detail see docs.
Eureka.
In 2.x, cake will automagically create a model for you and recognize the relationship.
$this->Video->UserVideo will generate a model that fits the HABTM join table users_videos. You can then interact with it just like a regular model and it will update the database.
I have a problem with using the ORM and creating a has and belongs to many self join using Kohana 2.3.4
I have this table
Tasks
ID | Title | etc......
I need to be able to link tasks to other tasks, a task can have multiple children, and multiple parents. So I was thinking of having this table.
Tasks_Tasks
ID | task_1_id | task_2_id
To link the tasks to each other, but I can't work out in Kohana how to set the foreign keys correctly for the relationships, or if it is possible at all?
Can anyone suggest an answer? Or even better, a better solution?
Instead of extending ORM you should be extending ORM_Tree. This is built for just this type of relationship. Take a look at system/libraries/ORM_Tree.php.