I am trying to write a customized relation that uses multiple tables, and this relation neeeds to join to the other tables by using a key from the first table.
Each table has the same columns.
So, let the story begin:
I am developing a online strategy based game. (PBBG - Persistant Browser Based Game)
I had a big table (30.000.000 rows) and I have split that table in 100 smaller tables (300.000 rows in each table).
The action happens within villages.
Details about the villages are stored within the village table.
Each village is situated within a continent, so there is a k number that gives me the continent that the village belongs to.
The village table has the following columns:
id - p.k.
x - x position on map
y - y position on map
k - f.k. continent id
name
user_id f.k.
points
I have 100 continents, that's why i have 100 new smaller tables.
I also have a table where I store the details for all the buildings.
The building details were stored within the village_building table.
So, from the original table village_building i now have village_building_k1, village_building_k2, ..., village_building_k100.
A village_building_k... table has the following columns:
id - p.k.
village_id - f.k.
building_id - f.k.
Now, I am trying to write a relation that jumps from model village to model village_building_kX where X should change when needed.
If I could use the k value from model village, that would give me the name of the table that i need to join.
If k = 2, the i need to jump from village to village_building_k2 using a concat
public function tableName() {
return 'village_building_k' . self::$k;
}
public function relations() {
return array(
'relation_village_building' => array(self::HAS_MANY, 'VillageBuildingKXSlaveM', 'village_id'),
);
}
$model_village->relation_village_building->building_id
Are you sure, that you need to solve this problem on framework level? I suggest to use here something like http://en.wikipedia.org/wiki/Shard_(database_architecture)
I think you need to add some shared databases. Due to your big database you can't manage it by your framework. Just try shared database for manage databases. Also you can try mysql cluster for manage database Follow this link and
this This may be helpful for you. Remember one frameworks are some set of functions so that you can to manipulate your function with your codes and supporting environment like databases and etc.
Actually you can try to pass db name from some global parameters, for example from session.
public function tableName()
{
return 'village_building_k' . Yii::app()->session['table'];
}
Problem is to save session for your gamer long time. If you'll get alot of users, it will become a problem.
If you have few tables with same structure, probably your database architecture is wrong. If it was sharding, you need to change AR for this.
Main problem in yii and sharding is that AR in __construct() calls getMetaData() to get table schema, so you need table name in that level.
Search on forums for solutions, i personally didn't found good article on english. Here is article about horizontal sharding realisation with Yii and Redis here. It's on Russian, but you can watch atleast some code.
And 30.000.000 table is ok, if you'll work right way with it, even in mysql.
Related
I have a general question about how to implement the best practice of model structure for an application I'm building in Laravel 5.
So, at the moment I have things set up like this:
'user' model and table: id, email, password, admin level - this is really just the info for authenticating login.
'user-details' model and table: id, userID (foreign key for user table id field), name, address etc - all the other details
'lesson-type' model and table: id, teacherID (foreign key for user-details table id field), lesson-label etc - info about different types of lessons
At the moment I have a Teacher Controller in which I'm passing through to the view:
- The info from the User table
- The info from the User-details table
- A list of different lesson types for the teacher from the Lesson-type table
But I kind of feel that all this should be tied together with one separate Teacher model which would extend the User-details model (and probably which in turn should extend the User model), but wouldn't have it's own table associated with it, but all the info pertaining to either updates for the User-details or the Lesson-types table would be stored in those relevant tables. Would this be correct?
(I should also say that users may alternatively be parents rather than teachers, and so would I would have a separate Parents model for all the properties and so on associated with parents)
I would then pass only the Teacher model object into the view and thus gain access to all the teacher info such as personal details and array of lesson types.
As I'm typing, this is sounding more and more to me like the right way to go, but it would be great to get some advice.
1 - technical implementation: I guess in the Teacher model, I'd populate all the relevant teacher into class variables (Name, array of lessons etc) in the constructor?
2 - am I over complicating this structure by having both Users AND Users details tables?
3 - Does what I'm proposing make the most structural sense in Laravel?
4 - just another thought I've just had, should the teacherID in the lesson-type table actually refer to the User table rather than the User-detail table... so user-detail and lesson-type would both be direct children of the user table??
Very much obliged for any help :)
You shouldn't extend models like that unless there is a clear inheritance. From a logical standpoint, it just doesn't make any sense since you'll have to overwrite most of what is on the User model anyway. And what you don't overwrite will be incorrectly mapped to the database because they are 2 completely different tables. What you actually want to do is utilize Eloquent relationships.
For clarity, I am assuming this basic structure:
users - id
teachers - id, user_id
user_details - id, user_id
lesson_types - id, teacher_id
Those should be 4 completely different models all interconnected using the Model::belongsTo() method. So the Teacher model would be
class Teacher extends Model {
public $table = 'teachers';
public function user() {
return $this->belongsTo('App\User');
}
}
When you query for a teacher, you can do Teacher::with('user')->get(). That will return all records from the teachers table and on each instance of the Teacher model, you'll be able to call $teacher->user and get the User instance associated with that teacher. That is a full model, not just extra data, so you have access to everything on the User Model, which is generally the main reason for extending
For your list of questions:
I may be misunderstanding you, but this isn't how an ORM works. I'd suggest going back and reading through the Eloquent docs (if you're running 5.0, I suggest reading 5.1's docs since they are much, much better)
It will depend on who you ask, but I tend to think so. If the data is clearly related and there is no reason for it to be shared across record types (for example, I generally have an addresses table that all records reference instead of having 5 address fields repeated on multiple tables), I believe it should all be on one table. It just makes it more difficult to manage later on if you have it in separate tables.
There will be those who disagree and think that smaller scopes for each table are better and it will likely allow for quicker queries on extremely large datasets, but I don't think it's worth the extra trouble in the end
No, as I have explained above
The teacher_id column should reference the teachers table, assuming that lessons belong to teachers and cannot belong to just any user in the system. Using the ORM, you'll be able to do $lesson->teacher->user->userDetails to get that data
I really think you need to go back and read through the Eloquent docs. Your understanding of how Eloquent works and how it is meant to be used seems very basic and you are missing much of the finer details.
Basics
Relationships
Laracasts - Laravel Fundamentals - You would benefit from watching Lesses 7-9, 11, 14, and 21
I'm currently working on an app backend (business directory). Main "actor" is an "Entry", which will have:
- main category
- subcategory
- tags (instead of unlimited sub-levels of division)
I'm pretty new to OOP but I still want to use it here. The database is MySql and I'll be using PDO.
In an attempt to figure out what database table structure should I use in order to support the above classification of entries, I was thinking about a solution that Wordpress uses - establish relationship between an entry and cats/subcats/tags through several tables (terms, taxonomies, relationships). What keeps me from this solution at the moment is the fact that each relationship of any kind is represented by a row in the relationships table. Given 50,000 entries I would have, attaching to a particular entry: main cat, subcat and up to 15 tags might slow down the app (or I am wrong)?
I then learned a bit about Table Data Gateway which seemed an excellent solution because I liked the idea of having one table per a class but then I read there is virtually no way of successful combating the impedence missmatch between the OOP and relational-mapping.
Are there any other approaches that you may see fit for this situation? I think I will be going with:
tblentry
tblcategory
tblsubcategory
tbltag
structure. Relationships would be based on the parent IDs but I+'m wondering is that enough? Can I be using foreign key and cascade delete options here (that is something I am not too familiar with and it seems to me as a more intuitive way of having relationships between the elements in tables)?
having a table where you store the relationship between your table is a good idea, and through indexes and careful thinking you can achieve very fast results.
since each entry must represent a different kind of link between two entities (subcategory to main entry, tag to subcategory) you need at least (and at the very most) three fields:
id1 (or the unique id of the first entity)
linkid (linking to a fourth table where each link is described)
id2 (or the unique id of the second entity)
those three fields can and should be indexed.
now the fourth table to achieve this kind of many-to-many relationship will describe the nature of the link. since many different type of relationship will exist in the table, you can't keep what the type is (child of, tag of, parent of) in the same table.
that fourth table (reference) could look like this:
id nature table1 table2
1 parent of entry tags
2 tag of tags entry
the table 1 field tells you which table the first id refers to, likewise with table2
the id is the number between the two fields in your relationship table. only the id field should be indexed. the nature field is more for the human reader then for joining tables or organizing data
I have a data hierarchy like this:
Country
-President // A Person
--Central
---Prime Minister // A Person
----State
-----Chief Minister // A Person
------District
-------District Minister // A Person
--------Area
---------Area Minister (AM) // A Person
So basically what i am trying to do here is, I want to Focus on Area Minister. As he will be soul of my application. Others are just the part of hierarchy. So lets say, if I am adding an Area Minister then he/she will be under an Area, and Area is under a District Minister and District Minister is under District and District is under Chief Minister and SO ON......
What I am doing here is adding these details for Area Minister into my MySQL Database So what i have done in database is, I have created the following tables:
Country // For adding entry
President // For adding president in country (So country_id is foreign key here)
Central // For adding central under President (So president_id is foreign key here)
...
...
...
...
...
Area // For adding area under District manager (So district_manager_id is foreign key here)
Area_Minister // For adding Area Minister (AM) in Area (So area_id is foreign key here)
The question is for fetching details of AM through **PHP**. I am fetching data's from **MySQL** and as AM is the core data which i am looking for, but the associated data's are also required at some points. But the core is AM - Data. However if I am going through this way, I am trying to fetch data's from 10 - TEN TABLES. Which i feel is pain.
Because to get the required data, I match Area ID in AM, then District Minister ID in Area and so on to the start level of hierarchy.
Is there any way that i can do to minimize this process. As the core data which i want is just AM i.e Area Manager's Data.
Any way that i can structure it well. Or maybe if any sought of help someone can provide me with insight that if this could be done through stored procedures.
I am just looking for a way to reduce the complexity of code in PHP and So many tables in MySQL, when core data is just there with AM only.
Kindly advice.
You have the basic table structure correct, and what you specified seems to be a reasonable structure for your table data. If you want to reduce the number of tables opened when you access an area manager, there are a few things you can do.
Use decorators. You can create an area manager object which loads only the bare minimum data needed when you fetch an area manager. Add methods to that object that fetch data from other tables and only run their queries when that data is needed. For example:
class AreaManager {
public function __construct() {
//do code which loads the basic table data for an area manager
}
public function getPresident() {
// do the SQL needed to load the president from the relational tables
}
}
Denormalize your data. You can run a script periodically which takes your data and puts it into a single table - so you have a structure like this:
area_manager_id | country_name | president | prime_minister | etc.
This script would have to be updated as you change your relationships but would allow you easy access to your data through a single query.
All that being said, you have set up relational tables the way they are supposed to be used and a 10 table query isn't usually the end of the world.
Your data wont be that bad as long as you index your foreign key and then using joins on what you are fetching will be fine.
In the end is not about your table because your table is properly structured with the proper relation key but rather your indexing optimization and the query that you will run.
I've done quit a bit of programming with php/mysql on small scale personal projects. However I'm working on my first commercial app that is going to allow customers or businesses to log in and perform CRUD operations. I feel like a total noob asking this question but I have never had to do this before and cannot find any relevant information on the net.
Basically, I've created this app and have a role based system set up on my data base. The problem that I'm running into is how to separate and fetch data for the relevant businesses or groups.
I can't, for example, set my queries up like this: get all records from example table where user id = user id, because that will only return data for that user and not all of the other users that are related to that business. I need a way to get all records that where created by users of a particular business.
I'm thinking that maybe the business should have an id and I should form my queries like this: get all records from example where business id = business id. But I'm not even sure if that's a good approach.
Is there a best practice or a convention for this sort data storing/fetching and grouping?
Note:Security is a huge issue here because I'm storing legal data.
Also, I'm using the latest version of laravel 4 if that's any relevance.
I would like to hear peoples thoughts on this that have encountered this sort problem before and how they designed there database and queries to only get and store data related to that particular business.
Edit: I like to read and learn but cannot find any useful information on this topic - maybe I'm not using the correct search terms. So If you know of any good links pertaining to this topic, please post them too.
If I understand correctly, a business is defined within your system as a "group of users", and your whole system references data belonging to users as opposed to data belonging to a business. You are looking to reference data that belongs to all users who belong to a particular business. In this case, the best and most extensible way to do this would be to create two more tables to contain businesses and business-user relations.
For example, consider you have the following tables:
business => Defines a business entity
id (primary)
name
Entry: id=4, name=CompanyCorp
user => Defines each user in the system
id (primary)
name
Entry: id=1, name=Geoff
Entry: id=2, name=Jane
business_user => Links a user to a particular business
user_id (primary)
business_id (primary)
Entry: user_id=1, business_id=4
Entry: user_id=2, business_id=4
Basically, the business_user table defines relationships. For example, Geoff is related to CompanyCorp, so a row exists in the table that matches their id's together. This is called a relational database model, and is an important concept to understand in the world of database development. You can even allow a user to belong to multiple different companies.
To find all the names of users and their company's name, where their company's id = 4...
SELECT `user`.`name` as `username`, `business`.`name` as `businessname` FROM `business_user` LEFT JOIN `user` ON (`user`.`id` = `business_user`.`user_id`) LEFT JOIN `business` ON (`business`.`id` = `business_user`.`business_id`) WHERE `business_user`.`business_id` = 4;
Results would be:
username businessname
-> Geoff CompanyCorp
-> Jane CompanyCorp
I hope this helps!
===============================================================
Addendum regarding "cases" per your response in the comments.
You could create a new table for cases and then reference both business and user ids on separate columns in there, as the case would belong to both a user and a business, if that's all the functionality that you need.
Suppose though, exploring the idea of relational databases further, that you wanted multiple users to be assigned to a case, but you wanted one user to be elected as the "group leader", you could approach the problem as follows:
Create a table "case" to store the cases
Create a table "user_case" to store case-user relationships, just like in the business_user table.
Define the user_case table as follows:
user_case => Defines a user -> case relationship
user_id (primary)
case_id (primary)
role
Entry: user_id=1, case_id=1, role="leader"
Entry: user_id=2, case_id=1, role="subordinate"
You could even go further and define a table with definitions on what roles users can assume. Then, you might even change the user_case table to use a role_id instead which joins data from yet another role table.
It may sound like an ever-deepening schema of very small tables, but note that we've added an extra column to the user_case relational table. The bigger your application grows, the more your tables will grow laterally with more columns. Trust me, you do eventually stop adding new tables just for the sake of defining relations.
To give a brief example of how flexible this can be, with a role table, you could figure out all the roles that a given user (where user_id = 6) has by using a relatively short query like:
SELECT `role`.`name` FROM `role` RIGHT JOIN `user_case` ON (`user_case`.`role_id` = `role`.`id`) WHERE `user_case`.`user_id` = 6;
If you need more examples, please feel free to keep commenting.
So I'm a visual designer type guy who has learned a respectable amount of PHP and a little SQL.
I am putting together a personal multimedia portfolio site. I'm using CI and loving it. The problem is I don't know squat about DB design and I keep rewriting (and breaking) my tables. Here is what I need.
I have a table to store the projects:
I want to do fulltext searcheson titles and descriptions so I think this needs to be MyISAM
PROJECTS
id
name (admin-only human readable)
title (headline for visitors to read)
description
date (the date the project was finished)
posted (timestamp when the project was posted)
Then I need tags:
I think I've figured this out. from researching.
TAGS
tag_id
tag_name
PROJECT_TAGS
project_id (foreign key PROJECTS TABLE)
tag_id (foreign key TAGS TABLE)
Here is the problem I have FOUR media types; Photo Albums, Flash Apps, Print Pieces, and Website Designs. no project can be of two types because (with one exception) they all require different logic to be displayed in the view. I am not sure whether to put the media type in the project table and join directly to the types table or use an intermediate table to define the relationships like the tags. I also thinking about parent-types/sub-types i.e.; Blogs, Projects - Flash, Projects - Web. I would really appreciate some direction.
Also maybe some help on how to efficiently query for the projects with the given solution.
The first think to address is your database engine, MyISAM. The database engine is how MySQL stores the data. For more information regarding MyISAM you can view: http://dev.mysql.com/doc/refman/5.0/en/myisam-storage-engine.html. If you want to have referential integrity (which is recommended), you want your database engine to be InnoDB (http://dev.mysql.com/doc/refman/5.0/en/innodb-storage-engine.html). InnoDB allows you to create foreign keys and enforce that foreign key relationship (I found out the hard way the MyISAM does not). MyISAM is the default engine for MySQL databases. If you are using phpMyAdmin (which is a highly recommended tool for MySQL and PHP development), you can easily change the engine type of the database (See: http://www.electrictoolbox.com/mysql-change-table-storage-engine/).
With that said, searches or queries can be done in both MyISAM and InnoDB database engines. You can also index the columns to make search queries (SELECT statements) faster, but the trade off will be that INSERT statements will take longer. If you database is not huge (i.e. millions of records), you shouldn't see a noticeable difference though.
In terms of your design, there are several things to address. The first thing to understand is an entity relationship diagram or an ERD. This is a diagram of your tables and their corresponding relationships.
There are several types of relationships that can exist: a one-to-one relationship, a one-to-many relationship, a many-to-many relationship, and a hierarchical or recursive relationship . A many-to-many relationship is the most complicated and cannot be produced directly within the database and must be resolved with an intermittent table (I will explain further with an example).
A one-to-one relationship is straightforward. An example of this is if you have an employee table with a list of all employees and a salary table with a list of all salaries. One employee can only have one salary and one salary can only belong to one employee.
With that being said, another element to add to the mix is cardinality. Cardinality refers to whether or not the relationship may exist or must exist. In the previous example of an employee, there has to be a relationship between the salary and the employee (or else the employee may not be paid). This the relationship is read as, an employee must have one and only one salary and a salary may or may not have one and only one employee (as a salary can exist without belonging to an employee).
The phrases "one and only one" refers to it being a one-to-one relationship. The phrases "must" and "may or may not" referring to a relationship requiring to exist or not being required. This translates into the design as my foreign key of salary id in the employee table cannot be null and in the salary table there is no foreign key referencing the employee.
EMPLOYEE
id PRIMARY KEY
name VARCHAR(100)
salary_id NOT NULL UNIQUE
SALARY
id PRIMARY KEY
amount INTEGER NOT NULL
The one-to-many relationship is defined as the potential of having more than one. For example, relating to your portfolio, a client may have one or more projects. Thus the foreign key field in the projects table client_id cannot be unique as it may be repeated.
The many-to-many relationship is defined where more than one can both ways. For example, as you have correctly shown, projects may have one or more tags and tags may assigned to one or more projects. Thus, you need the PROJECT_TAGS table to resolve that many-to-many.
In regards to addressing your question directly, you will want to create a separate media type table and if any potential exists whatsoever where a project is can be associated to multiple types, you would want to have an intermittent table and could add a field to the project_media_type table called primary_type which would allow you to distinguish the project type as primarily that media type although it could fall under other categories if you were to filter by category.
This brings me to recursive relationships. Because you have the potential to have a recursive relationship or media_types you will want to add a field called parent_id. You would add a foreign key index to parent_id referencing the id of the media_type table. It must allow nulls as all of your top level parent media_types will have a null value for parent_id. Thus to select all parent media_types you could use:
SELECT * FROM media_type WHERE parent_id IS NULL
Then, to get the children you loop through each of the parents and could use the following query:
SELECT * FROM media_type WHERE parent_id = {$media_type_row->id}
This would need to be in a recursive function so you loop until there are no more children. An example of this using PHP related to hierarchical categories can be viewed at recursive function category database.
I hope this helps and know it's a lot but essentially, I tried to highlight a whole semester of database design and modeling. If you need any more information, I can attach an example ERD as well.
Another posibble idea is to add columns to projects table that would satisfy all media types needs and then while editting data you will use only certain columns needed for given media type.
That would be more database efficient (less joins).
If your media types are not very different in columns you need I would choose that aproach.
If they differ a lot, I would choose #cosmicsafari recommendation.
Why don't you take whats common to all and put that in a table & have the specific stuff in tables themelves, that way you can search through all the titles & descriptions in one.
Basic Table
- ID int
- Name varchar()
- Title varchar()
etc
Blogs
-ID int (just an auto_increment key)
-basicID int (this matches the id of the item in the basic table)
etc
Have one for each media type. That way you can do a search on all the descriptions & titles at the one time and load the appropriate data when the person clicked through the link from a search page. (I assume thats the sort of functionality you mean when you say you want to be able to let people search.)