Hey,
I've got a bit of a problem right now trying to figure out how to resolve a specific many to many model in Mongo.
I have an event scheduling system for the CRM I am building that allows events to be assigned to both users and teams. These events are particular to each lead.
So for example, I have a call at 5:00pm Thursday with Jimmy Dolittle. My sales team also has a call Thursday at 7:00am with Bob Jones.
If this were SQL, I would just create a leads table, events table, users table, and teams table. I was thinking about putting the events in the users collection and in the teams collection but then the problem arises when I have a list of leads and want to display the callback date next to each lead. Referencing like that in Mongo is going to be sloooow with a list of 500 leads.
I was also thinking about storing the events in the leads collection, but that would mean I would have to do the same sort of search for leads with events assigned to a particular user or team (there might be 500,000 leads in the database but only 500 have events for a particular user.
This kind of relationship is just going to be a problem in Mongo. In that situation, I would probably write a function for connecting those objects at the application level. Whenever a connection is made, save the relationship in both objects. Then you can search either direction with ease. You'll have redundant data and that causes a risk of getting them out of sync, but that's the price you have to pay with a non-relational structure. Your updates won't be as fast since you'll have to update two docs, but your selects should be speedy.
As Tim Suggests a good idea would be to solve this at the app level.
What i'd do here is create a new collection 'Events' then store an array of _id's of related events inside the user and team objects from here it will be super fast to do a look up. It may mean a lot more queries but queries on the _id field alone are highly optimised and not very resource intensive (unless you have millions of events per user) so if a team has the app up they can see their events and if a user has theirs up they can see their events.
Also i recommend storing back links to the user and team _id's in the event object. Yes this is redundant data but its only a reference and if managed properly at the app level should keep the schema nice and tidy.
Best of Luck.
Related
I am calculating basketball stats and have the models Stat, User (which the basketball players are held within), Team, Stat_Meta, Game, Season, Substitution.
I have a view called statTable that can be added to any other view on the app. statTable basically just iterates through each player on the team and retrieves the calculation for each stat type (found in Stat_Meta model). Within those calculation, there are queries run for the Stat, Game, Season, etc. tables. By the time it iterates through every player and all their stats, we are looking at like 500 queries PER game (often we are going through like ~30 queries/view, so you do the math, it's bad).
My question: With the Laravel debug bar installed, I can see that in my test environment, I've got 3,116 queries running when loading the front page, and 2,432 of them are duplicates. It takes forever to load as well. So, how can I re-work this system of queries to reduce the number of them?
Full disclosure, I'm not a CS person, so efficiency isn't something I'm trained in. Right now, I'm super happy this even works, but not it is going to cost me an arm and a leg to do all these queries at scale (not to mention horrible UX).
You could do some optimization of your queries by using Laravel's eager loading. Definition of eager loading from official documentation:
When accessing Eloquent relationships as properties, the relationship
data is "lazy loaded". This means the relationship data is not
actually loaded until you first access the property. However, Eloquent
can "eager load" relationships at the time you query the parent model.
Eager loading alleviates the N + 1 query problem.
You can read some great examples from the link I provided. I believe this will optimize your queries a lot.
Beside eager loading, you should always aim to accomplish as much as you can with your queries instead of processing data with PHP, Laravel collections, etc.
I'm currently playing around with spark for laravel. I'd like to limit the amount of teams per user to 1. So basically I wantto forbid to create new teams or join other teams. I found the
CanJoinTeams Trait
But I'm actually not sure where to keep looking for changes. I'd love if someone could tell me where I have to look at and where I could overwrite necesseray functions.
By looking at code that someone forked on Github for Spark, it looks like one option is going to be to change that relationship between the User Model (code in CanJoinTeams Trait) and the Teams Model from Many to Many to One to Many, which is going to require you to any spot that uses information in the pivot table.
Another, possible option is that you perform a check in one or both models to see if the user already belongs to a team and if they do either ask if they want to change teams or throw some kind of error.
After searching online a good amount, I couldn't find anything that was exactly what I was looking for or that outlined how to create a web application with users. The closest thing I could find was this stackoverflow post, which outlined how to create a web application with users/roles on a SQL server.
I also found a couple alternatives from this post.
Create a simple relational database
make a 'meta-table'
I'm creating a web application on CodeIgniter that is a wine management tool, similar to CellarTracker. Each user has their own 'cellar', or collection of wines that they can perform the CRUD operations on. What would be a conventional way to make this. How should I structure the database? Are there any CI Libraries that would make User-management, roles, permissions, etc. easier?
(I have already found that TankAuth seems like its the best authentication library).
Here's how I would structure (part of) the database:
**users_table**
id
email
password
**wines_table**
id
user_id
My plan is to have each bottle of wine have a user's id associated with it. I doubt that this is the most secure or efficient way to handle a CRUD web application with many users though.
I have experience with the CRUD operations, but am not sure how I can connect them with a user-management system.
consider having a completely separate table just for wines that is not associated with users.
this table has wine parents and children. the parent contains fields for the winery, the name of the wine, location, etc. the children would be the specific years of the vintage.
so a user enters a wine name in his tracker. the system looks it up - if its not found then the user creates the record. system creates a parent record and a separate child record with the year.
if the wine is found - then the user can update the parent record, and either chose a year, or enter a new one.
system would then add a record to a separate table - users_wines - with the id of the parent (the wine) and the id of the child (the specific year)
this opens up many different possibilities including having wine lists that are shared by groups of users.
you also can easily change the relationship of the user and his wine bottle - because they are separate. the user can change status on users_wines - wine in cellar, drinking the wine, empty wine bottle - and this does not affect the record for the wine itself.
====
Edit - yeah mentioning the wine.com api would have been good info. that gets into much larger questions. suggest that you first define what your users are able to do. My first question is - what if a user does not like the name coming back from wine.com? and what if there is an actual mistake in the api? are you going to customer service a wrong wine name coming back from the api?
Or - Can a user rename a wine in their list? if they can then the data relationship with wine.com is going to be very different.
personally i would think it would be annoying to try and create a wine list, but you cant edit the names. it seems like thats removing too much control from the user.
however if this is an app for restaurants that need accountability - then having the wine name be unchangeable could be considered a feature.
so spend some time with your user roles and user stories. then let that inform the data relationships.
I've been using this site as a great resource since I started at school - but I've never had a problem this specific before and I was hoping for a bit of help. I was never the best when it came to figuring out database structures, and I've been tasked with creating a PHP/MySQL test engine with some rather specific specifications.
So you can better understand what I'm going for here - I am trying to take into account the following:
Administrator and student login are required, and provide different levels of access.
An administrator should be able to build one or more tests and assign it to one or more students.
For each question an administrator builds within a test, the administrator should be able to assign a point value.
A test should be able to present one or more questions.
Your application should support three basic questions types: true/false, multiple choice and fill in the blank.
Final results will display an overall score, as well as a student’s response to each question.
A student should be able to see final results for only test they have access to.
An administrator should see results for tests from multiple students.
Students are not allowed not retake the same test.
Mostly, I am trying to deal with the basic structure. I had five tables at the start, I condensed the Question/Answer section into one table - and excuse my poor attempt at switching around the relationships here, because I've had them 20 different ways it feels:
Am I on the right track? Any suggestions?
A good rule of thumb with database design in N-1. For all tables that have relationships, you should have N-1 relationships (where N is the number of tables). Tables with circular references are a no-no. Putting as the security components and just looking at test/questsion/answers, you want a design that has Some basic objects:
Admin (or teacher) table
Test table
Student Table
Question Table
You didn't mention it in your question, but if you want each test to be associated with a class that the teacher teaches, you will need that as well. With those four, or five, tables you should be able to create your relationships. Hint: Most of these relationships are many-to-many and, as such, will need an XRef table to resolve this. Post back what you do with this and we can look at what's next.
I am using the Data Mapper Pattern and I am wondering what is the best way to handle relationships with that pattern. I spent a lot of time searching for solutions in Google and Stack overflow, I found some but I am still not completely happy about them, especially in one special case that I will try to explain.
I am working with PHP so the examples of code that I will put are in PHP.
Let's say I have a table "team" (id, name) and a table "player" (id, name, team_id). This is a 1-N relationship.
By implementing the Data Mapper pattern, we will have the following classes: Team, TeamMapper, Player and PlayerMapper.
So far, everything is simple. What if we want to get all players from a team?
The first solution I found is to create a method getAllPlayers() in the Team class which will handle that with lazy loading and proxies. Then, we can retrieve the players of a team like that:
$players = $team->getAllPlayers();
The second solution I found is to directly use the PlayerMapper and pass the team ID as parameter. Something like:
$playerMapper->findAll(array('team_id' => $team->getId()));
But now, let's say that I want to display a HTML table with all the teams and with a column 'Players' with all of the players of each team. If we use the first solution I described, we will have to do one SQL query to get the list of teams and one query for each team to get the players, whcih means N+1 SQL queries where N is the number of teams.
If we use the second solutions I described, we can first retrieve all team IDs, put them in an array, and then pass it to the findAll method of the player mapper, something like that:
$playerMapper->findAll(array('team_id' => $teamIds));
In that case, we need to run only 2 queries. Much better. But I am still not very happy with that solution because the relationships are not described into the models and it is the developer who must know about them.
So my question is: are there others alternatives with the Data Mapper pattern? With the example I gave, is there a good way to select all teams with all players in just 2 queries with the description of the relationships into the model?
Thank you in advance!
If you look at Martin Fowler's text that describes how the DataMapper works, you'll see that you can use one query to get all the data that you need and then pass that data to each mapper, allowing the mapper to pick out only the data that it needs.
For you, this would be a query that joins from Team to Player, returning a resultset with duplicated Team data for each unique Player.
You then have to cater for the duplication in your mapping code by only creating new objects when the data changes.
I've done something similar where the equivalent would be the Team mapper iterating over the result set and, for each unique team pass the result set to the Player mapper so that it can create a player and then add the player to the team's collection.
While this will work, there are problems with this approach further downstream...
I have a possible solution to this problem that I have implemented successfully in one of my projects. It is not so complex and would use only 2 queries in the example described above.
The solution is to add another layer of code responsible for handling relationships.
For instance, we can put that in a service class (which can be used for other stuff as well, not only handling relationships).
So let's say that we have a class TeamService on top of Team and TeamMapper. TeamService would have a method getTeamsWithRelationships() which would return an array of Team objects. getTeamsWithRelationships() would use TeamMapper to get the list of teams. Then, with the PlayerMapper, it would get in only one query the list of players for these teams and set the players to the teams by using a setPlayers() method from the Team class.
This solution is quite simple and easy to implement, and it works well for all types of database relationships. I guess that some people may have something against it. If so, I would be interested to know what are the issues?