I am working on some type of private conversations site and I am storing conversations on a table (Messages are stored on another table)
That's the format:
Id: Auto incrementing
Title: Varchar, represents the name of the conversation
Members: Saves the ids of the members with access to the conversation
My problem is that the way that I'm storing members is literally horrible. I'm using user1Id;user2Id;user3Id.
The problem appears when trying to get all conversations of a specific player.
My question is: Is there any better way of storing the ids? (On one field if possible) and also being able to select all conversations of a specific id?
The "proper", normalized, database way would be to have another table, member_conversation_map with two columns - the member ID and the conversation ID, and query it with, e.g., an in operator:
SELECT *
FROM conversations
WHERE id IN (SELECT conversation_id
FROM member_conversation_map
WHERE member_id = 123)
If that's not an option, and you positively have to have the member IDs in a single cell in the database, at least use commas instead of semicolons as your delimiter. This will allow you to use MySQL's find_in_set:
SELECT *
FROM conversations
WHERE FIND_IN_SET('123', members) > 0
Why not make a member to conversation table? In there you can store members and conversations like this
member_id | conversation_id
1 | 1
2 | 1
3 | 1
1 | 2
2 | 2
Meaning member 1, 2 and 3 are part of conversation 1 and member 1 and 2 are of conversation 2
Thats how i do stuff like this and its working good this way.
If you still want to store them in one field, your way would be the best, since you can split them by their seperator and have all ids in an array that way. But as you pointed out, getting conversations by an user id is getting messy pretty quick using this method.
One way to store is using a json array: Define member_ids to VARCHAR and store priviliges ids like:
["11","1","2","3"]
Using json_decode you can get the members when required.
Related
I have a one-to-many relationship of rooms and their occupants:
Room | User
1 | 1
1 | 2
1 | 4
2 | 1
2 | 2
2 | 3
2 | 5
3 | 1
3 | 3
Given a list of users, e.g. 1, 3, what is the most efficient way to determining which room is completely/perfectly filled by them? So in this case, it should return room 3 because, although they are both in room 2, room 2 has other occupants as well, which is not a "perfect" fit.
I can think of several solutions to this, but am not sure about the efficiency. For example, I can do a group concatenate on the user (ordered ascending) grouping by room, which will give me comma separated strings such as "1,2,4", "1,2,3,5" and "1,3". I can then order my input list ascending and look for a perfect match to "1,3".
Or I can do a count of the total number of users in a room AND containing both users 1 and 3. I will then select the room which has the count of users equal to two.
Note I want to most efficient way, or at least a way that scales up to millions of users and rooms. Each room will have around 25 users. Another thing I want to consider is how to pass this list to the database. Should I construct a query by concatenating AND userid = 1 AND userid = 3 AND userid = 5 and so on? Or is there a way to pass the values as an array into a stored procedure?
Any help would be appreciated.
For example, I can do a group concatenate on the user (ordered ascending) grouping by room, which will give me comma separated strings such as "1,2,4", "1,2,3,5" and "1,3". I can then order my input list ascending and look for a perfect match to "1,3".
First, a word of advice, to improve your level of function as a developer. Stop thinking of the data, and of the solution, in terms of CSVs. It limits you to thinking in spreadsheet terms, and prevents you from thinking in Relational Data terms. You do not need to construct strings, and then match strings, when the data is in the database, you can match it there.
Solution
Now then, in Relational data terms, what exactly do you want ? You want the rooms where the count of users that match your argument user list is highest. Is that correct ? If so, the code is simple.
You haven't given the tables. I will assume room, user, room_user, with deadly ids on the first two, and a composite key on the third. I can give you the SQL solution, you will have to work out how to do it in the non-SQL.
Another thing I want to consider is how to pass this list to the database. Should I construct a query by concatenating AND userid = 1 AND userid = 3 AND userid = 5 and so on? Or is there a way to pass the values as an array into a stored procedure?
To pass the list to the stored proc, because it needs a single calling parm, the length of which is variable, you have to create a CSV list of users. Let's call that parm #user_list. (Note, that is not contemplating the data, that is passing a list to a proc in a single parm, because you can't pass an unknown number of identified users to a proc otherwise.)
Since you constructed the #user_list on the client, you may as well compute #user_count (the number of members in the list) while you are at it, on the client, and pass that to the proc.
Something like:
CREATE PROC room_user_match_sp (
#user_list CHAR(255),
#user_count INT
...
)
AS
-- validate parms, etc
...
SELECT room_id,
match_count,
match_count / #user_count * 100 AS match_pct
FROM (
SELECT room_id,
COUNT(user_id) AS match_count -- no of users matched
FROM room_user
WHERE user_id IN ( #user_list )
GROUP BY room_id -- get one row per room
) AS match_room -- has any matched users
WHERE match_count = MAX( match_count ) -- remove this while testing
It is not clear, if you want full matches only. In that case, use:
WHERE match_count = #user_count
Expectation
You have asked for a proc-based solution, so I have given that. Yes, it is the fastest. But keep in mind that for this kind of requirement and solution, you could construct the SQL string on the client, and execute it on the "server" in the usual manner, without using a proc. The proc is faster here only because the code is compiled and that step is removed, as opposed to that step being performed every time the client calls the "server" with the SQL string.
The point I am making here is, with the data in a reasonably Relational form, you can obtain the result you are seeking using a single SELECT statement, you don't have to mess around with work tables or temp tables or intermediate steps, which requires a proc. Here, the proc is not required, you are implementing a proc for performance reasons.
I make this point because it is clear from your question that your expectation of the solution is "gee, I can't get the result directly, I have work with the data first, I am ready and willing to do that". Such intermediate work steps are required only when the data is not Relational.
Maybe not the most efficient SQL, but something like:
SELECT x.room_id,
SUM(x.occupants) AS occupants,
SUM(x.selectees) AS selectees,
SUM(x.selectees) / SUM(x.occupants) as percentage
FROM ( SELECT room_id,
COUNT(user_id) AS occupants,
NULL AS selectees
FROM Rooms
GROUP BY room_id
UNION
SELECT room_id,
NULL AS occupants,
COUNT(user_id) AS selectees
FROM Rooms
WHERE user_id IN (1,3)
GROUP BY room_id
) x
GROUP BY x.room_id
ORDER BY percentage DESC
will give you a list of rooms ordered by the "best fit" percentage
ie. it works out a percentage of fulfilment based on the number of people in the room, and the number of people from your set who are in the room
I have a small problem with a php mysql query, I am looking for help.
I have a family tree table, where I am storing for each person his/her ancestors id separated by a comma. like so
id ancestors
10 1,3,4,5
So the person of id 10 is fathered by id 5 who is fathered by id 4 who is fathered by 3 etc...
Now I wish to select all the people who have id x in their ancestors, so the query will be something like:
select * from people where ancestors like '%x%'
Now this would work fine except, if id x is lets say 2, and a record has an ancestor id 32, this like query will retrieve 32 because 32 contains 2. And if I use '%,x,%' (include commas) the query will ignore the records whose ancestor x is on either edge(left or right) of the column. It will also ignore the records whose x is the only ancestor since no commas are present.
So in short, I need a like query that looks up an expression that either is surrounded by commas or not surrounded by anything. Or a query that gets the regular expression provided that no numbers are around. And I need it as efficient as possible (I suck at writing regular expressions)
Thank you.
Edit: Okay guys, help me come up with a better schema.
You are not storing your data in a proper way. Anyway, if you still want to use this schema you should use FIND_IN_SET instead of LIKE to avoid undesired results.
SELECT *
FROM mytable
WHERE FIND_IN_SET(2, ancestors) <> 0
You should consider redesigning your database structure. Add new table "ancestors" to database with columns:
id id_person ancestor
1 10 1
2 10 3
3 10 4
After -- use JOIN query with "WHERE IN" to choose right rows.
You're having this issue because of wrong design of database.First DBMS based db's aren't meant for this kind of data,graph based db's are more likely to fit for this kind of solution.
if it contain small amount of data you could use mysql but still the design is still wrong,if you only care about their 'father' then just add a column to person (or what ever you call it) table. if its null - has no father/unknown otherwise - contains (int) of his parent.
In case you need more then just 'father' relationship you could use a pivot table to contain two persons relationship but thats not a simple task to do.
There are a few established ways of storing hierarchical data in RDBMS. I've found this slideshow to be very helpful in the past:
Models for Hierarchical Design
Since the data deals with ancestry - and therefore you wouldn't expect it to change that often - a closure table could fit the bill.
Whatever model you choose, be sure to look around and see if someone else has already implemented it.
You could store your values as a JSON Array
id | ancestors
10 | {"1","3","4","5"}
and then query as follows:
$query = 'select * from people where ancestors like \'%"x"%\'';
Better is of course using a mapping table for your many-to-many relation
You can do this with regexp:
SELECT * FROM mytable WHERE name REGEXP ',?(x),?'
where x is your searched value
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
,ancestors VARCHAR(250) NOT NULL
);
INSERT INTO my_table VALUES(10,',1,3,4,5');
SELECT *
FROM my_table
WHERE CONCAT(ancestors,',') LIKE '%,5,%';
+----+-----------+
| id | ancestors |
+----+-----------+
| 10 | ,1,3,4,5 |
+----+-----------+
SELECT *
FROM my_table
WHERE CONCAT(ancestors,',') LIKE '%,4,%';
+----+-----------+
| id | ancestors |
+----+-----------+
| 10 | ,1,3,4,5 |
+----+-----------+
I am quite new to PHP and MySQL, but have experience of VBA and C++. In short, I am trying to count the occurrences of a value (text string), which can appear in 11 columns in my table.
I think I will need to populate a single-dimensional array from this table, but the table has 14 columns (named 'player1' to 'player14'). I want each of these 'players' to be entered into the one-dimensional array (if not NULL), before proceeding to the next row.
I know there is the SELECT DISTINCT statement in MySQL, but can I use this to count distinct occurrences across 14 columns?
For background, I am building a football results database, where player1 to player14 are the starting 11 (and 3 subs), and my PHP code will count the number of times a player has made an appearance.
Thanks for all your help!
Matt.
Rethink your database schema. Try this:
Table players:
player_id
name
Table games:
game_id
Table appearances:
appearance_id
player_id
game_id
This reduces the amount of duplicate data. Read up on normalization. It allows you to do a simple select count(*) from appearances inner join players on player_id where name='Joe Schmoe'
First of all, the database schema you're using is terrible, and you just found out a reason why.
That being said, I see no other way then to first get a list of all players by distinctly selecting the names of players into an array. Before each insertion, you would have to check if the name is already in the array (if it is already in, don't add it again).
Then, when you have the list of names, you would have to run an SQL statement for each player, adding up the number of occurences, like so:
SELECT COUNT(*)
FROM <Table>
WHERE player1=? OR player2=? OR player3=? OR ... OR player14 = ?
That is all pretty complicated, and as I said, you should really change your database schema.
This sounds like a job for fetch_assoc (http://php.net/manual/de/mysqli-result.fetch-assoc.php).
If you use mysqli, you would get each row as an associative array.
On the other hand the table design seems a bit flawed, as suggested before.
If you had on table team with team name and what not and one table player with player names.
TEAM
| id | name | founded | foo |
PLAYER
| id | team_id | name | bar |
With that structure you could add 14 players, which point at the same team and by joining the two tables, extract the players that match your search.
Apologies if this is really stupid but I don't have any experience in php and mysql to know how things should be done. I have a customer table in a mysql db and a group table:
customers - ID name email phone group
groups - ID name description
So I need to assign groups to customers if necessary, this can be more than one group to each customer. So e.g. customer 1 is in group 4,5,6
What way should I assign groups in the group column of the customer table. Should I just add the group ID's separated by commas, then just use explode when I need to get the individual ID's out?
Maybe this isn't the right approach at all, could someone enlighten me please. I would appreciate knowing the right way to do this, thanks.
Do not store multiple IDs in one column. This is a denormalization that will make it much harder to query and change your data, as well as hurting performance.
Instead, create a separate CustomerGroup table (with CustomerID and GroupID columns), and have one row per Customer/Group relationship.
Here is an example of tables to show how you should implement this :
Table 1 CONSUMERS:
id name email
1 john john#something.com
2 ray ray#something.com
Table 2 GROUPS :
id group_name description
1 music good music group
2 programming programmers
Table 3 CONSUMERS_GROUPS
consumer_id group_id
1 1
1 2
2 1
Now the table 3 is listing consumers ids which belong to which group id.
This type of relationship is called one to many relation where, one consumer can have many groups. Reverse might also be true where one group can have many consumers. In that case relationship is called many to many
Should I just add the group ID's separated by commas, then just use explode when I need to get the individual ID's out?
No! If you do that then you won't quickly be able to (for example) query for which users there are in a specific group.
Instead use a join table with two columns, each of which has a foriegn key constraint to the corresponding table.
group_id customer_id
4 1
5 1
6 1
In cases when some one needs to store more than one value in a in a cell, what approach is more desirable and advisable, storing it with delimiters or glue and exploding it into an array later for processing in the server side language of choice, for example.
$returnedFromDB = "159|160|161|162|163|164|165";
$myIdArray = explode("|",$returnedFromDB);
or as a JSON or PHP serialized array, like this.
:6:{i:0;i:1;i:1;i:2;i:2;i:3;i:3;i:4;i:4;i:5;i:5;i:6;}
then later unserialize it into an array and work with it,
OR
have a new row for every new entry like this
postid 12 | showto 2
postid 12 | showto 3
postid 12 | showto 5
postid 12 | showto 6
postid 12 | showto 8
instead of postid 12 | showto "2|3|4|6|8|5|".
OR postid 12 | showto ":6:{i:0;i:2;i:1;i:3;i:2;i:3;i:3;i:4;i:4;i:5;i:5;i:6;}".
Thanks, looking forward to your opinions :D
In cases when some one needs to store more than one value in a in a cell, what approach is more desirable and advisable, storing it with delimiters or glue and exploding it into an array later for processing in the server side language of choice, for example.
Neither. Oh goodness, neither! Edgar F. Codd is rolling in his grave right now.
Storing delimited data in a text field is no better than storing it in a flat file. The data becomes unqueryable. Storing PHP serialized data in a text field is even worse because then only PHP can parse the data.
You want a nice, happy, normalized database.
The thing you're trying to describe is a many-to-many relationship. Each user can maintain one or more posts. Likewise, each post can be maintained by one or more user. Right? Then something like this will work.
CREATE TABLE users (
user_id INTEGER PRIMARY KEY,
...
);
CREATE TABLE posts (
post_id INTEGER PRIMARY KEY,
...
);
CREATE TABLE user_posts (
user_id INTEGER REFERENCES users(user_id),
post_id INTEGER REFERENCES posts(post_id),
UNIQUE KEY(user_id, post_id)
);
-- All posts made by user 22.
SELECT posts.*
FROM posts, user_posts
WHERE user_posts.user_id = 22
AND posts.post_id = user_posts.post_id
-- All users that worked on post 47
SELECT users.*
FROM users, user_posts
WHERE user_posts.post_id = 47
AND users.user_id = user_posts.user_id
Most of the time the recommendation is that many-to-many relationships (such as posts to users) should have a mapping table with 1 row for each post-user combination (in other words, your "new row for every new entry" version).
It's more optimal for things like join queries, and lets you retrieve only the data you need.
You should only serialize data in the DB if the data is never needed to be processed by the DB. For example, you could serialize user ID in the user_id field if you never need to do a query with the user_id field; e.g. never selecting anything based on user.
If these are posts (blog/news/etc. posts?) then I'm pretty confident you'll need to be able to query them by user. Normalizing the user into another table would serve you:
CREATE TABLE posts (post_id, ....);
CREATE TABLE post_users (post_id, user_id, ...);
You can then get the users in a different query, or use group_concat: SELECT post_id, GROUP_CONCAT(user_id) FROM posts JOIN post_users USING (post_id) GROUP BY post_id. When you need to show user name, just join to the users table to get their name in the group concat.
From RDBMS point of view i would 'have a new row for every new entry'
Thats called m:n relationship table.
You can then query the data however you like.
If you need postid 12 | showto ":6:{i:0;i:2;i:1;i:3;i:2;i:3;i:3;i:4;i:4;i:5;i:5;i:6;}". you can do
SELECT postid, CONCAT(':',count(showto),':{i:',GROUP_CONCAT(showto SEPARATOR ';i:'),';}') AS showto
FROM tablename
GROUP BY postid
However if you only need the data in 1 form and not do any other kind of queries on that data then you may aswell store the string.