i am working on a project in which people can create a playlist and its stored in localStorage as objects. everything is client side for the moment.
so i will now like to take a leap forward, make a user login system (i can do it using php mysql and fb connect or oauth system, any other suggestions?). the problem is deciding if i make a sql database for each user and store their playlist (with media info) or is there any other way to go around. will handling a large number of databases be a trouble for me(in terms of speed)?
how about i create only one db as follows:
user database ---> one table containing{ user(primary key) pass someotherInfo} , then tables per USER {contains playlists) , 3rd table per playlist (containing userID and media info, what could be my primary key?)
example:
i have 10 registered user, each user has 2 playlists
1.table 1: 10 entries
2.table(s): username - playlists (10 tables) || i make one table with one field user other field playlist name
3.tables: each playlist - media info, owner (20 tables)
or is there a simpler way?
i hope my question is clear.
PS: i am new to php and database (so this might be very silly)
Surprised most answers seems to have missed the question, but I'll give this a try;
This is called data modeling (how you hobble a bunch of tables in a database together in order to express what you want in the best possible way), and don't feel silly for asking; there are people out there who spend all their waking hours tweaking and designing data models. They are hugely important to the well-being of any system, and they are, in truth, far more important that most people give them credit for.
It sounds like you're on the right path. It's always a good tip to define your entities, and create a table per each, so in this case you've got users and playlists and songs (for example). Define your tables thusly; USER, SONG, PLAYLIST.
The next thing is defining the names of fields and tables (and perhaps the simplistic names suggested above are, well, simplistic). Some introduce faux namespaces (ie. MYAPP_USER instead of just USER), especially if they know the data model will extend and expand in the same database in the future (or, some because they know this is inevitable), while others will just ram through whatever they need.
The big question will always be about normalization and various problems around that, balancing performance against applicability, and there's tons and tons of books written on this subject, so no way for me to give you any meaningful answer, but the gist of it for me is;
At what point will a data field in a table be worthy of its own table? An example is that you could well create your application with only one table, or two, or 6 depending on how you wish to split your data. This is where I think your question really comes in.
I'd say you're pretty much correct in your assumptions, the thing to keep in mind is consistent naming conventions (and there's tons of opinions of how to name identifiers). For your application (with the tables mentioned above), I'd do ;
USER { id, username, password, name, coffee_preference }
SONG { id, artist, album, title, genre }
PLAYLIST { id, userid }
PLAYLIST_ITEM { id, songid, playlistid, songorder }
Now you can use SQL you get all playlists for a user ;
SELECT * FROM PLAYLIST WHERE userid=$userid
Or get all songs in a playlist ;
SELECT * FROM SONG,PLAYLIST_ITEM WHERE playlist_item.playlistid=$playlist.id AND song.id=playlist_item.songid ORDER BY playlist_item.songorder
And so on. Again, tomes have been written about this subject. It's all about thinking clearly and semantically while jotting down a technical solution to it. And some people have only this as a career (like DBA's). There will be lots of opinions, especially on what I've written here. Good luck.
You can use either an SQL database like MYSQL or Postgresql or a NOSQL database like MongoDB. Each has it's pros and cons but since you seem like a beginner i am going to suggest MYSQL because it's what most beginners work with. Take a look at these articles
http://dev.mysql.com/tech-resources/articles/mysql_intro.html
http://www.redhat.com/magazine/007may05/features/mysql/
Of course you may feel free to do you own searching on The Big G as there are tons of resources out there.
Related
I'm trying to create a Like/Unlike system akin to Facebook's for an existing comments section of a website, and I need help in designing the system.
Currently, every product on the website has a comments section and members can post and like comments. I need to know each member has posted how many comments and each of his comments has received how many likes. Of course, I need to know who liked what comments too (partly so that I can prevent a user from liking a comment more than once) for analytical purposes.
The naive way of implementing a Like system to the current comments module is to create a new table in the database that has foreign keys to the CommentID and UserID. Then for every "like" given to a comment by a user, I would insert a row to this new table with the targeting comment ID and user ID.
While this might work, the massive amount of comments and users is going to cause this table to grow quickly and retrieving records from and doing counts on this huge table will become slow and inefficient. I can index either one of the columns, but I don't know how effective it would be. The website has over a million comments.
I'm using PHP and MySQL. For a system like this with a huge database, how should I designing a Like system so that it is more optimised and stable?
For scalability, do not include the count column in the same table with other things. This is a rare case where "vertical partitioning" is beneficial. Why? The LIKEs/UNLIKEs will come fast and furious. If the code to do the increment/decrement hits a table used for other things (such as the text of the Comment), there will be an unacceptable amount of contention between the two.
This tip is the first of many steps toward being able to scale to Facebook levels. The other tips will come, not from a free forum, but from the team of smart engineers you will have to hire to get to that level. (Hints: Sharding, Buffering, Showing Estimates, etc.)
Your main concern will be a lot of counts, so the easy thing to do is to keep a separate count in your comments table.
Then you can create a TRIGGER that increments/decrements the count based on a like/unlike.
That way you only use the big table to figure out if a user already voted.
I'm designing a PHP/MySQL solution for my youth athletic organization that will display schedules and standings for the teams participating in any given season. Unfortunately, I'm just learning the ropes when it comes to database design and having a bit of a challenge wrapping my brain around what should be a relatively simple concept. Perhaps someone can step in and clarify a few points?
What I'm unclear on is how exactly to structure the database(s) and tables to get started correctly. This is what I will ultimately need to set-up:
We have approximately 50 member schools. Each will need their own log-in ID and password. I was thinking about storing these in their own separate database because the accounts will be re-used again each season to represent teams and allow the coaches to log-in and update their game results.
Each season has two different sports. For example, the upcoming fall season has a girls volleyball league and a boys flag football league. Each league then has multiple divisions (divided by geographic region) in which the schools play their regular-season schedule. I'm not sure if each season needs its own separate database, each league, etc.
Each division's regular-season will be imported from a scheduling program (via an Excel spreadsheet) and displayed on its own webpage where schools can view their upcoming games and log-in to update results.
As game results are updated, I'm going to need to find a way to have those results reflected on a separate webpage where that division's league standings are displayed.
For a clearer demonstration of what I'm trying to do, have a look at these two mock pages I created for the schedules and standings data, respectively:
http://www.712jefferson.org/pal/schedules.html
http://www.712jefferson.org/pal/standings.html
This is where I'm looking for your guidance. How exactly should I structure the MySQL framework for all of this data?
I'm thinking a separate database for the school log-in ID's and then a separate database for each seasonal sport (e.g. Boys Volleyball 2013) with separate tables for each division and x-number of columns to represent the schedules and standings therein.
Does that sound about right? Am I thinking about it all wrong? Any assistance or guidance to help flesh out this thought process would be MUCH appreciated!
Thank you so much!
Ouch. You have taken on a big job. Are you absolutely sure that nothing you are going to do cannot be done by something that is already available? Well, if you are sure, read on.
Firstly, the hardest part of what you want to do is managing your user access. I'd advise that you start off by writing your user management module before you go any further.
For what you want, it seems likely that Drupal or one of the other senior CMS systems would be a great way to bootstrap the system. Drupal will handle your user management right out of the box (or with minimal problems) and you could write the rest of your code as static nodes. This also makes it easy to add blogs, forums, news and to manage mailing lists etc.
As stated in the comments above, you do need to keep your data together. it would be good to keep data for historical comparisons too.
If not extending a CMS, after you have got back from the psychiatrist, you will need something along the lines of:
header file to access the db and check for user authentication.
footer file to display your data
individual page files to present or obtain your data.
Database structure for handling the users (at minimum) should be IRO:
Person - details of individual users
username - link person to a username
email - email addresses
club - sports club details
password - passwords
logon - record of logon attempts
role - record of role of individuals in your site
permissions - list of required permissions to access areas of the site
role_permissions - default permissions for each role
person_role - link person to role
person_permissions - link person to permissions (only needed if some individuals need extra permissions not given routinely by their role)
club_person; person_email; - link people to clubs and to their email addresses.
To handle the matches you will need:
team - team name, group and club reference
grouping - list of groups eg by age.
divisions. - list of divisions
venue - list of venues. Include GPS!!!
match - division, grouping, team1, team2, venue, date, time
result - team1 reported result, team 2 reported result, approved result (you may need to intervene!) match.
As you can see you need a fair few tables but YOU MUST NOT try to do the fun stuff with the actual teams UNTIL you have your user access working properly.
What I have sketched out for you is a db in normal form. No textual data is duplicated and the data is easy to retrieve, index and display. I do feel this question is too broad for SO as designing a database for you is a bit out of scope but I do think the general format is useful.
Each table should only contain unique necessary data eg:
Person: personid int, surname, forename, style, whenadded, whoadded, inuse
email: emailid, email, whenadded, whoadded, inuse
email_person: emailpersonid,emailid,personid, whenadded,whoadded,inuse
This allows multiple people to share one email and multiple emails to be applied to one person with no text duplication. IDs should be type INT AUTO_INCREMENT PRIMARY KEY rather than SERIAL as this saves a whole lot of storage space and you'll never fill an INT in this application.
The other tables should be created in the same way. The whoadded and whenadded columns are optional and quite storage hungry but can be very useful. inuse is essential set this to a BOOL and you can remove teams without deleting them - the data is not lost. A whenremoved and whoremoved is useful too for audit.
A word on passwords - please ensure you store these as a SALTED HASH. If you do this, when your site is hacked, no-one will have the password that they also use for their internet banking exposed. People are often idiots. You have to look after them.
As I said, a bit out of scope so I'll end the answer there - it gives you as requested the basic outline of a 4th Normal Form Db that will be robust and extendable but leaves you to do the work. Why not ask more questions if the problem turns out to be too hard.
Good luck.
ADDED:
DIY Framework:
If you don't want to learn to use one of the existing frameworks or CMS, you will need to write your own. Oddly, this is actually very easy.
header.php:
<?PHP
$mysqli=new mysqli(credentials....)//connect to database and present a mysqli or pdo object.
session_start(); //open a session
//you will need to authenticate your session here - see below
?>
footer.php:
<HTML>
<HEAD>
<TITLE>
<?PHP echo $pagetitle;?>
</TITLE>
</HEAD>
<BODY>
<?PHP echo $content;?>
</BODY>
</HTML>
These are used by mypage.php:
<?PHP
require("header.php");
//do some stuff that generates $content
$pagetitle="mypage.php";
require("footer.php:);
?>
It should be stressed that this is the bare minimum you will need and is truly sucky - it is just presented to show how this should be begun, not an exemplar of ideal code. It will work though.
The key is creating a header that presents the variables you will need, such as a db connection, username, user logon status etc. and a footer you can enter details into to present the data. The footer is the only place where you combine HTML and PHP.
Use your $_SESSION to store information that needs to persist between pages.
These files can be as simple or complex as you like - I created my own ages ago that do several checks on the user and session and can display scripts, custom CSS files and such in the footer. It's not hard to do if you start simple and build on as you need. SO will be here to help you.
One word of caution: although you can start very simple, what you are trying to do has legs and will get out of hand. Please audit your code after you have it up and running to make sure that you have not inadvertently included security flaws. It is very easy to include these as you get into a project and need a quick fix and they can be devlishly hard to spot later on unless you are looking for them.
I am in your shoes, and the best solution I found was to use Backend as a Service (Baas) providers such as Parse.com or Stackmob.com or many others. They do complete database hosting for you plus your user management (login/register/forgot_password etc) so you don't have to worry about it. You access them via REST API and they also have PHP libraries on the Github.
Since you are new to database management, which is a non-trivial task, this solution may save you some headache.
Of course, you still have to decide how to organise your tables (or 'classes' in Parse, 'schema' in Stackmob etc) but they also provide help with their relation APIs. They have a pre-defined User table specifically designed for storing users.
Surely their API is not as flexible as SQL but it also safe from SQL injection, a downside of that flexibility. On the other hand, they also offer custom code to enhance that flexibility.
Edit.
Forgot to mention, those BaaS platforms also allow to store arrays as database fields, which can be really handy.
This can be a big build. You may want to try an existing service for league management. There's a few out there. I know a few friends who use Teamopolis and are very happy with the service, and I think it accomplishes everything you're looking for.
I'm creating a website and new to php and mysql. I just read as much as I could this morning and I've got the registration and login all working, along with a form a user can use to submit content.
My question specifically is how I should store the user submitted content. The content will basically be articles. As it stands now, I have a database for all of the users that stores their name, id, and password, but whenever someone makes an account I generate a new database with their to store any articles they submit. Are there any problems with this? I don't need the articles to be searchable at this point, but I would like to save the text of the articles, a category the article falls into, and the time it was published.
Is making a new database for every user that signs up a bad idea? Also, I was wondering about memory concerns. Are there any practical considerations I should take with regards to all of this content filling tons of server hard drive space?
When using databases the primary concern is SQL Injection. Use parametrised queries or escape all your data. Also in the users table don't store the plain text password. Use a hash and salt.
On the schema rather than creating a new database for each user I would use the same database with tables users (containing an id field) and an articles table containing the user id. This should be sufficent to know who the author of each article is.
I was a CMS-exclusive developer for 5 years, so I've jumped through these same hoops.
Your solution is fine if you don't mind having to fully qualify the Database names or dealing with multiple connection switching. My company does it for a very complex application, but I do have to mention that it's frustrating and annoying when the connection fails and we do get connection error issues quite often.
In my personal work, I prefer using just one database and adding a user or company id field to each entry. That way, if for some reason I want to aggregate from one account to the next, it's a piece of cake. Trying to search across DB's would definitely not be a fun task at all. The main content field (body, content, etc) is nearly always a MySQL Text field to avoid running out of field room. Ensure that when inserting you do some sort of data cleansing to avoid sql injection. Mysql_real_escape_string usually does the trick.
Space-wise, unless you grow a VERY popular site you're not going to have issues. I've run several hundred medium-sized CMS's on one small server with no issue.
Creating a new database for every user is definitely the wrong approach in possibly any conceivable scenario since it defies the purpose of relational databases.
What you should (probably) be doing is to create an articles table that contains articles (title, content etc.), and a user table that contains user data. Depending on your data model, you can either have the author data stored in the db row for every article (foreign key), or in case a single article can have multiple authors, you can store the connections (relationships) in a separate table.
I see many implementations such as the Facebook like, forum karma, mark as read on forum posts and other simple options and selections available to multiple users on a given item.
I know I can implement this in mysql by creating a table which links say post IDs to liker user IDs for say, a like system.
My problem is, on a page with lots of posts, I will have to make a lookup for every post. I use prepared statements so that makes it faster for me.
Is there another way to implement these systems, if not, are there optimisations like database types or other tweaks that can make this faster?
Basically, is there a powerful, fast implementation of a many to many database interaction.
*EDIT***
I'm using opera mini and so I have issues with the ajax and js for commenting
Right now, I have a table with two columns. One for user id and the other for post id. Both are indexed and are used in foreign key constraints.
I'm thinking of making a compound primary key across the two.
My main issue is for the karma. I allow users to vote on each post. The problem is, for each post, I need to get the total votes, determine if a user has voted to either allow the user to or not to vote.
My site allows many users to host their own sites and so I need to seriously optimize this.
Someone suggested I use memory tables for this.
NOTE**
I can't use memcached.
I strongly suggest using something else than a MySQL db. I've written an opensocial app which had both heavy writes and reads to a database. It all started with a MySQL DB, I even switched to a dedicated master slave replication setup. But to no avail, it was expensive and it didn't scale very well.
The final solution was to use a NoSQL db which made the most out of RAM. My decision was mongoDB which has an activy community and solved my problem very well. MongoDB proofed to be highly scalable.
Still a little hazy about what you got so far, but I'll start it off and keep adding stuff if need be:
Make sure you're Indexing
Minimum lookups -so you get the list
of posts that will pop up, use that
list to match the like's, if they've viewed the article etc.
Using numbers - make sure all your
comparisons are with numbers
If you're running queries, don't run a single query for each post
Is there a limit in your query? - make sure you use that
De-normalization is not a sin
You can partition your databases to decrease lookups (e.g. if data is older than 60 days and barely touched, move it to a secondary database/table, so the size of your table is not huge)
e.g. SELECT * FROM user_liked WHERE post_id IN (1,2,3)
instead of
SELECT * FROM user_liked WHERE post_id = 1
Philipp Keller wrote a bunch of articles on tag systems based on MYSQL a few years ago. Just as Like-ing, Tagging is establishing a many-to-many relationship between a thing (tag, article being liked) and a user. The logic in his articles should be directly applicable to your problem as well.
Check out the comments as well.
http://www.pui.ch/phred/archives/2005/04/tags-database-schemas.html
Database Schemas for Tagging solutions
http://www.pui.ch/phred/archives/2005/05/tags-with-mysql-fulltext.html
Abusing the MySQL FULLTEXT indices for tagging and tag search (requires MyISAM, I'd not go there).
http://www.pui.ch/phred/archives/2005/06/tagsystems-performance-tests.html
Performance Tests of tagging systems
I have been creating a web app and am looking to expand. In my web app I have a table for users which includes privileges in order to track whether a user is an administrator, a very small table for a dynamic content section of a page, and a table for tracking "events" on the website.
Being not very experienced with web application creation, I'm not really sure about how professionals would create systems of databases and tables for a web application. In my web app, I plan to add further user settings for each member of the website and even a messaging system. I currently use PHP with a MySQL database that I query for all of my commands, but I would be willing to change any of this if necessary. What would be the best wat to track content such as messages that are interpersonal and also specific user settings for each user. Would I want to have multiple databases at any point? Would I want to have multiple tables for each user, perhaps? Any information on how this is done or should be done would be quite helpful.
I'm sorry about the broadness of the question, but I've been wanting to reform this web app since I feel that my ideas for table usage are not on par with those that experienced programmers have.
Here's my seemingly long, hopefully not too convoluted answer to your question. I think I've covered most, if not all of your queries.
For your web app, you could have a table of users called "Users", settings table called "UserSettings" or something equally as descriptive, and messages in "PrivateMessages" table. Then there could be child tables that store extra data that is required.
User security can be a tricky thing to design and implement. Do you want to do it by groups (if you plan on having many users, making it easier to manage their permissions), or just assign individually due to a small user base? For security alone, you'd end up with 4 tables:
Users
UserSettings
UserGroups
UserAssignedGroups
That way you can have user info, settings, groups they can be assigned to and what they ARE assigned to separated properly. This gives you a decent amount of flexibility and conforms to normalization standards (as mentioned above by DrSAR).
With your messages, don't store them with the username, but rather the User ID. For instance, in your PrivateMessages table, you would have a MessageID, SenderUserID, RecipientUserID, Subject, Body and DateSent to store the most basic info. That way, when a user wants to check their received messages, you can query the table saying:
SELECT * FROM PrivateMessages WHERE RecipientUserID = 123556
A list of tables for your messages could be as such:
PrivateMessages
MessageReplies
The PrivateMessages table can store the parent message, and then the MessageReplies table can store the subsequent replies. You could store it all in one table, but depending on traffic and possibly writing recursive functions to retrieve all messages and replies from one table, a two table approach would be simplest I feel.
If I were you, I'd sit down with a pencil and paper, and write down/draw what I want to track in my database. That way you can then draw links between what you want to store, and see how it will come together. It helps me when I'm trying to visualise things.
For the scope of your web app you don't need multiple databases. You do need, however, multiple tables to store your data efficiently.
For user settings, always use a separate table. You want your "main" users table as lean as possible, since it will be accessed (= searched) every time a user will try to log in. Store IDs, username, password (hashed, of course) and any other field that you need to access when authenticating. Put all the extra information in a separate table. That way your login will only query a smaller table and once the user is authenticated you can use its ID to get all other information from the secondary table(s).
Messages can be trickier because they're a bigger order of magnitude - you might have tens or hundreds for each user. You need to design you table structure based on your application's logic. A table for each user is clearly not a feasible solution, so go for a general messages table but implement procedures to keep it to a manageable size. An example would be "archiving" messages older than X days, which would move them to another table (which works well if your users aren't likely to access their old messages too often). But like I said, it depends on your application.
Good luck!
Along the lines of Cristian Radu's comments: you need to split your data into different tables. The lean user table will (in fact, should) have one unique ID per user. This (unique) key should be repeated in the secondary tables. It will then be called a foreign key. Obviously, you want a key that's unique. If your username can be guaranteed to be unique (i.e. you require user be identified by their email address), then you can use that. If user names are real names (e.g. Firstname Sirname), then you don't have that guarantee and you need to keep a userid which becomes your key. Similarly, the table containing your posts could (but doesn't have to) have a field with unique userids indicating who wrote it etc.
You might want to read a bit about database design and the concept of normalization: (http://dev.mysql.com/tech-resources/articles/intro-to-normalization.html) No need to get bogged down with the n-th form of normalization but it will help you at this stage where you need to figure out the database design.
Good luck and report back ;-)