I am building a database in MySQL that will be accessed by PHP scripts. I have a table that is the activity stream. This includes everything that goes on on the website (following of many different things, liking, upvoting etc.). From this activity stream I am going to run an algorithm for each user depending on their activity and display relevant activity. Should I create another table that stores the activity for each user once the algorithm has been run on the activity or should I run the algorithm on the activity table every time the user accesses the site?
UPDATE:(this is what is above except rephrased hopefully in an easier to understand way)
I have a database table called activity. This table creates a new row every time an action is performed by a user on the website.
Every time a user logs in I am going to run an algorithm on the new rows (since the users last login) in the table (activity) that apply to them. For example if the user is following a user who upvoted a post in the activity stream that post will be displayed when the user logs in. I want the ability for the user to be able to access previous content applying to them. Would it be easiest to create another table that saved the rows that have already been run over with the algorithm except attached to individual users names? (a row can apply to multiple different users)
I would start with a single table and appropriate indexes. Using a union statement, you can perform several queries (using different indexes) and then mash all the results together.
As an example, lets assume that you are friends with user 37, 42, and 56, and you are interested in basketball and knitting. And, lets assume you have an index on user_id and an index on subject. This query should be quite performant.
SELECT * FROM activity WHERE user_id IN (37, 42, 56)
UNION DISTINCT
SELECT * FROM activity WHERE subject IN ("basketball", "knitting")
ORDER BY created
LIMIT 50
I would recommend tracking your user specific activities in a separate table and then upon login you could show all user activities that relate to them more easily. ie. So if a user is say big into baseball and hockey you could retrieve that from their recent activity, then got to your everything activities table and grab relevant items from it.
Related
I have some PHP codes and a database.
User can register using roll number, password and semester. I do plan to provide department as well later.
Teacher can add subjects based on semester. They can add class tests based on subjects.
In my simplest case, I have been able to login and see all subjects and all tests on pages 'view_subjects.php' and 'view_tests.php' respectively using SELECT * ..
I want to include a simple modification. I want to be able to show subjects and tests based on semester user is in, i.e filter.
I have created views I, II, III, IV where view names are semesters and are based on CREATE VIEW II AS SELECT * subjects WHERE semester='II';
And Views I, III, IV respectively. Edit- Realised that it is not necessary.
Based on views, I want to execute SELECT * FROM [viewname] on step 3. Eg: SELECT * FROM [I] , SELECT * FROM [II], ... SELECT * FROM [IV]
But the suitable statement out of the four has to be executed based on the user logging in.
Is my approach correct?
If so, how can I get the semester name after if he/she logs in
Otherwise, how should I approach this problem?
Edit My problem is to add a filter. I want users to see their subjects and tests based on the semesters they are in. Semester is an attribute of the student table along with roll_no and password.
I hope this makes sense, Im not going to write code as i dont know what you are working with there, but for situation like this, Following will be the basic idea.
Method one(i do not prefer this)
Add a current_semester field in student table, where you store semester id. This is to be updated every time a student completes a semester.
before loading your information page, save the value of current_semester in a variable, and use something like, "SELECT * FROM [semester table] WHERE id = [current_semester]"
This will give you the current semester of his/her semester where ever you want but it will not save his semester history.
Advised Method(using different table to save student's semester history)
create new table "student_semester" with id, semester_id(foreign key from associated semester), student_id(foreign key from associated student), start_date and any other info that you may want.
simply get the latest record associated with that user which will be the current semester
this method will allow you to store semester history each student
Advise: Use Laravel, Eloquent makes this sort of work extremely easier.
I have to develop a shopping site where I need to maintain data like "Add to Cart" and "Previous Orders" for a single user, for N number of users in a database. E.g. if there is a user named "Mark" and I need to store his 5 items previously added to cart and 4 items as previous order and there are say 100 users like him whose data I need to store, How should I solve this problem? I am using PHP and MySQL.
Each user is assigned a unique ID (you can exploit MySQL's AUTO_INCREMENT for this purpose when creating the table of users.
Then, for items the users select, you create another table including the User_ID (from the previous table), the Item_ID (from a similar table for items), the status (that would indicate the status of the item for the specific user) and an optional field that would contain the order number when the user places the order (for one or more items).
In summary, and since you are actually asking for help in designing your database, I would suggest you sit with a pencil and a piece of paper and make lists of informational elements that you need, marking the one-to-one relation between them (when applicable). This will (approximately) suggest what needs to be arrange in a record of a table (e.g. all the stable details of a user would normally go together into a single table).
For now, the system should have 6 different user levels.
Each level will be gained upon user activity, for example:
Level 1 - When user register
Level 2 - When user completes a mission
Level 3 - When user completes more than one mission
Level 4 - When user donate > $X amount of money
Level 5 - When user write more than 50 comments in blog
Level 6 - When user complete quiz
And now... I'm wondering, what's the best database schema to achieve this? I should keep track of all actions related to user's activities, that's why I though about xp_events table... Something like this:
id # primary key
event # type of event, e.g. 'register', 'complete_quiz', etc.
user_id # id of user
delta # number of "exp" which will be gained after specific action
And... in users table I will keep record of current level and "exp" which each user has earned until now.
When user makes any activity, I will call a trigger which will check if user have new level unlocked.
But... I'm aware that in long term (e.g. if more levels are added), this isn't optimal solution.
Looking forward for any suggestions.
I see two obvious possibilities here.
One is to have an event table like you say, with a user ID, event ID, dollar amount (for donations), probably a date/time, maybe other data. If the only reason why you are keeping any of this data is to determine each user's level, this is simple and effective.
If you're keeping track of this data for other purposes also, you probably want to separate it into multiple tables. In such a case you would likely have other data you need to keep for each event. Like for a donation you would need dollar amounts, which I assume don't apply to comments and missions. For comments you likely need the text of the comment and some indication of the thread this comment is on or what it's subject is. For a mission -- I don't know what a "mission" is in this context, but you likely want some information about the type of mission and where it was or who they were supposed to kill or whatever. Most of this data would not be applicable to events of different types. A comment probably doesn't have a dollar amount, a mission doesn't have a thread, etc. So you'd end up with a lot of irrelevant data and bunches of null fields.
Im building a classifieds website of adverts where I want to store a count of the number of views of each advert which I want to be able to display in a graph at a later date by day and month etc.. for each user and each of their adverts. Im just struggling with deciding how best to implement the mysql database to store potentially a large amount of data for each advert.
I am going to create a table for the page views as follows which would store a record for each view for each advert, for example if advert (id 1) has 200 views the table will store 200 records:
Advert_id (unique id of advert)
date_time (date and time of view)
ip_address ( unique ip address of person viewing advert)
page_referrer (url of referrer page)
As mentioned I am going to create the functionality for each member of the site to view a graph for the view statistics for each of their adverts so they can see how many total views each of their adverts have had, and also how many views their advert has had each day (between 2 given dates) and also how many views per month each advert has had. I'll do this by grouping by the date_time field.
If my site grows quite large and for example has 40,000 adverts and each advert has on average 3,000 page views, that would mean the table has 120 Million records. Is this too large ? and would the mysql queries to produce the graphs be very slow?
Do you think the table and method above is the best way to store these advert view statistics or is there a better way to do this?
Unless you really need to store all that data it would probably be better to just increment the count when the advert is viewed. So you just have one row for each advert (or even a column in the row for the advert).
Another option is to save this into a text file and then process it offline but it's generally better to process data as you get it and incorporate that into your applications process.
If you really need to save all of that data then rotating the log table weekly maybe (after processing it) would reduce the overhead of storing all of that information indefinitely.
I was working with website with 50.000 unique visitors per day, and i had same table as you.
Table was growthing ~200-500 MB/day, but i was able to clean table every day.
Best option is make second table, count visitors every day, add result to 2nd table, and flush 1st table.
first table example:
advert_id
date & time
ip address
page referrer
second table example (for graph):
advert_id
date
visitors
unique visitors
Example SQL query to count unqiue visitors:
SELECT
advert_id,
Count(DISTINCT ip_address),
SUBSTRING(Date,1,10) as Date
FROM
adverts
GROUP BY
advert_id,
Date
Problem is not even perfomance (MySQL ISAM Engine is quite smart and fast), problem is storage such big data.
90% statistics tools (even google analytics or webalyzer) is making graphs only once per day, not in real-time.
And quite good idea is store IP as INT using function ip2long()
I'm building a website that constructs both site-wide and user-specific activity feeds. I hope that you can see the structure below and share you insight as to whether my solution is doing the job. This is complicated by the fact that I have multiple types of users that right now are not stored in one master table. This is because the types of users are quite different and constructing multiple different tables for user meta-data would I think be too much trouble. In addition, there are multiple types of content that can be acted upon, and multiple types of activity (following, submitting, commenting, etc.).
Constructing a site-wide activity feed is simple because everything is logged to the main feed table and I just build out a list. I have a master feed table in MySQL that simple logs:
type of activity;
type of target entity;
id of target entity;
type of source entity (i.e., user or organization);
id of source entity.
(This is just a big reference table that points the script generating the feed to the appropriate table(s) for each feed entry).
In generating the user-specific feed, I'm trying to figure out some way to join the relationship table with the feed table, and using that to parse results. I have a relationships table, comprised of 'following' relationships, that is similar to the feed table. It is simpler though b/c only one type of user is allowed to follow other content types/users.
user/source id;
type of target entity;
id of target entity.
Columns 2 & 3 in the feed and follow table are the same, and I have been trying to use various JOIN methodologies to match them up, and then limit them by any relationships in the follow table that the user has. This is has not been very successful.
The basic query I am using is:
SELECT *
FROM (`feed` as fe) LEFT OUTER JOIN `follow` as fo
ON `fe`.`feed_target_type` = `fo`.`follow_e_type`
AND fo.follow_e_id = fe.feed_target_id
WHERE `fo`.`follow_u_id` = 1 OR fe.feed_e_id = 1
AND fe.feed_e_type = 'user'
ORDER BY `fe`.`feed_timestamp` desc LIMIT 10
This query also attempts to grab any content that the user has created (which data is logged in the feed table) that the user is, in effect, following by default.
This query seems to work, but it took me sometime to get to it and am pretty sure I'm missing a more elegant solution. Any ideas?
The first site I made with an activity feed had a notifications table where activities were logged, and then friends actions were pulled from that. However a few months down the line this hit millions of records.
The solution I am programming now pulls latest "friends" activities from separate tables and then orders by date. The query is at home, can post the example later if interested?