I have an application (More likely a quiz app) where i have saved all my 1000 quizzes in MySQL database, I want to retrieve a random question from this table when a user request one, I can easily do it using the RAND() function in MySQL.. my problem is , I don't want to give the same question two or more times to a user, how can i keep a record of retrieved questions? Do I have to create tables for each and every users? won't that increase the load time?? please help me, any help would be a big favor ..
-regards
If you want it for a short time, use the user's $_SESSION for that.
If you need the long term ( say tomorrow, not to ask the same questions) - you'll have to create additional table for usersToQuestions, where you'll store the user id and the questions the user had been already asked.
Retrieving a question in both cases would require a simple IN condition:
SELECT * FROM questions
WHERE id not IN ('implode(",", $_SESSION["asked"])')
SELECT * FROM questions
WHERE id not IN (
SELECT question_id FROM questions2users WHERE userid = 123
)
my problem is , I don't want to give the same question two or more times to a user,
how can i keep a record of retrieved questions? Do I have to create tables for each
and every users? won't that increase the load time?
Yes, but possibly not so much.
You keep a single extra table with userId, questionId and insert there the questions already asked to the various users.
When you ask question 123 to user 456, you run a single INSERT
INSERT INTO askedQuestions (userId, questionId) VALUES (456, 123);
Then you extract questions from questions with a LEFT JOIN
SELECT questions.* FROM questions
LEFT JOIN askedQuestions ON (questions.id = askedQuestions.questionId AND askedQuestions.userId = {$_SESSION['userId']} )
WHERE askedQuestions.userId IS NULL
ORDER BY RAND() LIMIT 1;
if you keep askedQuestions indexed on (userId, questionId), joining will be very efficient.
Notes on RAND()
Selecting on a table like this should not done with ORDER BY RAND(), which will retrieve all the rows in the table before outputting one of them. Normally you would choose a questionId at random, and select the question with that questionId, and that would be waaaay faster. But here, you have no guarantee that the question has not been already asked to that user, and the faster query might fail.
When most questions are still free to ask, you can use
WHERE questions.questionId IN ( RAND(N), RAND(N), RAND(N), ... )
AND askedQuestions.userId IS NULL LIMIT 1
where N is the number of questions. Chances are that at least one of the random numbers you extract will still be free. The IN will decrease performances, and you will have to strike a balance with the number of RANDs. When questions are almost all asked, chances of a match decrease, and your query might return nothing even with many RANDs (also because RANDs will start yielding duplicate IDs, in what is known as the Birthday Paradox).
One way to achieve the best of both worlds could be to fix a maximum number of attempts, say, three (or better still, based on the number of questions left over).
For X times you generate (in PHP) a set of Y random ids betweeen 1 and 1000, and try to retrieve (userId, questionId) from askedQuestions. The table is thin and indexed, so this is really fast. If you fail, then the extracted questionId is random and free, and you can run
SELECT * FROM questions WHERE id = {$tuple['questionId']};
which is also very fast. If you succeed X times, i.e., for X times, all Y random questionIds are registered as being already asked, then you run the full query. Most users will be served almost instantly (two very quick queries), and only a few really dedicated users will require more processing. You might want to set some kind of alerting to warn you of users running out of questions.
One solution is to add an ID column in the question table and when you serve it to a user you check that ID with the list of questions that you served the user.
You can use in memory data structure like List to keep track of the questions that are served to a particular user. This way, you only need array of Lists instead of tables to get the job done.
Related
I am creating a MCQ quiz based on php and mysql. Here are the structures of my main tables:
quiz table: quiz id, quiz_category
category table: id, title...
questions table: id, quiz id, categoryid, title...
answers table: id, question id...
To start things, I have the tables populated with 150+ quizzes, 4 categories, 14000+ questions and rightanswers for each.
To save time, for each question, the right answer is pulled from the answers table https://stackoverflow.com/editing-helpalongwith 3 other random answers .
Now when I was testing it with just two quizzes, it worked fine. But with 150 quizzes, several problems have cropped up:
the database is slow and for later quizzes takes forever to load questions
the randomization of answers is not working anymore - along with the right answer, the other options show the same entry, making it easy for the user to guess the right answer.
You can see the code I am working with in my previous Stackoverflow query. https://stackoverflow.com/questions/14826573/randomising-questions-and-answers-php-quiz-not-working
Any idea about what the ideal queries should be for the quiz program to work?
I will provide some tips on how to improve performance, however these will be generic and may not be complete.
From briefly looking at your PHP and SQL statements from your previous question, there are a few logical places for an index. To add an index please reefer to the MySQL manual for more information
$sql4="select * from answers where question_id=".$row2['id'];
question_id should have an index
$sql2="select * from questions where quiz_id=".$_SESSION['quizid'];
quiz_id should have an index
Adding these two indexes will also improve selectivity on this
$sql3="select * from answers where question_id in (select id from
questions where quiz_id =$row2[quiz_id]) order by rand()";
This will help as previously you would have been performing a full table scan for each query.
Your other issue is that you have a loop and on each iteration you are sending commands to query the database, you should collect all the information at once before the loop and then iterate using that rather than sending individual queries each iteration.
I have recently written a survey application that has done it's job and all the data is gathered. Now i have to analyze the data and i'm having some time issues.
I have to find out how many people selected what option and display it all.
I'm using this query, which does do it's job:
SELECT COUNT(*)
FROM survey
WHERE users = ? AND table = ? AND col = ? AND row = ? AND selected = ?
GROUP BY users,table,col,row,selected
As evident by the "?" i'm using MySQLi (in php) to fetch the data when needed, but i fear this is causing it to be so slow.
The table consists of all the elements above (+ an unique ID) and all of them are integers.
To explain some of the fields:
Each survey was divided into 3 or 4 tables (sized from 2x3 to 5x5) with a 1 to 10 happiness grade to select form. (questions are on the right and top of the table, then you answer where the questions intersect)
users - age groups
table, row, col - explained above
selected - dooooh explained above
Now with the surveys complete and around 1 million entries in the table the query is getting very slow. Sometimes it takes like 3 minutes, sometimes (i guess) the time limit expires and you get no data at all. I also don't have access to the full database, just my empty "testing" one since the costumer is kinda paranoid :S (and his server seems to be a bit slow)
Now (after the initial essay) my questions are: I left indexing out intentionally because with a lot of data being written during the survey, it would be a bad idea. But since no new data is coming in at this point, would it make sense to index all the fields of a table? How much sense does it make to index integers that never go above 10? (as you can guess i haven't got a clue about indexes). Do i need the primary unique ID in this table? I
I read somewhere that indexing may help groups but only if you group by the first columns in a table (and since my ID is first and from my point of view useless can i remove it and gain anything by it?)
Is there another way to write my query that would basically do the same thing but in a shorter period of time?
Thanks for all your suggestions in advance!
Add an index on entries that you "GROUP BY" or do "WHERE". So that's ONE index incorporating users,table,col,row and selected in your case.
Some quick rules:
combine fields to have the WHERE first, and the GROUP BY elements last.
If you have other queries that only use part of it (e.g. users,table,col and selected) then leave the missing value (row, in this example) last.
Don't use too many indexes/indeces, as each will slow the table to updates marginally - so on really large system you need to balance queries with indexes.
Edit: do you need the GROUP BY user,col,row as these are used in the WHERE. If the WHERE has already filtered them out, you only need group by "selected".
I'm currently working on a medium-sized web project, and I've ran into a problem.
What I want to do is display a question, together with an image. I have a (global) list of questions, and a (global) list of images, all questions should be asked for all images.
As far as the user can see the question and image should be chosen at random. However the statistics from the answers (question/image-pair) will be used for research purposes. This means that all the question/image-pair must be chosen such that the answers will be distributed evenly across all question, and across all images.
A user should only be able to answer a specific question/image-pair one time.
I am using a mysql database and php. Currently, i have three database tables:
tbl_images (image_id)
tbl_questions (question_id)
tbl_answers (answer_id, image_id, question_id, user_id)
The other columns are not related to this specific problem.
Solution 1:
Track how many times each image/question has been used (add a column in each table). Always choose the image and question that has been asked the least.
Problem:
What I'm actually interested in is distribution among questions for an image and vice versa, not that each question is even globally.
Solution 2:
Add another table, containing all question/image-pairs along with how many times it has been asked. Choose the lowest combination (first row if count column is sorted by ascending order).
Problem:
Does not enforce that the user can only answer a question once. Also does not give the appearance that the choice is random to the user.
Solution 3:
Same as #2, but store question/image/user_id in table.
Problem:
Performance issues (?), a lot of space wasted for each user. There will probably be semi-large amounts of data (thousands of questions/images and atleast hundreds of users).
Solution 4:
Choose a question and image at true random from all available. With a large enough amount of answers they will be distributed evenly.
Problem:
If i add a new question or image they will not get more answers than the others and therefore never catch up. I want an even amount of statistics for all question/image-pairs.
Solution 5:
Weighted random. Choose a number of question/image pairs (say about 10-100) at true random and pick the best (as in, lowest global count) of these that the user has not answered.
Problem:
Does not guarantee that a recently added question or image gets a lot of answers quickly.
Solution #5 is probably the best once I've come up with so far.
Your input is very much appreciated, thank you for your time.
From what I understand of your problem, I would go with #1. However, you do not need a new column. I would create an SQL View instead becuase it sounds like you'll need to report on things like that anyway. A view is basically a cached select, but acts similar to a table. Thus you would create a view for keeping the total of each question answered for each image:
DROP VIEW IF EXISTS "main"."view_image_question_count";
CREATE VIEW "view_image_question_count" AS
SELECT a.image_id, a.question_id, SUM(b.question_id) as "total"
FROM answer AS a
INNER JOIN answer AS b ON a.question_id = b.question_id
GROUP BY a.image_id, a.question_id;
Then, you need a quick and easy way to get the next best image/question combo to ask:
DROP VIEW IF EXISTS "main"."view_next_best_question";
CREATE VIEW "view_next_best_question" AS
SELECT a.*, user_id
FROM view_image_question_count a
JOIN answer USING( image_id, question_id )
JOIN question USING(question_id)
JOIN image USING(image_id)
ORDER BY total ASC;
Now, if you need to report on your image to question performace, you can do so by:
SELECT * FROM view_image_question_count
If you need the next best image+question to ask for a user, you would call:
SELECT * FROM view_next_best_question WHERE user_id != {USERID} LIMIT 1
The != {USERID} part is to prevent getting a question the user has already answered. The LIMIT optimizes to only get one.
Disclaimer: There is probably a lot that could be done to optimize this. I just wanted to post something for thought.
Also, here is the database dump I used for testing. http://pastebin.com/yutyV2GU
On a social network I am working on in PHP/MySQL, I have a friends page, it will show all friends a user has, like most networks do. I have a friend table in MySQL, it only has a few fields. auto_ID, from_user_ID, to_friend_ID, date
I would like to make the friends page have a few different options for sorting the results,
By auto_ID which is basically in the order a friend was added. It is just an auto increment id
new friends by date, will use the date field
By friends name, will have a list in alphabetical order.
The alphabetical is where I need some advice. I will have a list of the alphabet A-Z, when a user clicks on K it will show all the user's name starting with K and so on. The trick is it needs to be fast so doing a JOIN on the user's table is not an option, even though most will argue it is fast, it is not the performance I want for this action. One idea I had is to add an extra field to my friendship table and store the first letter of the users name in it. User's can change there name at anytime so I would have to make sure this is updated on possible thousands of records, anytime a user changes there name.
Is there a better way to do this?
Well if you don't want to do a join, then storing the user's name or initials on the friendships table is really your only other viable option. You mention the problem of having to update thousands of records every time a name changes, but is this really a problem? Unless you're talking about a major social networking site like Facebook, or maybe MySpace, does the average user really have enough friends to make this problematic? And then you have to multiply that by the probability that a user will change their name, which I would imagine isn't something that happens very often for each user.
If those updates are in fact non-trivial, you could always background or delay that to happen during non-peak times. Sure you would sacrifice up-to-the-second accuracy, but really, would most users even notice? Probably not.
Edit: Note, my answer above really only applies if you already have those levels of users. If you are still basically developing your site, just worry about getting it working, and worry about scaling problems when they become real problems.
You could also look at a caching solution like memcached. You can have a background process that is always updating a memcached hash and then when you want this data it is already in memory.
I'd just join on the table that contains the name and then sort on the name. Assuming a pretty normal table layout:
Table Person:
ID,
FirstName,
LastName
Table Friend:
auto_ID,
from_user_ID,
to_friend_ID,
date
You could do things like:
Select person.id, person.firstname, person.lastname, friend.auto_id
from Friend
left join on person where person.id = friend.to_friend_ID
where friend.from_user_ID = 1
order by person.lastname, person.firstname
or
Select person.id, person.firstname, person.lastname, friend.auto_id
from Friend
left join on person where person.id = friend.to_friend_ID
where friend.from_user_ID = 1
order by friend.date desc
I'd really recommend adding a column in the friend table to keep the first letter around, no need to duplicate data like that (and have to worry about keeping it in sync), that's what joins are for.
I'm working on a PHP app that has several objects that can be commented on. Each comment can be voted on, with users being able to give it +1 or -1 (like Digg or Reddit). Right now I'm planning on having a 'votes' table that has carries user_id and their vote info, which seems to work fine.
The thing is, each object has hundreds of comments that are stored in a separate comments table. After I load the comments, I'm having to tally the votes and then individually check each vote against the user to make sure they can only vote once. This works but just seems really database intensive - a lot of queries for just the comments.
Is there a simpler method of doing this that is less DB intensive? Is my current database structure the best way to go?
To be clearer about current database structure:
Comments table:
user_id
object_id
total_votes
Votes table:
comment_id
user_id
vote
End Goal:
Allow user to vote only once on each comment with least # of MySQL queries (each object has multiple comments)
To make sure that each voter votes only once, design your Votes table with these fields—CommentID, UserID, VoteValue. Make CommentID and UserID the primary key, which will make sure that one user gets only one vote. Then, to query the votes for a comment, do something like this:
SELECT SUM(VoteValue)
FROM Votes
WHERE CommentID = ?
Does that help?
Why don't you save the totaled votes for every comment? Increment/decrement this when a new vote has happened.
Then you have to check if the user has voted specifically for this comment to allow only one vote per comment per user.
You can put a sql join condition which returns all the votes on comments made by the current user for this object, if you get no rows, the user hasn't voted. That is just slightly different from you checking each comment one by one in the program.
as far as the database structure is concerned, keeping these things separate seems perfectly logical. vote { user_id, object_id, object_type, vote_info...)
You may be already doing this, sorry but I couldn't interpret from you post if that was the case.