Pattern Values in sql database/php - php

I'm working on this little enrollment system and i have this problem that if i want to enroll the student in my system the value of year/lvl should only increase, For example Grade1 once enrolled will be grade 2 and not decrease to preparatory nor double value to grade 3. So far my data entry is flat out inserting varchars. would appreciate any help on how can i do a pattern of progression and how should my database look like. Thanks.

I wouldn't suggest that you use varchars where you really mean numbers. It makes some things oh-so-much easier.
mysql> create table studentLevels(id int(3) primary key auto_increment, name varchar(100));
Query OK, 0 rows affected (0.02 sec)
mysql> desc studentLevels;
+-------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+----------------+
| id | int(3) | NO | PRI | NULL | auto_increment |
| name | varchar(100) | YES | | NULL | |
+-------+--------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
mysql> create table studentEnrolled (studentID int(3),subjectID int(3), grade int(3));
Query OK, 0 rows affected (0.00 sec)
mysql> desc studentEnrolled;
+-----------+--------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------+------+-----+---------+-------+
| studentID | int(3) | YES | | NULL | |
| subjectID | int(3) | YES | | NULL | |
| grade | int(3) | YES | | NULL | |
+-----------+--------+------+-----+---------+-------+
3 rows in set (0.00 sec)
mysql> create table subjects (id int(3), subGrade int(3));
Query OK, 0 rows affected (0.05 sec)
mysql> desc subjects;
+----------+--------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------+------+-----+---------+-------+
| id | int(3) | YES | | NULL | |
| subGrade | int(3) | YES | | NULL | |
+----------+--------+------+-----+---------+-------+
2 rows in set (0.00 sec)
Now, with this structure, you have a table to identify your students nicely, you can do a lot with the primary key here as needed (link it to other tables, add functionality that you aren't aware you want right now later - and the data is normalized). You have a neat table that defines what the student is enrolled in - linked back to the student. And lastly, you have another neat succinct table that holds subject information.
Looking at the grade column that is used to see where a student is at, you can use one of the following:
When a student completes a course, you can update the grade like this:
update
studentEnrolled a,
subjects b
set
a.grade=b.subGrade
where
a.subjectID=b.id
and a.studentID=:ID
The above will use a bound parameter from your application and the code will simply update the student grade to the subject that was just passed.
If you want students to be able to go back and do courses that have a lower grade, but don't want their "grade" to to drop, you can easily modify the above to be this:
update
studentEnrolled a,
subjects b
set
a.grade=b.subGrade
where
a.subjectID=b.id
and a.grade<subGrade
and a.studentID=:ID
The update will now only modify a record where the student gained a "grade".

Related

How to efficiently calculate averages from a big table?

I have a table called ratings with the following fields:
+-----------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------+------+-----+---------+----------------+
| rating_id | bigint(20) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| movie_id | int(11) | NO | | NULL | |
| rating | float | NO | | NULL | |
+-----------+------------+------+-----+---------+----------------+
Indexes on this table:
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| ratings | 0 | PRIMARY | 1 | rating_id | A | 100076 | NULL | NULL | | BTREE | | |
| ratings | 0 | user_id | 1 | user_id | A | 564 | NULL | NULL | | BTREE | | |
| ratings | 0 | user_id | 2 | movie_id | A | 100092 | NULL | NULL | | BTREE | | |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
I have another table called movie_average_ratings which has the following fields:
+----------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+---------+------+-----+---------+-------+
| movie_id | int(11) | NO | PRI | NULL | |
| average_rating | float | NO | | NULL | |
+----------------+---------+------+-----+---------+-------+
As it is obvious by this point I want to calculate the average rating of movies from ratings table and update the movie_average_ratingstable. I tried the following SQL query.
UPDATE movie_average_ratings
SET average_rating = (SELECT AVG(rating)
FROM ratings
WHERE ratings.movie_id = movie_average_ratings.movie_id);
Currently, there are around 10,000 movie records and 100,000 rating records and I get Lock wait timeout exceeded; try restarting transaction error. The number of records can grow significantly so I don't think increase timeout is a good solution.
So, how can I write 'scalable' query to acheive this? Is iterating the movie_average_ratings table records and calculate averages individually the most efficient solution to this?
Without an explain, it's hard to be clear on what's holding you up. It's also not clear that you will get a performance improvement by storing this aggregated data as a denormalized table - if the query to calculate the ratings executes in 0.04 seconds, it's unlikely querying your denormalized table will be much faster.
In general, I recommend only denormalizing if you know you have a performance problem.
But that's not the question.
I would do the following:
delete from movie_average_ratings;
insert into movie_average_ratings
Select movie_ID, avg(rating)
from ratings
group by movie_id;
I just found something in another post:
What is happening is, some other thread is holding a record lock on
some record (you're updating every record in the table!) for too long,
and your thread is being timed out.
This means that some of your records are locked you can force unlock them in the console:
1) Enter MySQL mysql -u your_user -p
2) Let's see the list of locked tables mysql> show open tables where in_use>0;
3) Let's see the list of the current processes, one of them is locking
your table(s) mysql> show processlist;
4) Kill one of these processes mysql> kill put_process_id_here;
You could redesign the movie_average_ratings table to
movie_id (int)
sum_of_ratings (int)
num_of_ratings (int)
Then, if a new rating is added you can add it to movie_average_ratings and calculate the average if needed

Proper way of handling and preventing deadlocks

I've understood that deadlocks occur when an sql query tries to lock an already locked row and I'm currently experiencing deadlocks. here's my sql query below:
INSERT INTO transactions (product_id, category, C, amount, date)
SELECT 'SomeProduct', 'SomeCategory', v.username, 10, '2016-3-31' FROM balance v
WHERE v.username = 'SomeUsername' AND v.balance + 10 >= 0
balance is a virtual table that sums transactions to get user's balance.
This error usually is noticed when having a reasonable amount of users which makes it hard to test, any tips on how to avoid deadlocks or any possible solution because I'm inserting rows into the transaction table in a very numerous way and looking to solve it!
I've also tried tried to catch the exception, but I couldn't create a loop that would redo the query until it is finished.
General answer
Deadlocks can only occur when you have two or more resources, two or more processes, and the processes lock the resources in different order.
Say, process 1 wants to lock resource A, then B, then C. Process 2 wants to lock B, then A, then C.
This may lead to a dead lock if 1 gets A, then 2 gets B, then 1 waits for B and 2 waits for A - indefinitely.
The solution is, thankfully quite simple: anytime if a process needs to lock two or more resources, it must do so in a "sorted" fashion. In this example,
if process 2 also gets A, then B, then C, a deadlock can never happen.
Specific answer
I your case, you seem to be locking different table rows within one transaction in more or less random order. Try to find out how to release locks with mysql and make sure you are only holding as many as you actually need. If you need to hold more than one at a time, try to order your requests in some way.
Hard to tell without knowing more about your code... the first Google hit for "mysql deadlock" shows some promising stuff though: https://www.percona.com/blog/2014/10/28/how-to-deal-with-mysql-deadlocks
I have create a sample table with 2 field. id has primary key
MariaDB [who]> DESC mynum;
+-------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| num | float | YES | | NULL | |
+-------+------------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
I have filled with 100000 records
MariaDB [who]> SELECT * FROM mynum LIMIT 10;
+----+----------+
| id | num |
+----+----------+
| 1 | 0.47083 |
| 2 | 0.670773 |
| 3 | 0.941373 |
| 4 | 0.69455 |
| 5 | 0.648627 |
| 6 | 0.159488 |
| 7 | 0.851557 |
| 8 | 0.779321 |
| 9 | 0.341933 |
| 10 | 0.371704 |
+----+----------+
10 rows in set (0.00 sec)
MariaDB [who]> SELECT count(*) FROM mynum;
+----------+
| count(*) |
+----------+
| 100000 |
+----------+
1 row in set (0.02 sec)
Now i select row and calculate +10 to the id. You see that he must read ALL rows
MariaDB [who]> EXPLAIN SELECT * FROM mynum WHERE id +10 > 20;
+------+-------------+-------+------+---------------+------+---------+------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+--------+-------------+
| 1 | SIMPLE | mynum | ALL | NULL | NULL | NULL | NULL | 100464 | Using where |
+------+-------------+-------+------+---------------+------+---------+------+--------+-------------+
1 row in set (0.00 sec)
and now i compare the id with a constant. You can see the only reads the row they use and use a index
MariaDB [who]> EXPLAIN SELECT * FROM mynum WHERE id < 10;
+------+-------------+-------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | mynum | range | PRIMARY | PRIMARY | 4 | NULL | 9 | Using where |
+------+-------------+-------+-------+---------------+---------+---------+------+------+-------------+
1 row in set (0.00 sec)

MySQL is not correctly selecting rows (sometimes)

This is an update to this question, wherein I was casting around trying to work out what on earth was going on:
MySQL sometimes erroneously returns 0 for count(*)
I ended up accepting an answer there because it did answer the question I posed ("why might this happen") even though it didn't answer the question I really wanted to know about ("why is this happening to me"). But I've managed to narrow things down a little bit on the latter question, and think I can definitively say that something is wrong in a way that I don't understand and have never seen before.
The issue has been really difficult to debug because, for reasons beyond my comprehension, logging in to the database automagically fixes it. However, today I managed to trigger the problematic state while having an open MySQL session in a terminal. Here are some queries and the subsequent responses taken from that session:
First, this is my table layout:
mysql> describe forum_posts;
+-----------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------+------+-----+---------+----------------+
| post_id | int(11) | NO | PRI | NULL | auto_increment |
| thread_id | int(11) | YES | MUL | NULL | |
| forum_id | int(11) | YES | MUL | NULL | |
| user_id | int(11) | YES | MUL | NULL | |
| moderator | tinyint(1) | NO | | 0 | |
| message | mediumtext | YES | MUL | NULL | |
| date | int(11) | NO | MUL | NULL | |
| edited | int(11) | YES | | NULL | |
| deleted | tinyint(1) | YES | MUL | 0 | |
| bbcode | tinyint(1) | NO | | 1 | |
+-----------+------------+------+-----+---------+----------------+
10 rows in set (0.00 sec)
Now, lets look at how many posts there are in a given forum thread:
mysql> SELECT count(post_id) as num FROM `forum_posts` where thread_id=5243;
+-----+
| num |
+-----+
| 195 |
+-----+
1 row in set (0.00 sec)
OK, but I only want forum posts that don't have the deleted flag set:
mysql> SELECT count(post_id) as num FROM `forum_posts` where thread_id=5243 and deleted=0;
+-----+
| num |
+-----+
| 0 |
+-----+
1 row in set (0.06 sec)
mysql> select post_id,deleted from forum_posts where thread_id=5243 and deleted=0;
Empty set (0.06 sec)
OK, lets just double-make-sure that they aren't actually all deleted:
mysql> select post_id,deleted from forum_posts where thread_id=5243;
+---------+---------+
| post_id | deleted |
+---------+---------+
| 104081 | 0 |
| 104082 | 0 |
[snip]
| 121162 | 0 |
| 121594 | 0 |
+---------+---------+
195 rows in set (0.00 sec)
Every row in that table has 'deleted' set to 0, and yet adding and deleted=0 to the query yields no results. Until I open a new session by logging in to MySQL again from a terminal window, after which I can once again properly select rows where 'deleted' is 0.
What on earth?
UPDATES:
#miken32 in the comments below suggested I try an EXPLAIN SELECT ..., so:
mysql> explain select post_id,deleted from forum_posts where thread_id='5243' and deleted=0;
+----+-------------+-------------+-------------+-------------------+-------------------+---------+------+------+--------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+-------------+-------------------+-------------------+---------+------+------+--------------------------------------------------------------+
| 1 | SIMPLE | forum_posts | index_merge | thread_id,deleted | thread_id,deleted | 5,2 | NULL | 97 | Using intersect(thread_id,deleted); Using where; Using index |
+----+-------------+-------------+-------------+-------------------+-------------------+---------+------+------+--------------------------------------------------------------+
1 row in set (0.00 sec)
Based on the comment that using FORCE KEY alters the result from the query, it is very likely that we are dealing with the merge optimizer bug. EXPLAIN of the original query shows the optimization is done by selecting from the deleted key, then from the post_id key, then merging the results. When we force to bypass that code, the problem goes away.
The steps from the point:
try it on the same data with the most recent 5.6 version of MySQL
if the issue reproduces, try to isolate it to the most minimal test case, visit http://bugs.mysql.com/ and report the bug
Exorcise the daemons and ghosts! Add this index to avoid any "merge" bug:
INDEX(deleted, thread_id) and DROP the key on just deleted
An index on a flag is almost always useless. This time it was worse than useless.
This wil be cheaper, faster, and safer than FORCE INDEX.

Why I can't get the warning messages as phpmyadmin does?

I can even use abc as the value of a DATETIME column .
Why is it not doing validating at all?
UPDATE
About how phpmyadmin gets the warning:
By reading its code,I find it's from show warnings,but I can't reproduce it in command line MySQL.
mysql> UPDATE `test`.`uu3` SET `id` = 'x',dt='ab1c' WHERE `uu3`.`id` =3 AND `uu3`.`id2` =4;
Query OK, 0 rows affected (0.00 sec)
Rows matched: 0 Changed: 0 Warnings: 0
mysql> SHOW WARNINGS;
Empty set (0.00 sec)
mysql> desc `test`.`uu3`;
+-------+------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+-------+
| id | int(10) unsigned | NO | | NULL | |
| id2 | int(10) unsigned | NO | | NULL | |
| dt | datetime | NO | | NULL | |
+-------+------------------+------+-----+---------+-------+
3 rows in set (0.02 sec)
No, it does the proper validation.
When I tried to add a value "abc" to the field "date_time" (whose data type is "DATETIME"), then I got this error "Warning: #1264 Out of range value adjusted for column 'date_time' at row 1".
However, the row got inserted successfully, but the value of that field got changed automatically to "0000-00-00 0000:00:00".
I'm using the version 3.3.10.2 of phpMyAdmin.
Hope it helps.

Recursive-ish query for tags?

I have a table of tags that can be linked to other tags and I want to "recursively" select the tags in order of arrangement. So that when a search is made, we get the immediate (1-level) results and then carry on down to say 5-levels so that we always have a list of tags no matter if there wasn't enough exact matches on level 1.
I can manage this fine with making multiple queries until I get enough results, but surely there is a better, optimized, way via a one-trip query?
Any tips will be appreciated.
Thanks!
Results:
tagId, tagWord, child, child tagId
'513', 'Slap', 'Hog Slapper', '1518'
'513', 'Slap', 'Corporal Punishment', '147'
'513', 'Slap', 'Impact Play', '1394'
Query:
SELECT t.tagId, t.tagWord as tag, tt.tagWord as child, tt.tagId as childId
FROM platform.tagWords t
INNER JOIN platform.tagsLinks l ON l.parentId = t.tagId
INNER JOIN platform.tagWords tt ON tt.tagId = l.tagId
WHERE t.tagWord = 'slap'
Table Layouts:
mysql> explain tagWords;
+---------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+---------------------+------+-----+---------+----------------+
| tagId | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| tagWord | varchar(45) | YES | UNI | NULL | |
+---------+---------------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
mysql> explain tagsLinks;
+----------+---------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+---------------------+------+-----+---------+-------+
| tagId | bigint(20) unsigned | NO | | NULL | |
| parentId | bigint(20) | YES | | NULL | |
+----------+---------------------+------+-----+---------+-------+
2 rows in set (0.00 sec)
AFAIK Mysql doesn't have any mechanism for querying data recursively
Oracle has Connected By construct and Sql Server has CTE(Common Table Expressions).
But Mysql,
Read Here and Here
Here are the options that I consider each time I find myself in a situation when I need to query hierarchical data.
Nested Sets
Path enumeration
Explicit joins (when the maximum level is known)
Vendor Extensions (SQL Server CTE, Oracle Connect by etc)
Stored Procedures
Suck it up

Categories