mysql> describe break;
+-------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+----------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| start | datetime | YES | | NULL | |
| end | datetime | YES | | NULL | |
| duration | datetime | YES | | NULL | |
| date | datetime | YES | | NULL | |
| employee_id | int(11) | NO | MUL | NULL | |
+-------------+----------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
I would like to calculate (and update) the 'duration' column using the start and end values from the above table. For example:
UPDATE break SET duration=TIMEDIFF(start, end) WHERE employee_id=1;
Alas, this sets the duration column to 0000-00-00 00:00:00
mysql> select * from break\G;
*************************** 1. row ***************************
id: 30
start: 2013-08-06 15:43:17
end: 2013-08-06 15:55:42
duration: 0000-00-00 00:00:00
date: 2013-08-06 15:43:17
employee_id: 1
1 row in set (0.00 sec)
help
From the documentation (emphasis mine):
TIMEDIFF() returns expr1 – expr2 expressed as a time value.
Change duration to type TIME and it should work.
Note that you are creating redundancy in your schema. This is usually a Bad Thing. What if end or start change and you do not update duration? Better use a view and return the TIMEDIFF calculated from end and start from there.
Change durration to type TIME and then update your code
UPDATE break SET duration=TIMEDIFF(end, start) WHERE employee_id=1;
or you will be getting a negative durration
UPDATE: in response to another answer you can make a trigger that sets the duration every time the row is updated
DROP TRIGGER IF EXISTS `break_trigger`;
DELIMITER //
CREATE TRIGGER `break_trigger` AFTER UPDATE ON `break`
FOR EACH ROW BEGIN
SET duration=TIMEDIFF(end, start) WHERE employee_id= NEW.employee_id;
END
//
DELIMITER ;
Related
I have a table called ratings with the following fields:
+-----------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------+------+-----+---------+----------------+
| rating_id | bigint(20) | NO | PRI | NULL | auto_increment |
| user_id | int(11) | NO | MUL | NULL | |
| movie_id | int(11) | NO | | NULL | |
| rating | float | NO | | NULL | |
+-----------+------------+------+-----+---------+----------------+
Indexes on this table:
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| ratings | 0 | PRIMARY | 1 | rating_id | A | 100076 | NULL | NULL | | BTREE | | |
| ratings | 0 | user_id | 1 | user_id | A | 564 | NULL | NULL | | BTREE | | |
| ratings | 0 | user_id | 2 | movie_id | A | 100092 | NULL | NULL | | BTREE | | |
+---------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
I have another table called movie_average_ratings which has the following fields:
+----------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------+---------+------+-----+---------+-------+
| movie_id | int(11) | NO | PRI | NULL | |
| average_rating | float | NO | | NULL | |
+----------------+---------+------+-----+---------+-------+
As it is obvious by this point I want to calculate the average rating of movies from ratings table and update the movie_average_ratingstable. I tried the following SQL query.
UPDATE movie_average_ratings
SET average_rating = (SELECT AVG(rating)
FROM ratings
WHERE ratings.movie_id = movie_average_ratings.movie_id);
Currently, there are around 10,000 movie records and 100,000 rating records and I get Lock wait timeout exceeded; try restarting transaction error. The number of records can grow significantly so I don't think increase timeout is a good solution.
So, how can I write 'scalable' query to acheive this? Is iterating the movie_average_ratings table records and calculate averages individually the most efficient solution to this?
Without an explain, it's hard to be clear on what's holding you up. It's also not clear that you will get a performance improvement by storing this aggregated data as a denormalized table - if the query to calculate the ratings executes in 0.04 seconds, it's unlikely querying your denormalized table will be much faster.
In general, I recommend only denormalizing if you know you have a performance problem.
But that's not the question.
I would do the following:
delete from movie_average_ratings;
insert into movie_average_ratings
Select movie_ID, avg(rating)
from ratings
group by movie_id;
I just found something in another post:
What is happening is, some other thread is holding a record lock on
some record (you're updating every record in the table!) for too long,
and your thread is being timed out.
This means that some of your records are locked you can force unlock them in the console:
1) Enter MySQL mysql -u your_user -p
2) Let's see the list of locked tables mysql> show open tables where in_use>0;
3) Let's see the list of the current processes, one of them is locking
your table(s) mysql> show processlist;
4) Kill one of these processes mysql> kill put_process_id_here;
You could redesign the movie_average_ratings table to
movie_id (int)
sum_of_ratings (int)
num_of_ratings (int)
Then, if a new rating is added you can add it to movie_average_ratings and calculate the average if needed
I've understood that deadlocks occur when an sql query tries to lock an already locked row and I'm currently experiencing deadlocks. here's my sql query below:
INSERT INTO transactions (product_id, category, C, amount, date)
SELECT 'SomeProduct', 'SomeCategory', v.username, 10, '2016-3-31' FROM balance v
WHERE v.username = 'SomeUsername' AND v.balance + 10 >= 0
balance is a virtual table that sums transactions to get user's balance.
This error usually is noticed when having a reasonable amount of users which makes it hard to test, any tips on how to avoid deadlocks or any possible solution because I'm inserting rows into the transaction table in a very numerous way and looking to solve it!
I've also tried tried to catch the exception, but I couldn't create a loop that would redo the query until it is finished.
General answer
Deadlocks can only occur when you have two or more resources, two or more processes, and the processes lock the resources in different order.
Say, process 1 wants to lock resource A, then B, then C. Process 2 wants to lock B, then A, then C.
This may lead to a dead lock if 1 gets A, then 2 gets B, then 1 waits for B and 2 waits for A - indefinitely.
The solution is, thankfully quite simple: anytime if a process needs to lock two or more resources, it must do so in a "sorted" fashion. In this example,
if process 2 also gets A, then B, then C, a deadlock can never happen.
Specific answer
I your case, you seem to be locking different table rows within one transaction in more or less random order. Try to find out how to release locks with mysql and make sure you are only holding as many as you actually need. If you need to hold more than one at a time, try to order your requests in some way.
Hard to tell without knowing more about your code... the first Google hit for "mysql deadlock" shows some promising stuff though: https://www.percona.com/blog/2014/10/28/how-to-deal-with-mysql-deadlocks
I have create a sample table with 2 field. id has primary key
MariaDB [who]> DESC mynum;
+-------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| num | float | YES | | NULL | |
+-------+------------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
I have filled with 100000 records
MariaDB [who]> SELECT * FROM mynum LIMIT 10;
+----+----------+
| id | num |
+----+----------+
| 1 | 0.47083 |
| 2 | 0.670773 |
| 3 | 0.941373 |
| 4 | 0.69455 |
| 5 | 0.648627 |
| 6 | 0.159488 |
| 7 | 0.851557 |
| 8 | 0.779321 |
| 9 | 0.341933 |
| 10 | 0.371704 |
+----+----------+
10 rows in set (0.00 sec)
MariaDB [who]> SELECT count(*) FROM mynum;
+----------+
| count(*) |
+----------+
| 100000 |
+----------+
1 row in set (0.02 sec)
Now i select row and calculate +10 to the id. You see that he must read ALL rows
MariaDB [who]> EXPLAIN SELECT * FROM mynum WHERE id +10 > 20;
+------+-------------+-------+------+---------------+------+---------+------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+------+---------------+------+---------+------+--------+-------------+
| 1 | SIMPLE | mynum | ALL | NULL | NULL | NULL | NULL | 100464 | Using where |
+------+-------------+-------+------+---------------+------+---------+------+--------+-------------+
1 row in set (0.00 sec)
and now i compare the id with a constant. You can see the only reads the row they use and use a index
MariaDB [who]> EXPLAIN SELECT * FROM mynum WHERE id < 10;
+------+-------------+-------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+-------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | mynum | range | PRIMARY | PRIMARY | 4 | NULL | 9 | Using where |
+------+-------------+-------+-------+---------------+---------+---------+------+------+-------------+
1 row in set (0.00 sec)
This is an update to this question, wherein I was casting around trying to work out what on earth was going on:
MySQL sometimes erroneously returns 0 for count(*)
I ended up accepting an answer there because it did answer the question I posed ("why might this happen") even though it didn't answer the question I really wanted to know about ("why is this happening to me"). But I've managed to narrow things down a little bit on the latter question, and think I can definitively say that something is wrong in a way that I don't understand and have never seen before.
The issue has been really difficult to debug because, for reasons beyond my comprehension, logging in to the database automagically fixes it. However, today I managed to trigger the problematic state while having an open MySQL session in a terminal. Here are some queries and the subsequent responses taken from that session:
First, this is my table layout:
mysql> describe forum_posts;
+-----------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------+------+-----+---------+----------------+
| post_id | int(11) | NO | PRI | NULL | auto_increment |
| thread_id | int(11) | YES | MUL | NULL | |
| forum_id | int(11) | YES | MUL | NULL | |
| user_id | int(11) | YES | MUL | NULL | |
| moderator | tinyint(1) | NO | | 0 | |
| message | mediumtext | YES | MUL | NULL | |
| date | int(11) | NO | MUL | NULL | |
| edited | int(11) | YES | | NULL | |
| deleted | tinyint(1) | YES | MUL | 0 | |
| bbcode | tinyint(1) | NO | | 1 | |
+-----------+------------+------+-----+---------+----------------+
10 rows in set (0.00 sec)
Now, lets look at how many posts there are in a given forum thread:
mysql> SELECT count(post_id) as num FROM `forum_posts` where thread_id=5243;
+-----+
| num |
+-----+
| 195 |
+-----+
1 row in set (0.00 sec)
OK, but I only want forum posts that don't have the deleted flag set:
mysql> SELECT count(post_id) as num FROM `forum_posts` where thread_id=5243 and deleted=0;
+-----+
| num |
+-----+
| 0 |
+-----+
1 row in set (0.06 sec)
mysql> select post_id,deleted from forum_posts where thread_id=5243 and deleted=0;
Empty set (0.06 sec)
OK, lets just double-make-sure that they aren't actually all deleted:
mysql> select post_id,deleted from forum_posts where thread_id=5243;
+---------+---------+
| post_id | deleted |
+---------+---------+
| 104081 | 0 |
| 104082 | 0 |
[snip]
| 121162 | 0 |
| 121594 | 0 |
+---------+---------+
195 rows in set (0.00 sec)
Every row in that table has 'deleted' set to 0, and yet adding and deleted=0 to the query yields no results. Until I open a new session by logging in to MySQL again from a terminal window, after which I can once again properly select rows where 'deleted' is 0.
What on earth?
UPDATES:
#miken32 in the comments below suggested I try an EXPLAIN SELECT ..., so:
mysql> explain select post_id,deleted from forum_posts where thread_id='5243' and deleted=0;
+----+-------------+-------------+-------------+-------------------+-------------------+---------+------+------+--------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------+-------------+-------------------+-------------------+---------+------+------+--------------------------------------------------------------+
| 1 | SIMPLE | forum_posts | index_merge | thread_id,deleted | thread_id,deleted | 5,2 | NULL | 97 | Using intersect(thread_id,deleted); Using where; Using index |
+----+-------------+-------------+-------------+-------------------+-------------------+---------+------+------+--------------------------------------------------------------+
1 row in set (0.00 sec)
Based on the comment that using FORCE KEY alters the result from the query, it is very likely that we are dealing with the merge optimizer bug. EXPLAIN of the original query shows the optimization is done by selecting from the deleted key, then from the post_id key, then merging the results. When we force to bypass that code, the problem goes away.
The steps from the point:
try it on the same data with the most recent 5.6 version of MySQL
if the issue reproduces, try to isolate it to the most minimal test case, visit http://bugs.mysql.com/ and report the bug
Exorcise the daemons and ghosts! Add this index to avoid any "merge" bug:
INDEX(deleted, thread_id) and DROP the key on just deleted
An index on a flag is almost always useless. This time it was worse than useless.
This wil be cheaper, faster, and safer than FORCE INDEX.
I'm creating a portfolio website that has galleries that contain images. I want the user of this portfolio to be able to order the images within a gallery. The problem itself is fairly simple I'm just struggling with deciding on a solution to implement.
There are 2 solutions I've thought of so far:
Simply adding an order column (or priority?) and then querying with an ORDER BY clause on that column. The disadvantage of this being that to change the order of a single image I'd have to update every single image in the gallery.
The second method would be to add 2 nullable columns next and previous that simply store the ID of the next and previous image. This would then mean there would be less data to update when the order was changed; however, it would be much more complex to set up and I'm not entirely sure how I'd actually implement it.
Extra options would be great.
Are those options viable?
Are there better options?
How could / should they be implemented?
The current structure of the two tables in question is the following:
mysql> desc Gallery;
+--------------+------------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+------------------+------+-----+-------------------+-----------------------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| title | varchar(255) | NO | | NULL | |
| subtitle | varchar(255) | NO | | NULL | |
| description | varchar(5000) | NO | | NULL | |
| date | datetime | NO | | NULL | |
| isActive | tinyint(1) | NO | | NULL | |
| lastModified | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+--------------+------------------+------+-----+-------------------+-----------------------------+
mysql> desc Image;
+--------------+------------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+------------------+------+-----+-------------------+-----------------------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| galleryId | int(10) unsigned | NO | MUL | NULL | |
| description | varchar(250) | YES | | NULL | |
| path | varchar(250) | NO | | NULL | |
| lastModified | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+--------------+------------------+------+-----+-------------------+-----------------------------+
Currently there is no implementation of ordering in any form.
while 1 is a bit ugly you can do:
UPDATE table set order=order+1 where order>='orderValueOfItemYouCareAbout';
this will update all the rest of the images and you wont have to do a ton of leg work.
As bart2puck has said and I stated in the question, option 1 is a little bit ugly; it is however the option I have chosen to go with to simplify the solution all round.
I have added a column (displayOrder int UNSIGNED) to the Image table after path. When I want to re-order a row in the table I simply swap rows around. So, if I have 3 rows:
mysql> SELECT id, galleryId, description, displayOrder FROM Image ORDER BY displayOrder;
+-----+-----------+----------------------------------+--------------+
| id | galleryId | description | displayOrder |
+-----+-----------+----------------------------------+--------------+
| 271 | 20 | NULL | 1 |
| 270 | 20 | Tracks leading into the ocean... | 2 |
| 278 | 20 | NULL | 3 |
+-----+-----------+----------------------------------+--------------+
3 rows in set (0.00 sec)
If I want to re-order row 278 to appear second rather than third, I'll simply swap it with the second by doing the following:
UPDATE Image SET displayOrder =
CASE displayOrder
WHEN 2 THEN 3
WHEN 3 THEN 2
END
WHERE galleryId = 20
AND displayOrder BETWEEN 2 AND 3;
Resulting in:
mysql> SELECT id, galleryId, description, displayOrder FROM Image ORDER BY displayOrder;
+-----+-----------+----------------------------------+--------------+
| id | galleryId | description | displayOrder |
+-----+-----------+----------------------------------+--------------+
| 271 | 20 | NULL | 1 |
| 278 | 20 | NULL | 2 |
| 270 | 20 | Tracks leading into the ocean... | 3 |
+-----+-----------+----------------------------------+--------------+
3 rows in set (0.00 sec)
One possible issue that some people may find is that you can only alter the position by one place with this method, i.e. to move image 278 to appear first I'd have to make it second, then first, otherwise the current first image would appear third.
I'm working on this little enrollment system and i have this problem that if i want to enroll the student in my system the value of year/lvl should only increase, For example Grade1 once enrolled will be grade 2 and not decrease to preparatory nor double value to grade 3. So far my data entry is flat out inserting varchars. would appreciate any help on how can i do a pattern of progression and how should my database look like. Thanks.
I wouldn't suggest that you use varchars where you really mean numbers. It makes some things oh-so-much easier.
mysql> create table studentLevels(id int(3) primary key auto_increment, name varchar(100));
Query OK, 0 rows affected (0.02 sec)
mysql> desc studentLevels;
+-------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+--------------+------+-----+---------+----------------+
| id | int(3) | NO | PRI | NULL | auto_increment |
| name | varchar(100) | YES | | NULL | |
+-------+--------------+------+-----+---------+----------------+
2 rows in set (0.00 sec)
mysql> create table studentEnrolled (studentID int(3),subjectID int(3), grade int(3));
Query OK, 0 rows affected (0.00 sec)
mysql> desc studentEnrolled;
+-----------+--------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------+------+-----+---------+-------+
| studentID | int(3) | YES | | NULL | |
| subjectID | int(3) | YES | | NULL | |
| grade | int(3) | YES | | NULL | |
+-----------+--------+------+-----+---------+-------+
3 rows in set (0.00 sec)
mysql> create table subjects (id int(3), subGrade int(3));
Query OK, 0 rows affected (0.05 sec)
mysql> desc subjects;
+----------+--------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------+------+-----+---------+-------+
| id | int(3) | YES | | NULL | |
| subGrade | int(3) | YES | | NULL | |
+----------+--------+------+-----+---------+-------+
2 rows in set (0.00 sec)
Now, with this structure, you have a table to identify your students nicely, you can do a lot with the primary key here as needed (link it to other tables, add functionality that you aren't aware you want right now later - and the data is normalized). You have a neat table that defines what the student is enrolled in - linked back to the student. And lastly, you have another neat succinct table that holds subject information.
Looking at the grade column that is used to see where a student is at, you can use one of the following:
When a student completes a course, you can update the grade like this:
update
studentEnrolled a,
subjects b
set
a.grade=b.subGrade
where
a.subjectID=b.id
and a.studentID=:ID
The above will use a bound parameter from your application and the code will simply update the student grade to the subject that was just passed.
If you want students to be able to go back and do courses that have a lower grade, but don't want their "grade" to to drop, you can easily modify the above to be this:
update
studentEnrolled a,
subjects b
set
a.grade=b.subGrade
where
a.subjectID=b.id
and a.grade<subGrade
and a.studentID=:ID
The update will now only modify a record where the student gained a "grade".