I have thousands of entries in my database, each with 9 digits or less. I would like to do a mass update, find all rows with digits less than 9 and add 0's to make them equal 9 digits.
For example, my table looks like:
ID | Number
---------------
0 | 489379
1 | 854744329
2 | 56456669
I would like to make it look like:
ID | Number
---------------
0 | 000489379
1 | 854744329
2 | 056456669
How would I do this with a MySQL query?
The lpad function should solve your issue:
SELECT `id`, LPAD(`Number`, 9, '0')
FROM mytable
To answer the question in the comment, this can also be applied in an update statement:
UPDATE mytable
SET `Number` = LPAD(`Number`, 9, '0')
WHERE LENGTH(`Number`) < 9
Use a case statement
update table
set column1 = case when length(column1) = 4 then concatenate('00000', column1)....
Have an element in the case statement for every possible length of the column. Kind of manual, and there is likely an easier way, but this is one possibility.
Related
I have a MariaDB table with an auto-incremented index, but also a "sortorder" field that controls the, well, sort order, when data is queried and displayed.
E.g.
id title sortorder
1 this 10
2 that 30
3 other 20
4 something 25
So far, so good. I'd like to create a function to automatically re-order these though - well, not re-order, but redo the values of the sortorder column per the existing order. The desired outcome from the above after running the function would be this:
id title sortorder
1 this 10
2 that 40
3 other 20
4 something 30
Is this something that can be done with an SQL statement in MariaDB (I have not found anything for that yet), or do I need to do it in my (php) application?
The logic for the new sort order values is based on the ordering by the sortorder column.
The reason for renumbering is that the sort order values are going to be manually maintained in the application, but it may be occasionally helpful to start with a clean slate. Users will be trained to "leave some room" in the values to allow for future edits.
On day one, "sortorder" will get (manually) populated with, say, 10, 20, 30, etc. Or possibly 100, 200, 300, etc. So that if they need to reorder things in the future, this will allow changing one item's sortorder value to say 25, to put it between the items with 20 and 30. Make sense?
But eventually, it's possible that the users could paint themselves into a corner, or at any rate make things confusing for themselves. It would be nice to build them a button that simply goes through the rows, and re-sets all the sortorder values, to preserve the existing row order but to make the values of sortorder be spaced evenly by intervals anew.
This would require some subqueries to be written inside. Steps of what I did are as follows:
Table name I used is tt. You need to change it according to your table name.
First is to get all rows in sorted order of sortorder column.
Second, declare a variable, say #serial_no and keep incrementing it by 1 on every selected row. This is an old school technique but I find it more readable.
Assign new sortorder values in this new parent select query. For now, I have just multiplied it's serial number(as in rank) by 10. You can adjust accordingly.
In your update query, inner join current copy of the table being updated with this select query and update the new sortorder column values correctly by matching them on id column.
Snippet:
update tt A
inner join (
select id, title, (#serial_no := #serial_no + 1) as serial_no,#serial_no * 10 as `sortorder`
from (
select *
from tt
order by sortorder asc
) temp_derived,(SELECT #serial_no := 0) as sn
) B
on A.id = B.id
set A.sortorder = B.sortorder
Update:
I just realised the control is completely shifted from user to DB. If you wish to update multiple rows with their new sortorder values, I wish to propose a workaround technique since I have never seen updating multiple rows with new values submitted from user in bulk(happy to learn if there exists one).
You need to map old values with new values, say in an associative array in PHP.
Start a DB transaction in MySQL.
Insert all new rows in bulk.
Delete all previous old rows in one go with IDs sent from PHP (in a prepared statement preferably with the previously mapped assoc array)
Commit the transaction.
Rollback ofcourse if something goes wrong.
Update: This solution works in MySQL 8.0, but not in MariaDB, because MariaDB's support for CTE doesn't support UPDATE statements. I'll leave this solution here for readers who use MySQL, but it doesn't work for MariaDB.
mysql> select * from NoOneEverNamesTheirTableInSqlQuestions;
+----+-----------+-----------+
| id | title | sortorder |
+----+-----------+-----------+
| 1 | this | 10 |
| 2 | that | 30 |
| 3 | other | 20 |
| 4 | something | 25 |
+----+-----------+-----------+
mysql> with cte as (
select id, row_number() over (order by sortorder) * 10 as new_sortorder
from NoOneEverNamesTheirTableInSqlQuestions
)
update NoOneEverNamesTheirTableInSqlQuestions join cte using (id)
set sortorder = new_sortorder;
Query OK, 2 rows affected (0.01 sec)
Rows matched: 4 Changed: 2 Warnings: 0
mysql> select * from NoOneEverNamesTheirTableInSqlQuestions;
+----+-----------+-----------+
| id | title | sortorder |
+----+-----------+-----------+
| 1 | this | 10 |
| 2 | that | 40 |
| 3 | other | 20 |
| 4 | something | 30 |
+----+-----------+-----------+
In MariaDB you can do:
update t
join (
select id, 10 * row_number() over (order by sortorder) as rn10
from t
) x on x.id = t.id
set t.sortorder = x.rn10;
Result:
id title sortorder
--- ---------- ---------
1 this 10
2 that 40
3 other 20
4 something 30
See running example at db<>fiddle.
Currently we have have following combined different data i.e one with comma separated and one without comma separated in same table.
#1
id | values
------------
1 | 1,2,3
2 | 4,5,6
3 | 7,8,9
4 | 1
5 | 2
6 | 3
7 | 4
8 | 5
Let's say I want to search value 1 from the table and count it. Which is the most optimal way of querying for such kind of data structure ? Is it using :
WHERE value =
WHERE value LIKE
WHERE value IN
Or Do I need to change my database and have all values to be in same format ?
Any help or advice on best practices would be appreciated.
Never store multiple values in one column!
Please normalize your data. That makes it way easier to query and it is also faster and can make use of indexes.
I agree that normalizing your data is the way to go to get optimal performance. But not changing the schemma, you can't really use = because you may have multiple numbers. One way to count the number of occurrences for 1 in those comma separated values is using FIND_IN_SET():
SELECT SUM(CASE
WHEN FIND_IN_SET(1, `values`)
THEN 1
ELSE 0
END) AS count1
FROM tab1
sqlfiddle demo
This is an expansion of my original question located here:
How do I pull all rows from a table with one unique field and specific values for another field?
I have a table with two fields: user_id and skill_id.
I want to pull out all rows that have a skill_id of a certain number but I have a large number of skill_id's to search for (~30). I was using the self-join suggestion presented in the question linked above but with so many skills to look for, that query is proving extremely slow.
How can I look for a large number of skill_ids without bogging down the query?
EDIT:
Here's an example of what I'm looking for. Using the table below, I want to pull out all rows of users that have skill_id of 10 AND 11 AND 12, etc. (except I'd be looking for more like 30 skills at a time).
TABLE
user_id | skill_id
=====================
1 | 10
1 | 11
1 | 12
1 | 13
2 | 10
2 | 12
2 | 13
3 | 15
3 | 16
4 | 10
5 | 45
5 | 46
If I understand your question well, below query might help you. Assuming (user_id, skill_id) is UNIQUE or PK.
SELECT user_id
FROM tab
WHERE skill_id IN (30 entries)
GROUP BY user_id
HAVING SUM(skill_id IN (30 entries)) = 30;
You can test here. http://www.sqlfiddle.com/#!2/f73dfe/1/0
select user_id
from table
where skill_id IN (10,11,12...)
make suer skill_is is indexed
I have a table which contains a standard auto-incrementing ID, a type identifier, a number, and some other irrelevant fields. When I insert a new object into this table, the number should auto-increment based on the type identifier.
Here is an example of how the output should look:
id type_id number
1 1 1
2 1 2
3 2 1
4 1 3
5 3 1
6 3 2
7 1 4
8 2 2
As you can see, every time I insert a new object, the number increments according to the type_id (i.e. if I insert an object with type_id of 1 and there are 5 objects matching this type_id already, the number on the new object should be 6).
I'm trying to find a performant way of doing this with huge concurrency. For example, there might be 300 inserts within the same second for the same type_id and they need to be handled sequentially.
Methods I've tried already:
PHP
This was a bad idea but I've added it for completeness. A request was made to get the MAX() number for the item type and then add the number + 1 as part of an insert. This is quick but doesn't work concurrently as there could be 200 inserts between the request for MAX() and that particular insert leading to multiple objects with the same number and type_id.
Locking
Manually locking and unlocking the table before and after each insert in order to maintain the increment. This caused performance issues due to the number of concurrent inserts and because the table is constantly read from throughout the app.
Transaction with Subquery
This is how I'm currently doing it but it still causes massive performance issues:
START TRANSACTION;
INSERT INTO objects (type_id,number) VALUES ($type_id, (SELECT COALESCE(MAX(number),0)+1 FROM objects WHERE type_id = $type_id FOR UPDATE));
COMMIT;
Another negative thing about this approach is that I need to do a follow up query in order to get the number that was added (i.e. searching for an object with the $type_id ordered by number desc so I can see the number that was created - this is done based on a $user_id so it works but adds an extra query which I'd like to avoid)
Triggers
I looked into using a trigger in order to dynamically add the number upon insert but this wasn't performant as I need to perform a query on the table I'm inserting into (which isn't allowed so has to be within a subquery causing performance issues).
Grouped Auto-Increment
I've had a look at grouped auto-increment (so that the number would auto-increment based on type_id) but then I lose my auto-increment ID.
Does anybody have any ideas on how I can make this performant at the level of concurrent inserts that I need? My table is currently InnoDB on MySQL 5.5
Appreciate any help!
Update: Just in case it is relevant, the objects table has several million objects in it. Some of the type_id can have around 500,000 objects assigned to them.
Use transaction and select ... for update. This will solve concurrency conflicts.
In Transaction with Subquery
Try to make index on column type_id
I think by making index on column type_id it will speed up your subquery.
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
,type_id INT NOT NULL
);
INSERT INTO my_table VALUES
(1,1),(2,1),(3,2),(4,1),(5,3),(6,3),(7,1),(8,2);
SELECT x.*
, COUNT(*) rank
FROM my_table x
JOIN my_table y
ON y.type_id = x.type_id
AND y.id <= x.id
GROUP
BY id
ORDER
BY type_id
, rank;
+----+---------+------+
| id | type_id | rank |
+----+---------+------+
| 1 | 1 | 1 |
| 2 | 1 | 2 |
| 4 | 1 | 3 |
| 7 | 1 | 4 |
| 3 | 2 | 1 |
| 8 | 2 | 2 |
| 5 | 3 | 1 |
| 6 | 3 | 2 |
+----+---------+------+
or, if performance is an issue, just do the same thing with a couple of #variables.
Perhaps an idea to create a (temporary) table for all rows with a common "type_id".
In that table you can use auto-incrementing for your num colomn.
Then your num shoud be fully trustable.
Then you can select your data and update your first table.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Create a trigger that updates a column on one table when a column in another table is updated
I got a database table with these values
UpVotes | DownVotes | PercentVotes
8 | 2 | 80
10 | 0 | 100
5 | 5 | 50
560 | 34 | 94
is it possible to get the mysql database to work like this:
UpVotes | DownVotes | PercentVotes
8 | 2 | (UpVotes/(UpVotes + DownVotes)) x 100 = 80
10 | 0 | (UpVotes/(UpVotes + DownVotes)) x 100 = 100
5 | 5 | (UpVotes/(UpVotes + DownVotes)) x 100 = 50
560 | 34 | (UpVotes/(UpVotes + DownVotes)) x 100 = 94
automatically without having to update it via any script? Just like in excel when you can then change the UpVotes value or DownVotes value and the PercentVotes value automatically becomes the new correct percent of upvotes according to the total votes.
Sorry for my bad english but i hope you get my point. :)
You can use a trigger for that. Something like this:
delimiter |
CREATE TRIGGER calc_percentages BEFORE INSERT on your_table
FOR EACH ROW
BEGIN
SET NEW.PercentVotes = NEW.UpVotes/(NEW.UpVotes + NEW.DownVotes);
END
|
delimiter ;
It will run on every insert and add the calculated value. If you want it to update after an update then just add another trigger for after update instead of before insert.
Since the percentage information is a calculated value you can calculate it in your selects on-the-fly and don't store it in your DB. But if you really need the performance gain then use a trigger to store it.
You would want to run something like this on the select, you may need to check the math though.
select UpVotes, DownVotes ((UpVotes / (UpVotes + DownVotes)) * 100) as PercentVotes from my_table;
Yes it is possible.
You can use triggers. Just set trigger on update and insert and it will change value of column.
if you are using triggers then try something like this
delimiter |
CREATE TRIGGER insertPercentVotes AFTER INSERT ON YourTableName
// calculate percentage here
// Execute Update query here
END;
|
delimiter //
CREATE TRIGGER ins_votepercent BEFORE INSERT ON votetable FOR EACH ROW
SET NEW.PercentVotes = (NEW.UpVotes/(NEW.UpVotes+NEW.DownVotes)) * 100
END;//
CREATE TRIGGER upd_votepercent BEFORE UPDATE ON votetable FOR EACH ROW
SET NEW.PercentVotes = (NEW.UpVotes/(NEW.UpVotes+NEW.DownVotes)) * 100
END;//
See MySQL Manual