everyone! I'm making a simple todo app. I stopped on the one problem. I want to allow users to change the order of elements in a list (saving this to database).
One of first idea was:
Create a column (order) and change it every time when user do something.
It's good when we have a few records, but what with bigger number?
My thought:
id | name | order
1 | lorem| 1
2 | ipsum| 2
3 | dolor| 3
When user change "dolor" to first position, script must update all of records.
This isn't the best solution I think.
Anyone can share the knowledge how to optimize that?
I will be grateful!
You could use a column called next or previous. This is called a linked list, or if you use both, a double linked list. See:
https://en.wikipedia.org/wiki/Doubly_linked_list
Moving a record up one step in a database table would involve two steps:
Remove the record from the order.
Insert the record back into the order.
In all you would always need about five record changes for a double linked list, and a minimum of three records for a linked list.
If you want to store this data in a database, then an "ordering" column is appropriate.
Whenever you update or insert into the table, you will need to update this column (deletes are unnecessary). In general, you will need to update all the rows after the changed row. A trigger can do this work.
Cycling through rows is probably fine for a few dozen or even a few hundred rows (depending on how powerful your database is). So, depending on the length of the list, this is likely to be fine.
Any enhancements depend on additional factors. Some that I can think of:
How large will the lists really be?
What kind of transformations are most import? (Swaps? Inserts? deletes? updates?)
Will the transformations happen in bulk?
Will multiple users be changing the list at the same time.
Related
Using PHP and mySQL, I need to add multiple values in 2 mysql tables : the first table would have the more important informations and the second one would have the less important informations about each items.
To be more clear, ONE element would have his informations split in two tables.
(I need this for some reasons but two of them are : having the first table the less weight possible, and the second table would store datas that will be erase after a short time (meanwhile the first table keeps all the datas it stored).)
In the best scenario, I'd like to add a row in each table about one item/element with the same id in each table. Something like this :
Table 1 id|data_1_a|data_1_b|...
Table 2 id|data_2_a|data_2_b|...
So if I add an element which get the ID "12345" in the table 1, it adds the datas in the table 2 with the same ID "12345".
To achieve this, I think of two solutions :
Create the ID myself for each element (instead of having an auto_increment on table 1). The con is that it would probably be better to check if the ID doesn't already exist in the tables everytime I generate an ID...
Add the element on table 1, get its ID with $db->lastInsertId(); and use it to add the element's datas on table 2. The con is that I have to add one element by one element to get all the IDs, while most of the time I want to add a lot of elements (like one, two or three hundreds !) at once
Maybe there's a better way to achieve this ?
lastInsertId() reports the first value generated by the last INSERT statement executed. It's reliable to assume that when you insert many rows, they are given consecutive id values following that first value. For example, the MySQL JDBC driver relies on this assumption, so it can report the set of id values generated.
This assumption breaks only if you deliberately set innodb_autoinc_lock_mode=2 (interleaved). See https://dev.mysql.com/doc/refman/8.0/en/innodb-auto-increment-handling.html for details about that.
But if it were my task, I would still choose to use a single table. When you find you don't need some of the columns anymore, use UPDATE to set them to NULL. This will eliminate the problems you're facing with assuring the same id is used across two tables.
I have this problem where I want to create a data page where you can see how you've progressed in losing weight. I start by collecting their weight, but in order for me to actually do stuff with the data, I need multiple values.
For example - Below is a picture of my colums, where vaegtUsers and hoejdeUsers is the weight and height, however, I can't really figure out how to store multiple weight values in one single column? Is there some way to get around this? I've done a bit of research and almost all say it's not possible. Should I just add a new column for each "new" weight ID or keep creating tables for each individual user? Or should I do something completely else?
Since it appears a single user may have multiple weights at different times, you have a one to n relationships. You should create a second table (for instance, Measure) which refers to your User table.
This table could contain columns such as ID, UserId, MeasureDate, and weight.
You could also include height measure in this table if your users are not yet fully grown, and therefore susceptible to have varying height at different points in time. Otherwise, height could be stored inside user table.
On a side note, i would advise you to check database normalization for relational databases.
I have some issues with sugarcrm.
As you know that sugarcrm table, they do have ID (which is a unique string), they not run by sequential. e.g
4bab37e4-798a-e01c-75de-4e4397f358b7
For example, I would like to copy the table sugarcrm.accounts to something.accounts, in something.accounts I added some custom file on it for another PHP process usage.
Now the problem is, my sugarcrm table got huge records there, I plan to run it batch by batch, each time I would like to copy 10,000 records to my somthing.accounts.
However, sugarcrm.accounts, their ID, not in sequential, how do I know offset parameter?
I do not want to amend sugarcrm table / or add a temporary table in sugarcrm. (e.g sugarcrm.account_index), it might caused me having problem to do the upgrade.
So anyone have any idea, how do I get the index number? Is MySQL got hidden index?
Or anyone have better idea to do the database table copy another database table?
One way is the following:
- Select all rows from sugarcrm.accounts and order by date_created ascending.
- Use limit to only select a subset of the rows (store the offset from batch to batch)
- Copy the subset of rows to something.accounts
If new records are added later they will still be copied, since they will be last in the set. However, if you delete records in sugarcrm.accounts while running the batch jobs, then you need to change the offset as well, since you might omit some rows.
Another way, if the two databases/tables are in the same MySQL instance, is to join the two tables, and select the next 10.000 which doesn't exist in something.accounts.
Problem
In a web application dealing with products and orders, I want to maintain information and relationships between former employees (users) and the orders they handled. I want to maintain information and relationships between obsolete products and orders which include these products.
However I want employees to be able to de-clutter the administration interfaces, such as removing former employees, obsolete products, obsolete product groups etc.
I'm thinking of implementing soft-deletion. So, how does one usually do this?
My immediate thoughts
My first thought is to stick a "flag_softdeleted TINYINT NOT NULL DEFAULT 0" column in every table of objects that should be soft deletable. Or maybe use a timestamp instead?
Then, I provide a "Show deleted" or "Undelete" button in each relevant GUI. Clicking this button you will include soft-deleted records in the result. Each deleted record has a "Restore" button. Does this make sense?
Your thoughts?
Also, I'd appreciate any links to relevant resources.
That's how I do it. I have a is_deleted field which defaults to 0. Then queries just check WHERE is_deleted = 0.
I try to stay away from any hard-deletes as much as possible. They are necessary sometimes, but I make that an admin-only feature. That way we can hard-delete, but users can't...
Edit: In fact, you could use this to have multiple "layers" of soft-deletion in your app. So each could be a code:
0 -> Not Deleted
1 -> Soft Deleted, shows up in lists of deleted items for management users
2 -> Soft Deleted, does not show up for any user except admin users
3 -> Only shows up for developers.
Having the other 2 levels will still allow managers and admins to clean up the deleted lists if they get too long. And since the front-end code just checks for is_deleted = 0, it's transparent to the frontend...
Using soft-deletes is a common thing to implement, and they are dead useful for lots of things, like:
Saving a user's data when they deleted something
Saving your own data when you delete something
Keep a track record of what really happened (a kind of audit)
etcetera
There is one thing I want to point out that almost everyone miss, and it always comes back to bite you in the rear piece. The users of your application does not have the same understanding of a delete as you have.
There are different degrees of deletions. The typical user deletes stuff when (s)he
Made a misstake and want to remove the bad data
Doesn't want to see something on the screen anymore
The problem is that if you don't record the intention of the delete, your application cannot distinguish between erronous data (that should never have been created) and historically correct data.
Have a look at the following data:
PRICES | item | price | deleted |
+------+-------+---------+
| A | 101 | 1 |
| B | 110 | 1 |
| C | 120 | 0 |
+------+-------+---------+
Some user doesn't want to show the price of item B, since they don't sell that item anymore. So he deletes it. Another user created a price for item A by misstake, so he deleted it and created the price for item C, as intended. Now, can you show me a list of the prices for all products? No, because either you have to display potentially erronous data (A), or you have to exclude all but current prices (C).
Of course the above can be dealt with in any number of ways. My point is that YOU need to be very clear with what YOU mean by a delete, and make sure that there is no way for the users to missunderstand it. One way would be to force the user to make a choice (hide/delete).
If I had existing code that hits that table, I would add the column and change the name of the table. Then I would create a view with the same name as the current table which selects only the active records. That way none of the existing code woudl break and you could have the soft delete column. If you want to see the deleted record, you select from the base table, otherwise you use the view.
I've always just used a deleted column as you mentioned. There's really not much more to it than that. Instead of deleting the record, just set the deleted field to true.
Some components I build allow the user to view all deleted records and restore them, others just display all records where deleted = 0
Your idea does make sense and is used frequently in production but, to implement it you will need to update quite a bit of code to account for the new field. Another option could be to archive (move) the "soft-deleted" records to a separate table or database. This is done frequently as well and makes the issue one of maintenance rather than (re)programming. (You could have a table trigger react to the delete to archive the deleted record.)
I would do the archiving to avoid a major update to production code. But if you want to use deleted-flag field, use it as a timestamp to give you additional useful info beyond a boolean. (Null = not deleted.) You might also want to add a DeletedBy field to track the user responsible for deleting the record. Using two fields gives you a lot of info tells you who deleted what and when. (The two extra field solution is also something that can be done in an archive table/database.)
The most common scenario I've come across is what you describe, a tinyint or even bit representing a status of IsActive or IsDeleted. Depending on whether this is considered "business" or "persistence" data it may be baked into the application/domain logic as transparently as possible, such as directly in stored procedures and not known to the application code. But it sounds like this is legitimate business information for your needs so would need to be known throughout the code. (So users can view deleted records, as you suggest.)
Another approach I've seen is to use a combination of two timestamps to show a "window" of activity for a given record. It's a little more code to maintain it, but the benefit is that something can be scheduled to soft-delete itself at a pre-determined time. Limited-time products can be set that way when they're created, for example. (To make a record active indefinitely one could use a max value (or just some absurdly distant future date) or just have the end date be null if you're ok with that.)
Then of course there's further consideration of things being deleted/undeleted from time to time and tracking some kind of audit for that. The flag approach knows only the current status, the timestamp approach knows only the most recent window. But anything as complex as an audit trail should definitely be stored separately than the records in question.
Instead I would use a bin table in which to move all the records deleted from the other tables. The main problem with the delete flag is that with linked tables you will definitely run into a double key error when trying to insert a new record.
The bin table could have a structure like this:
id, table_name, data, date_time, user
Where
id is the primary key with auto increment
table_name is the name of the table from which the record was deleted
data contains the record in JSON format with name and value of all fields
date_time is the date and time of the deletion
user is the identifier of the user (if the system provides for it) who performed the operation
this method will not only save you from checking the delete flag at each query (immagine the ones with many joins), but will allow you to have only the really necessary data in the tables, facilitating any searches and corrections using SQL client programs
I'm trying to figure out how to best keep revision/history information on revisions to multiple rows of data, in case for some reason we need to revert to that data.
This is the general sort of layout:
item
---------------
id
title
etc...
region
---------------
id
title
etc...
release_type
-----------------
id
title
etc...
items_released_dates_data
---------------------
item_id
region_id
release_type_id (these three form the primary key)
date
So you can have one release date per item + region_id + release_type and we basically only track the date (For the purposes of this question the 'date' could be a number, a string, or whatever. I'm certain to run into this issue again)
Changes are submitted in bulk, when new data is added everything in items_released_dates_data where item_id=your_id is first deleted then an insert statement adds the new values (perhaps this isn't the best way to do this?)
My thought was to create a table like:
items_release_dates_data_history
-------------------------------------
item_id
timestamp
description
raw_data
Making description a short summary of what was updated, and including the data in some format like json or xml or something that could be quickly decoded on the client side to give the user a review of the changes and a choice to revise to a given version. Then every entry to items_released_dates_data also requires an entry to items_released_dates_data_history (doesn't sound like a question does it? :| )
I've read something about mysql triggers that would be helpful here, but quite frankly I don't know a thing about them so I'm working with what I understand.
My question is, am I following the right path to version this stuff, and is there any advice/best practices anyone can give me on how to improve this method?
I second Alex Miller's comment. Everything you write make sense so far.
I'd strongly recommend looking into triggers though, despite your reservations. They're fairly easy to grasp, and make for a very powerful tool in such scenarios. Using triggers you can store a copy of the row into a separate table each time a record is updated (or deleted). If you want to go all fancy you can, within the trigger, compare the incoming data to the existing data, and write only what has changed.
Also consider the Archive storage engine instead of MyISAM or InnoDB for these kinds of tables - they're made for this kind of job.
Also, the search phrase you're probably looking for is "audit trail".
I'd say that you're definitely on the right track. Although, you may want to store the region ID in the history so you can check release history based on a region rather than just by entire items.
As for the delete+insert, that's fine as long as you don't end up with too much traffic, as those are both locking actions. There is a lot of time used when inserting or deleting a row to update the index. If you're using a MyISAM table, it's also going to halt all reads on the table until those actions complete. Update will as well, but for a much shorter time. InnoDB will only lock the row, so that's not really a concern.