I got a Table which stores objects. An Object can be anything from a chair to a employee. An Object got an ObjectID, which is a 10 characters code-39 barcode label on the Object.
Many Objects already have a Label, thus an ObjectID assinged to them. Some have Prefixes, e.g. "9000000345" might be a Desk or "0000000895" might be a folder with invoices.
When People start a new Folder for example, they take pre-printed Barcode Labels
and put them on it. The pre-printed Barcode Labels are generated by a Printer which just increases a number by 1 and zerofills it to 10 Digits and then prints it as code-39.
All Most of the objects are stored in Excel Sheets. They now should be migrated into a MySQL Database.
Now, the System should also be able to create objects on its own. Objects created by the System have a leading "1" e.g. "1000000426".
The Problem: How do I get the next ObjectID for Auto generated Objects?
I cant really use AUTO_INCREMENT because there are also non-auto-generated rows in the table.
Another Thing to say is that the 'ObjectID' field has to be CHAR(10) because for special occasions there were alphanumeric prefixes used like "T1" -> "T100003158"
My Table when using AUTO_INCREMENT:
| ID | Created | ObjectID | Parent | Title | Changed | Note |
|----|-------------|--------------|--------|-------------|-------------|------|
| 1 | <timestamp> | "1000000001" | NULL | "Shelf 203" | <timestamp> | NULL |
| 2 | <timestamp> | "9000000458" | NULL | "Lamp" | <timestamp> | NULL |
| 3 | <timestamp> | "1000000003" | NULL | "Shelf 204" | <timestamp> | NULL |
The ObjectID of the last Object in the table should be "1000000002" not "1000000003"
I hope I could explain the Problem well enough.
Naive solution can be:
SELECT CAST(ObjectID AS UNSIGNED) + 1 FROM yourTable WHERE ObjectId LIKE "1%" ORDER BY ObjectID DESC LIMIT 1
Basically search for all Object ID starting with 1xxxx then sort them (because its zero padded we can still sort) and then cast result to int and increment it.
Might be faster to cast to int first and then do between. Rest would be the same
Related
I'm making small commenting app written in PHP as backend, React as frontend and PostgreSQL as database. I have table comment which holds all comments and it is self referencing table.
\d+ comment:
Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+--------------------------+-----------+----------+-------------------------------------+----------+--------------+-------------
id | bigint | | not null | nextval('comment_id_seq'::regclass) | plain | |
website_page_id | bigint | | not null | | plain | |
author_id | bigint | | not null | | plain | |
parent_id | bigint | | | | plain | |
content | text | | | | extended | |
deleted_date | timestamp with time zone | | | | plain | |
updated_date | timestamp with time zone | | not null | | plain | |
created_date | timestamp with time zone | | not null | | plain |
On the client side I make request to get all comments, backend makes recusrive query to database to grab all comments and return them in appropriate format, then I render it.
Here is JSON of parent comment:
{
id: 1
author_id: 1
content: "Some content"
created_date: "2019-05-29 06:11:43+00"
depth: 0
parent_id: null
replies: [...]
updated_date: "2019-05-29 06:11:43+00"
website_page_id: null
}
So each comment as depth parameter, which I use to define identation (I don't nest comments recursively like comment -> replies -> comment -> replies, it is only comment and all its replies. I do extra processing on backend to make this form, PostgreSQL returns just data as it is with depth definition.
I have a form for creating new comments and replies to existing comments. So far replies can nest as far as it can go (not sure about database limitations).
Here are my concerns:
I don't want to nest forever as it kills performance (I assume). Does it really? Also, it is resonable to limit it up to n level by default so it does not go off the screen on the client side.
Not sure where and how to make limitation. Whether it should be on the database level, backend or client side?
I had only one idea how to solve it, but so far it does not seem to be elegant solution. Here it is:
Ignore that it nests on the database level and just limit identation on client side, so if I defined 5 level as maximum, then anything above that would have 5 level identation. It works, but it does not help the database performace.
I am pretty sure there are other possible ways to do this, help would be appreciated!
Recursive queries (when they take advantage of index) are really fast. It will probably take more time to nest the results in Javascript. The nesting limitation is more for the UI and not very difficult to fetch:
with recursive
comment_node (comment_id, parent_id, level) as (
select comment_id, comment_parent_id, 1::int4 as level
from comment
where website_page_id = $*
union all
select c.comment_id, c.comment_parent_id, parent.level + 1 as level
from comment as c
inner join comment_node as parent
on parent.comment_id = c.parent_id
and parent.level < 5
)
select c.comment_id, cn.level, c.comment_parent_id, c.content, a.name, ...
from comment as c
join comment_node as cn
using (comment_id)
join author as a
using (author_id)
Limiting the insertion of comments with a nesting level of 5 or more is probably not a meaningful database constraint as it does not break the data consistency.
I am trying to build a table of this structure using Laravel's migrations feature:
------------------------------------
| Data | Rules | Allow | Restrict |
------------------------------------
| item1 | rule1 | 1,3 | null |
| item2 | rule2 | null | 2,5,6 |
------------------------------------
As in, for each entry either Allow or Restrict must possess a not null value, but not both. I've found this comment which sounds like the condition I need, but I need to express it in a format understandable to Laravel.
I think there are 2 good solutions
Seperate the data in 2 tables
Table1 -> data, rules, constraint (FK)
Table2-> id (PK, referenced by Table1), content (allow/restrict numbers), isAllow(bool)
That way you do the constraint in the database. This is the better solution, because now you don't have null values in your database. Normalisation is a good thing.
Use the event listener to check before insert
https://laravel.com/docs/5.6/eloquent#events
public function creating(Table1 $t1)
{
// check if 1 is null and 1 is not null before creating
}
i have an id coloumn which is integer and auto incremented type.
The problem is when ever i delete a row the continuity of the number breaks.
+----------------------+----+
| name | id |
+----------------------+----+
| mashable | 1 |
| Behance | 2 |
| Techcrunch | 3 |
| flipkart | 4 |
+----------------------+----+
FOR EXAMPLE if i delete the row with id=2, then i output in id will be
+----------------------+----+
| name | id |
+----------------------+----+
| mashable | 1 |
| Techcrunch | 3 |
| flipkart | 4 |
+----------------------+----+
but i want it to be like :
+----------------------+----+
| name | id |
+----------------------+----+
| mashable | 1 |
| Techcrunch | 2 |
| flipkart | 3 |
+----------------------+----+
How to do it ??
To directly answer your question, here's how you fix those gaps in sequential numeric fields: Fixing gaps in mysql table row id after we delete some of them
But let's be careful here for a moment.
Let's assume id is your primary key. ID's are usually the point of reference to an object, because auto-generated ID's are unique. Call it a convention.
That means that If ANY part of your code depends on the id column, your application will break.
If you NEED to do this, then use some other field as main reference. Perhaps an unique name field or something similar.
If ID is NOT your primary key, then you probably should've chosen another name for it to begin with. Anyway, in this case, the chances of you breaking anything are much smaller.
Notice that I said smaller, but not zero. We don't know your application, so it's possible that your code uses id for something important, and that'll mean trouble for you.
How much faster (in %) sql will be if I will avoid to used built-in mysql date and time functions ?
What do I mean ? For example: SELECT id FROM table WHERE WEEKOFYEAR(inserted)=WEEKOFYEAR(CURDATE())
MySQL has a lot of buil-in function to work with date and time, and they are suitable as well. But what about peromance ?
Above sql can be rewritten without built-in functions, like: SELECT id FROM table WHERE inserted BETWEEN 'date for 1 day of particular week 00:00:00' AND 'last day of particular week 23:59:59', server side code become worse :( but on db side we could use indexes
I see two problems for usage built-in functions:
1. indexes
I did small test
mysql> explain extended select id from table where inserted between '2013-07-01 00:00:00' and '2013-07-01 23:59:59';
+----+-------------+-------+-------+---------------+------+---------+------+------+----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------+---------------+------+---------+------+------+----------+--------------------------+
| 1 | SIMPLE | table | range | ins | ins | 4 | NULL | 7 | 100.00 | Using where; Using index |
+----+-------------+-------+-------+---------------+------+---------+------+------+----------+--------------------------+
mysql> explain extended select id from table where date(inserted)=curdate();
+----+-------------+-------+-------+---------------+------+---------+------+--------+----------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+-------+---------------+------+---------+------+--------+----------+--------------------------+
| 1 | SIMPLE | table | index | NULL | ins | 4 | NULL | 284108 | 100.00 | Using where; Using index |
+----+-------------+-------+-------+---------------+------+---------+------+--------+----------+--------------------------+
First one took 0.00 sec second one was running after first one and took 0.15. Everything was made with small anout of data.
and second problem, is
time to call that functions
If in table I have 1 billion records it means that WEEKOFYEAR, DATE whatever... would be called so many times, so many records do we have, right ?
So the question will it bring real profit if I will stop to work with mysql built-in date and time functions ?
Using a function of a column in a WHERE clause or in a JOIN condition will prevent the use of indexes on the column(s), if such indexes exist. This is because the raw value of the column is indexed, as opposed to the computed value.
Notice the above does not apply for a query like this:
SELECT id FROM atable WHERE inserted = CURDATE(); -- the raw value of "inserted" is used in the comparison
And yes, on top of that, the function will be executed for each and every row scanned.
The second query is running the date function on every row in the table, while the first query can just use the index to find the rows it needs. Thats where the biggest slowdown would be. Look at the rows column in the explain output
I'm trying to create a table like this:
lives_with_owner_no from until under_the_name
1 1998 2002 1
3 2002 NULL 1
2 1997 NULL 2
3 1850 NULL 3
3 1999 NULL 4
2 2002 2002 4
3 2002 NULL 5
It's the Nermalization example, which I guess is pretty popular.
Anyway, I think I am just supposed to set up a dependency within MySQL for the from pending a change to the lives_with table or the cat_name table, and then set up a dependency between the until and from column. I figure the owner might want to come and update the cat's info, though, and override the 'from' column, so I have to use PHP? Is there any special way I should do the time stamp on the override (for example, $date = date("Y-m-d H:i:s");)? How do I set up the dependency within MySQL?
I also have a column that can be generated by adding other columns together. I guess using the cat example, it would look like:
combined_family_age family_name
75 Alley
230 Koneko
132 Furrdenand
1,004 Whiskers
Should I add via PHP and then input the values with a query, or should I use MySQL to manage the addition? Should I use a special engine for this, like MemoryAll?
I disagree with the nermalization example on two counts.
There is no cat entity in the end. Instead, there is a relation (cat_name_no, cat_name), which in your example has the immediate consequence that you can't tell how many cats named Lara exist. This is an anomaly that can easily be avoided.
The table crams two relations, lives_with_owner and under_the_name into one table. That's not a good idea, especially if the data is temporal, as it creates all kinds of nasty anomalies. Instead, you should use a table for each.
I would design this database as follows:
create table owner (id integer not null primary key, name varchar(255));
create table cat (id integer not null primary key, current_name varchar(255));
create table cat_lives_with (
cat_id integer references cat(id),
owner_id integer references owner(id),
valid_from date,
valid_to date);
create table cat_has_name (
cat_id integer references cat(id),
name varchar(255),
valid_from date,
valid_to date);
So you would have data like:
id | name
1 | Andrea
2 | Sarah
3 | Louise
id | current_name
1 | Ada
2 | Shelley
cat_id | owner_id | valid_from | valid_to
1 | 1 | 1998-02-15 | 2002-08-11
1 | 3 | 2002-08-12 | 9999-12-31
2 | 2 | 2002-01-08 | 2001-10-23
2 | 3 | 2002-10-24 | 9999-12-31
cat_id | name | valid_from | valid_to
1 | Ada | 1998-02-15 | 9999-12-31
2 | Shelley | 2002-01-08 | 2001-10-23
2 | Callisto | 2002-10-24 | 9999-12-31
I would use a finer grained date type than just year (in the nermalization example having 2002-2002 as a range can really lead to messy query syntax), so that you can ask queries like select cat_id from owner where '2000-06-02' between valid_from and valid_to.
As for the question of how to deal with temporal data in the general case: there's an excellent book on the subject, "Developing Time-Oriented Database Applications in SQL" by Richard Snodgrass (free full-text PDF distributed by Richard Snodgrass), which i believe can even be legally downloaded as pdf, Google will help you with that.
Your other question: you can handle the combined_family_age either in sql externally, or, if that column is needed often, with a view. You shouldn't manage the content manually though, let the database calculate that for you.