I have a query that on every user purchase gets currently highest receipt_counter number from receipts table in order to create new receipt. receipt_counter is not unique in the table because it resets every year.
receipt_counter is just an integer that is used in generating receipt_label that looks like "pos_id"-"receipt_counter".
There is a possibility that people can buy a product simultaneously on the same point of sale (pos_id).
Function that gets new receipt_counter looks like this:
SELECT (MAX(receipt_counter) + 1) as next_receipt_counter FROM receipts
The problem is when multiple people are buying a product simultaneously, which triggers generating new receipt (along with receipt number), sometimes a collision occurs (multiple people get same receipt number) because there is some delay between retrieving receipt counter and inserting new receipt into DB.
Is there a best practice to deal with this kind of problems? Do I need to use some kind of deadlock, or is my initial idea flawed and I need to change tactic for generating receipt counter all together?
EDIT: receipt_counter needs to be a sequential number without gaps.
there is some delay between retrieving receipt counter and inserting new receipt into DB
You can change your software in order to instead or retrieving the ID without creating the actual receipt, it creates the receipt (with "pending" state or something like this) and then retrieve its ID. In the moment you currently create the receipt, you would just set its status to "active" or something.
Doing it this way you get rid of this time gap between getting and ID and storing the record, which in my point of view, is the main source of your problems.
You can create separate table for id only and enable auto_increment on that id column. Then add receipt in 2 steps - first add new record to id table, to receive back generated id. Then add actual receipt using received id. Then when you need just truncate table with id's when you want to reset the increment counter.
Does the receipt_counter need to be an increasing number without gaps?
If an increasing large number with gaps is okay, how about generating a number out of the current date/time? If you go down to milliseconds or nanoseconds, the chance of a collision is pretty low.
For example:
2013-11-13 13:08:15.012 -> 1113130815012
(I omitted the year because you said the number is reset every year anyway)
Related
My site hosted in a shared hosting. It's a POS application (PHP, Codeigniter). It has several users. Everyone is generating invoice. Invoice number is incremental. That is when a user submit a invoice form, it fetches the last invoice number then increment it by one and then create a new row with new invoice number. This process some time (very rarely) duplicate invoice number generated when users submits the form pretty much same time.
One possible way is that make invoice unique. But if it happens again, user will see an exception or formatted error message.
I don't want show error to my users. Because when they submit the invoice form , it contains sales information that they have written. If they loose it because of this warning, they feel disturbed. AJAX will not work. Direct submit is working here(for invoice submission ).
Can SQL lock be applied for this situation? I have no idea about SQL locking.
If your concern is not performance an inefficient way to do would be something like
Insert your invoice number as null/zero and have another query update that
like
INSERT INTO invoices (id, invoice_number) VALUES (10001, null);
UPDATE invoices SET invoice_number = id WHERE invoice_number IS NULL;
For locks you can look into SELECT ... FOR UPDATE that would lock the last read row, and also inserts from other connections are also blocked but its better you try it on your DB as this depends on your Mysql version and isolation levels set.
What is the best way to generate consecutive values when you have a load balanced database and instances of your application ?
For example, i have a load balanced mysql database.
My PHP application, is deployed with docker and has 3 containers
I have to generate consecutive ids. I cannot use auto increment because i have to generate unique ids depending on relations (For example, i have to generate a unique bill number depending on witch society it is related)
My bill can be generated but not emmited. I must generate the unique value when the bill is emitted.
TRIGGER ON UPDATE is the good solution or not ?
Thnks for your answers
I would save the current id in a db table.
Each time you want to increase the id, do the following:
Start a transaction
Block the id row in the db table: in mysql use FOR UPDATE
Read the current id
increase the id
generate the bill with the id
Store the id back to the db
Commit the transaction
I'd go for MAX(id)+1
You can get the next number in the sequence with a query like:
SELECT COALESCE(MAX(id),0) + 1 FROM bill
WHERE society = 'XYZ'
You'll have to take steps to ensure that two processes don't generate the same number and that can be complicated but not insurmountable.
Personally, I would always avoid a trigger. I've never used trigger and not regretted it later.
Together with my team, I am working on a functionality to generate invoice numbers. The requirements says that:
there should be no gaps between invoice numbers
the numbers should start from 0 every year (the together with the year we will have a unique key)
the invoice numbers should grow accordinlgy to the time of the creation of the invoices
We are using php and postgres. We tought to implement this in the following way:
each time a new invoice is persisted on the database we use a BEFORE INSERT trigger
the trigger executes a function that retrieves a new value from a postgres sequence and writes it on the invoice as its number
Considering that multiple invoices could be created during the same transaction, my question is: is this a sufficiently safe approach? What are its flaws? How would you suggest to improve it?
Introduction
I believe the most crucial point here is:
there should be no gaps between invoice numbers
In this case you cannot use a squence and an auto-increment field (as others propose in the comments). Auto-increment field use sequence under the hood and nextval(regclass) function increments sequence's counter no matter if transaction succeeded or failed (you point that out by yourself).
Update:
What I mean is you shouldn't use sequences at all, especially solution proposed by you doesn't eliminates gap possibility. Your trigger gets new sequence value but INSERT could still failed.
Sequences works this way because they mainly meant to be used for PRIMARY KEYs and OIDs values generation where uniqueness and non-blocking mechanism is ultimate goal and gaps between values are really no big deal.
In your case however the priorities may be different, but there are couple things to consider.
Simple solution
First possible solution to your problem could be returning new number as maximum value of currently existing ones. It can be done in your trigger:
NEW.invoice_number =
(SELECT foo.invoice_number
FROM invoices foo
WHERE foo._year = NEW._year
ORDER BY foo.invoice_number DESC NULLS LAST LIMIT 1
); /*query 1*/
This query could use your composite UNIQUE INDEX if it was created with "proper" syntax and columns order which would be the "year" column in the first place ex.:
CREATE UNIQUE INDEX invoice_number_unique
ON invoices (_year, invoice_number DESC NULLS LAST);
In PostgreSQL UNIQUE CONSTRAINTs are implemented simply as UNIQUE INDEXes so most of the times there no difference which command you will use. However using that particular syntax presented above, makes possible to define order on that index. It's really nice trick which makes /*query 1*/ quicker than simple SELECT max(invoice_number) FROM invoices WHERE _year = NEW.year if the invoice table gets bigger.
This is simple solution but has one big drawback. There is possibility of race condition when two transactions try to insert invoice at the same time. Both could acquire the same max value and the UNIQUE CONSTRAINT will prevent the second one from committing. Despite that it could be sufficient in some small system with special insert policy.
Better solution
You may create table
CREATE TABLE invoice_numbers(
_year INTEGER NOT NULL PRIMARY KEY,
next_number_within_year INTEGER
);
to store next possible number for certain year. Then, in AFTER INSERT trigger you could:
Lock invoice_numbers that no other transaction could even read the number LOCK TABLE invoice_numbers IN ACCESS EXCLUSIVE;
Get new invoice number new_invoice_number = (SELECT foo.next_number_within_year FROM invoice_numbers foo where foo._year = NEW.year);
Update number value of new added invoice row
Increment UPDATE invoice_numbers SET next_number_within_year = next_number_within_year + 1 WHERE _year = NEW._year;
Because table lock is hold by the transaction to its commit, this probably should be the last trigger fired (read more about trigger execution order here)
Update:
Instead of locking whole table with LOCK command check link provided by Craig Ringer
The drawback in this case is INSERT operation performance drop down --- only one transaction at the time can perform insert.
My setup:
Mysql and PHP
System Scenario:
I have more than 10 Type of system Users:
For example :Customer and Employee
Everytime a customer or employee added to the system, the system will automatically generate ID to each user based on current date.
Ex (Customer):
Today is June 20,2015 and this customer is the 3rd to sign up. So his
ID would be 06202015-03. So everytime a user (any type of user) signup
the sequence number will increment by 1 in a day basis only. Every next day
the sequence counter will be back to 0.
General Question: Given my concern of ID generation is solved, is it a good practice to pre-process the next sequence #? I mean the system will just pullout the next sequence number saved on the db table? or should I just process the next sequence number only until a new user is signing up?
UPDATE (Added best possible scenario) :
Example Date: June 20,2015
Customer 1 signup = Generated ID would be 06202015-01
Customer 2 signup = Generated ID would be 06202015-02
and so on...
Worst possible scenario during signup:
2 or more user signing up simoltaneously
If customer1 is deleted (by admin) on that same day and customer2 signed up, the customer 2 should get the #1 id (06202015-01) and not *-02 as the customer1 is being deleted already.
.
I would like to know the best way to generate a sequence number efficiently:
Is stored procedure would be the best fit for this? or should I use #2?(see below)
Is it a good practice to just process the next sequence number (using PHP function) everytime a user signed up?
The #2 process is I think the best and easier way to process auto ID generation but I'm thinking WHAT IF 2 or more users
simultaneously singing up?
On my latest update, the sequence is obviously predictable. My only concern is what is the best or efficient way to get the sequence number. Is it thru stored procedure or using php script function given the worst scenarios stated.
General Question: Given my concern of ID generation is solved, is it a good practice to pre-process the next sequence #? I mean the system will just pullout the next sequence number saved on the db table? or should I just process the next sequence number only until a new user is signing up?
If the id is dependent on the date an user signs up, you can't predict the next id because you don't know when the next user will sign up (unless you are a clairvoyant).
To make it easier to obtain the next value I would split the id into two columns, a column with the date and a column with the sequence, then u can use:
IFNULL((SELECT MAX(sequence) FROM usertable WHERE signup_date = CURRENT_DATE), 0) + 1
Imo there's no best practise, it's a personal preference.
There's also a third option, a before insert trigger.
To avoid duplicates add an unique index with both columns.
In addition you can lock the table:
LOCK TABLES user_table WRITE;
/* CALL(sproc) or INSERT statement, or SELECT and INSERT statements */
UNLOCK TABLES;
With a write lock no other session can access the table untill the lock is released (it will wait)
Just a quickey. I am developming website, where you can buy credits and spend them later for things on the website.
My question is, is it ok to store amount of credits with user (user table, column credits and iteger amount) or it is necessary (or just better) to have separate table with user id and amount ?
Thanks
Both actually.
Considering that you'll be dealing with monetary transactions to get those credits, you want to be able to get a log of all transactions (depending of the laws in your country, you will NEED this). Therefore you'll need a credits_transactions table.
user_id, transaction_id, transaction_details, transaction_delta
Since programmatically calculating your current credit balance will be too costly for users with a lot of transactions, you'll also need a credit_balance row in your user table for quick access. Use triggers to automatically update that column whenever a row is inserted from credits_transactions (technically, update and delete shouldn't be allowed in that table). Here's is the code for the insert trigger.
CREATE TRIGGER ct_insert
AFTER INSERT ON credits_transactions
BEGIN
UPDATE users SET credit_balance = credit_balance + NEW.transaction_delta WHERE user_id = NEW.user_id;
END
;;
I also have sites containing credits and found it easiest to store them in the user table, mostly because you need access to it on every page (when the user is logged in). It is only an integer so will not do much harm. I think actually creating a new table for this value might be worse perfomance wise because it needs an index aswel.
A good rule of thumb is to create a user table for the info you need on every page, and normalise the data you dont need on every page (for example adress information, descriptions etc).
Edit:
Seeing the other reactions,
If you want to have transaction logs aswel I would store them seperately as they are mainly for logging (or if the user wants to view them). Calculating them on the fly from the log is fine for smaller sites but if you really have to squeeze performance just store the actual value in the user table.
If you store in separate table, you can keep log of changing the credits. If you store in column, you will have only the current amount of credits.
If you want to keep a record of Credits History Log like
how many credit bought today.
how many spend yesterday.
what did you bought with credits
I think its better to put this in a separate table. In this way you can get these kind of results by applying mathematical operations.
Credits are like money. If a user needs to purchase them, then they are money. Money is tracked using accounts. Account has associated transactions, deposits and withdrawals -- and balance. Search the SO or google for database and account. Here are just a few examples:
one
two
three
I'd have a table which stores the purchases and bought credits, with user id.
Then calculate each time based on this, it should be fast if it's indexed, this way you will be able to easily have a purchase history.