I need to create a Raffle system where users can enter the number of tickets they want to buy and pay with credit cards to participate in a raffle. A raffle has a limited number of tickets let's say 1000. Anyone can enter any number of tickets he wants to buy, of course the number should be less or equal to 1000. There is no login in the system so I'm seeing this following as a race condition:
One user enters 998 tickets to buy and another one enters 5 tickets, if both users click on submit on the same time and I process both requests is this going to be a race condition? If yes, has anyone came across a similar case and is there any way to avoid this?
Thanks.
Many programmers will approach this by checking the database for sufficient tickets and if found then updating the ticket sales. This leaves a window between the read and write parts of the operation where the race condition exists.
User A reads (reads 1000 tickets)
User B reads (reads 1000 tickets)
User A writes(reduces tickets left by 998, writes 2)
User B writes (reduces tickets left by 5, writes 995)
We're left with 1003 tickets 'sold', and a remaining balance of 995 tickets to sell. This is clearly not acceptable.
What's required is an atomic test-and-set operation (atomic, because it's indivisible)
Fortunately, databases treat single queries as atomic, and also provide a mechanism for the test-and-set requirement
Consider the query
UPDATE `raffles` set `ticketsLeft` = `ticketsLeft`- 995 WHERE `raffleID` = 'someId' and `ticketsLeft` >= 995
This will test for sufficient tickets and deduct the tickets sold all as part of one query. There's no window in which a race can exist.
But, how does the program know that the update succeeded?
The database handles that too. The program asks the database for the number of rows affected by the UPDATE query. If the update succeeds one row is affected. If it fails (no raffleId or insufficient tickets) no rows are affected.
Thus, the program executes its query, and checks the rows affected. If the answer is 1 the ticket sale succeeded. If the answer is zero, the ticket sale failed. The program handles these two possibilities and carries on.*
For PHP there are two interfaces to MySQL: see mysqli::affected_rows(), or PDOStatement::rowcount() for the details.
Other databases have similar constructions.
* Data integrity is assured here by applying a UNIQUE index to the raffleId column, guaranteeing there will only be one matching raffle, or none.
Related
I'm running a high throughput application in which many auction-like processes happen every second and budgets are spent based on auction-winning bids.
However, there's a slight latency between the moment a bid is placed and the time when the auction is decided, so what started happening is this:
two similar auctions are won by the same highest bidder in the same second, but the total bids for the 2 auctions exceed the budget of the user, so by the time the second auction tries to get the bid, the first one already got it and the user now ends up with a negative balance.
I do understand this is a complicated problem, but I would like to know your opinions.
Some details about my application might offer insights into what the right tradeoffs might be:
the bids are, generally, much smaller than the user budget, so not allowing the users to spend the last few cents of their $5 budget might not be a big issue, but there's no limit to the bids, so this might not, in fact, help decreasing risk (might work for the low-bid transactions, but as soon as the user places, say, a $3 bid, the budget can still go from $5 to -$7 in a second, by winning just 4 auctions with this higher than normal bid).
not allowing bids higher than a certain budget ratio can also be an acceptable solution, but this might affect the user experience quite a lot and the bid/budget ratio will be rather arbitrary. The lower the ratio, the safer the budget and the worse user experience, but there's still no guarantee the budget won't be exceeded.
booking the bid for a certain time can also be an option, but with 100k users bidding multiple times per second, the solution for booking all the bids and releasing the funds can become quite complex
simply not paying for the second transaction can also be a (rather crappy and unfair) solution, but it might limit the incentive to abuse this bug.
I realize these details are all direct attempts to solve the problem, but none of them is good enough and I'm curious what your opinion is and if there are any better solution to the Double Spending problem. Unlike the solution found for the Bitcoin protocol, in my case the exchange of "goods" is done in real time and I cannot revert it once there are no more funds available (the goods, in my case, are website visits, so they're instantaneous, can't be stored until the transaction is settled and can't be delayed).
This might work: change the rule for winning an auction.
Now, I guess, a user wins an auction if she's the highest bidder at the conclusion of the auction. You can change the rule so the user must BOTH be the high bidder AND have sufficient budget to cover the bid. If both criteria aren't met, the next highest bidder with enough budget wins.
This is easy to explain to users. "to win, you need enough in your budget. If you're losing auctions you can increase your budget."
Implementation wise, you can use a database transaction to handle the "win" operation. Inside a single database transaction, debit the winning buyer's budget and credit the seller's account.
You can use SQL sequences like this:
START TRANSACTION;
SELECT budget
FROM user
WHERE id = nnnn
AND budget >= bid
FOR UPDATE;
/* if you got a result do this */
UPDATE user SET budget=budget-bid
WHERE id= nnnn;
UPDATE seller SET balance=balance+bid
WHERE ID = sssss;
UPDATE auction
SET winner=nnnn, winning_bid=bid,
closetime=NOW()
WHERE auction = aaaa;
COMMIT;
/* if you got no result from the SELECT do this */
ROLLBACK;
You can use distributed locks to solve the double-spending /double booking issue. You need to acquire at least three locks that would update
User's budget i.e. bidder's account balance
Seller's account balance
Auction
You can also create a bid ledger to verify the user's spending, seller account balance, and user's wallet balance. You can run a cron job hourly or every 10 minutes to verify any error and notify support/dev to check for potential errors.
Your logic could be like this
Procedure BidProcessor
input: user, bid, seller
// this should get locks for auction, user wallet balance and seller balance
success = acquireLocks( bid.auction_id, user.id, seller.id )
if success then
hasSufficientFund = user.hasSufficientFund( bid.amount )
if hasSufficientFund then
ExecuteBid( user, bid, seller )
releaseLocks( bid.auction_id, user.id, seller.id )
else
releaseLocks( bid.auction_id, user.id, seller.id )
endif
else
releaseLocks( bid.auction_id, user.id, seller.id )
endif
Each executor can use their identity id as a lock value to avoid releasing someone else lock.
Procedure ExecuteBid
input: user, bid, seller
Description: Run a SQL transaction to update all related entities
START TRANSACTION;
UPDATE user SET budget=budget-bidAmount WHERE id= user.id;
UPDATE seller SET balance=balance+bidAmount WHERE ID = seller.id;
UPDATE auction SET winner=bid.id, closed_at=NOW() WHERE id = bid.auction_id;
UPDATE bid SET processed_at = NOW(), status='WON' WHERE id = bid.id;
COMMIT;
if commit fails then do a rollback
ROLLBACK;
I'm trying to create an application that has the ability to sell gift cards but I'm fearing the consistency of data, so let me explain what I mean:
I have 3 tables
transactions(id, created_date),
cards(id, name) and
vouchers(id, code, card_id, transaction_id).
The database contains many cards and vouchers and each voucher belongs to one card.
The user will want to select a card to buy and choose a quantity.
So, the application will select vouchers according to the selected card and LIMIT according to quantity then creating a new transaction and adding the transaction_id into the selected vouchers to flag them as bought cards for this new transaction.
So, what I am afraid of is what if multiple users sent the same buying request for the same card at the exact same time, then will some data collision occur? and what is the best approach to fix this?
I am currently using MySQL 8 Community Edition with InnoDB engine for all tables.
What I am searching for is if I can create some kind of a queue that activates transactions one by one without collision even if it means that the users will have to wait until they get to their turn in the queue.
Thanks in advance
This is a job for MySQL transactions. With SQL you can do something like this.
START TRANSACTION;
SELECT id AS card_id FROM cards WHERE name = <<<<chosen name>>>> FOR UPDATE;
--do whatever it takes in SQL to complete the operation
COMMIT;
Multiple php processes, may try to do something with the same row of cards concurrently. Using START TRANSACTION combined with SELECT ... FOR UPDATE will prevent that by making the next process wait until the first process does COMMIT.
Internally, MySQL has a queue of waiting processes. As long as you do the COMMIT promptly after you do BEGIN TRANSACTION your users won't notice this. This most often isn't called "queuing" at the application level, but rather transaction consistency.
Laravel / eloquent makes this super easy. It does the START TRANSACTION and COMMIT for you, and offers lockForUpdate().
DB::transaction(function() {
DB::table('cards')->where('name', '=', $chosenName)->lockForUpdate()->get();
/* do whatever you need to do for your transaction */
});
currently I am running into a problem and I am breaking my head over it (although I might be over thinking it).
Currently i have a table in my SQL DB with some products and the amount in stock. People can visit the product page, or order it (or update it if you are an admin). But now I am affraid of race conditions.
The order process happens as following:
1) The session starts an Transaction.
2) It gets the current amount of units available.
3) It checks that the amount to order is available, and it substract the amount.
4) It updates the product table with the new "total amount" value. Here is the code very short(without using prepared statements etc. etc.).
BEGIN;
SELECT amount FROM products WHERE id=100;
$available=$result->fetch_array(MYSQLI_NUM)[0];
if($order<=$available){
$available-=$order;
UDPATE products SET amount=$available WHERE id=100;
}
//error checking and then ROLLBACK or COMMIT
My question now is:
What do i do to prevent dirty readings in step 2, and so the write back of wrong values in step 4?
example: If 1 person orders 10 things of product A, and while it is at step 3, the second person also orders 5 things of product A. So in step 2 it will still get the "old" value and work with that, and thus restores an incorrect number in step 4.
I know i can use "SELECT.... FOR UPDATE" which will put an exclusive lock on the row, but this also prevents an normal user who is just checking the availability(on the product page) to prevent instantaneously loading, while I rather have them to load the page quick than an on the second accurate inventory. So basically i want the read-lock only to apply to clients who will update the value in the same transaction.
Is what I want possible, or do I need to work with what i got?
Thanks in advance!
There are two ways you can address the problem:
You can use a function in MySQL, that shall update the stocks and give an error"Sorry, you product just went out of stock!", whenever the balance after deduction goes below 0.
OR (preferred way)
You can use locking in MySQL. In this case, it shall be a write lock.The write lock shall disable other read requests(BY second person), till the lock is released(BY First Person).
I hope that helps you!
I'm a bit new to coding in general and seem to be struggling to wrap my mind around how to store data effectively for my application. (I'm attempting this in Laravel, with mySql)
I realise this question might lean towards being opinion-specific, but I am really looking for obvious pointers on false assumptions I have made, or a nudge in the direction of best-practices.
I will be tracking a number of messages, as if they were credits in a bulk email management system. One message-batch could use an undetermined number of credits to fire off a full batch of messages.
Groups of users will be able to send messages if they have credits to do so, One solution I have thought of is to have a table: id, user_group_id, debt/credit, reference_code - Where user_group_id would be linked to the group to which the user belongs, the debit/credit column could hold a positive or negative number (of a message related transaction), and the reference_code would track the type of transaction. Debits/Credit transactions would come about where the user_group account received new credits (purchased block of new credits), or in the case of a debits example, where batches of messages had been sent.
All this background leads to my question .. I still don't hold a single value for the available number of credits a user_group has. Before being able to send a new batch, should I be running a database query each time that sums all the "accounting" of positive and negative transactions to determine whether a user is "in the black" to be able to send further message batches, or should I have an additional table and keeps the result of their available credits total separately?
If I do store the total-available-credits value by itself, when should this single value be updated, at the end of every message-related task my application performs ? *user adds new credits, update total - user sends batch, update total.. etc.
This is an interesting question. Opinionated as you pointed out but nonetheless interesting. Database design is not my strong suit but here is how I would do it.
First, ensure integrity with INNODB tables and foreign key constraints. I would keep the total remaining credits each user group has in the user group table. You cold then create a transaction table with a Transaction ID, the User Group ID, and the credits used for that transaction so that you could follow each user group's transaction history.
Just a thought. Like I said, I'm by no means an expert. It may be useful however, to have a log of some sort so that you could verify transactions later in case of a credit discrepancy. You could always use this later to recalculate the remaining credits to ensure the numbers align.
Since these transactions may be important for credit/billing purposes, you may also want to turn off MySQL's auto commit and use the commit and rollback features to ensure your data stays in tact in case of an error.
" should I be running a database query each time that sums all the "accounting" of positive and negative transactions to determine whether a user is "in the black" to be able to send further message batches "
YES
I have another case.
I got the bus with 27 seats
then, if people pick the seat 4, so nobody can take that seat 4.
my question:
how design database contain 27 seats? I guess using looping until 27 with PHP
how to show in form, selection form contain the un-booked seat?
how to prevent if other people take same seat?
thanks.
Database:
Have a BusType table with properties that would include the number of seats,cost,etc.
Have a BusBookedSeats table that will hold a FK relationship to the Bus in question and keep a running total of the number of seats it has remaining and/or booked.
Form:
Query the database and take the number of total seats - the number of booked seats. Add further validation to show the location of the seat.
Prevention:
If the query returns a value greater than 0 (meaning taken in this instance) warn the user that it is taken and inform them to take another seat. Or don't even display the seat to them.
For your database:
You have a table with busses and available seat numbers
You have a table for reservations
You have a table which has a relation with a reservation on a specific seat on a specific bus
How to show the available seats:
Do a query for all reserved seats
Draw a picture of the bus and mark all reserved seats
How to prevent double reservation:
You need some kind of transaction routine but it is hard to tell, when to lock a seat and when to unlock it (e.g. a user doesn't responds for some time). Usually you only reserve it at the last step. So making sure you have not too many steps or the selection is one of the last steps, you reduce the number of conflicts.
These are indeed only some very basic tips, but the whole problem is too complex to anwser it in short. If you have any specific questions you might want to aks them separately.
From a DB Perspective, based on what i understand of your requirments, i would suggest something on these lines :-
Bus Table - containing an entry for each individual bus. Has a column that indicates the seating capacity of this bus.
User Table - containing an entry for each user who can book a seat on a bus.
BusUser Table - transaction table containing a record for seat on a bus eg: if bus X has 27 = seating capacity, this table will have 27 records - each with the corresponding seat number associated with it.
From a UI perspective, you display the bus user contents. When a user selects one of the records on the UI, you associate the user with that record in BusUser table. BusUser - will have a column indicating the user associated with that bus instance.
Thats the general idea. Hope that gives some pointers for you to start off on..
Make sure you have some sort of a primary key which involves the time of the bus leaves etc along with the unique identifier for each bus (composite primary key). You can only have the unique identifier of the bus as that will limit each bus to be only ride once :)
The 27 seats of a bus i think should be handled by your application. Perhaps, insert 27 rows first for each bus and update when someone books a seat. This way, you can let the user perhaps select which seat they want as well. (Should be done from your GUI).
Of course you should have a status for each seat which you can use to find out whether it is reserved or not.
This is a VERY basic idea and I have not included any diagrams or anything of that sort. I hope I have not spoiled your opportunity to design the database on your own :)
A database lie In Sane has suggested is a good idea. Stopping duplicate bookings really isn't that difficult - you just need to choose a method.
Queue system - When people order you'd append them to the tail of a queue and then have a seperate script to pull people from the head. You'd then be able to evaluate each order at a steady pace, one by one.
Database level - If you're going to use MySQL you'd probabaly be better off using a engine like InnoDB which supports transactions (introduction: http://www.databasejournal.com/features/mysql/article.php/3382171/Transactions-in-MySQL.htm)