I hope my question is suitable for this site and is not too broad but I am having trouble designing the following architecture:
I have two sites. Site 1 is responsible for transferring credits between users. Site 2 is responsible for providing these users with services/products which can be paid with the credits they own in Site 1.
Lets say I have 1000 credits on Site 1. I have a service/product which costs 50 credits on Site 2 and a user wants to purchase it with his amount of credits he owns in Site 1.
Both sites communicate with REST. So for example when a user wants to purchase a service/product Site 2 prepares its request and sends it to Site 1 which makes the transaction and confirms to Site 2 that the transaction was successful (e.g. the user had enough credits for the service/product and those credits were successfully transferred to the destination)
Now here's the tricky part. In Site 1 I have the following logic:
Begin transaction
update user set credits -= 50 where id = 1
update user set credits += 50 where id = 2
REST CALL (Site 2) Success
Site 2 response - OK, commit transaction
Commit
Since REST is a call to a different site the transaction might take some time to complete. In the mean time is the whole table locked for any transactions or are the rows for user 1 and user 2 locked? Is this the proper way to implement my logic? Am I missing something?
Thank you in advance.
This is in response to your question on Casey's answer:
Yes, as long as you do it like this:
Site 2:
Customer logs in.
Ask Site 1 for credit total & transaction history (GET request) for this user (user 1).
(Any awaiting transactions which receive 'transaction succeeded' responses are made available for download/dispatch)
Use credit total to enable "Buy" buttons for things that can be afforded.
Customer clicks a Buy button
generate a transaction ID unique to site 2, store in database along with details of who bought what, when they did so, with state = pending. tell user transaction has been received and they should be notified soon whether it was successful (HTTP 202 response)
POST purchase request to Site 1, including authentication (don't want forged requests to cause people to spend money they don't want to spend) and the transaction ID.
Site 1
validate authentication
verify that Site 2's transactionID has not been used before by site 2. if it has, return an error, if not:
Begin transaction
update user set credits -= 50 where id = 1
update user set credits += 50 where id = 2
insert into transactions remoteSiteID = 'Site2', remoteTransactionID = tID, user = 1
You would not need the remoteSiteID field if site2 is the only site using credits from site1
Commit
REST CALL (Site 2) Success
Site 2:
EITHER:
1. Receive REST success call, make purchase available for download/dispatch, display some message to user saying purchase processing complete. Update local transaction record, state=succeeded.
OR
2. Site 2 is down. Transaction success will be noted next time background polling process runs (which checks status of purchase requests awaiting responses) or next time customer logs in (in which case poll is initiated too--step 3 in first list)
If you have not received a response to a transaction, perform a GET using the transaction ID. If the response is an error, Site 1 did not receive the original request, Site 2 is free to repeat the transaction (POST) request. If the response is 'transaction failed' then the user didn't have enough credits, update transaction record on site 2 accordingly. if result is 'transaction succeeded' record that too.
if a transaction fails N number of times, or a certain period elapses since the user clicked the button (say 5 minutes) then Site 2 stops retrying the purchase.
Since you're using primary keys and not ranges, it should be row-level locking. There's also the concept of a shared vs exclusive lock. Shared locks allow other processes to still read the data while an exclusive is used in an update/delete scenario and blocks all others from reading it.
On the logic in general.. if there's really only one place storing the credits and one place reading them, how important is it to sync in realtime? Would 3, 5, or 10 seconds later be sufficient? If Site 1 is completely down, do you want to allow for Site 2 to still work?
Personally, I would restructure things a bit:
A user creates an account on Site 1.
The first time a transaction is done on Site 2, it validates the account exists on S1 and gets the number of credits.. and keeps it.
Whenever a transaction is done on Site 2, you check the local credit count (cache) first and if there are enough credits you return a 202 Accepted response code. It basically means "hey, we accepted this but aren't done with it yet."
You can immediately allow the user to continue at this point.
But at the same time you did the local credit check, you made another request to S1 with the actual transaction.
Hopefully, that service can give you a success/failure message and the update definitive/official credit count.
Using that, you update the local transaction status to 204 NO Content (success) and update the cache for the next time.
Since S1 is always returning the definitive current credit count, you don't have to worry about maintaining the credit count on S2 in a 100% accurate way.. you can just wait for it from S1.
If you're really nervous about it, you could have a job that runs every N hours that polls S1 requesting updates on every account updated during that N hours.
Related
I have a question about concurrent request a user from different browser
Imagine we have balance in wallet for just buy one product, if a user request a product at the same time with two different browser, can user buy a product two times? if it is possible, how can I prevent the second action?
example:
user A balance : 100$
user A ----> mozila ----> request ----> product A50(price 100$)
user A ----> chrome ----> request ----> product A50(price 100$)
above request happen at the same time after that some process the amount of wallet decrease
You should perform these operations in SQL TRANSACTIONs having an appropriate isolation level. All of the operations performed within the transaction will be "all or nothing," which means that all of the changes take effect if you COMMIT and none of them do if you instead ROLLBACK. Furthermore, if two transactions attempt to touch the same row, one of them will either be forced to wait or will be turned away. Also, the other transaction will not see anything that hasn't yet been committed.
For instance, if you want to "deduct money from the user's account and apply it to an order," you would perform both updates in one transaction. So, "if everything worked, both updates happened instantaneously." And, "if it didn't work and the transaction was rolled back, nothing changed anywhere."
But it's important that you also test the user's balance within the same transaction! (Otherwise, there would be a "race" between testing the balance and proceeding with the sale.) Your logic might be something like this pseudocode:
BEGIN TRANSACTION with proper isolation level
Retrieve user's account.
If there isn't enough money:
ROLLBACK
exit
Else:
UPDATE user account to withdraw money.
UPDATE the invoice to show payment.
INSERT a new entry into the (financial ...) transaction log table.
COMMIT
This works as intended because the entire set of operations that occurs within the transaction is "atomic."
SQL servers vary slightly in their implementation of transactions but here is a web-page on the topic (covering MS SQL Server):
https://www.sqlserverlogexplorer.com/types-of-transaction-isolation-level/
I've created a IPN code that allows paypal to update my database for a item purchased. I am thinking ahead though that the person buying my item, might not confirm upon checkout, or that while they are confirming check out, someone else might click the buy it now button as well before it's confirmed via paypal.
The point to all this is, I want to figure out the best way to fix this. My idea is after they click the buy now button it sets the item to pending, and they have 5 minutes to confirm the purchase or it's sent back to available.
Please help.. I can provide my code if needed.
I'd suggest using something like Resque - there is a PHP port (https://github.com/chrisboulton/php-resque).
When the user goes to PayPal, set the status of the product to be 'reserved', and create a Resque job that's scheduled to run 5 minutes from now.
If the user buys the product, change the status from 'reserved' to 'sold'.
When the Resque job runs after 5 minutes, it checks whether the status is 'reserved', and if it is, it changes it back to 'available'.
If the status is 'sold', then it does nothing.
If you don't want to use Resque, then you could do something a bit more simple like a cron job that selects all products that with a status of 'reserved' where the status was set more than 5 minutes ago. Then you change that status to 'available' again. You can run that cronjob every 60 seconds.
However, this can get into complicated race conditions, and it will be less flexible, so I'd implement a queue-based system if you can.
First sorry that I cant comment, cause of my lvl here. So your idea with locking it is user friendly and 5 minutes would be okay. But I would PayPal update my stock, when the payment is finished. who pays first... But I would program a ajax script which checks the stock for the user, who has this item in the basket and when the stock changed to 0 i would do an alert message like
alert('Item Xy changed his state to 0');
I'm developing an order system with PHP + mySQL. There's a high chance that multiple users will create new orders with same product at the same time.
This is the scenario: in MySQL DB, there's a table named "inventory", in which all products' item numbers and available quantity are listed.
Now,
Product A's inventory is 500 pcs.
User1 is working on a new order with 300 pcs of product A, but he hasn't submitted yet. However, there's an AJAX script to send a query to table "inventory" and since the inventory qty is 500 pcs, so it returns 300 pcs as available for this order.
User2 is working on another new order now, while User1 still hasn't submitted his order. User2 wants 400 pcs of product A, and since 300 pcs from User1 has't been submitted, AJAX function returns 400 pcs are available for User2's order.
Now here's the problem. Either User1 or User2 submits his order first, the another user will not have enough quantity to fulfill his order as the result returned by AJAX.
I want to know what is the best (or normally) practice to handle this type of task.
I have thought about PDO transaction handling (not sure if it will work, have never used it, I'm quite new), or maybe should I just forget about AJAX but do the inquiry when user submit the form?
Also I had thought about putting a "check" button on the page then after the user finished inputting the order, they hit "check" to send an inquiry to DB then get the most 'updated' quantity, maybe it's soon enough before the user hits submit.
Thanks.
Thwo things you have to understand first:
There is no such thing like "PDO transaction"
Transactions are inapplicable here anyway.
What you actually need is simple reservation system. Or a notification one.
In case of reservation, you may mark 300 items as "reserved". Then all others may either content with remained 200 or wait till reservation gets released or stock refunded.
While an example of notification system you may see right here on this very site: in case there is a user who is working on the answer, and question happened to be closed, an ajax notification is shown to the user, saying he has not enough items to reserve anymore.
there are 2 ways:
1) you reserve the stock as user1 puts it in their basket
(and deal with timeouts / abandoned orders and moving stock back to inventory and a way to notify the users when items come back in stock)
this means that user2 can't add them if user1 has them.
(you could physically remove the number from the inventory, or you could add another column with 'reserved' in it - e.g. 200 in stock and 300 reserved)
2) you do the stock check + reserve when the user has begun the checkout process and notify the users if the items aren't in stock and you don't have to deal with abandoned orders as they haven't reserved the stock yet.
I have developed e-commerce systems based on both, and #1 is, imho the best - as a user I want to know if the things are in stock as I add them to my basket
This isn't a matter of what you're using to make your query (though using PDO is still a good idea.) When the user finally submits their order, your php script should perform an extra check that the requested amount is in stock. If not, the user should presented with an error that the amount of product is no longer avaiable.
Use ajax to check the stock periodically so the user gets a javascript notification when something in their order is no longer available.
I'm setting up a simple 'buy now' transaction from a website with these major steps:
Select product from price list
Review selection (amounts, tax etc)
Process Payment on Paypal
Receipt / Thank you
At the moment, i'm storing a database record in step 2 - which potentially means there will be a number of records where no payment is received as people decide not to go ahead with their purchase after all. These records are of no real use since i'll use Google Analytics to track how successful the checkout flow is.
I'm using Paypal IPN to verify the authenticity of the payments and log them against the records inserted at step 2 - however, could I feasibly rely solely on the data from the IPN transactions to populate the database in the first place, thus removing the need to store them at step 2 and have to do database cleanup to remove transactions that never completed?
I personally can see no reason why I wouldn't - the IPN contains all the data I need about the payment and probably more besides, and Paypal will resend IPNs for several days if they don't go through first time due to server glitchery, but am I missing anything else important?
Obviously the number one consideration is that no transactions get lost or aren't logged so that no customer unhappiness ensues!
It's important to do a 2 way validation like you have.
You save the order info (total, quantity) before the user leaves your system towards paypal. When ipn come back you validate the request (it must be from paypal ip or whatever), you validate that it's a successful transaction then your step 2 enters the scene. You validate if the total returned from paypal ipn is the same with the total that was saved before the user left (Paypal sometime may return partial payments, the user may grab the post data and do his own post from a modified html with a lower total set). Step 2 should also store the user_id of the buyer so you must compare that too.
here's a sample layer (no programming language just a dummy code):
if request comes from paypal:
# query the order
if order.total == request.total && order.user_id == request.custom:
payment may come in...
As the designer and administrator of a system that has processed over 600,000 PayPal payments in the last three years - relying exclusively on IPN will allow some errors to slip through the cracks.
Real data:
Total transactions No IPN Invalid IPN Duplicate IPN
year 1 170,000 + 2 101 0
year 2 205,000 + 54 15 3
year 3 230,000 + 20 24 13
Fortunately, our system is structured with PDT (Payment Data Transfer) as a 'backup' so we didn't lose any transaction data or have unhappy customers. Note: PDT can't be relied upon exclusively either - in fact, early this year, there was a major problem with the reliability of PDT returns.
The most common 'invalid' IPN returns are an HTML error page or truncated results ... I can provide samples if desired.
The best choice is a combination of both IPN and PDT (with your 'cart' data stored in your DB as you are). Either the IPN processs or the PDT process can create the transaction (and delete the 'cart' data record in the DB). The second process to arrive will then not have a 'cart' entry from which to record a transaction.
NOTE: - as you noted in your final solution to use a custom field - be aware there is a length limitation to the custom field and it can be truncated upon being returned to you.
I have not relied solely on IPN to do this, but PayPal will log failures to contact your server if it fails and is supposed to retry later, although I only ever had failures in development and never verified the retry. I just trust them on this one.
For a typical e-commerce site, yes you can -- it's fairly reliable. If nuclear reactors will melt down and people will die, then no you can't -- I've seen problems with it, but very infrequently.
I have developed a number of eCommerce sites, and in practice you always want to record what you can in case of any 'accidents'. You own data is probably more informative.
Like you said, yes you can do this, but I would suggest that it is not a great idea.
I'm creating a simple online shop with PHP integrated with PayPal that sells unique items. What I'm wondering is how other shops deal with multiple people attempting to go through the payment process with the same item.
This is my current draft strategy:
Items have three status types: available, on hold & sold.
As a user moves to the PayPal payment page it checks the status of all the items in the cart to ensure they're available. It also sets the item's status to "on hold" until they either come back after payment is confirmed or it times out (10 minutes? Not sure what this should be).
Is this standard practice or is there a more practical way I should be going about this?
Thanks in advance!
Have a look at Dell's UK outlet. When someone adds a system to their shopping basket it is held and not available to other customers. If it isn't purchased, the item is removed from the basket after 15mins of inactivity and is then available to other customers.
I would say the first part of your strategy is correct - as you move to the payment page, flag all the products as 'on hold'
When the user has finished the payment, you will get a postback from Paypal which lets you know if the authorisation was successful or not (and possibly also allows you to check the CSC/CVV2 result), and at that point you have the option of either accepting the payment, or rejecting it.
On receipt of the postback you should also check whether the items are still on hold. If they have timed out you can reject the payment and display a message 'sorry - timeout exceeded' or somesuch.
This method also allows you to work out an ideal timeout period if you keep track of how often customers run into the timeout, so you can extend the timeout from (eg) 5 to 10 minutes if too many are timing out, or shorten it if none are timing out.
This is a fairly common issue with fixed inventory systems such as venue, transport/airline tix etc.
I like the airline model where once you get the itinery you want and click select, you get a page with passenger info with a message saying, seats on hold and you now have xx(10/15) minutes to complete the purchase. Everything becomes explicit at that time. For other unique/one of a kind item, I'd think a message on any page, that the user clicks on, saying you have xx(mins) remaining to complete the purchase would be a big motivator for "on the edge" buyers !
woot.com is notorious for this problem but their solution works well. After payment information is verified the user is brought to a page with a small amount of text saying something like "your order is in, we are verifying inventory".
It looks like the paypal API has a "RefundTransaction" message so something like that might not be possible. But the user experience could be awkward if your going to the paypal website then coming back to your site.
This is much like booking theatre tickets or similar online and yes the way you describe is generally the way it works. At some point the item is "reserved" in the system and either the customer completes the transaction or the item is released after some time for others to buy.
Of course at what point you reserve the item (when it's added to the cart, at the point you send them off to pay etc) is up to you. I would expect putting it in the cart would be the best choice as it makes it less likely someone will build up a basket of stuff only to find half of it is no longer available at the checkout.