Hopefully I can do this without writing a book... I'm working on a system for a Surveying company. They're also throwing in an accounting section, and one of the functionalities it needs is to find jobs that are not completely paid off.
A few points to be made:
A 'job' is a project, essentially. There are different types of surveys.
A job can have more than one "type" of survey in it, thus having more than one price that is calculated into the job's total price.
Payments are made on individual jobs (the client might send 30 checks if they have 30 jobs, it's an accounting thing)
Job types are dynamic (they can create/delete them from the system panel) - so I can't hardcode these things
Here's the applicable database structure:
table jobs: job_id, client_id
table job_types: type_id, type_name
table job_type_assoc: id_job, type_id, price
table payments: payment_id, job_id, amount
Note: payments aren't made on each job type, but the JOB as a whole (again, as opposed to the client's account having a "balance").
I need to somehow pull jobs where the total of price from job_type_assoc is less than the total of amount in payments.
I don't know if this is possible in mysql alone or if php would be more efficient - also, their old system has about 340,000 jobs in it. Granted that they won't have the accounting info from then, they do have a lot of work and the new entries will build quickly, AND if I somehow do it in PHP where I end up querying the jobs table a lot, it might get messy.
One possible way you can do this is with a clever combination of subqueries.
select * from
(select job_id,
(select ifnull(sum(job_type_assoc.price),0) from job_type_assoc
where jobs.job_id = job_type_assoc.id_job) as amount_due,
(select ifnull(sum(payments.amount),0) from payments
where jobs.job_id = payments.job_id) as payments
from jobs) as outstanding
where payments< amount_due;
This shouldn't be too bad a query to run, if you have properly indexed job_type_assoc and payments.
Please note that I haven't tested this exact query, so it might need tweaking. I've run the same query against tables on my local DB.
Related
I m looking for advices in report generating per client, per product, per salesman, in a given period, using mysql and php. I m currently working on an application with sales and i was wondering what is the best way to structure this function. I m planning on hosting in a shared hosting server, so i m worried about efficiency. So here are the two ways i m looking things:
1- add entries of sales then once a user specifies a period of the sales report, executing a mysql query that will sum up everything then feed it to the php to arrange it a send response back
2- fetch all entries then calculate sums and everything in php
3- on every sales, add the totals on a report table for each day, then another table for each client, then for each product then for each salesman. And once a user requires a report, fetch the totals in each table and sum up the totals (the problem here is if a sale has been edited or deleted, i have to re-adjust the totals for all tables)
I m looking for suggestions for the best way to do this, also all the data are on the same table and segregated by user_id so for example if there are 1000 users with 1000 sales, the table sales will have 1000000 rows with corresponding user_id.
Thanks in advance
We run a high transaction volume insurance system.
The goal is to update a single flat table of 200 columns on any changes in a customers profile.
The customers profile is stored in a complex model of around 400 tables.
e.g If the customer makes a single premium payment, it goes across 15+ tables but we want the flat table to update say 5 out of 200 columns to reflect some representative data off this payment for the customer
We are considering using triggers, that is not working out well in testing as the current MySQL server (AWS Aurora) is not able to deal with it.
The main application is PHP Code-igniter
Any one has any suggestions that can help?
"If the customer makes a single premium payment ..." --
That should touch about 2 tables.
A record of the payment (who, when, how much; not much more)
Some kind of status for the user.
The first of those tables is a running "log" of what happened. The second is the current state of things.
Having 400 tables or 200 columns, or touching 15 tables for a simple operation -- each of these indicate poor design.
It is OK, even normal, to have a few (not 400) JOINs to reconstruct things on the SELECT side.
I am developing a MySQL DB for an online jewelry shopping website. Every jewelry may have different stones, metals, … Their net prices will change everyday (e.g., every ounce of gold may be X today and Y tomorrow). Product A may be more expensive than B today, but cheaper tomorrow. Every time a user requests the products list, the website automatically calculates the price of every product.
How can users sort products by its price, considering that net price is not stored in a field in DB and every time will be calculated (using weights, net fees, …) by a function named func($id), where $id is the id of the product.
An illustration to the question:
Assuming you have all of your data in your DB, there is no reason you can't ORDER BY any kind of expression you want.
If, say, you had a table products (pid, name), productContents (pid, type, amount), and contentPrices (type, price), you could run something like
SELECT a.name, SUM(b.amount*c.price) AS totalPrice FROM products a
LEFT JOIN productContent b ON a.pid = b.pid
LEFT JOIN contentPrices c ON b.type=c.type
ORDER BY totalPrice DESC;
EDIT to work with the updated question schema:
SELECT p.title, p.metal_weight*m.fee + p.stone_weight*s.fee AS totalPrice FROM Products p
LEFT JOIN Metals m ON p.metal_id=m.id
LEFT JOIN Stones s ON p.stone_id=s.id
ORDER BY totalPrice DESC;
I would do it in two ways, but first of all, if you want a good performance, store the price value in a Field. I don't think it changes every second, does it?
So first, if you can make your Price calculator function in MySQL, create a Stored Procedure for it and run this on the table periodically (for example every day at 00:00) and store the value in a field. You can find how to schedule an event in MySQL here.
If you can't do this, stay with php, but it also should called periodically to calculate everything. You can do it with cron on Unix (if you have the rights). This can be scheduled on a remote server too, so if you have somewhere a server where you can use cron, simply call the updater.php with wget, for example:
wget -c http://jewerly-shop.com/update_prices.php && rm -rf update_prices.php
But if you need further help, we need more info (like the function that does the calculating and the table structures).
I am building a platform on php/mysql that allows users to but for a percentage of a product. Basically they choose what percentage they want to bid for and the markup price they'd like to bid at e.g. 1.2.
The issue I have is that if there are several bids placed simultaneously for the same product, I need to queue the bids so when processed only the valid ones go through. For example, if there is 20% available at a value of 1.2 and two users simultaneously bid for 20% at 1.2, the process for each bid would be:
1----- select current bids
2----- php processing to work out if bid is still available
3----- place bid
The issue here is if both of the 'check if availables' happen before either of the 'place bids' then both bids will go through and effectively cause a stock issue.
Is there a way of queuing this process so the whole of steps 1-3 happen for a bid before the next one can run.
My initial thought was to use a cache table to store all the bids and then run a cron every few seconds which will process in order of ID from the cache table and notify accordingly but this seems like it may not be viable on a high traffic system as the server would get hammered over it.
Any help much appreciated
If two bids can never be identical I would solve it at the data layer. Create a unique key on your bid table based on item_id, bid_value and whatever else needs to be unique. The second insert will fail and you can let user know they were beaten to the post
I solved this at PHP level by injecting the bid as an 'accepted bid' first so that any subsequent bids would fail. Then I ran the check to see if the bid was ok, if not the bid gets removed, if it is the table gets updated as required in place for new bids.
I'm actually worried about designing the perfect tables to handle transactions and commissions. especially that we really don't want tables to be out of sync in any case.
Basically this is like ebay, where merchants sell to buyers and we get a commission out of each transaction.
The app is already built using Codeigniter, and it has:
Merchants(id,name,balance)
Transactions(id,merchant_id,buyer_id,amount,transaction_id)
Now what other tables to make if needed to make us able to fetch our revenue and the seller's revenue, and what's the best DB mechanism to process a refund?
And should we make another table of each deposit to the merchant's balance rather than +x the balance row.
Whenever you are dealing with any financial tasks, always store as much information about it as possible.
There are two ways of keeping all your tables in sync.
Database level - using options such as stored procedures and triggers
Application level - Use transactions (innoDB) and make sure that the transactions are atomic. (I would suggest to go with this)
Regarding the database design -
Add a column commission to Transactions that tracks your commission. Data for this can be either calculate beforehand when pricing the item or on the fly during the transaction.
The buyer pays product cost + commission, seller gets product cost and you get commission. If your commission is a variable (not a flat rate and changes from item to item), you could have a column in the Items table which stores it. Things should be trivial from now on. You just have to maintain a History table that keeps a list of transactions with respect to the user. It would look something like
AccountHistory(id, transaction_type, transaction_id, amount)
You could split AccountHistory into two - one for incoming money and one for outgoing money.
So everytime a transaction takes place the following happens
Entry into to Transactions table.
Entry into AccountHistory for buyer
Update buyer balance
Entry into AccountHistory for seller
Update seller balance
All this should be atomic.
For rollbacks, you could add a column transaction_type to distinguish rollbacks from normal transaction or you could maintain a separate table.
Having worked with a database where the design had financial transactions scattered around a bunch of different tables, I can tell you that it is much better to have all transactions in a single table. You should add transaction type to your transaction table (and a look up table for the transaction types).
Then each transaction can have a separate line for commission and seller product cost. Transaction that are undone later would get entered in as negative amounts. It is also helpful for these types of things to add a parent transaction field so that you can easily identify which transaction it was changing.
All data entry that forms part of a transaction should be in a stored proc and have transactions so that if one part of the insert fails, the entire thing is immediately rolled back. This is critical. It needs to be in a stored proc so that no one (except systems dbas) has direct access to tables that store financial records.
When you start to think of handling financial transactions the very worst thing you can do is design for the user interface alone. You need to design for the financial reporting you will need to have. You need to consult the people who will be auditing your finances to see what information they need and what internal controls they expect you to have in place to prevent internal as well as external fraud.