I built an application to keep track of the sales. In my customers view, I want a column with total sales per customer, but as the customer base is growing, the list is loading more and more slowly. This is what I did (simplified):
Controller:
$customers = App\Customer::get();
View:
#foreach ($customers as $customer)
{{ $customer->name }} {{ $customer->totalSales() }}
#endforeach
Model:
public function totalSales()
{
$invoiceLines = InvoiceLine::whereHas('invoice', function ($query) {
$query->where('customer_id', $this->id);
})->get();
$sales = $invoiceLines->reduce(function ($carry, $invoiceLine) {
return $carry + ($invoiceLine->quantity * $invoiceLine->pricePerUnit);
});
return $sales ?: 0;
}
What would be the best way to make this view/report more "scalable"?
I have been thinking in creating command that calculates the total sales per customer overnight and put the result in the customer table, but that means that the numbers won't be accurate during the day...
this seems like a very interesting problem.
I have been thinking in creating command that calculates the total
sales per customer overnight and put the result in the customer table
this is a good option.
but that means that the numbers won't be accurate during the day...
You can keep the numbers accurate by doing the following:
by incrementing the customers table count every time a invoice is made.
This should work for total sales.
Make sure you have an index on the customer_id column.
Search for ways to do a "SQL SUM on 2 columns using laravel".
Try and find some way to do "SQL SUM on 2 with a GROUP BY. Doing this will replace #2
A good way to speed up your application is to avoid making calls to the database in a loop. That is what #3 is suggesting (the loop in this case is the #foreach in your View and the database call is the InvoiceLine::...->get(); in totalSales()
Adding the index (if missing) and reducing the # of calls to the DB will yield the best results.
I have limited knowledge of Laravel but one way to do this with raw SQL would be:
SELECT c.name, ts.totalSales
FROM customer c
INNER JOIN (
SELECT customer_id, SUM(quantity * pricePerUnit) as totalSales
FROM invoice
GROUP BY customer_id
) ts ON c.id = ts.customer_id
You can see how all the data you're trying to print is pulled at once? I assume you'd want to try and write that using Laravel's Eloquent thingy.
Based on the answers above, I came to the following solution:
I created an event: App\Events\InvoiceSaved which is dispatched every time an invoice is "touched" (created, updated or deleted). The App\Events\InvoiceSaved event will calculate the total sales for the customer and add the result to the customers table (extra field total_sales). Now I can just query my customers table and need to query a relation. The loading time dropped from 7 seconds to 0.5 second!
Related
In the application, we have a feature where it will display all the years as tab. Each tab has amount(where the amount is the summarized) but at the same time it is have data.
Scenario
Just to give a background about the application, it is a car seller
Imagine you are the customer, you started to use this application way back 2019. You clicked this module to check the summarized of each year
Let's assume we have 2500 data for each year.
In current year I want to display the total amount of all previous years
2023 tab is the active, from 2019 - 2022 you have sold 300,000 each year
In getting the previous amount, I used the DB raw sum function to get the summation.
Question: Does DB raw sum function lessen the heavy query to get the summation?
You can accomplish this with eloquent, without having to load all of the inner relationship data.
Normally programmers do the following:
Model:
public function children()
{
return $this->hasMany(Children::class);
}
In the Controller:
$models = Model::with('children')->get();
Then they load the data normally in blade and count:
#foreach($models as $model)
<p>{{ $model->children->count() }}</p>
#endforeach
Instead you can do the following:
In the Controller:
$models = Model::withCount(['children'])->get();
In the blade you load the data with the count:
#foreach($models as $model)
<p>{{ $model->children_count }}</p>
#endforeach
The withCount method will place a {relation}_count attribute on the resulting models
Use Barry DebugBar to test out the speed with heavy data, you will notice that doing it this way will reduce heavily the time needed to load the relationship, since you are not loading all of the inner relationship, rather you are just counting the inner relationship and displaying it.
For withCount Reference: https://laravel.com/docs/9.x/eloquent-relationships#aggregating-related-models
For withSum Reference: https://laravel.com/docs/9.x/eloquent-relationships#other-aggregate-functions
You can accomplish the same thing using withSum() and you can do this for deeper level of relationships as well.
Create and maintain a summary table, with year, and the sum for that year.
Working from that table, the query would be much faster, even if you had to do multiple summing.
Or, that table could have the cumulative amount since day-1. Then, the sum for "prior to 2023" is "cumulative through 2022" minus the "cumulative through 2018".
Summary Tables
I have 3 tables (customers, invoices, invoice_items).
A customer can have multiple invoices and an invoice can have multiple items.
This is my current schema:
customers
id
name
timestamps
invoices
id
customer_id
timestamps
invoice_items
id
invoice_id
price
timestamps
Note: timestamps are created_at and updated_at fields. No need to worry.
as you can see, I am not storing the total amount of an invoice in the invoices table. I let the front-end calculate it.
Now, my client has 1000+ customers and they want to show all customers in the homepage along with the total sum of all invoices they have.
ID | Customer name | Total Invoice (sum of all invoices)
Right now I am using Laravel, so my query looks like this:
$customers = Customer::with('invoices.items')->get();
What's the best way to do this? Performance wise?
Should I just store the total amount of each invoice in the invoices table to avoid querying the items and calculate the sum? Something like:
invoices
id
customer_id
total
timestamps
Imagine a customer having thousand invoices and each invoice has multiple items. The query will be very slow and I have to calculate every thing in the front-end.
Also, when an invoice has been made, it can't be changed. So the total for an invoice is static.
Ofcourse, when you add a total column on invoice table then you should get a boost in the performance, as you will not have to go through all the invoice_items and add them up. This is good if you need to add items to the invoice only at once and do not need to be updated later. This way, you will need to add the total while generating the invoice at once.
It looks good apart from the timestamps.
Can you not just use one timestamp column in your invoices table?
Why do you need the same column in all 3 tables?
Also, I'm sure you can calculate total in your php script.
You don't need to store the total that you can calculate through a count() in php.
Adding total column in invoice table will be definitely good for performance and it will also help fetch result through eloquent. However, if performance is not something important here, we can do it without it. Your SQL query should be like (might need little adjustment)
SELECT c.*, i.*, sum(ii. price) as total_invoice_price
FROM customers c
INNER JOIN invoices i on i.customer_id = c.id
INNER JOIN invoice_items ii on ii.invoice_id = i.id
GROUP BY i.id
Once you have expected result through SQL, you can run it using DB::table in laravel.
This is classic case of WORM (Write Once Read Many).
Lets look into pros and cons of both the approaches:
Case 1: Not storing the total in invoices table
Pros: Simple and easy. No data redundancy.
Cons: Slow performance and consuming resources(memory and cpu) every time you compute the total
Case 2: Storing the total in invoice table
Pros: Better performance because "complex" invoice total computation
happens only once and subsequent reads are faster
Cons: Data redundancy and requires some extra storage
Now if you look into the cons of both the approaches, case 2 is obviously better because extra storage can be bought easily but performance can't be even if you increase RAM and CPU upto some extent.
Quick question to which I haven't been able to find a solution for myself, nor by googling it.
On my dashboard view I show my payment plans, simply by doing $plans = Plan::all() and I return that to the view. Nothing fancy there. But, on my actual dashboard view, I want to show the total amount in dollars from the transactions that have been done. So on my view, I do the following (stripped example):
#foreach($plans as $plan)
{{ $plan->paidAmount() }}
#endforeach
Which is using the following method from my model:
private function paidAmount()
{
return $this->transactions->sum('amount');
}
Which basically just grabs the total amount of all related transactions to that plan, and displays it on the page. However, this generates a new query for each of my plans. Let's say I have 5 plans, it will generate 5 additional queries for each amount I need to be displayed.
Same goes for the remaining amount that I want to display:
private function paidAmount()
{
return $this->amount - $this->paidAmount();
}
Which will generate an additional 5 queries. So for a simple table in which I display some information,a whopping 11 queries are being executed.
Is there any way that I can include the paidAmount and remainingAmount in my original query? I know there should be a way, I just haven't been able to find it yet. So just something along the lines of:
$plans = Plan::with('paidAmount')->get();
You can use sub queries inside your main query Like this to calculate the sum
Like this -
$plans = Plan::all('*', DB::RAW("(SELECT SUM(amount) FROM transactions WHERE plan_id=plans.id ) as amount") );
This way you will already have sum when rendering in views
I am using Laravel and have a music table, an order_items table and a users table. The music items all have a user and a user can have multiple music items. I am trying to get the top 100 selling music, but I want to return only one music item per user (the top selling one of all their music sales).
My code is:
$items = DB::table('order_items')
->join('music', 'music.id', '=', 'order_items.music_id)
->groupBy('music_id')
->select('order_items.user_id', 'order_items.music_id', DB::raw('count(*) as sold'))
->orderBy(DB::raw('sold_count'), 'desc');
$items = collect($items)->unique('user_id')->values()->take(100);
This does seem to return the data that I need, however the first query returns more than 35K records so takes a while to run. I can place a limit on the initial query however then I can not guarantee that there will be 100 items once they are grouped by user.
Is there a better way to do this? Can it be run in one query?
I've been working on a shopping cart application and now I've come to the following issue..
There is a User, a Product and a Cart object.
The Cart table only contains the following columns: id, user_id, product_id and timestamps.
The UserModel hasMany Carts (because a user can store multiple products).
The CartModel belongsTo a User and CartModel hasMany Products.
Now to calculate the total products I can just call: Auth::user()->cart()->count().
My question is: How can I get the SUM() of prices (a column of product) of the products in cart by this User?
I would like to accomplish this with Eloquent and not by using a query (mainly because I believe it is a lot cleaner).
Auth::user()->products->sum('price');
The documentation is a little light for some of the Collection methods but all the query builder aggregates are seemingly available besides avg() that can be found at http://laravel.com/docs/queries#aggregates.
this is not your answer but is for those come here searching solution for another problem.
I wanted to get sum of a column of related table conditionally.
In my database Deals has many Activities
I wanted to get the sum of the "amount_total" from Activities table where activities.deal_id = deal.id and activities.status = paid
so i did this.
$query->withCount([
'activity AS paid_sum' => function ($query) {
$query->select(DB::raw("SUM(amount_total) as paidsum"))->where('status', 'paid');
}
]);
it returns
"paid_sum_count" => "320.00"
in Deals attribute.
This it now the sum which i wanted to get not the count.
I tried doing something similar, which took me a lot of time before I could figure out the collect() function. So you can have something this way:
collect($items)->sum('amount');
This will give you the sum total of all the items.
you can do it using eloquent easily like this
$sum = Model::sum('sum_field');
its will return a sum of fields,
if apply condition on it that is also simple
$sum = Model::where('status', 'paid')->sum('sum_field');
You can pass this as UserModel attribute. Add this code to UserModel.
public function getProductPriceAttribute(){
return $this->cart()->products()->sum('price');
}
I assume something like this:
UserModel has a one to many relationship with a CartModel named cart
CartModel has a one to many relationship with ProductModel named products
And then you can get sum price of the product like this:
Auth::user()->product_price
Since version 8, there is a withSum method on Eloquent, so you could use this.
Auth::user()->withSum('products', 'price')->products_sum_price;
This won't load all products into memory and then sum it up with collection method. Rather it will generate a sub query for the database, so it's quicker and uses less memory.
Also using query builder
DB::table("rates")->get()->sum("rate_value")
To get summation of all rate value inside table rates.
To get summation of user products.
DB::table("users")->get()->sum("products")
For people who just want to quickly display the total sum of the values in a column to the blade view, you can do this:
{{ \App\Models\ModelNameHere::sum('column_name') }}
You can also do it for averages:
{{ \App\Models\ModelNameHere::avg('column_name') }}
Min:
{{ \App\Models\ModelNameHere::min('column_name') }}
Max:
{{ \App\Models\ModelNameHere::max('column_name') }}
To get the Count of a table:
{{ \App\Models\ModelNameHere::count() }}