hello i have this function
//Get Total Number of Personal Leads By User ID & Requested week Through Each Day
//Get Total Number of Personal Leads By User ID & Requested week Through Each Day
$personalleads = \DB::table('leads')
->where('owned_by_id', $id) // User ID
->where('lead_source_id', 7) // 7 = Personal Lead
->whereBetween('created_at', [$weeks[$week]]) // get week through GET request & Query the data
->select(\DB::raw('DATE(created_at) as date'), \DB::raw('count(*) as pleads'))
->groupBy('date')
->get(); // Get All Data
//Get Total Number of leads Created by Managers By User ID & Requested week Through Each Day
$managerleads = \DB::table('leads')
->where('owned_by_id', $id) // User ID
->where('lead_source_id', 3) // 3 = Manager Lead
->whereBetween('created_at', [$weeks[$week]]) // get week through GET request & Query the data
->select(\DB::raw('DATE(created_at) as date'), \DB::raw('count(*) as mleads'))
->groupBy('date')
->get(); // Get All Data
//Get Total Number of leads Created by Admins By User ID & Requested week Through Each Day
$adminleads = \DB::table('leads')
->where('owned_by_id', $id) // User ID
->where('lead_source_id', 4) // 4 = Admin Lead
->whereBetween('created_at', [$weeks[$week]]) // get week through GET request & Query the data
->select(\DB::raw('DATE(created_at) as date'), \DB::raw('count(*) as aleads'))
->groupBy('date')
->get(); // Get All Data
i want to return the data of all of them
like
return $adminleads+personalleads+managerleads;
i know this is invalid but i want to display all data at once
this is the output i get when i return $personalleads only
[{"date":"2019-02-10","pleads":1},{"date":"2019-02-12","pleads":1},{"date":"2019-02-14","pleads":1}]
how can i make it something like
[{"date":"2019-02-10","pleads":1,"aleads":1,"mleads":1},{"date":"2019-02-12","pleads":1,"aleads":1,"mleads":1},{"date":"2019-02-14","pleads":1,"aleads":1,"mleads":1}]
Thank you very much
Two options you can try to join these values:
PHP will loop the results and join the data.
The queries handle the joining of data (which I cannot wrap my head around at this moment).
Not sure which one would be better. PHP looks much easier however, depenending on the system and size, my guess is that mysql can handle this much more efficient.
PHP
$pLeads = ...
$mLeads = ...
$aLeads = ...
$allLeads = $pLeads->merge($mLeads)->merge($aLeads); // Maybe there is a shorter variant.
$leads = [];
$types = ['pleads', 'mleads', 'aleads'];
$allLeads->each(function ($lead) {
// Make sure there is an array present with the date property.
data_fill($leads, $lead->date, [
'date' => $lead->date,
]);
// At this point our array looks like:
// ['2019-11-06' => ['date' => '2019-11-06']]
// Get the the type of lead.
$type = $types[array_search(array_keys($lead))];
// Set the lead amount for the type.
// Not sure if "$lead->date.$type" works. Make sure you end up with a key like 2019-11-06.pleads
data_set($leads, "$lead->date.$type", $lead->$type);
// At this point our array looks like:
// ['2019-11-06' => ['date' => '2019-11-06', 'pleads' => 1]]
});
// Remove the keys from the array.
$leads = array_flatten($leads);
// At this point our array looks like:
// [['date' => '2019-11-06', 'pleads' => 1, 'mleads' => 2, 'aleads' => 3]]
Query
SELECT
IFNULL(p.date, m.date)
pleads,
mleads
FROM (
(
SELECT
DATE(created_at) as date,
COUNT(*) pleads
FROM
leads
WHERE
...
GROUP BY
date
) p
RIGHT JOIN
(
SELECT
DATE(created_at) as date,
COUNT(*) mleads
FROM
leads
WHERE
...
GROUP BY
date
) m ON p.date = m.date
);
Both solutions have not been tested fully.
Related
So this is my case. I have a main table payment_transactions having almost 1 million records.
I am getting the data from this table with joins and where clauses and there is Laravel paginate method,
But this method takes too much time and after investigation, I found that its count method is taking time like 4 to 5 seconds just for counting.
So how can I optimize and speed up this query, especially is there any way to improve paginate method speed?
Note: I can't use simplePaginate because there is a datatable on frontend and I need a total count for that.
So for paginate, two queries run
1 is the main query and other one is for count, and I felt that, the count query is taking much time.
Here is the count query after getQueryLog
select count(*) as aggregate from `payment_transactions`
left join `users` as `U` on `U`.`id` = `payment_transactions`.`user_id`
left join `coupons` as `C`
on `C`.`id` = `payment_transactions`.`coupon_id`
where `payment_transactions`.`refund_status` = 'NO_REFUND'
and `payment_transactions`.`transaction_type`
in ('BOOKING','SB_ANPR','QUERCUS_ANPR','CANDID_ANPR','SB_TICKET',
'ORBILITY_TICKET','TOPUP,CREDIT','DEBIT','GIFT')
and `payment_transactions`.`status` != 'INITIATED'
Here is my code example:
//Get Transactions data
public function adminTransactions(Request $request)
{
$selectableFields = [
'payment_transactions.id', 'payment_transactions.transaction_id AS transaction_id',
'payment_transactions.refund_status',
'payment_transactions.created_at', 'payment_transactions.response_data', 'payment_transactions.status',
'payment_transactions.transaction_type', 'payment_transactions.payment_mode','payment_transactions.payment_source',
'payment_transactions.subtotal', 'payment_transactions.service_fees', 'C.coupon_code','C.amount AS coupon_value',
DB::raw("IF(payment_transactions.refund_remarks='NULL','-NA-',payment_transactions.refund_remarks) as refund_remarks"),
DB::raw("IF(payment_transactions.transaction_type='TOPUP' AND payment_transactions.coupon_id IS NOT NULL
AND payment_transactions.coupon_id!=0,
payment_transactions.amount + C.amount,payment_transactions.amount) as amount"),
DB::raw("CONCAT(U.first_name,' ',U.last_name) AS username"), 'U.id AS user_id',
DB::raw("JSON_UNQUOTE(json_extract(payment_transactions.response_data, '$.description')) AS description"),
DB::raw("payment_transactions.invoice_id"),
DB::raw("JSON_UNQUOTE(json_extract(payment_transactions.response_data, '$.Data.PaymentID')) AS upay_payment_id"),
];
return PaymentTransactions::select($selectableFields)
->with('homeScreenMessages:payment_transaction_id,from_name,message,amount')
->leftJoin('users AS U', 'U.id', '=', 'payment_transactions.user_id')
->leftJoin('coupons AS C', 'C.id', '=', 'payment_transactions.coupon_id')
->where(DB::raw("CONCAT(U.first_name,' ',U.last_name)"), 'like', "%{$request->input('query')}%")
->orWhere('U.id', $request->input('query'))
->orWhere("U.phone_number", "LIKE", "%" . $request->input('query') . "%")
->orWhere("U.email", "LIKE", "%" . $request->input('query') . "%")
->orWhere('payment_transactions.id', $request->input('query'))
->orWhere('payment_transactions.transaction_id', $request->input('query'));
}
//Paginate function
public function paginationCalculate($queryObject, $request) {
$draw = $request->get('draw');
$start = $request->get("start");
$rowperpage = $request->get("length"); // Rows display per page
$columnIndex_arr = $request->get('order');
$columnName_arr = $request->get('columns');
$order_arr = $request->get('order');
$columnIndex = $columnIndex_arr[0]['column']; // Column index
$columnName = $columnName_arr[$columnIndex]['name']; // Column name
$columnSortOrder = $order_arr[0]['dir']; // asc or desc
$pageNumber = ($start + $rowperpage) / $rowperpage;
if(!empty($columnName)) {
$queryObject->orderBy($columnName, $columnSortOrder);
}
$records = $queryObject->paginate($rowperpage, ['*'], 'page', $pageNumber)->toArray();
return array(
"draw" => intval($draw),
"recordsFiltered" => $records['total'],
"recordsTotal" => $records['total'],
"data" => $records['data']
);
}
I don't think you want "LEFT". With LEFT, the COUNT will count payment transactions even if there is no matching user and/or coupon.
Or maybe there are multiple users or coupons for each payment_transaction? In this case, the COUNT will be inflated. To fix this, simply remove the two extra tables.
Regardless of the situation, add this composite (and covering) index to payment_transactions:
INDEX(refund_status, transaction_type, status, user_id, coupon_id)
refund_type must be first since it is tested with '='.
If that is not "fast enough", then I must ask you "Who cares what the exact number when there are a million rows?" Do you ever see a search engine giving an exact number of hits? Does that imprecision bother you?
That is, rethink the requirement to calculate the exact number of rows. Or even an approximate number.
You mentioned "pagination". Please show us how that is done. OFFSET is an inefficient way since it rescans the previously seen rows again and again. Instead, 'remember where you left off': Pagination
You HAVE to adding INDEX to your MySQL database. Makes it lighting fast!
If you are using Navicat, click on the Index tab, and add the Column name and Name it the same. You WILL notice a huge difference.
I'm trying to get what should seem like a simple SQL query to work in my Laravel project, I've got a front-end web application that allows a user to pull data from various pre-defined tables and a selectable list of columns.
This works fine if I don't attempt to select and groupBy together, equally, I need someway of grouping baed on whether the user wants to group data by a day, or a month for instance.
My POST request looks like, where each array item inside the parent array is a table:
[
[
table: 'my_table',
columns: ['event_category', 'event_action', 'event_count', 'created_at'] ...
filterBy: [
['event_category', 'my category'],
['event_action', 'some action']
],
orderBy: {
field: 'created_at',
direction: 'desc'
}
]
]
Each row in my_table contains an event_count column, which contains a number, so if there's 5 rows for a particular day with different event_count numbers, I need to add up all of those event_count entries and group them by that day
Here's my function and query:
public function findDataFromSources(Request $request)
{
$request_data = $request->all();
$realData = [
'size' => 0,
'results' => [],
'filtered' => []
];
foreach ($request_data as $key => $findable) {
// NOTE: this works for retrieving data that isn't grouped
// $res = DB::table($findable['table'])
// ->select($findable['columns'])
// ->where($findable['filterBy'])
// ->orderBy($findable['orderBy']['field'], $findable['orderBy']['direction'])
// ->take(20)
// ->get();
// TODO: this isn't grouping for some reason...
$res = DB::table($findable['table'])
->select($findable['columns'], DB::raw('sum(event_count) as total_events'))
->groupBy('created_at')
->orderBy($findable['orderBy']['field'], $findable['orderBy']['direction'])
->get();
$realData['size'] += count($res);
array_push($realData['results'], $res);
}
$data = [
'success' => true,
'message' => 'Your chosen data sources and fields',
'sources' => $realData
];
}
What am I missing? The error I'm getting is:
SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #1 of SELECT list is not in GROUP BY clause and contains nonaggregated column
->select($findable['columns']
You cannot SELECT fields/columns that are not in GROUP BY without aggregate function and your $findable['columns'] probably contain fields that are not in GROUP BY.
You can use ANY_VALUE() like #RobbinBenard commented for example
->select(DB::raw('ANY_VALUE(' .$findable['columns'][0] .')'))
You can loop through the columns and use addSelect() to add those columns.
$query = DB::table($findable['table'])
->select(DB::raw('sum(event_count) as total_events'));
foreach($findable['columns'] as $column) {
$query->addSelect(DB::raw('ANY_VALUE(' .$column .')'));
}
$res = $query->groupBy('created_at')
->orderBy($findable['orderBy']['field'], $findable['orderBy']['direction'])
->get();
I am attempting to fetch a count of users who have registered each month, separated by month in the results - so, January - 22, February - 36, March- 56, so on and so forth limited to this month and the last five.
However, I can't seem to get the query correct as it relates to CakePHP 3's query builder. This is what I have so far:
$query = $this->find('all');
$query->select(['count' => $query->func()->count('id'), 'day' => 'DAY(dateRegistered)']);
$query->where(['YEAR(dateRegistered)' => date('Y')]);
$query->group(['MONTH(dateRegistered)']);
$query->enableHydration(false);
return $query->all();
This seems to return the user count for the month, but that's all, and not in an easily graph-able format. I have seen some queries in raw SQL so it should be possible, but I would like to use the native query controls in Cake. Could someone help fix this?
Try this:
$query = $this->find();
$res = $query
->select([
'month'=>'monthname(created)',
'count'=>$query->func()->count('*'),
])
->where('year(created) = year(curdate())')
->group('month')
->enableHydration(false);
pr($res->toArray());
I am trying to achieve something like this:
I am using Laravel, and for example to display the results I would do a foreach on the array and display the title, the body etc.
Example
#foreach($resArray as $res)
{{$res->Title}}
#endforeach
And that will display the title of each result in the array, I also have the date in $res->startDate but I am not sure how to list each result with same date under their specific date.
You probably didn't understood..
So for example if I have two notifications from 10/07/2017 and one from 11/07/2017 they will display as this:
10/07/2017
- notification 1
- notification 2
11/07/2017
- notification
I was thinking at an if statement but what statement
if($res->startDate) what so this won't work either, I was thinking to store them in arrays, for example array of date 11/07/2017 and display those arrays but would that even work...
Couldn't find too much from google as I am not too sure how to google this in a good maneer.
EDIT 1
I tried doing this:
$notifResult = notifications::where('userID', '=', $userID)->select('*')->groupBy(['date'])->get();
dd($notifResult);
but it didn't work, first of all, I had 3 results in database, it only got 2 of them and it didn't even group them by date, the two results that are listed before are from different days...
EDIT 2
I added toArray to it and this is what I've got:
still, it only picks two results...
One naive but simple solution is to sort the results by start date and then in the foreach check if the previous date (store it in temp var) is different from the current one, and if true display the new date.
$prevDate = '';
#foreach($resArray as $res)
#if($prevDate != '' && $prevDate != $res->startDate) {
//do sth to break the html and display the new date
#endif
{{$res->Title}}
{{$prevDate = $res->startDate;}}
#endforeach
if i understand your problem you must use group by
$collection = collect([
['date' => '10/07/2017', 'title' => 'notification 1'],
['date' => '10/07/2017', 'title' => 'notification 2'],
['date' => '11/07/2017', 'title' => 'notification'],
]);
$grouped = $collection->groupBy('date');
$grouped->toArray();
dd($grouped);
Hello i get stuck on this proble, hope you guys can give me some advice to solve this matter. So, i want to show graph that count basen on month, how many keluhan (complain) on that month. so the table will be like january has 3 complain, febuary has 5 complain, etc (example)
public function lihatkeluhan(){
$halaman="tindaklayanan";
$keluhan_list=DB::table('keluhans')
->select(DB::raw('id,tanggal,produk,username,area,masalah,status'))->get();
$count = count($keluhan_list); //this still not count based on month
$population = Lava::DataTable();
$population->addDateColumn('Year')
->addNumberColumn('Keluhan')
->addRow(['?',$count]);
Lava::LineChart('Population', $population, [
'title' => 'Tahun : 2017',
]);
return view('layanankonsumen.daftarkeluhan',compact('halaman','keluhan_list','lava'));
}
try query like this.
it will gives you groupby result with count of that particular month,
no need for $count = count($keluhan_list); because you will get count in result.
$keluhan_list=DB::table('keluhans')
->select(DB::raw('id,tanggal,produk,username,area,masalah,status,count(*) as count'))
->get()
->groupBy(function($date) {
return Carbon::parse($date->created_at)->format('m'); // grouping by months
});