can we avoid eloquent in large no of records - php

Question

I am using laravel eloquent and have condition where thousands of records inside database so when using eloquent relationship my query is executing slow. Should i avoid using eloquent in these satuations or any other way?
here is mysql query
$leads=Lead::select($col)
->join("gc_od_leads_detail as ld", "gc_od_leads.leads_id", "=", "ld.ld_leads_id")
->join("gc_od_chat as c", "gc_od_leads.leads_chat_id", "=", "c.chat_id")
->join("gc_od_group as g", "c.chat_group_id", "=", "g.group_octachat_id")
->where('c.chat_tags','sales')
->whereIn('c.chat_group_id',$filter['groups']);
if(!empty($filter['keyword'])) {
$leads=$leads->where(function ($q) use ($filter) {
$q->where('ld_name','like', "%".$filter['keyword']."%")
->orWhere('ld_email','like', "%".$filter['keyword']."%")
->orWhere('ld_phoneno','like', "%".$filter['keyword']."%");
});
}
if(!empty($filter['startDate']) && !empty($filter['endDate'])){
$leads=$leads->whereBetween('leads_created_date', [$filter['startDate']." 00:00:00",$filter['endDate']." 23:59:59"]);
}
$leads=$leads->orderBy('leads_created_date','desc');
return $leads;
}
I have more than 500 000 recordings in side messages and chats table. i changed query in eloquent and debugged it
Query:
Lead::select('leads_id','leads_chat_id')->with(["detail"=>function($q){
$q->select("ld_leads_id");
}])->with(["chat"=>function($q){
$q->select("chat_id")->where(['chat_status'=>1]);
}])->where("leads_status",1)->get();
Debuging Ouput
array:3 [▼
0 => array:3 [▼
"query" => "select `leads_id`, `leads_chat_id` from `gc_od_leads` where `leads_status` = ?"
"bindings" => array:1 [▼
0 => 1
]
"time" => 14.85
]
1 => array:3 [▼
"query" => "select `ld_leads_id` from `gc_od_leads_detail` where `gc_od_leads_detail`.`ld_leads_id` in (2278918, 2278919, 2278920, 2278921, 2278922, 2278923, 2278924, 22789 ▶"
"bindings" => []
"time" => 0.59
]
2 => array:3 [▼
"query" => "select `chat_id` from `gc_od_chat` where `gc_od_chat`.`chat_id` in (3496457, 3496458, 3496459, 3496460, 3496461, 3496462, 3496463, 3496464, 3496465, 3496466, 34 ▶"
"bindings" => array:1 [▶]
"time" => 4.21
]
]
i above output you can see that it get all records of leads first then going to lead detail and chats table if i only want to find out the leads having chat status =1 it will still query all leads this is what slowing my query
where we use join it will not work in this way i think which will save time and space both that's my i post this question i think a lot of people having same problem and no one discuss this point

Answer 1

Let's take a look at one part of this.
if(!empty($filter['keyword'])) {
$leads=$leads->where(function ($q) use ($filter) {
$q->where('ld_name','like', "%".$filter['keyword']."%")
->orWhere('ld_email','like', "%".$filter['keyword']."%")
->orWhere('ld_phoneno','like', "%".$filter['keyword']."%");
});
}
This keyword-matching scheme is inherently, and disastrously, slow. It's slow in both Eloquent and native SQL. There's no way it can work in MySQL without doing a full table scan. That is, it must examine every row of your table looking for matches and cannot, in MySQL, exploit any indexed lookup scheme. Why?
column LIKE 'constant%'
can look at an index on column and quickly find any value starting with 'constant'. But
column LIKE '%constant%'
has to look at every value in the table. The leading % makes the index lookup useless.
In MySQL you would be wise to investigate MySQL's FULLTEXT searching as a way of handling your keyword lookup. (Recent versions of postgreSQL can handle this sort of query directly with a different kind of index, but not MySQL.)