I have got a relationship where I select all row's based on the category, however I need to exclude some of these if they are within a sub query.
/** #var \Illuminate\Database\Query\Builder $images */
$images = $vehicle->images()
->whereIn('image_category', $website->image_categories)
->orderBy('seq', 'ASC');
$images->whereNotIn('id', static function ($q) {
return $q->select('id')
->whereIn('image_category', [0, 99])
->groupBy('seq')
->having(DB::raw('count(`seq`)'), '>', 1);
});
dd($images->toSql(), $images->getBindings());
So above is my code, nearly works as I want it, however it seems that the $q variable doesn't have the table name within the query, below is the query outputted:
select
*
from
`vehicle_images`
where
`vehicle_images`.`vehicle_id` = ?
and `vehicle_images`.`vehicle_id` is not null
and `image_category` in (?, ?)
and `id` not in (
select
`id`
where
`image_category` in (?, ?)
group by
`seq`
having
count(`seq`) > ?
)
order by
`seq` asc
This is the relationship:
public function images()
{
return $this->hasMany(VehicleImage::class);
}
You can specify what table you want to use.
$images->whereNotIn('id', static function ($q) {
return $q->select('id')->from('{CORRECT_TABLE_NAME_HERE}')
->whereIn('image_category', [0, 99])
->groupBy('seq')
->having(DB::raw('count(`seq`)'), '>', 1);
});
I don't know what exactly the table name should be, hence the placeholder.
I'm trying to get customer specific users who don't have an owner role, but it also skips users who don't have any role. Users can have one or multiple roles. I want to get all users either having multiple roles or no role at all, but if the user contains an owner role then only that user should be ignored.
Note: I am using spatie/laravel-permission which gets users roles from model has roles intermediate table
Here is my scope query
public function scopeForCompany(EloquentBuilder $query, string $customerId): EloquentBuilder
{
$query->where(function (EloquentBuilder $q) {
$q->doesntHave('roles');
$q->orHas('roles');
});
$query->whereHas('roles', function (EloquentBuilder $q) {
$q->whereNotIn('name', ['owner']);
});
return $query->where('customer_id', $customerId);;
}
here is the test
public function it_apply_query_scope_to_get_customer_specific_users_only(): void
{
$model = new User;
// create non customer users
\factory(User::class, 2)->create();
$customer = \factory(Customer::class)->create();
foreach (['owner', 'admin', 'user'] as $role) {
$role = \factory(Role::class)->create(['name' => $role]);
$user = \factory(User::class)->create(['customer_id' => $customer->id]);
$user->roles()->save($role);
}
$scopedUsers = $model->newQuery()->forCompany($customer->id)->get();
$nonScopedUsers = $model->newQuery()->get();
static::assertCount(2, $scopedUsers); // Failed asserting that actual size 0 matches expected size 2.
static::assertCount(5, $nonScopedUsers);
}
Debug: here is the row query:
"select * from `users` where (not exists (select * from `roles` inner join `model_has_roles` on `roles`.`id` = `model_has_roles`.`role_id` where `users`.`id` = `model_has_roles`.`model_uuid` and `model_has_roles`.`model_type` = ?) or exists (select * from `roles` inner join `model_has_roles` on `roles`.`id` = `model_has_roles`.`role_id` where `users`.`id` = `model_has_roles`.`model_uuid` and `model_has_roles`.`model_type` = ?)) and exists (select * from `roles` inner join `model_has_roles` on `roles`.`id` = `model_has_roles`.`role_id` where `users`.`id` = `model_has_roles`.`model_uuid` and `model_has_roles`.`model_type` = ? and `name` not in (?)) and `customer_id` = ? and `users`.`deleted_at` is null"
This is what i tried first but didn't worked
return $query->whereHas('roles', function (EloquentBuilder $query): void {
$query->whereNotIn('name', ['owner']);
})->where('customer_id', $customerId);
Any help would be appreciated thanks
You need to do some Or logic for this to happen. I break the query up into 3 pieces.
The statement: "I want to get all users either having multiple roles"
$query->has('roles', '>=', 2);
Next you want all with no roles: "or no role at all".
$query->doesntHave('roles');
And lastly your query correctly filter out where the role cannot be the owner.
$query->whereHas('roles', function (EloquentBuilder $query): void {
$query->whereNotIn('name', ['owner']);
})
Putting it all together doing something like, with a sub where query. To proper do the Or logic you want.
$query->where(function($builder){
$builder->has('roles', '>=', 2);
$builder->whereHas('roles', function (EloquentBuilder $query): void {
$query->whereNotIn('name', ['owner']);
})
});
$builder->orDoesntHave('roles');
In pseudo logical statements this would look something similar to like:
(roles.each.name != 'owner' && count(roles) >= 2) || empty(roles)
Let's see if this help your case, else post the toSql() of the builder and let's figure it out. It's a fairly complex query builder logic this is doing.
i want get all posts with last three comment on each post. my relation is
public function comments()
{
return $this->hasMany('App\Commentpostfeed','post_id')->take(3);
}
This would return only 3 comments total whenever I called it instead of 3 comments per post.
i use this way :
1 :
Postfeed::with(['comment' => function($query) {
$query->orderBy('created_at', 'desc')->take(3); }]);
2 :
$postings = Postfeed::with('comments')->get();
but getting same result. please help me out for this problem.
Can you try like that ?;
Postfeed::with('comment')->orderBy('id','desc')->take(3);
Using plain mysql (If using Mysql) query you can get 3 recent comments per post using following query which rejoins comment table by matching created_at
SELECT p.*,c.*
FROM posts p
JOIN comments c ON p.`id` = c.`post_id`
LEFT JOIN comments c1 ON c.`post_id` = c1.`post_id` AND c.`created_at` <= c1.`created_at`
GROUP BY p.`id`,c.`id`
HAVING COUNT(*) <=3
ORDER BY p.`id`,c.`created_at` DESC
Sample Demo
Using laravel's query builder you can write similar to
$posts = DB::table('posts as p')
->select('p.*,c.*')
->join('comments c', 'p.id', '=', 'c.post_id')
->leftJoin('comments as c1', function ($join) {
$join->on('c.post_id', '=', 'c1.post_id')->where('c.created_at', '<=', 'c1.created_at');
})
->groupBy('p.id')
->groupBy('c.id')
->having('COUNT(*)', '<=', 3)
->orderBy('p.id', 'asc')
->orderBy('c.created_at', 'desc')
->get();
You can create a scope in the BaseModel like this :
<?php
class BaseModel extends \Eloquent {
/**
* query scope nPerGroup
*
* #return void
*/
public function scopeNPerGroup($query, $group, $n = 10)
{
// queried table
$table = ($this->getTable());
// initialize MySQL variables inline
$query->from( DB::raw("(SELECT #rank:=0, #group:=0) as vars, {$table}") );
// if no columns already selected, let's select *
if ( ! $query->getQuery()->columns)
{
$query->select("{$table}.*");
}
// make sure column aliases are unique
$groupAlias = 'group_'.md5(time());
$rankAlias = 'rank_'.md5(time());
// apply mysql variables
$query->addSelect(DB::raw(
"#rank := IF(#group = {$group}, #rank+1, 1) as {$rankAlias}, #group := {$group} as {$groupAlias}"
));
// make sure first order clause is the group order
$query->getQuery()->orders = (array) $query->getQuery()->orders;
array_unshift($query->getQuery()->orders, ['column' => $group, 'direction' => 'asc']);
// prepare subquery
$subQuery = $query->toSql();
// prepare new main base Query\Builder
$newBase = $this->newQuery()
->from(DB::raw("({$subQuery}) as {$table}"))
->mergeBindings($query->getQuery())
->where($rankAlias, '<=', $n)
->getQuery();
// replace underlying builder to get rid of previous clauses
$query->setQuery($newBase);
}
}
And in the Postfeed Model :
<?php
class Postfeed extends BaseModel {
/**
* Get latest 3 comments from hasMany relation.
*
* #return Illuminate\Database\Eloquent\Relations\HasMany
*/
public function latestComments()
{
return $this->comments()->latest()->nPerGroup('post_id', 3);
}
/**
* Postfeed has many Commentpostfeeds
*
* #return Illuminate\Database\Eloquent\Relations\HasMany
*/
public function comments()
{
return $this->hasMany('App\Commentpostfeed','post_id');
}
}
And to get the posts with the latest comments :
$posts = Postfeed::with('latestComments')->get();
Ps :
Source
For many to many relationships
You can do it like this,
Postfeed::with('comments',function($query){
$query->orderBy('created_at', 'desc')->take(3);
})
->get();
I want to apply a where condition to relation. Here's what I do:
Replay::whereHas('players', function ($query) {
$query->where('battletag_name', 'test');
})->limit(100);
It generates the following query:
select * from `replays`
where exists (
select * from `players`
where `replays`.`id` = `players`.`replay_id`
and `battletag_name` = 'test')
order by `id` asc
limit 100;
Which executes in 70 seconds. If I manually rewrite query like this:
select * from `replays`
where id in (
select replay_id from `players`
where `battletag_name` = 'test')
order by `id` asc
limit 100;
It executes in 0.4 seconds. Why where exists is the default behavior if it's so slow? Is there a way to generate the correct where in query with query builder or do I need to inject raw SQL? Maybe I'm doing something wrong altogether?
replays table has 4M rows, players has 40M rows, all relevant columns are indexed, dataset doesn't fit into MySQL server memory.
Update: found that the correct query can be generated as:
Replay::whereIn('id', function ($query) {
$query->select('replay_id')->from('players')->where('battletag_name', 'test');
})->limit(100);
Still have a question why exists performs so poorly and why it is the default behavior
Try this:
mpyw/eloquent-has-by-non-dependent-subquery: Convert has() and whereHas() constraints to non-dependent subqueries.
mpyw/eloquent-has-by-join: Convert has() and whereHas() constraints to join() ones for single-result relations.
Replay::hasByNonDependentSubquery('players', function ($query) {
$query->where('battletag_name', 'test');
})->limit(100);
That's all. Happy Eloquent Life!
The reason for laravel has(whereHas) sometimes slowly is that implemented with where exists syntax.
For example:
// User hasMany Post
User::has('posts')->get();
// Sql: select * from `users` where exists (select * from `posts` where `users`.`id`=`posts`.`user_id`)
The 'exists' syntax is a loop to the external table, and then queries the internal table (subQuery) every time.
However, there will be performance problems when the users table has a large amount of data, because above sql select * from 'users' where exists... unable to use index.
It can use where in instead of where exists here without damaging the structure.
// select * from `users` where exists (select * from `posts` where `users`.`id`=`posts`.`user_id`)
// =>
// select * from `users` where `id` in (select `posts`.`user_id` from `posts`)
This will greatly improve performance!
I recommend you try this package hasin, in the above example, you can use the hasin instead of the has.
// User hasMany Post
User::hasin('posts')->get();
// Sql: select * from `users` where `id` in (select `posts`.`user_id` from `posts`)
The hasin just only use where in syntax instead of where exists compared with the framework has, but everywhere else is the same, such as parameters and call mode even the code implementation, and can be used safely.
whereHas performance is poor on tables without index, put index on it and be happy!
Schema::table('category_product', function (Blueprint $table) {
$table->index(['category_id', 'product_id']);
});
This is related to the mysql not to the laravel. You can perform the same thing you wanted from the above with the both options, joins and the subqueries. Subqueries are generally much slower than joins.
Subqueries are:
less complicated
elegant
easier to understand
easier to write
logic separation
and the above facts are why ORMs like eloquent are using suquries. but there are slower! Especially when you have many rows in the database.
Join version of your query is something like this :
select * from `replays`
join `players` on `replays`.`id` = `players`.`replay_id`
and `battletag_name` = 'test'
order by `id` asc
limit 100;
but now you must change select and add group by and be careful on many other things, but why is this so it is beyond that answer. New query would be :
select replays.* from `replays`
join `players` on `replays`.`id` = `players`.`replay_id`
and `battletag_name` = 'test'
order by `id` asc
group by replays.id
limit 100;
So that are the reasons why join in more complicated.
You can write raw query in laravel, but eloquent support for join queries are not well supported, also there are no much packages that can help you with that, this one is for example : https://github.com/fico7489/laravel-eloquent-join
WhereHas() query is really as slow as lazy turtle, so I created and still using a trait that I glue to any laravel model which required a simple join requests. This trait make a scope function whereJoin(). You can just pass there a joined model class name, where clause params and enjoy. This trait take care of table names and related details in query. Well, it's for my personal use and ofc feel free to modify this monstruosity.
<?php
namespace App\Traits;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Database\Query\JoinClause;
/** #mixin Model */
trait ModelJoinTrait
{
/**
* #param string|\Countable|array $on
* #param $column
* #param $whereOperator
* #param $value
* #param Model $exemplar
* #return array
*/
function _modelJoinTraitJoinPreset($on, $column, $whereOperator, $value, $exemplar){
$foreignTable = $exemplar->getTable();
$foreignId = $exemplar->getKeyName();
$localTable = $this->getTable();
$localId = $this->getKeyName();
//set up default join and condition parameters
$joinOn =[
'local' => $localTable.'.'.$localId,
'foreign'=> $foreignTable.'.'.$foreignId,
'operator' => '=',
'type'=>'inner',
'alias'=>'_joint_id',
'column'=>$column,
'where_operator'=>$whereOperator,
'value'=>$value
];
//config join parameters based on input
if(is_string($on)){
//if $on is string it treated as foreign key column name for join clause
$joinOn['foreign'] = $foreignTable.'.'.$on;
} elseif (is_countable($on)){
//if $is array or collection there can be join parameters
if(isset($on['local']) && $on['local'])
$joinOn['local'] = $localTable.'.'.$on['local'];
if(isset($on['foreign']) && $on['foreign'])
$joinOn['foreign'] = $localTable.'.'.$on['foreign'];
if(isset($on['operator']) && $on['operator'])
$joinOn['operator'] = $on['operator'];
if(isset($on['alias']) && $on['alias'])
$joinOn['alias'] = $on['alias'];
}
//define join type
$joinTypeArray = ['inner', 'left', 'right', 'cross'];
if(is_countable($on) && isset($on['type']) && in_array($on['type'], $joinTypeArray))
$joinOn = $on['type'];
return $joinOn;
}
/**
* #param Model $exemplar
* #param string|array|\Countable $joinedColumns
* #param string|array|\Countable $ownColumns
* #param string $jointIdAlias
* #return array
*/
function _modelJoinTraitSetColumns($exemplar, $joinedColumns, $ownColumns, $jointIdAlias = '_joint_id')
{
$foreignTable = $exemplar->getTable();
$foreignId = $exemplar->getKeyName();
$localTable = $this->getTable();
$localId = $this->getKeyName();
if(is_string($joinedColumns))
$foreignColumn = ["$foreignTable.$joinedColumns"];
else if(is_countable($joinedColumns)) {
$foreignColumn = array_map(function ($el) use ($foreignTable) {
return "$foreignTable.$el";
}, $joinedColumns);
} else {
$foreignColumn = ["$foreignTable.*"];
}
if(is_string($ownColumns))
$ownColumns = ["$localTable.$ownColumns"];
elseif(is_countable($ownColumns)) {
$ownColumns = array_map(function ($el) use ($localTable) {
return "$localTable.$el";
}, $ownColumns);
} else {
$ownColumns = ["$localTable.*"];
}
$columns = array_merge($foreignColumn, $ownColumns);
if($foreignId == $localId){
$columns = array_merge(["$foreignTable.$foreignId as $jointIdAlias"], $columns);
}
return $columns;
}
/**
* #param Builder $query
* #param string|array|\Countable $on
* #param Model $exemplar
*/
function _modelJoinTraitJoinPerform($query, $on, $exemplar){
$funcTable = ['left'=>'leftJoin', 'right'=>'rightJoin', 'cross'=>'crossJoin', 'inner'=>'join'];
$query->{$funcTable[$on['type']]}($exemplar->getTable(),
function(JoinClause $join) use ($exemplar, $on){
$this->_modelJoinTraitJoinCallback($join, $on);
}
);
}
function _modelJoinTraitJoinCallback(JoinClause $join, $on){
$query = $this->_modelJoinTraitJoinOn($join, $on);
$column = $on['column'];
$operator = $on['where_operator'];
$value = $on['value'];
if(is_string($column))
$query->where($column, $operator, $value);
else if(is_callable($column))
$query->where($column);
}
/**
* #param JoinClause $join
* #param array|\Countable $on
* #return JoinClause
*/
function _modelJoinTraitJoinOn(JoinClause $join, $on){
//execute join query on given parameters
return $join->on($on['local'], $on['operator'], $on['foreign']);
}
/**
* A scope function used on Eloquent models for inner join of another model. After connecting trait in target class
* just use it as ModelClass::query()->whereJoin(...). This query function forces a select() function with
* parameters $joinedColumns and $ownColumns for preventing overwrite primary key on resulting model.
* Columns of base and joined models with same name will be overwritten by base model
*
* #param Builder $query Query given by Eloquent mechanism. It's not exists in
* ModelClass::query()->whereJoin(...) function.
* #param string $class Fully-qualified class name of joined model. Should be descendant of
* Illuminate\Database\Eloquent\Model class.
* #param string|array|\Countable $on Parameter that have join parameters. If it is string, it should be foreign
* key in $class model. If it's an array or Eloquent collection, it can have five elements: 'local' - local key
* in base model, 'foreign' - foreign key in joined $class model (default values - names of respective primary keys),
* 'operator' = comparison operator ('=' by default), 'type' - 'inner', 'left', 'right' and 'cross'
* ('inner' by default) and 'alias' - alias for primary key from joined model if key name is same with key name in
* base model (by default '_joint_id')
* #param Closure|string $column Default Eloquent model::where(...) parameter that will be applied to joined model.
* #param null $operator Default Eloquent model::where(...) parameter that will be applied to joined model.
* #param null $value Default Eloquent model::where(...) parameter that will be applied to joined model.
* #param string[] $joinedColumns Columns from joined model that will be joined to resulting model
* #param string[] $ownColumns Columns from base model that will be included in resulting model
* #return Builder
* #throws \Exception
*/
public function scopeWhereJoin($query, $class, $on, $column, $operator = null, $value=null,
$joinedColumns=['*'], $ownColumns=['*']){
//try to get a fake model of class to get table name and primary key name
/** #var Model $exemplar */
try {
$exemplar = new $class;
} catch (\Exception $ex){
throw new \Exception("Cannot take out data of '$class'");
}
//preset join parameters and conditions
$joinOnArray = $this->_modelJoinTraitJoinPreset($on, $column, $operator, $value, $exemplar);
//set joined and base model columns
$selectedColumns = $this->_modelJoinTraitSetColumns($exemplar, $joinedColumns, $ownColumns, $joinOnArray['alias']);
$query->select($selectedColumns);
//perform join with set parameters;
$this->_modelJoinTraitJoinPerform($query, $joinOnArray, $exemplar);
return $query;
}
}
You can use it like this (Model Goods in example have a dedicated extended data model GoodsData with hasOne relationship between them):
$q = Goods::query();
$q->whereJoin(GoodsData::class, 'goods_id',
function ($q){ //where clause callback
$q->where('recommend', 1);
}
);
//same as previous exmple
$q->whereJoin(GoodsData::class, 'goods_id',
'recommend', 1); //where clause params
// there we have sorted columns from GoodsData model
$q->whereJoin(GoodsData::class, 'goods_id',
'recommend', 1, null, //where clause params
['recommend', 'discount']); //selected columns
//and there - sorted columns from Goods model
$q->whereJoin(GoodsData::class, 'goods_id',
'recommend', '=', 1, //where clause params
['id', 'recommend'], ['id', 'name', 'price']); //selected columns from
//joined and base model
//a bit more complex example but still same. Table names is resolved
//by trait from relevant models
$joinData = [
'type'=>'inner' // inner join `goods_data` on
'local'=>'id', // `goods`.`id`
'operator'=>'=' // =
'foreign'=>'goods_id', // `goods_data`.`goods_id`
];
$q->whereJoin(GoodsData::class, $joinData,
'recommend', '=', 1, //where clause params
['id', 'recommend'], ['id', 'name', 'price']); //selected columns
return $q->get();
Resulting SQL query will be like this
select
`goods_data`.`id` as `_joint_id`, `goods_data`.`id`, `goods_data`.`recommend`,
`goods`.`id`, `goods`.`name`, `goods`.`price` from `goods`
inner join
`goods_data`
on
`goods`.`id` = `goods_data`.`goods_id`
and
-- If callback used then this block will be a nested where clause
-- enclosed in parenthesis
(`recommend` = ? )
-- If used scalar parameters result will be like this
`recommend` = ?
-- so if you have complex queries use a callback for convenience
In your case there should be like this
$q = Replay::query();
$q->whereJoin(Player::class, 'replay_id', 'battletag_name', 'test');
//or
$q->whereJoin(Player::class, 'replay_id',
function ($q){
$q->where('battletag_name', 'test');
}
);
$q->limit(100);
To use it more efficiently, you can go like this:
// Goods.php
class Goods extends Model {
use ModelJoinTrait;
//
public function scopeWhereData($query, $column, $operator = null,
$value = null, $joinedColumns = ['*'], $ownColumns = ['*'])
{
return $query->whereJoin(
GoodsData::class, 'goods_id',
$column, $operator, $value,
$joinedColumns, $ownColumns);
}
}
// -------
// any.php
$query = Goods::whereData('goods_data_column', 1)->get();
PS I dont run any automated tests for this so be careful in use. It works just fine in my case, but there may be unexpected behaviour in yours.
I think performance does not depend on whereHas only it depends on how many records you have selected
Plus try to optimize your mysql server
https://dev.mysql.com/doc/refman/5.7/en/optimize-overview.html
and also Optimize your php server
and if you have faster query why don't you use raw query object from larval
$replay = DB::select('select * from replays where id in (
select replay_id from players where battletag_name = ?)
order by id asc limit 100', ['test']
);
You can use left join
$replies = Replay::orderBy('replays.id')
->leftJoin('players', function ($join) {
$join->on('replays.id', '=', 'players.replay_id');
})
->take(100)
->get();
I remember the old days "discovering" it in a query using the Eloquent, if I used the with Laravel do aninner join.
Today by chance I checked the queries of a project and ...
[2014-11-20 23:21:16] sql.INFO: select * from `ocurrences` where `ocurrences`.`deleted_at` is null order by RAND() limit 4 {"bindings":[],"time":3.58,"name":"mysql"} []
[2014-11-20 23:21:16] sql.INFO: select * from `users` where `users`.`id` in ('7') {"bindings":["7"],"time":0.49,"name":"mysql"} []
[2014-11-20 23:21:16] sql.INFO: select * from `users` where `users`.`id` = '7' limit 1 {"bindings":["7"],"time":0.51,"name":"mysql"} []
[2014-11-20 23:21:16] sql.INFO: select * from `tags` limit 5 {"bindings":[],"time":0.41,"name":"mysql"} []
In this case, I'm doing a query like this:
/**
* Get random ocurrences for home
* #return mixed
*/
public static function randomForHome()
{
return static::with('user')
->orderByRaw('RAND()')
->limit(4)
->get();
}
What is wrong and/or how do I do joins with Eloquent?
I found the solution in a Laracasts video (thanks Jeffrey!).
I need to use join('users', 'users.id', '=', 'ocurrences.user_id'). It's a bit logical but I thought with do joins too.
Anyway, here's the final solution:
/**
* Get random ocurrences for home
* #return Eloquent
*/
public static function randomForHome()
{
return static::join('users', 'users.id', '=', 'ocurrences.user_id')
->orderByRaw('RAND()')
->limit(4)
->get();
}
Thanks guys.