Simple way to compare manytomany relationships? Laravel 5.3 - php

Question

I have a set of relationships that looks like this:
The users and agencies have a lot of data stored in them, in addition to the pivot tables you see there.
What I'd like to do is find the agencies or users that match the preferences of the currently logged in individual, whether they are an agency or user.
It's a straight comparison, so nothing fancy. But I honestly have no idea how to write the eloquent query to account for the pivot tables. Can someone point me in the right direction?
I'm looking at something like this, which is understandably failing:
$loggedinagency = Auth::user()->id;
$match_user_agency = User::with('work_prefs')
->where('work_prefs', 'like',
Agency::find($loggedinagency)->work_prefs)
->get();
Edit: The relationships are declared like so:
User:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class User extends Model {
protected $table = 'users';
public function work_prefs() {
return $this->belongsToMany('App\Work_Prefs', 'preference_user', 'user_id', 'preference_id');
}
}
Agency:
<?php
namespace App;
class Agency extends Authenticatable
{
use Notifiable;
protected $table='agencies';
public function work_prefs() {
return $this->belongsToMany('App\Work_Prefs', 'agency_preference', 'agency_id', 'preference_id');
}
}
Work Preferences:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class Work_Prefs extends Model
{
protected $table = 'work_prefs';
public function user() {
return $this->belongsToMany('App\User', 'preference_user', 'preference_id', 'user_id');
}
public function agency() {
return $this->belongsToMany('App\Agency', 'agency_preference', 'preference_id', 'agency_id');
}
}