Complex Database Queries in yii2 with Active Record - php

Question

TL;DR
I have a query that works in RAW SQL but i have had little success recreating it with query builder or active record.
I am working on a web application based off of the yii2 advanced application template. I have written a database query and implemented it with findbysql() returning the correct records but am having trouble translating this into active record.
I originally wanted to allow the user to modify (filter) the results by means of a search form(user & date), however i have since realized that implementing filters on the gridview with active records would be smoother.
I have gotten simple queries to work however am unsure of how to implement one with this many joins. Many examples used sub queries but my attempts failed to return any records at all. I figured before I attempt filters i need to transcribe this query first.
videoController.php
public function actionIndex()
{
$sql = 'SELECT videos.idvideo, videos.filelocation, events.event_type, events.event_timestamp
FROM (((ispy.videos videos
INNER JOIN ispy.cameras cameras
ON (videos.cameras_idcameras = cameras.idcameras))
INNER JOIN ispy.host_machines host_machines
ON (cameras.host_machines_idhost_machines =
host_machines.idhost_machines))
INNER JOIN ispy.events events
ON (events.host_machines_idhost_machines =
host_machines.idhost_machines))
INNER JOIN ispy.staff staff
ON (events.staff_idreceptionist = staff.idreceptionist)
WHERE (staff.idreceptionist = 182)
AND (events.event_type IN (23, 24))
AND (events.event_timestamp BETWEEN videos.start_time
AND videos.end_time)';
$query = Videos::findBySql($sql);
$dataProvider = new ActiveDataProvider([
'query' => $query,
]);
return $this->render('index', [
'dataProvider' => $dataProvider,
]);
}
Failed Attempt
public function actionIndex()
{
$query = Videos::find()
->innerJoin('cameras', 'videos.cameras_idcameras = cameras.idcameras')
->innerJoin('host_machines', 'cameras.host_machines_idhost_machines = host_machines.idhost_machines')
->innerJoin('events', 'events.host_machines_idhost_machines = host_machines.idhost_machines')
->innerJoin('staff', 'events.staff_idreceptionist = staff.idreceptionist')
->where('staff.idreceptionist = 182')
->andWhere(['events.event_type' => [23,24]])
->andwhere(['between', 'events.event_timestamp', 'videos.start_time', 'videos.end_time']);
$dataProvider = new ActiveDataProvider([
'query' => $query,
]);
return $this->render('index', [
'dataProvider' => $dataProvider,
]);
}
Portion of View
<?= GridView::widget([
'dataProvider' => $dataProvider,
'columns' => [
['class' => 'yii\grid\SerialColumn'],
'idvideo',
'event_type',
'event_timestamp',
'filelocation',
//['class' => 'yii\grid\ActionColumn'],
],
]); ?>
Please let me know if i need to be more specific or include any additional information.
Thanks ahead

Answer 1

i will assume, based on the question you asked here you liked in comments that you provided the entire query
(no other fields, that you took out just to show sample code)
therefore, if you only need only the fields specified in SELECT statement, you can optimize your query quite a bit:
first off, you're joining with host_machines only to link cameras and events, but have the same key host_machines_idhost_machines on both, so that's not needed, you can directly:
INNER JOIN events events
ON (events.host_machines_idhost_machines =
cameras.host_machines_idhost_machines))
secondly, the join with ispy.staff, the only used field is idreceptionist in WHERE clause, that field exists in events as well so we can drop it completly
the final query here:
SELECT videos.idvideo, videos.filelocation, events.event_type, events.event_timestamp
FROM videos videos
INNER JOIN cameras cameras
ON videos.cameras_idcameras = cameras.idcameras
INNER JOIN events events
ON events.host_machines_idhost_machines =
cameras.host_machines_idhost_machines
WHERE (events.staff_idreceptionist = 182)
AND (events.event_type IN (23, 24))
AND (events.event_timestamp BETWEEN videos.start_time
AND videos.end_time)
should output the same records as the one in your question, without any identitcal rows
some video duplicates will still exists due to one to many relation between cameras and events
now to the yii side of things,
you have to define some relations on the Videos model
// this is pretty straight forward, `videos`.`cameras_idcameras` links to a
// single camera (one-to-one)
public function getCamera(){
return $this->hasOne(Camera::className(), ['idcameras' => 'cameras_idcameras']);
}
// link the events table using `cameras` as a pivot table (one-to-many)
public function getEvents(){
return $this->hasMany(Event::className(), [
// host machine of event => host machine of camera (from via call)
'host_machines_idhost_machines' => 'host_machines_idhost_machines'
])->via('camera');
}
the VideoController and the search function itself
public function actionIndex() {
// this will be the query used to create the ActiveDataProvider
$query =Video::find()
->joinWith(['camera', 'events'], true, 'INNER JOIN')
->where(['event_type' => [23, 24], 'staff_idreceptionist' => 182])
->andWhere('event_timestamp BETWEEN videos.start_time AND videos.end_time');
$dataProvider = new ActiveDataProvider([
'query' => $query,
]);
return $this->render('index', [
'dataProvider' => $dataProvider,
]);
}
yii will treat each video as a single record (based on pk), that means that all video duplicates are
removed. you will have single videos, each with multiple events so you wont be able to use 'event_type'
and 'event_timestamp' in the view but you can declare some getters inside Video model to show that info:
public function getEventTypes(){
return implode(', ', ArrayHelper::getColumn($this->events, 'event_type'));
}
public function getEventTimestamps(){
return implode(', ', ArrayHelper::getColumn($this->events, 'event_timestamp'));
}
and the view use:
<?= GridView::widget([
'dataProvider' => $dataProvider,
'columns' => [
['class' => 'yii\grid\SerialColumn'],
'idvideo',
'eventTypes',
'eventTimestamps',
'filelocation',
//['class' => 'yii\grid\ActionColumn'],
],
]); ?>
edit:
if you want to keep the video duplicates, declare the two columns from events inside Video model
public $event_type, $event_timestamp;
keep the original GridView setup, and add a select and indexBy this to the query inside VideoController:
$q = Video::find()
// spcify fields
->addSelect(['videos.idvideo', 'videos.filelocation', 'events.event_type', 'events.event_timestamp'])
->joinWith(['camera', 'events'], true, 'INNER JOIN')
->where(['event_type' => [23, 24], 'staff_idreceptionist' => 182])
->andWhere('event_timestamp BETWEEN videos.start_time AND videos.end_time')
// force yii to treat each row as distinct
->indexBy(function () {
static $count;
return ($count++);
});
update
a direct staff relation to Video is currently somewhat problematic since that is more than one table away from it.
there's an issue about it here
however, you add the staff table by linking it to the Event model,
public function getStaff() {
return $this->hasOne(Staff::className(), ['idreceptionist' => 'staff_idreceptionist']);
}
that will allow you to query like this:
->joinWith(['camera', 'events', 'events.staff'], true, 'INNER JOIN')
Filtering will require some small updates on the controller, view and a SarchModel
here's a minimal implementation:
class VideoSearch extends Video
{
public $eventType;
public $eventTimestamp;
public $username;
public function rules() {
return array_merge(parent::rules(), [
[['eventType', 'eventTimestamp', 'username'], 'safe']
]);
}
public function search($params) {
// add/adjust only conditions that ALWAYS apply here:
$q = parent::find()
->joinWith(['camera', 'events', 'events.staff'], true, 'INNER JOIN')
->where([
'event_type' => [23, 24],
// 'staff_idreceptionist' => 182
// im guessing this would be the username we want to filter by
])
->andWhere('event_timestamp BETWEEN videos.start_time AND videos.end_time');
$dataProvider = new ActiveDataProvider(['query' => $q]);
if (!$this->validate())
return $dataProvider;
$this->load($params);
$q->andFilterWhere([
'idvideo' => $this->idvideo,
'events.event_type' => $this->eventType,
'events.event_timestamp' => $this->eventTimestamp,
'staff.username' => $this->username,
]);
return $dataProvider;
}
}
controller:
public function actionIndex() {
$searchModel = new VideoSearch();
$dataProvider = $searchModel->search(Yii::$app->request->queryParams);
return $this->render('test', [
'searchModel' => $searchModel,
'dataProvider' => $dataProvider,
]);
}
and the view
use yii\grid\GridView;
use yii\helpers\ArrayHelper;
echo GridView::widget([
'dataProvider' => $dataProvider,
'filterModel' => $searchModel,
'columns' => [
['class' => 'yii\grid\SerialColumn'],
'idvideo',
'filelocation',
[
'attribute' => 'eventType', // from VideoSearch::$eventType (this is the one you filter by)
'value' => 'eventTypes' // from Video::getEventTypes() that i suggested yesterday
// in hindsight, this could have been named better, like Video::formatEventTypes or smth
],
[
'attribute' => 'eventTimestamp',
'value' => 'eventTimestamps'
],
[
'attribute' => 'username',
'value' => function($video){
return implode(', ', ArrayHelper::map($video->events, 'idevent', 'staff.username'));
}
],
//['class' => 'yii\grid\ActionColumn'],
],
]);

Answer 2

My recommendation would be to have 2 queries. The first one to get the ids of the videos that fit your search, the second query theone that uses those ids and feeds your $dataProvider.
use yii\helpers\ArrayHelper;
...
public function actionIndex()
{
// This is basically the same query you had before
$searchResults = Videos::find()
// change 'id' for the name of your primary key
->select('id')
// we don't really need ActiveRecord instances, better use array
->asArray()
->innerJoin('cameras', 'videos.cameras_idcameras = cameras.idcameras')
->innerJoin('host_machines', 'cameras.host_machines_idhost_machines = host_machines.idhost_machines')
->innerJoin('events', 'events.host_machines_idhost_machines = host_machines.idhost_machines')
->innerJoin('staff', 'events.staff_idreceptionist = staff.idreceptionist')
->where('staff.idreceptionist = 182')
->andWhere(['events.event_type' => [23,24]])
->andwhere(['between', 'events.event_timestamp', 'videos.start_time', 'videos.end_time'])
// query the results
->all();
// this will be the query used to create the ActiveDataProvider
$query = Videos::find()
// and we use the results of the previous query to filter this one
->where(['id' => ArrayHelper::getColumn($searchResults, 'id')]);
$dataProvider = new ActiveDataProvider([
'query' => $query,
]);
return $this->render('index', [
'dataProvider' => $dataProvider,
]);
}