How to retrieve all media entries in Kaltura, beyond the 10k mark? - php

Question

I am using the php5.3 SDK: https://github.com/kaltura/KalturaGeneratedAPIClientsPHP53
We have 90k media entries, but I can only got 20k entries. The following code is straight forward. Could anyone help me out?
// Main entry point
public function mywrite(Route $route, Console $console)
{
// Max records is 500, the range cannot be too big.
$range = 3600 * 24;
$this->__mywrite($route, $console, $range);
}
// Count how many objects we can get
// $veryStartDate == 1446173087, sep 2015
// $maxDate == 1526469375, may 2018
public function __mywrite($route, $console, $range) {
$configObj = $this->readMyWriteHistoryConfigFile();
$lastProcessObj = $this->readMyWriteLastProcessFile();
//
$veryStartDate = $configObj->veryStartDate;
$maxDate = $configObj->maxDate;
// Set start Date
$startDate = $veryStartDate;
$endDate = $startDate + $range;
//
$totalCount = 0;
while($startDate <= $maxDate) {
$objs = $this->listMediaByLastPlay($startDate, $endDate);
$totalCount += count($objs);
echo "\n$startDate - $endDate:\n";
echo "\n". count($objs). "\n";
$startDate = $endDate + 1;
$endDate = $endDate + $range;
} // end while loop
// we get like 25k records, but we have 90k records....
echo "\ncount: $totalCount\n";
}
// we call the client and get records by start last play date and end last play date
public function listMediaByLastPlay($startDate, $endDate) {
// Page size
$pageSize = 1000;
// Client with admin
$client = $this->getClient(\KalturaSessionType::ADMIN);
// media
$mediaObj = $client->media;
// Set a range to pull, order by last played at
$filter = new \KalturaMediaEntryFilter();
$filter->lastPlayedAtGreaterThanOrEqual = $startDate;
$filter->lastPlayedAtLessThanOrEqual = $endDate;
$filter->orderBy = '+lastPlayedAt';
// We still want more records
$pager = new \KalturaFilterPager();
$pager->pageSize = $pageSize;
// now list.....
$arr = $mediaObj->listAction($filter, $pager)->objects;
$buf = array();
foreach($arr as $k => $v) {
$t = array();
$t['dataUrl'] = $v->dataUrl;
$t['flavorParamsIds'] = $v->flavorParamsIds;
$t['plays'] = $v->plays;
$t['views'] = $v->views;
$t['lastPlayedAt'] = $v->lastPlayedAt;
$buf[] = $t;
}
return $buf;
}

Answer 1

You're iterating on the first page of each response, there might be more that one page.
The kaltura ListResponse has a totalCount property.
so you code should something like:
$pager = new \KalturaFilterPager();
$pageIndex = 1;
$entriesGot = 0;
$buf = array();
do
{
$pager->pageSize = $pageSize;
$pager->pageIndex = $pageIndex++;
// now list.....
$response = $mediaObj->listAction($filter, $pager);
$arr = $response->objects;
$entriesGot += count($arr);
foreach($arr as $k => $v) {
$t = array();
$t['dataUrl'] = $v->dataUrl;
$t['flavorParamsIds'] = $v->flavorParamsIds;
$t['plays'] = $v->plays;
$t['views'] = $v->views;
$t['lastPlayedAt'] = $v->lastPlayedAt;
$buf[] = $t;
}
}while($entriesGot < $response->totalCount);