Get all youtube videos from a channel (some videos are missing) - php

Question

I'm using the v3 Google API for Youtube:
$url = 'https://www.googleapis.com/youtube/v3/search?part=id&channelId=' . $channelID . '&maxResults=50&order=date&key=' . $API_key;
I've set up a script which should give me all videos from a given channel ID. For some channels I get all videos, for some a few are missing (compared with the number of videos shown directly in Youtube), and for bigger channel I get a max. result of 488 videos despite there are more.
The pageToken is a strange thing. For example a channel has 955 videos. I get 18 pages with 50 items per page (that would be 900 videos). Some of them are playlists but if I subtract the 23 playlists I still have 877 videos. If I remove duplicates I only have 488 results! The totalResults in the JSON output shows me 975 results!?
This is my recursive function:
function fetchAllVideos($parsed_json){
$foundIds = array();
if($parsed_json != ''){
$foundIds = getVideoIds($parsed_json);
$nextPageToken = getNextPageToken($parsed_json);
$prevPageToken = getPrevPageToken($parsed_json);
if($nextPageToken != ''){
$new_parsed_json = getNextPage($nextPageToken);
$foundIds = array_merge($foundIds, fetchAllVideos($new_parsed_json));
}
if($prevPageToken != ''){
$new_parsed_json = getNextPage($prevPageToken);
$foundIds = array_merge($foundIds, fetchAllVideos($new_parsed_json));
}
}
return $foundIds;
}
I call it with $videoIds = fetchAllVideos($parsed_json); and $parsed_json is the result from the first URL which I retrieve. Can you see an error here?
Does anybody know how the number of videos are counted, which are directly shown in Youtube? Has anybody managed to get a full list which correspond to the number in Youtube?

Answer 1

This script goes through selecting a 60 day period at a time and retrieves the results for it, then adds it to the existing data array. By doing this there are no limitations to how many videos are allowed, though it may take some time to trawl larger YouTube channels with a couple thousand videos. Make sure you set the API_KEY, timezone, username, start date (should begin before the first video on the channel), and period (set by default to 60 * 60 * 24 * 60, which is 60 days in seconds. This will need to be lower if the frequency of videos is higher than about 50 for 60 days.) (5184000 seconds).
*All of this is commented within the script.
date_default_timezone_set("TIMEZONE");
//youtube api key
$API_KEY = "YOUR API KEY";
function search($searchTerm,$url){
$url = $url . urlencode($searchTerm);
$result = file_get_contents($url);
if($result !== false){
return json_decode($result, true);
}
return false;
}
function get_user_channel_id($user){
global $API_KEY;
$url = 'https://www.googleapis.com/youtube/v3/channels?key=' . $API_KEY . '&part=id&forUsername=';
return search($user,$url)['items'][0]['id'];
}
function push_data($searchResults){
global $data;
foreach($searchResults['items'] as $item){
$data[] = $item;
}
return $data;
}
function get_url_for_utc_period($channelId, $utc){
//get the API_KEY
global $API_KEY;
//youtube specifies the DateTime to be formatted as RFC 3339 formatted date-time value (1970-01-01T00:00:00Z)
$publishedAfter = date("Y-m-d\TH:i:sP",strval($utc));
//within a 60 day period
$publishedBefore_ = $utc + (60 * 60 * 24 * 60);
$publishedBefore = date("Y-m-d\TH:i:sP",$publishedBefore_);
//develop the URL with the API_KEY, channelId, and the time period specified by publishedBefore & publishedAfter
$url = 'https://www.googleapis.com/youtube/v3/search?part=snippet&type=video&key=' . $API_KEY . '&maxResults=50&channelId=' . $channelId . '&publishedAfter=' . urlencode($publishedAfter) . '&publishedBefore=' . urlencode($publishedBefore);
return array("url"=>$url,"utc"=>$publishedBefore_);
}
//the date that the loop will begin with, have this just before the first videos on the channel.
//this is just an example date
$start_date = "2013-1-1";
$utc = strtotime($start_date);
$username = "CHANNEL USERNAME NOT CHANNEL ID";
//get the channel id for the username
$channelId = get_user_channel_id($username);
while($utc < time()){
$url_utc = get_url_for_utc_period($channelId, $utc);
$searchResults = search("", $url_utc['url']);
$data = push_data($searchResults);
$utc += 60 * 60 * 24 * 60;
}
print "<pre>";
print_r($data);
print "</pre>";
//check that all of the videos have been accounted for (cross-reference this with what it says on their youtube channel)
print count($data);

Answer 2

https://gdata.youtube.com/feeds/api/users/USERNAME_HERE/uploads?max-results=50&alt=json&start-index=1 did the trick. It's a JSON feed where you have to loop until you get less than 50 results.
Edit:
This should be the script I used:
ini_set('max_execution_time', 900);
function getVideos($channel){
$ids = array();
$start_index = 1;
$still_have_results = true;
if($channel == ""){
return false;
}
$url = 'https://gdata.youtube.com/feeds/api/users/' . $channel . '/uploads?max-results=50&alt=json&start-index=' . $start_index;
$json = file_get_contents($url);
$obj = json_decode($json);
while($still_have_results){
foreach($obj->feed->entry as $video){
$video_url = $video->id->{'$t'};
$last_pos = strrpos($video_url, '/');
$video_id = substr($video_url, $last_pos+1, strlen($video_url) - $last_pos);
array_push($ids, $video_id);
}
$number_of_items = count($obj->feed->entry);
$start_index += count($obj->feed->entry);
if($number_of_items < 50) {
$still_have_results = false;
}
$url = 'https://gdata.youtube.com/feeds/api/users/' . $channel . '/uploads?max-results=50&alt=json&start-index=' . $start_index;
$json = file_get_contents($url);
$obj = json_decode($json);
}
return $ids;
}
$videoIds = getVideos('youtube');
echo '<pre>';
print_r($videoIds);
echo '</pre>';
Now I made a test, but I didn't gathered 100% of the videos. Nevertheless, the best option I came up with.