Running out of YouTube quota what are my options? - php

I am working customer relationship department and creating an app which is replying to every YouTube comment.
So what i am making right now is basically a script which pull comment data from YouTube Data API v3. This script is a looping script which is being called every 30 seconds, however you may know the YouTube API has a quota limit and I keep hitting it.
I am open for any solution:
Do I have to apply for more quota to YouTube
How much quota should i apply ( basically im only pulling comment data, from who, id , and timestamp ) or sis there any other way.
my code
<?php
if (!file_exists(__DIR__ . '/youtube_vendor/autoload.php')) {
throw new Exception(sprintf('Please run "composer require google/apiclient:~2.0" in "%s"', __DIR__));
}
require_once __DIR__ . '/youtube_vendor/autoload.php';
include "mysql.php";
$db = new db();
$client = new Google_Client();
$client->setAuthConfig('client_secret.json');
$client->addScope('https://www.googleapis.com/auth/youtube.readonly');
$client->addScope('https://www.googleapis.com/auth/youtube.force-ssl');
$client->setRedirectUri('your_url');
// offline access will give you both an access and refresh token so that
// your app can refresh the access token without user interaction.
$client->setAccessType('offline');
// Using "consent" ensures that your application always receives a refresh token.
// If you are not using offline access, you can omit this.
$client->setPrompt("consent");
$client->setIncludeGrantedScopes(true); // incremental auth
$auth_url = $client->createAuthUrl();
if(isset($_GET['code'])) {
// id index exists
$client->authenticate($_GET['code']);
$access_token = $client->getAccessToken();
// var_dump($access_token);
// echo "<br><br>";
// file_put_contents("received.txt",var_dump($access_token));
// $access_token = file_get_contents("received.txt");
// // $file = json_decode($fb);
// var_dump($access_token);
// serialize your input array (say $array)
$serializedData = serialize($access_token);
// save serialized data in a text file
file_put_contents('youtube_access_token.txt', $serializedData);
// at a later point, you can convert it back to array like:
$recoveredData = file_get_contents('youtube_access_token.txt');
// unserializing to get actual array
$access_token = unserialize($recoveredData);
// you can print your array like
print_r($access_token);
echo "<br>";
$client->setAccessToken($access_token);
$service = new Google_Service_YouTube($client);
// $channel = $youtube->channels->listChannels('snippet', array('mine' => $mine));
// var_dump($channel);
$queryParams = [
'maxResults' => 25,
'mine' => true
];
$arrayComment = array();
$arrayReplies = array();
$responseVideo = $service->activities->listActivities('snippet,contentDetails', $queryParams);
foreach($responseVideo['items'] as $video)
{
$db->insert_youtube_video($video['snippet']['channelId'],$video['snippet']['channelTitle'],$video['snippet']['publishedAt'],$video['snippet']['title'],$video['snippet']['description'],$video['snippet']['thumbnails']['standard']['url'],$video['contentDetails']['upload']['videoId']);
$queryParams = [
'videoId' => $video['contentDetails']['upload']['videoId']
];
$responseComment = $service->commentThreads->listCommentThreads('snippet,replies', $queryParams);
foreach($responseComment['items'] as $comment)
{
$db->insert_youtube_comment($comment['snippet']['topLevelComment']['snippet']['authorChannelUrl'],$comment['snippet']['topLevelComment']['snippet']['authorDisplayName'],$comment['snippet']['topLevelComment']['snippet']['authorProfileImageUrl'],$comment['snippet']['topLevelComment']['snippet']['publishedAt'],$comment['snippet']['topLevelComment']['snippet']['updatedAt'],$comment['snippet']['topLevelComment']['snippet']['textDisplay'],$comment['snippet']['topLevelComment']['snippet']['videoId'],$comment['snippet']['topLevelComment']['id']);
$queryParams = [
'parentId' => $comment['snippet']['topLevelComment']['id']
];
$responseReplies = $service->comments->listComments('snippet', $queryParams);
foreach ($responseReplies['items'] as $replies)
{
$db->insert_youtube_replies($replies['snippet']['authorChannelUrl'],$replies['snippet']['authorDisplayName'],$replies['snippet']['authorProfileImageUrl'],$replies['snippet']['publishedAt'],$replies['snippet']['updatedAt'],$replies['snippet']['textDisplay'],$replies['snippet']['videoId'],$comment['snippet']['topLevelComment']['id'],$replies['id']);
}
$arrayReplies[] = $responseReplies;
}
$arrayComment[] = $responseComment;
}
}
else
{
echo $auth_url;
}
?>
<textarea style="width:100%;height:300px"><?php print_r($responseVideo['items']); ?><?php print_r($arrayComment); ?><?php print_r($arrayReplies); ?></textarea>
<script>
setTimeout(function () { window.location.reload(); }, 15*1000);
document.write(new Date());
</script>

Do I have to apply for more quota to YouTube
if you want to avoid hitting the quota and continue to make the number of requests you are making now. Yes you need to apply for a quota extension.
I would apply as soon as possible it can take a long time to get an extension.
How much quota should i apply ( basically im only pulling comment data, from who, id , and timestamp ) or sis there any other way.
That is up to you. How many requests are you making every day. You will need to do some math then pad it to ensure for future in crease of your application.

Related

Using an API key Why can I max retrieve 20k videos? Also why only public videos?

On
YouTube API to fetch all videos on a channel
found compact sample PHP under heading "Here is the code that will return all video ids under your channel".
Program shown below.
I expanded the program to fetch various attributes of each video, including ACCESS.
I have a channel with over 20,000 videos and large quote.
The program ran nicely and produced a .csv with video attributes.
It ran for about 2 hours and 10 minutes and stopped at 20,000 videos. In addition it only picked up PUBLIC videos.
How can the above two issues be remedied?
<?php
// FROM https://stackoverflow.com/questions/18953499/youtube-api-to-fetch-all-videos-on-a-channel/70071113#70071113
$baseUrl = 'https://www.googleapis.com/youtube/v3/';
// https://developers.google.com/youtube/v3/getting-started
$apiKey = 'API_KEY';
// If you don't know the channel ID see below
$channelId = 'CHANNEL_ID';
$params = [
'id'=> $channelId,
'part'=> 'contentDetails',
'key'=> $apiKey
];
$url = $baseUrl . 'channels?' . http_build_query($params);
$json = json_decode(file_get_contents($url), true);
$playlist = $json['items'][0]['contentDetails']['relatedPlaylists']['uploads'];
$params = [
'part'=> 'snippet',
'playlistId' => $playlist,
'maxResults'=> '50',
'key'=> $apiKey
];
$url = $baseUrl . 'playlistItems?' . http_build_query($params);
$json = json_decode(file_get_contents($url), true);
$videos = [];
foreach($json['items'] as $video)
$videos[] = $video['snippet']['resourceId']['videoId'];
while(isset($json['nextPageToken'])){
$nextUrl = $url . '&pageToken=' . $json['nextPageToken'];
$json = json_decode(file_get_contents($nextUrl), true);
foreach($json['items'] as $video)
$videos[] = $video['snippet']['resourceId']['videoId'];
}
print_r($videos);
?>
In addition it only picked up PUBLIC videos.
The code you are currently using uses an API key. API keys are used to access public data only.
If you want to access private data then you will need to be authorized using Oauth2 as a user who has access to the private videos.
It ran for about 2 hours and 10 minutes and stopped at 20,000 videos.
This question is a little harder for me to answer as i cant test it i don't have a YouTube Channel with 20k videos.
I can guess that as you are using an api key there is a limit to the number of videos they will let you download with an api key. They probably dont want people downloading all public videos on YouTube.
I suggest that you try and authorize it with Oauth2 and see if the limit is still there.
php example with Oauth2.
Create installed app credentials see video
make sure to enable the YouTube data api under libary.
change the channelId in this code to your own
Code
<?php
/**
* Sample PHP code for youtube.search.list
* See instructions for running these code samples locally:
* https://developers.google.com/explorer-help/guides/code_samples#php
*/
if (!file_exists(__DIR__ . '/vendor/autoload.php')) {
throw new Exception(sprintf('Please run "composer require google/apiclient:~2.0" in "%s"', __DIR__));
}
require_once __DIR__ . '/vendor/autoload.php';
$client = new Google_Client();
$client->setApplicationName('API code samples');
$client->setScopes([
'https://www.googleapis.com/auth/youtube.force-ssl',
]);
// TODO: For this request to work, you must replace
// "YOUR_CLIENT_SECRET_FILE.json" with a pointer to your
// client_secret.json file. For more information, see
// https://cloud.google.com/iam/docs/creating-managing-service-account-keys
$client->setAuthConfig('YOUR_CLIENT_SECRET_FILE.json');
$client->setAccessType('offline');
// Request authorization from the user.
$authUrl = $client->createAuthUrl();
printf("Open this link in your browser:\n%s\n", $authUrl);
print('Enter verification code: ');
$authCode = trim(fgets(STDIN));
// Exchange authorization code for an access token.
$accessToken = $client->fetchAccessTokenWithAuthCode($authCode);
$client->setAccessToken($accessToken);
// Define service object for making API requests.
$service = new Google_Service_YouTube($client);
$queryParams = [
'channelId' => 'UCyqzvMN8newXIxyYIkFzPvA',
'forMine' => false
];
$response = $service->search->listSearch('snippet', $queryParams);
print_r($response);
Note: I dont have php installed on this machine so i cant test it but this should be close. Let me know if you have any issues.

"Login with Google" in PHP - Google+ API shutdown migration - how to migrate away from plus.people.get?

I got a warning email from Google reminding me of Google+'s EOL which is supposed to break my current "Login with Google", but I am unsure what exactly should I change.
Let me show you my (simplified) login code:
google-login.php
new class {
public function __construct() {
$state = mt_rand();
$client = new Google_Client();
$client->setApplicationName(Config::Google['app_name']);
$client->setClientId(Config::Google['id']);
$client->setClientSecret(Config::Google['secret']);
$client->setRedirectUri(sprintf('https://%s/members/google-callback.php', $_SERVER['HTTP_HOST']));
$client->setScopes(['profile', 'email']);
$client->setState($state);
$_SESSION['state'] = $state;
$url = $client->createAuthUrl(); // $url = https://accounts.google.com/o/oauth2/auth?response_type=code&access_type=online&client_id=CLIENT_ID.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fread2me.online%2Fmembers%2Fgoogle-callback.php&state=1588245f23f2a&scope=profile%20email&approval_prompt=auto
header ("location: $url");
}
};
google-callback.php
new class {
private $newUser = false;
public function __construct() {
if (!isset($_GET['state']) || $_GET['state'] != $_SESSION['state'])
die('State mismatch.');
$client = new Google_Client();
$client->setApplicationName(Config::Google['app_name']);
$client->setClientId(Config::Google['id']);
$client->setClientSecret(Config::Google['secret']);
$client->setRedirectUri(sprintf('https://%s/members/google-callback.php', $_SERVER['HTTP_HOST']));
$client->setScopes(['profile', 'email']);
$plus = new Google_Service_Plus($client);
if (isset($_GET['code'])) {
$client->fetchAccessTokenWithAuthCode($_GET['code']);
$_SESSION['token'] = $client->getAccessToken();
}
if (isset($_SESSION['token'])) {
$client->setAccessToken($_SESSION['token']);
}
if (!$client->getAccessToken() || $client->isAccessTokenExpired()) {
$state = mt_rand();
$client->setState($state);
$_SESSION['state'] = $state;
$url = $client->createAuthUrl();
header ("location: $url");
}
try {
$me = $plus->people->get('me');
} catch (Google_Exception $e) {
\Rollbar::report_message($e->getMessage());
print_r($e->getMessage());
return;
}
$accessToken = $client->getAccessToken()['access_token'];
$email = $me->getEmails()[0]->getValue();
$name = $me->getDisplayName();
$avatar = $me->getImage()->getUrl();
$id = $me->getId();
if ($this->isEmailInSystem($email) === false) {
$this->newUser = true;
$this->addUser($email, $name, 'google', $accessToken, $id, $avatar);
}
header ("location: " . '/');
}
};
Now, I'm going through at what seems to be the up-to-date Sign In guide for PHP, but I am not sure what to change - any ideas?
Thanks
The best migration is to move from the Plus API to the People API, which provides access to the user's profile in a similar (tho not quite identical) way.
You would replace the creation of the $plus object with a new Goolge_Service_PeopleService object. Something like
$people = new Google_Service_PeopleService( $client );
Getting the profile is more involved since you need to specify which fields from the profile you want to get. But you might do it something like
$profile = $people->people->get(
'people/me',
array('personFields' => 'names,emailAddresses,photos')
);
The first parameter needs to be "people/me" to specify that you're requesting the authorized user's profile.
The second is an array of query parameters. You need to specify the "personFields" that you want from the list of what is available (scroll down on this page till you see the description of the available fields) and specify this as a comma separated list in a string. In my example above, I illustrate getting the name, email addresses, and photos. But consult the list and experiment.
The exact fields you get from the result in $profile will be different than those you got from $plus, but they should match the fields you requested. Check the values and exactly how they're structured.
I ran into the same issue as Google+ APIs shutting down on March 7, 2019.
Make sure Google People API is enable in your google console
I used google-api-php-client Library.
Once you have an access token here is code to get the person object using people API
$accessToken = 'REPLACE_WITH_ACCESS_TOKEN';
$clientId = 'REPLACE_WITH_CLIENT_ID';
$clientSecret = 'REPLACE_WITH_CLIENT_SECRET';
$developerKey = 'REPLACE_WITH_DEVELOPER_KEY';
$client = new Google_Client();
$client->setApplicationName("Application Name");
$client->setClientId($clientId . '.apps.googleusercontent.com');
$client->setClientSecret($clientSecret);
$client->setDeveloperKey($developerKey);
$client->setScopes(['https://www.googleapis.com/auth/userinfo.email','https://www.googleapis.com/auth/userinfo.profile']);
$client->setAccessToken($accessToken);
$guzzleClient = new \GuzzleHttp\Client(array( 'curl' => array( CURLOPT_SSL_VERIFYPEER => false, ), ));
$client->setHttpClient($guzzleClient);
$people = new Google_Service_PeopleService( $client );
if ($client->getAccessToken()) {
try {
$me = $people->people->get(
'people/me',
array('personFields' => 'emailAddresses,names,photos')
);
$id = preg_replace('/[^0-9]/', '', $me->getResourceName());
$email = $me->getEmailAddresses()[0]->value;
$name = $me->getNames()[0]->displayName;
$avtar = $me->getPhotos()[0]->getUrl();
} catch (Google_Exception $e) {
// error
echo $e->getMessage();
}
}
I also disabled Google+ API to make sure the application is not using it anymore anywhere.
With latest version of Google API PHP Client you can fetch profile details from Google_Client object itself.
$token = $client->fetchAccessTokenWithAuthCode($_GET['code']);
$attributes = $client->verifyIdToken($token['id_token'], GOOGLE_CLIENT_ID);
print_r($attributes);
Refer this article.
Obviously, the lines
$plus = new Google_Service_Plus($client);
and
$me = $plus->people->get('me');
You need to use google email API, see https://developers.google.com/gmail/api/quickstart/php , so the first line will be
$service = new Google_Service_Gmail($client);
and second ... hmmm ... not sure there WILL be any avatar after removing of google plus ...

Ebay: Auth token is invalid. Validation of the authentication token in API request failed

I'm trying to use Ebay PHP SDK to connect to Ebay and fetch sellers selling item. For this I used following steps:
Step 1: Get authorize token and code for logged-in user. I used following code to implement.
use \DTS\eBaySDK\OAuth\Services as OauthService;
use \DTS\eBaySDK\OAuth\Types as OauthType;
use \DTS\eBaySDK\Constants;
use \DTS\eBaySDK\Trading\Services;
use \DTS\eBaySDK\Trading\Types;
use \DTS\eBaySDK\Trading\Enums;
$service = new OauthService\OAuthService([
'credentials' => $config['sandbox']['credentials'],
'ruName' => $config['sandbox']['ruName'],
'sandbox' => true
]);
$oauthParam = [
'client_id' => $config['sandbox']['credentials']['appId'],
'redirect_uri' => $config['sandbox']['redirect_uri'],
'response_type' => 'code',
'scope' => 'https://api.ebay.com/oauth/api_scope'
];
$urlParam = '';
$query = [];
foreach($oauthParam as $key => $param) {
$query[] = "$key=$param";
}
$urlParam = '?' . implode('&', $query);
$url = 'https://signin.sandbox.ebay.com/authorize' . $urlParam;
#session_start();
if(isset($_SESSION['ebay_oauth_token'])) {
$token = $_SESSION['ebay_oauth_token']['code'];
}
else {
if(isset($_GET['code'])) {
$token = $_GET['code'];
$_SESSION['ebay_oauth_token']['code'] = $token;
$request = new OauthType\GetUserTokenRestRequest();
$request->code = $token;
$response = $service->getUserToken($request);
if ($response->getStatusCode() !== 200) {
//Error
} else {
$_SESSION['ebay_oauth_token']['access_token'] = $response->access_token;
}
} else {
#header('location: ' . $url);
}
}
$userOauthToken = $_SESSION['ebay_oauth_token']['access_token'];
The above code is working as expected. That is the user is redirected to Sign In Page to authorize himself and get the set of Code and Access Token.
Step 2: Fetch Selling Items using code obtained from Step #1. I've used following code to implement the functionality.
$request->RequesterCredentials = new Types\CustomSecurityHeaderType();
$request->RequesterCredentials->eBayAuthToken = $token; //Obtained from Step 1
$request->ActiveList = new Types\ItemListCustomizationType();
$request->ActiveList->Include = true;
$request->ActiveList->Pagination = new Types\PaginationType();
$request->ActiveList->Pagination->EntriesPerPage = 10;
$request->ActiveList->Sort = Enums\ItemSortTypeCodeType::C_CURRENT_PRICE_DESCENDING;
$pageNum = 1;
do {
$request->ActiveList->Pagination->PageNumber = $pageNum;
$response = $service->getMyeBaySelling($request);
if (isset($response->Errors)) {
//Error Output
}
if ($response->Ack !== 'Failure' && isset($response->ActiveList)) {
foreach ($response->ActiveList->ItemArray->Item as $item) {
//Output response
}
}
$pageNum += 1;
} while ({condition});
I'm having problem in Step #2. It is generating Invalid Token while running the code.
I would highly appreciate if anyone help me.
You are mixing the requests. In the second part of your code, the request belongs to the Trading API that uses the Auth'n'Auth token, and you are trying to make the call using the OAuth token. These 2 tokens are different and work for different APIs.
You have 2 options.
Either keep the second part of your code, which appears to be correct, actually, but use the Auth'n'Auth token (that you can generate from the developer account). In this case, the first part is useless.
Keep the first part of your code, and delete the second part. In this case, you need to rewrite your second part of code using the OAuth API instead of the Trading API.

Setting up Google Reporting API to work in cronjob

I've managed to set up a PHP script that grabs data from our Google Analytics account and can run this great in the browser but I'm struggling to figure out how to allow this to work inside of the cronjob script instead.
Here's my code:
<?php
// Load the Google API PHP Client Library.
require_once dirname(__FILE__) . '/analytics/google-api-php-client-2.2.0/vendor/autoload.php';
session_start();
$client = new Google_Client();
$client->setAuthConfig(dirname(__FILE__) . '/analytics/client_secrets.json');
$client->setAccessType("offline"); //offline access
$client->addScope(Google_Service_Analytics::ANALYTICS_READONLY);
if (isset($_SESSION['access_token']) && $_SESSION['access_token']) {
print "<br />start: ".date("H:i:s")."<br />";
// Set the access token on the client.
$client->setAccessToken($_SESSION['access_token']);
$sessionid = $_SESSION['access_token']['access_token'];
// Create an authorized analytics service object.
$analytics = new Google_Service_AnalyticsReporting($client);
$view_id = urlencode("ga:XXXXXX");
$start_date = "yesterday"; //can be words or dates in Y-m-d
$end_date = "yesterday"; //can be words or dates in Y-m-d
$metrics = urlencode("ga:transactions");
$dimensions = urlencode("ga:transactionId,ga:date,ga:orderCouponCode");
$sort = "ga:date"; //for DESC use -FIELD e.g. -ga:date
$sampling = "HIGHER_PRECISION"; //https://developers.google.com/analytics/devguides/reporting/core/v3/reference#samplingLevel
$limit = "10000";
$filters = urlencode("ga:orderCouponCode!=(not set)");
$params = "ids=$view_id&start-date=$start_date&end-date=$end_date&metrics=$metrics&dimensions=$dimensions&filters=$filters&sort=$sort&samplingLevel=$sampling&max-results=$limit&access_token=$sessionid";
$data = file_get_contents("https://www.googleapis.com/analytics/v3/data/ga?$params");
$json_data = json_decode($data);
print "<pre>";
print_r($json_data);
print "</pre>";
} else {
// Handle authorization flow from the server.
if (! isset($_GET['code'])) {
$auth_url = $client->createAuthUrl();
header('Location: ' . filter_var($auth_url, FILTER_SANITIZE_URL));
} else {
$client->authenticate($_GET['code']);
$_SESSION['access_token'] = $client->getAccessToken();
$redirect_uri = 'http://' . $_SERVER['HTTP_HOST'] . '/cronjobs/ga_coupon_codes.php';
header('Location: ' . filter_var($redirect_uri, FILTER_SANITIZE_URL));
}
}
print "<br />end: ".date("H:i:s")."<br />";
When my cronjob runs, the servers send me an email to say that it has finished running. I'm expecting to see both the "start" and "end" times printed in the response, as well as the JSON data, but the only thing that's coming back in the "end" time, leading me to believe it's not running through the main bit of it.
I know the main reason why it's not working is due to the authentication bit. In a cronjob I cannot use header('location: URL') but I can't figure out what to change this to in order to make it work.
When I set up the account, it was created as a service account and I can see from within the API console that I have a service account key.
I've also got an OAuth 2.0 client ID (which is what is in the client screts JSON file). If I run this in my browser I'm getting the connection to work, even after the hour runs out (where it's getting a new access token).
I've googled around for this for quite some time and found that adding "offline access" should help but obviously I'm still using the header() function. I can't find examples for me to use that are understandable to me and I'd really like to get this working in a cronjob.

Khan Academy PHP Oauth Code

Im really struggling w/ the OAuth for Khan Academy. This is for my class website (Im a teacher) and I want to pull in user data on particular students. If I could do the OAUTH I would be fine. Im using PHP.
There seems to be many librarys out there, I have been playing w/ Google Oauth (located here http://code.google.com/p/oauth-php/source/browse/trunk/example/client/twolegged.php)
I can formulate the token request fine, although when I call it in the script, it seems like it tries to redirect to another page and gets blocked there.
http://myonlinegrades.com/prealg/khan/oauth-php/example/client/twoleggedtest.php
Im really struggling - Id love any help you might offer.
Code below:
<?php
include_once "../../library/OAuthStore.php";
include_once "../../library/OAuthRequester.php";
// Test of the OAuthStore2Leg
// uses http://term.ie/oauth/example/
$key = '*********';//'<your app's API key>';
$secret = '***********';//'<your app's secret>';
$callBack = "http://myonlinegrades.com/prealg/test2.php5";
$url = 'http://www.khanacademy.org/api/auth/request_token';
$options = array('consumer_key' => $key, 'consumer_secret' => $secret);
OAuthStore::instance("2Leg", $options);
$method = "GET";
//$params = null;
$params = array(oauth_consumer_key => $key,oauth_callback=>$callBack);
try
{
// Obtain a request object for the request we want to make
$request = new OAuthRequester($url, $method, $params);
// Sign the request, perform a curl request and return the results,
// throws OAuthException2 exception on an error
// $result is an array of the form: array ('code'=>int, 'headers'=>array(), 'body'=>string)
$result = $request->doRequest();
$response = $result['body'];
if ($response != 'oauth_token=requestkey&oauth_token_secret=requestsecret')
{
echo 'Error! $response ' . $response;
}
else
{
}
var_dump($response);
}
catch(OAuthException2 $e)
{
echo "Exception" . $e->getMessage();
}
?>
Not sure this is what you're looking for, but I put together a simple example of doing oAuth with Khan Academy using the Temboo SDK: take a look at https://github.com/matthewflaming/temboo-experiments/tree/master/KhanAcademyOauth
(Full disclosure: I work at Temboo)

Categories