Exponential Backoff With Google API PHP Client Library - php

Question

I am running a web app to get a list of users with Google API PHP Client Library 2.0.3. and save them to a CSV file and at the same time, I am tracking the process on screen. The code I am using is the following:
$pageToken = null;
$optParams = array(
"customer" => "my_customer",
"maxResults" => 500,
"orderBy" => "email",
"sortOrder" => "ASCENDING"
);
try {
$usernum = 0;
do {
if ($pageToken){
$optParams['pageToken'] = $pageToken;
}
$results = $service->users->listUsers($optParams);
$pageToken = $results->getNextPageToken();
$users = $results->getUsers();
foreach ($users as $user) {
$usernum++;
$csvfileusers = array($user->getPrimaryEmail());
fputcsv($savecsv, $csvfileusers);
$usersemails = $user->getPrimaryEmail();
print "<li>".$usernum." - <font color='#9dd7fb'>".$usersemails."</font></li>";
}
} while($pageToken);
} catch (Exception $e) {
print "An error occurred: " . $e->getMessage();
}
Everything works fine. The problem is that from time to time I am getting { error: { errors: [ { domain: global, reason: backendError, message: Service unavailable. Please try again } ], code: 503, message: Service unavailable. Please try again } }
I know this means that I am sending requests to Google Server too fast hence I need to implement an exponential backoff solution. My problem is that I don't know how to do that. Would someone be kind enough to provide me an example on how to do that using the PHP Client Library? I know that I might be able to figure this out at the long term but if I can get some help I will greatly appreciate it.

Answer 1

Unfortunately, the documentation is lacking for the actual backoff implementation. However, the Google_Task_Runner class outlines the backoff implementation process. You can see how it does it here.
However, based on what you're doign you don't actually want to implement a exponential backoff procedure in general networking terms. You're really wanting to just throttle the request so you don't slam the API. Depending on how many $pageToken you're iterating over, you could just do a sleep before doing the next while iteration.
Additionally, what does $pageToken = $results->getNextPageToken(); become after one request? Becuase you're setting that from the response rather than decrementing it programatically, which may be causing an infinute loop or something of that nature.

Answer 2

So after 20 days of trying and investigating and thanks to the information provided by #kyle, I came up with this exponential backoff solution.
$attemptNum = 1; // retry attempt
$expBackoff = false; // exponential backoff rety indicator
do {
if($attemptNum <= 4) {
try {
$usernum = 1;
do {
if ($pageToken){
$optParams['pageToken'] = $pageToken;
}
$results = $service->users->listUsers($optParams);
$pageToken = $results->getNextPageToken();
$users = $results->getUsers();
foreach ($users as $user) {
$csvfileusers = array($user->getPrimaryEmail());
fputcsv($savecsv, $csvfileusers);
$usersemails = $user->getPrimaryEmail();
print "<li>".$usernum." - <font color='#9dd7fb'>".$usersemails."</font></li>";
$usernum++;
}
} while($pageToken);
} catch (Exception $e) {
$err503ReasonA = "Service unavailable"; // Service unavailable.
$err503ReasonB = "Backend Error"; //Backend Error
$exception = $e->getMessage();
if(strpos($exception, $err503Reason) !== false || strpos($exception, $err503ReasonB) !== false){
$expBackoff = true;
$sleeptime = $retrynum * 1; //retrynum * seconds
sleep($sleeptime);
$retrynum++;
} else {
$expBackoff = false;
print "There was an error -> $exception";
}
}
} else {
$expBackoff = false;
}
} while ($expBackoff);
I have been trying this solution for two days now and it has worked like a charm. Hopefully this might be helpful for someone else. I´m happy now. :)