Is there a way to sideload (load multiple API calls at the same time) API calls to lessen the impact on API call limits, using PHP?
For example, we're using the EchoNest API to gather information on musicians. When the artist page on our site is accessed, we run multiple functions which each call a different API method that returns the specific data that we need. Everything works and looks awesome!
Here are a few (abbreviated) methods that we're calling that each count against our call limit:
function artistPageNews() {
$artist_name = $_GET['artistname'];
$results = iTunes::search($artist_name, array(
'entity' => 'musicVideo'
))->results;
$echonest_api_key = "OUR_API_KEY";
// News Method
$echonest_news = 'http://developer.echonest.com/api/v4/artist/news?api_key='.$echonest_api_key.'&name='.str_replace(" ", "+", $artist_name).'&format=json&results=2&start=0';
$echonest_news_json = file_get_contents($echonest_news);
$news_json = json_decode($echonest_news_json);
$news_entry = $news_json->response->news;
foreach ($news_entry as $news) {
// Do Magic Stuff Here...
}
}
function artistPageVideos() {
$artist_name = $_GET['artistname'];
$results = iTunes::search($artist_name, array(
'entity' => 'musicVideo'
))->results;
$echonest_api_key = "OUR_API_KEY";
// Videos Method
$echonest_videos = 'http://developer.echonest.com/api/v4/artist/video?api_key='.$echonest_api_key.'&name='.str_replace(" ", "+", $artist_name).'&format=json&results=6&start=0';
$echonest_videos_json = file_get_contents($echonest_videos);
$videos_json = json_decode($echonest_videos_json);
$videos_entry = $videos_json->response->video;
foreach ($videos_entry as $video) {
// Do More Magic Stuff Here...
}
}
We have maybe about 7 (or more) of these methods that are called on each Artist page load. Obviously this can mean trouble when lots of people are viewing the artist pages every hour.
I understand that there's a way to store the more static information into a database and use that info instead of calling the API methods on every request. I am currently exploring that option. But I also read here that there may be a way to 'sideload' the API calls so that you can make multiple requests at one time. In that example, they're using Curl. I'm trying to do this with PHP.
curl https://{subdomain}.zendesk.com/api/v2/help_center/fr/articles.json?include=users \
-v -u {email_address}:{password}
Can anyone help me get started with this or perhaps recommend a better way to do this, such as storing this information into a database or table and pulling from that instead of calling the API every time?
Thanks in advance.
Related
I'm just looking at the Microsoft Graph API PHP SDK to get a bunch of resources, notably Users.
Looking a the SDK docs, there's 2 ways to get users, one using the createRequest() method and the other using the createCollectionRequest() method.
The docs suggests using the createCollectionRequest() and then just doing a while loop, array_merge and getPage() to create an array.
while (!$docGrabber->isEnd()) {
$docs = array_merge($docs,$docGrabber->getPage());
}
The issue is, I have a collection of ~50,000 users, so this method isn't particularly efficient.
I guess the biggest issue, i that the above example (using the while loop) is to avoid using the #odata.nextLink that the API returns.
But, what if we actually want to use this, instead of returning every single record in a single array?
Thanks
Instead of using getPage() and that sample, you can access the nextlink with something like this:
$url = "/users";
// Get the first page
$response = $graph->createCollectionRequest("GET", $url)
->setPageSize(50)
->execute();
if ($response->getNextLink())
{
$url = $response->getNextLink();
// TODO: remove https://graph.microsoft.com/v1.0 part of nextlink
} else {
// There are no more pages.
return null;
}
// get the next page, page size is already set in the next link
$response = $graph->createCollectionRequest("GET", $url)
->execute();
Information
I've started using the Asana API to make our own task overview in our CMS. I found an API on github which helps me a great deal with this.
As I've mentioned in an earlier question, I wanted to get all tasks for a certain user. I've managed to do this using the code below.
public function user($id)
{
if (isset($_SERVER['HTTP_X_REQUESTED_WITH']) &&
($_SERVER['HTTP_X_REQUESTED_WITH'] == 'XMLHttpRequest')) {
$this->layout = 'ajax';
}
$asana = new Asana(array(
'apiKey' => 'xxxxxxxxxxxxxxxxxxxx'
));
$results = json_decode($asana->getTasksByFilter(array(
'assignee' => $id,
'workspace' => 'xxxxxxxxxx'
)));
if ($asana->responseCode != '200' || is_null($results)) {
throw new \Exception('Error while trying to connect to Asana, response code: ' . $asana->responseCode, 1);
}
$tasks = array();
foreach ($results->data as $task) {
$result = json_decode($asana->getTaskTags($task->id));
$task->tags = $result->data;
$tasks[] = $task;
}
$user = json_decode($asana->getUserInfo($id));
if ($asana->responseCode != '200' || is_null($user)) {
throw new \Exception('Error while trying to connect to Asana, response code: ' . $asana->responseCode, 1);
}
$this->render("tasks", array(
'tasks' => $tasks,
'title' => 'Tasks for '.$user->data->name
));
}
The problem
The above works fine, except for one thing. It is slower than a booting Windows Vista machine (very slow :) ). If I include the tags, it can take up to 60 seconds before I get all results. If I do not include the tags it takes about 5 seconds which is still way too long. Now, I hope I am not the first one ever to have used the Asana API and that some of you might have experienced the same problem in the past.
The API itself could definitely be faster, and we have some long-term plans around how to improve responsiveness, but in the near-to-mid-term the API is probably going to remain the same basic speed.
The trick to not spending a lot of time accessing the API is generally to reduce the number of requests you make and only request the data you need. Sometimes, API clients don't make this easy, and I'm not familiar with the PHP client specifically, but I can give an example of how this would work in general with just the plain HTTP queries.
So right now you're doing the following in pseudocode:
GET /tasks?assignee=...&workspace=...
foreach task
GET /task/.../tags
GET /users/...
So if the user has 20 tasks (and real users typically have a lot more than 20 tasks - if you only care about incomplete and tasks completed in the last, say, week, you could use ?completed_since=<DATE_ONE_WEEK_AGO>), you've made 22 requests. And because it's synchronous, you wait a few seconds for each and every one of those requests before you start the next one.
Fortunately, the API has a parameter called ?opt_fields that allows you to specify the exact data you need. For example: let's suppose that for teach task, all you really want is to know the task ID, the task name, the tags it has and their names. You could then request:
GET /tasks?assignee=...&workspace=...&opt_fields=name,tags.name
(Each resource included always brings its id field)
This would allow you to get, in a single HTTP request, all the data you're after. (Well, the user lookup is still separate, but at least that's just 1 extra request instead of N). For more information on opt_fields, check out the documentation on Input/Output Options.
Hope that helps!
I need to get the state and country from the visitor IP. I will be using the country info to showcase custom made products. As for the state info it will not be used for the same purpose but only for record keeping to track the demand.
I have found on this site an instance of using the ipinfo.io API with this example code:
function ip_details($ip) {
$json = file_get_contents("http://ipinfo.io/{$ip}/json");
$details = json_decode($json);
return $details;
}
However, since I do not need the full details, I see that the site does allow to just grab single fields. So I am considering using these 2:
1) ipinfo.io/{ip}/region
2) ipinfo.io/{ip}/country
like so:
function ip_details($ip) {
$ip_state = file_get_contents("http://ipinfo.io/{$ip}/region");
$ip_country = file_get_contents("http://ipinfo.io/{$ip}/country");
return $ip_state . $ip_country;
}
OR would I be better off going with:
function ip_details($ip) {
$json = file_get_contents("http://ipinfo.io/{$ip}/geo");
$details = json_decode($json);
return $details;
}
The last one has the "/geo" in the url to slim down the selection from the first one with "/json". Currently I am leaning to the second option above by using 2 file_get_contents but wanted to know if it is slower than the last one having it in an array. Just want to minimize the load time. Or if any other method can be given it would be much appreciated.
In short, go for your second option, with a single request (file_get_contents makes a get request when parsed a url):
The result is a simple array, access the details you want via its key:
function ip_details($ip) {
$json = file_get_contents("http://ipinfo.io/{$ip}/geo");
$details = json_decode($json);
return $details;
}
$ipinfo = ip_details('86.178.xxx.xxx');
echo $ipinfo['country']; //GB
//etc
Regarding speed difference - 99% of the overhead is network latency, so making ONE request and parsing the details you need will be much faster than making 2 separate requests for individual details
I'm using Live Reporting Google APIs to retrieve active users and display the data inside a mobile application. On my application I'd like to make a HTTP request to a PHP script on my server which is supposed to return the result.
However I read on Google docs that it's better not to request data using APIs more often than 30 seconds.
I prefer not to use a heavy way such as a cron job that stores the value inside my database. So I'd like to know if there's a way to cache the content of my PHP scrpit na dmake it perform an API request only when the cache expires.
Is there any similar method to do that?
Another way could be implementing a very simple cache by yourself.
$googleApiRequestUrlWithParameter; //This is the full url of you request
$googleApiResponse = NULL; //This is the response by the API
//checking if the response is present in our cache
$cacheResponse = $datacache[$googleApiRequestUrlWithParameter];
if(isset($cacheResponse)) {
//check $cacheResponse[0] for find out the age of the cached data (30s or whatever you like
if(mktime() - $cacheResponse[0] < 30) {
//if the timing is good
$googleApiResponse = $cacheResponse[1];
} else {
//otherwise remove it from your "cache"
unset($datacache[$googleApiRequestUrlWithParameter]);
}
}
//if you do no have the response
if(!isset($googleApiResponse)) {
//make the call to google api and put the response in $googleApiResponse then
$datacache[] = array($googleApiRequestUrlWithParameter => array(mktime(), $googleApiResponse)
}
If you data are related to the user session, you could store $datacahe into $_SESSION
http://www.php.net/manual/it/reserved.variables.session.php
ortherwise define $datacache = array(); as a global variable.
There is a lot of way of caching things in PHP, the simple/historic way to manage cache in PHP is with APC http://www.php.net/manual/book.apc.php
Maybe I do not understard correctly your question.
Greetings,
I already have a working connection to the AD and can search and retrieve information from it. I've even developed a recursive method by which one can retrieve all groups for a given user. However, I'd like to avoid the recursion if possible. One way to do this is to get the tokenGroups attribute from the AD for the user, which should be a list of the SIDs for the groups that the specified user has membership, whether that membership be direct or indirect.
When I run a search for a user's AD information, though, the tokenGroups attribute isn't even in it. I tried specifically requesting that information (i.e., specifying it using the fourth parameter to ldap_search) but that didn't work, either.
Thanks,
David Kees
Solved my own problem and thought I'd put the answer here so that others might find it. The issue was using the ldap_search() function. The answer was to use the ldap_read() function instead of ldap_search(). The difference is the scope of the request. The search function uses a scope of "sub" (i.e., subtree) while the read function uses "base." The tokenGroups information can only be found when using a scope of "base" so using the correct PHP function was the key.
As I mentioned above, I was working from someone else code in perl to create my solution and the perl script used a function named "search" to do it's LDAP requests which lead me down wrong path.
Thanks to those who took a peek at the question!
--
As per the requests in the comments, here's the basics of the solution in code. I'm extracting from an object that I use so this might not be 100% but it'll be close. Also, variables not declared in this snipped (e.g. $server, $user, $password) are for you to figure out; I won't know your AD credentials anyway!
$ldap = ldap_connect($server);
ldap_bind($ldap, $user, $password);
$tokengroups = ldap_read($ldap, $dn, "CN=*", array("tokengroups")));
$tokengroups = ldap_get_entries($ldap, $tokengroups);
At this point, $tokengroups is our results as an array. it should have count index as well as some other information. To extract the actual groups, you'll need to do something like this:
$groups = array();
if($tokengroups["count"] > 0) {
$groups = $tokengroups[0]["tokengroups"];
unset($groups["count"]);
// if you want the SID's for your groups, you can stop here.
// if you want to decode the SID's then you can do something like this.
// the sid_decode() here: http://www.php.net/manual/en/function.unpack.php#72591
foreach($groups as $i => &$sid) {
$sid = sid_decode($sid);
$sid_dn = ldap_read($ldap, "<SID=$sid>", "CN=*", array("dn"));
if($sid_dn !== false) {
$group = ldap_get_entries($ldap, $sid_dn);
$group = $group["count"] == 1 ? $group[0]["dn"] : NULL;
$groups[$i] = $group;
}
}
}
That's the basics. There's one caveat: you'll probably need to work with the individual or individuals who manage AD accounts at your organization. The first time I tried to get this running (a few years ago, so my memory is somewhat fuzzy) the account that I was given did not have the appropriate authorization to access the token groups information. I'm sure there are other ways to do this, but because I was porting someone else's code for this specific solution, this was how I did it.