Rate limit. Twitter API - php

I'm working on a small and simple code which basically does some tweets filtering. The problem is that I'm hitting the request limit of Twitter API and I would like to know if there is a workaround or if what I want to do just cannot be done.
First, I type a twitter username to retrieve the ID's of people this user follows.
$user_id = $_GET["username"];
$url_post = "http://api.twitter.com/1/friends/ids.json?cursor=-1&screen_name=" . urlencode($user_id);
$following = file_get_contents($url_post, true);
$json = json_decode($following);
$ids = $json->ids;
Twitter API responds with a list of ID's.
Here comes the problem. The next step is to make a request to find out username, profile picture and description for each one of those ID's.
$following = array();
foreach ($ids as $value)
{
$build_url = 'http://api.twitter.com/1/users/lookup.json?user_id=' . $value . '';
$following[] = $build_url;
}
foreach ($following as $url)
{
$data_names = file_get_contents($url, true); //getting the file content
$json_names = json_decode($data_names);
foreach ($json_names as $tweet) {
$name = $tweet->name;
$description = $tweet->description;
echo '<p>';
echo $name . '<br>';
echo $description;
echo '</p>';
}
}
If the user follows 50 people it works. But if he follows, let's say, 600 hundred, that would be 600 hundred request (for username, description and profile pic) to Twitter API which exceeds the limit.
Is there any way to workaround this o it just cannot be done?
Thank you!

You can and should request users/lookup API endPoint with 100 userIds at a time, instead of doing one request per twitter ID. cf. https://dev.twitter.com/docs/api/1.1/get/users/lookup
You have to replace your forEach loop (foreach ($following as $url)) by a recursive function.
At the end of the function, check the number of hits remaining before calling it again (cf. this link to see how to know the time remining until you get rate limited).
If there is no hit left, sleep 15 minutes before calling the function again, otherwise do the call again.
There is plenty of information on how to do this, use Google and search existing stackOverflow questions.

Related

Get info from API/URL

I have the URL https://android.rediptv2.com/ch.php?usercode=5266113827&pid=1&mac=02:00:00:00:00:00&sn=&customer=GOOGLE&lang=eng&cs=amlogic&check=3177926680
which outputs statistics.
For example:
[{"id":"2972","name":"MBC 1","link":"http://46.105.112.116/?watch=TR/mbc1-ar&token=RED_cexVeBNZ8mioQnjmGiYNEg==,1643770076.5266113827&t=1&s=2&p=1&c=BR&r=1351&lb=1","epg":"https://epg.cdnrdn.com/MBC1En.ae-20220201.xml","dvr":"disabled","language":"Arabic","category":"TOP 100","logo":"http://files.rednetcontent.com/chlogo/mbc1.png"},{"id":"1858","name":"MBC 2","link":"http://46.105.112.116/?watch=TN/mbc2-ar&token=RED_cexVeBNZ8mioQnjmGiYNEg==,1643770076.5266113827&t=1&s=2&p=1&c=BR&r=1351&lb=1","epg":"https://epg.cdnrdn.com/MBC2En.ae-20220201.xml","dvr":"disabled","language":"Arabic","category":"TOP 100","logo":"http://files.rednetcontent.com/chlogo/mbc2.png"},{"id":"1859","name":"MBC 3","link":"http://46.105.112.116/?watch=TN/mbc3-ar&token=RED_cexVeBNZ8mioQnjmGiYNEg==,1643770076.5266113827&t=1&s=2&p=1&c=BR&r=1351&lb=1","epg":"https://epg.cdnrdn.com/-20220201.xml","dvr":"disabled","language":"Arabic","category":"TOP 100","logo":"http://files.rednetcontent.com/chlogo/mbc3.png"}]
I want to get the value of link count.
Can anyone help?
I tried to do:
<?php
$content = file_get_contents("https://android.rediptv2.com/ch.php?usercode=5266113827&pid=1&mac=02:00:00:00:00:00&sn=&customer=GOOGLE&lang=eng&cs=amlogic&check=3177926680");
$result = json_decode($content);
print_r( $result->link );
?>
But it didn't work.
Put the JSON in an editor and you'll see that it's an array and not an object with the link attribute. This is why you cannot access it directly. You have to loop over the items and then you'll be able to access the link property of one of the items. If you need to access the link by id, as you asked 4 months later, then just create a dictionnary in an array indexed by id and containing just the interesting data you need.
PHP code:
<?php
// The result of the request:
$content = <<<END_OF_STRING
[{"id":"2972","name":"MBC 1","link":"http://46.105.112.116/?watch=TR/mbc1-ar&token=RED_cexVeBNZ8mioQnjmGiYNEg==,1643770076.5266113827&t=1&s=2&p=1&c=BR&r=1351&lb=1","epg":"https://epg.cdnrdn.com/MBC1En.ae-20220201.xml","dvr":"disabled","language":"Arabic","category":"TOP 100","logo":"http://files.rednetcontent.com/chlogo/mbc1.png"},{"id":"1858","name":"MBC 2","link":"http://46.105.112.116/?watch=TN/mbc2-ar&token=RED_cexVeBNZ8mioQnjmGiYNEg==,1643770076.5266113827&t=1&s=2&p=1&c=BR&r=1351&lb=1","epg":"https://epg.cdnrdn.com/MBC2En.ae-20220201.xml","dvr":"disabled","language":"Arabic","category":"TOP 100","logo":"http://files.rednetcontent.com/chlogo/mbc2.png"},{"id":"1859","name":"MBC 3","link":"http://46.105.112.116/?watch=TN/mbc3-ar&token=RED_cexVeBNZ8mioQnjmGiYNEg==,1643770076.5266113827&t=1&s=2&p=1&c=BR&r=1351&lb=1","epg":"https://epg.cdnrdn.com/-20220201.xml","dvr":"disabled","language":"Arabic","category":"TOP 100","logo":"http://files.rednetcontent.com/chlogo/mbc3.png"}]
END_OF_STRING;
$items = json_decode($content);
echo '$items = ' . var_export($items, true) . "\n\n";
// Create a dictionnary to store each link accessible by id.
$links_by_id = [];
// Loop over all items:
foreach ($items as $i => $item) {
// Show how to access the current link.
echo "Link $i = $item->link\n";
// Fill the dictionary.
$links_by_id[(int)$item->id] = $item->link;
}
// To access the first one:
echo "\nFirst link = " . $items[0]->link . "\n";
// Example of access by id:
// The id seems to be a string. It could probably be "1895" or "zhb34" or whatever.
// (If they are only numbers, we could convert the string to an integer).
$id = "1859";
echo "\nAccess with id $id = " . $links_by_id[$id] . "\n";
Test it here: https://onlinephp.io/c/e8ab9
Another important point: You are getting a 403 Forbidden error on the URL you provided. So typically, you will not obtain the JSON you wanted.
As I explained in the comment below, I think that you will not be able to access this page without having a fresh URL with valid query parameters and/or cookies. I imagine you obtained this URL from somewhere and it is no longer valid. This is why you'll probably need to use cURL to visit the website with a session to obtain the fresh URL to the JSON API. Use Google to find some examples of PHP scraping/crawling with session handling. You'll see that depending on the website it can get rather tricky, especially if some JavaScript comes into the game.

How to display this JSON data 'item' only once in this foreach loop (PHP)

I have a small function that grabs the avatars of comment authors on a specified video. It simply loops through the JSON data returned by YouTube API v3 commentThreads method.
The only issue is that sometimes an author has commented more than once, so my function is displaying the authors avatar more than once. I'd like to only display it one time, and on to the next avatar.
Here's a picture of what I mean:
Currently my function looks like this:
function videoCommentAvatars($video) {
// Parse YouTube video ID from the url
if (preg_match('%(?:youtube(?:-nocookie)?\.com/(?:[^/]+/.+/|(?:v|e(?:mbed)?)/|.*[?&]v=)|youtu\.be/)([^"&?/ ]{11})%i', $video, $match)) {
$video_id = $match[1];
}
// Gather Video stats with YouTube API v3
$api_key = "API_KEY_HERE";
$JSON = file_get_contents('https://www.googleapis.com/youtube/v3/commentThreads?part=snippet&videoId='.$video_id.'&key='.$api_key);
$json_data = json_decode($JSON, true);
if (!empty($json_data)) {
foreach ($json_data['items'] as $data) {
// Create variables that hold info
$author_name = $data['snippet']['topLevelComment']['snippet']['authorDisplayName']; // Author Name
$author_avatar = $data['snippet']['topLevelComment']['snippet']['authorProfileImageUrl']; // Author Avatar
$author_channel = $data['snippet']['topLevelComment']['snippet']['authorChannelUrl']; // Author Channel URL
echo '<span class="comment-author-avatar">';
echo '<a target="_blank" href="'.$author_channel.'" title="'.$author_name.'"><img width="50" alt="'.$author_name.'" class="comment-author-thumb-single" src="'.$author_avatar.'"></a>';
echo '</span>';
}
}
}
Everything works fine, but there's no way to check if an avatar has been displayed yet. I thought about using an array maybe? Adding each avatar URL to the array, and checking the array to see if the key exists. But that seems like overkill for something that's seemingly more simple. Does anyone have a clever way of checking the foreach loop for duplicates?
to check for duplicates in an array, you have a few options. Firstly, to get rid of any before looping, you can use array_unqiue($array) which will return an array that does not repeat it's values.
Or if you do need initial access to all values in the loop, and to do something if a repeat is present, you can have another array that acts as a record to see if they appear more than once.
$record = array();
foreach ($json_data['items'] as $data) {
// Create variables that hold info
$author_name = $data['snippet']['topLevelComment']['snippet']['authorDisplayName']; // Author Name
if(!in_array($author_name, $record)){
$author_avatar = $data['snippet']['topLevelComment']['snippet']['authorProfileImageUrl']; // Author Avatar
$author_channel = $data['snippet']['topLevelComment']['snippet']['authorChannelUrl']; // Author Channel URL
echo '<span class="comment-author-avatar">';
echo '<a target="_blank" href="'.$author_channel.'" title="'.$author_name.'"><img width="50" alt="'.$author_name.'" class="comment-author-thumb-single" src="'.$author_avatar.'"></a>';
echo '</span>';
$record[] = $author_name;
}
}
You can try a multi array with a unique index in your loop.
$author[$data['snippet'][...]['authorDisplayName']['avatar'] = $data['snippet'][...]['authorProfileImageUrl'];
So only unique results in the array.

Are Facebook Names Publicly Accesible Using Facebook API

Are the names of profiles on Facebook publicly accessible, as if I would NOT need to log into Facebook to access them?
I am intending to store a large amount of names as a small piece of a larger project. I feel as if scraping Facebook for names would be a relatively simple task using the Facebook Graph API, but I am a little confused.
I found another tutorial online at http://jilltxt.net/?p=2810 which described an easy way of finding any Facebook profile picture using one simple line:
https://graph.facebook.com/USER-ID/picture?type=large
This was very helpful because I am able to use a range of ID numbers and a small amount of PHP to gather large amounts of profile pictures as seen on my test page here: http://www.joshiefishbein.com/fi/photobook.php
But what I am unfamiliar with is how I go from collecting pictures to names in this one simple line. Is it possible? Is there another (better) way?
Here's the code I am working with. The range of ID's are just an example.
function gen_pix($min, $max, $quantity) {
$numbers = range($min, $max);
shuffle($numbers);
$x_arr = array_slice($numbers, 0, $quantity);
foreach ($x_arr as $key => $value) {
$username = "https://graph.facebook.com/" . $value . "/";
$json = json_decode(file_get_contents($username), true);
echo $json["name"];
}
}
$x = 337800042;
$y = 337800382;
$z = 1;
gen_pix($x,$y,$z);
I've gotten a little farther with this code, I can echo $username and I get the URL that I am looking for (for example https://graph.facebook.com/337800382/) but I do not get anything after that. json_decode isn't working seemingly.
In the same way you are pulling the profile picture, you can get the basic information of a user with their ID.
This page provides a list of data that is always publicly accessible.
So you need to make a GET request to pull back the JSON, like so...
https://graph.facebook.com/{user-id}/
For example https://graph.facebook.com/586207189/ pulls back my basic information. So your PHP would look like this
$json = json_decode(file_get_contents("https://graph.facebook.com/$user_id/"), true);
echo $json["name"];
PHP fiddle here
Update: Based on the code above, it's worth adding an IF to catch invalid Facebook IDs. Facebook IDs may not be sequential so not every one will return a name or image.
Updated code:
<?php
function gen_pix($min, $max, $quantity) {
$numbers = range($min, $max);
shuffle($numbers);
$x_arr = array_slice($numbers, 0, $quantity);
foreach ($x_arr as $key => $value) {
$username = "https://graph.facebook.com/" . $value . "/";
$json = json_decode(file_get_contents($username), true);
if (!isset($json['name'])) {
echo "Invalid ID<br />";
}
else {
echo $json["name"]. '<br />';
}
}
}
$x = 337800042;
$y = 337800382;
$z = 50;
gen_pix($x,$y,$z);
?>
PHP Fiddle here
It's also worth noting that pulling that much data from the graph is going to take a while. Have a look at doing batch requests to speed things up a bit. More info here

Facebook comments loop very very slow

I have this function in order to retrieve the count of Facebook comments to blog posts:
function comment_count($url) {
$json = json_decode(file_get_contents('https://graph.facebook.com/?ids=' . $url));
return ($json->$url->comments) ? $json->$url->comments : 0;
}
However if I insert it in a loop fetching the results of a query in order to retrieve five posts on a page, this function is seriously affecting the speed of the website (the page takes up to 6-7 seconds to load).
Is there a way to avoid this? Why is it so slow?
Thanks
Pass in a comma separated list of URLs to the ids parameter to get all the counts at once, or alternatively, cache them on the server side and use those values.
Example: https://graph.facebook.com/?ids=http://www.google.com,http://www.bing.com,http://www.yahoo.com
This is specified in Facebook's Graph API Reference under the section "selection"
An example implementation follows:
<?php
function comment_count($urls) {
$json = json_decode(file_get_contents('https://graph.facebook.com/?ids=' . implode(',', array_map("rawurlencode", $urls))));
$output = Array();
foreach($json as $url=>$data)
{
$output[$url] = isset($data->comments) ? $data->comments : 0;
}
return $output;
}
var_dump(comment_count(Array('http://www.facebook.com/', 'http://www.google.com')));
I hope this helps!

How to delete all Facebook Graph API apprequests?

$request_url ="https://graph.facebook.com/".$uid."/apprequests?".$access_token;
$requests = file_get_contents($request_url);
This gets all the requests for a user. But how do I delete all of them at once? Facebook only has an example for deleting them one by one.
Thanks!
You can't delete multiple items in a single operation (like you can with, say, SQL). You will need to iterate to some degree to specify the unique URL for each request. What you can do is batch up your operations into a single request to Graph API.
More info here at FB.
I know this question is old, but I found it searching on Google so I thought anyone could need an answer now.
The best method for me, is to send all the request ids to Facebook on a single api call.
Your request ids must be on the format requestid_userid, like 12345_67890, supposing you have all the IDs inside an array ($array_of_request_ids), the code (PHP) would be like this:
$ids = implode(',', $array_of_request_ids);
$facebook->api("/?ids={$ids}", 'DELETE');
That should delete all the requests.
if($requests) {
foreach($requests as $key => $data) {
$request_url = "https://graph.facebook.com/" .
$data['id'] . "?" . $access_token;
$requests = file_get_contents($request_url);
//Delete a request.
$delete_url = $request_url . "&method=delete";
$result = file_get_contents($delete_url);
}
}

Categories