Facebook Batch API insight requests - php

For a project I have to grab the insights of a page over a long period of time (e.g. 1-2 years) of facebook.
I first tried to do a single request but it turned out that only requesting
/PAGE_ID/insights?since=xxx&until=xxx
doesn't return all the data I want (it somehow supresses data as if there's some limit to the size of the answer).
I then tried to split up the date range (e.g. 01.04.2011-01.04.2011 -> 01.04.2011-01.08.2011-01.12.2011-01.04.2011) which as well, didn't quite work like I wanted it to.
My next approach was to request only the insight values I need, like 'page_stories, page_impressions, ...'. The requests looked like this
/PAGE_ID/insights/page_impressions/day?since=xxx&until=xxx
This actually worked but not with ajax. It sometimes seemed to drop some requests (especially if I changed the browser tab in google chrome) and I need to be sure that all requests return an answer. A synchronus solution would just take way too much time considering that one requests needs at least 2 seconds and with a date range of 2 years I may have about 300 single requests this just takes way too long to complete.
Lastly I stumbled over facebook ability to do batch requests which is exactly what I need. It can pack up to 50 requests in one call which significantly lowers the bandwith. And here's where I'm stuck. The facebook api gives some examples on how to use it but none of them worked when I tested them in the Graph Explorer and via the php facebook api sdk. I tried to pack this request
PAGE_ID/insights/page_fan_adds/day?since=1332486000&until=1333695600
into a batch request but failed.
It seems that the api is bugged. It's always giving me this error when I'm using a question mark '?' in the 'relative_url' field.
{
"error": {
"message": "batch parameter must be a JSON array",
"type": "GraphBatchException"
}
}
Here is what I tried:
These give the 'must be a JSON array' error:
?batch=[{"method":"GET","relative_url":"/PAGE_ID/insights/page_fan_adds/day?since=1332486000&until=1333695600"}]
These two actually return data but they are ignoring the parameters:
?batch=[{"method":"GET","relative_url":"/PAGE_ID/insights/page_fan_adds/day","body":"since=1332486000 until=1333695600"}]
?batch=[{"method":"GET","relative_url":"/PAGE_ID/insights/page_fan_adds/day","body":"since=1332486000,until=1333695600"}]
?batch=[{"method":"GET","relative_url":"/PAGE_ID/insights/page_fan_adds/day","body":{"since":"1332486000","until":"1333695600"}}]
And this one tells me that this is an 'Unsupported post request':
?batch=[{"method":"POST","relative_url":"/PAGE_ID/insights/page_fan_adds/day","body":"since=1332486000 until=1333695600"}]
Can someone help?

I finally found the solution to my problem. It's not mentioned in the facebook documentation but for this request
?batch=[{"method":"GET","relative_url":"/PAGE_ID/insights/page_fan_adds/day?since=1332486000&until=1333695600"}]
to properly work we have to use a function like
urlencode()
to encode the json part. This way the querys work like a charm. A php example:
$insights = $facebook->api('?batch=['.urlencode('{"method":"GET","relative_url":"/PAGE_ID/insights/page_fan_adds/day?since=1332572400&until=1333782000"}').']'
,'post',array('access_token' => $this->facebook->getAccessToken()));
which results in this:
?batch=[%7B%22method%22%3A%22GET%22%2C%22relative_url%22%3A%22%2FPAGE_ID%2Finsights%2Fpage_fan_adds%2Fday%3Fsince%3D1300086000%26until%3D1307862000%22%7D]

This example is for using an array of IDs to make a batch request with urlencoding.
$postIds = [
'XXXXXXXXXXXXXXX_XXXXXXXXXXXXXXX',
'XXXXXXXXXXXXXXX_XXXXXXXXXXXXXXX',
'XXXXXXXXXXXXXXX_XXXXXXXXXXXXXXX',
'XXXXXXXXXXXXXXX_XXXXXXXXXXXXXXX',
'XXXXXXXXXXXXXXX_XXXXXXXXXXXXXXX',
];
$queries = [];
foreach( $postIds as $postId ) {
$queries[] = [
'method' => 'GET',
'relative_url' => '/' . $postId . '/comments?summary=1&filter=stream&order=reverse_chronological',
];
}
$requests = $facebook->post( '?batch=' . urlencode( json_encode( $queries ) ) )->getGraphNode();

Related

Google translate API POST 11second delay?

EDIT: 11.5 seconds for 28 messages
Single requests work fine. This code below takes 11 seconds, measured using postman setting route in API to access.
Am I doing something wrong? I feel as though it shouldn't take 11 seconds even without cache.
$xs = ChatMessage::where('chat_room_id','=',$roomId)
->with('user')
->orderBy('created_at','DESC')
->get();
foreach($xs as $r){
$translate = new TranslateClient([
'key' => 'xxxxxxxxxxxxxxxxxxxxxxx'
]);
$result = $translate->translate($r->message_english, [
'target' =>'es',
'source' => 'en',
]);
$r->message = $result['text'];
}
return $xs;
I think you can easily speed the process by just moving the client out of the for each loop. You are creating a client each time you iterate. That's not optimal. You should be able to reuse the client per translate call. That should speed your translation process. You can find samples of this usage in the official github client project
Here is a pseudo code sample:
client = new TranslateClient()
foreach(message in messages)
result = client.translate(message)
print(result)
Also, how long is your translated text? You should pass the whole text to be translated into a single call (as long as the supported library allows) So you also reduce the calls done to the API.
If you still have issues you can go by using multiple request in parallel as mentioned in the comments.
Some useful links about this:
PHP Documentation Overview
Translate Client

Pulling Bright Local API Data into my Ruby on Rails App - API Docs written in PHP

I'm trying to build a rails app that pulls data from several different SEO tool API's. For Bright Local (see their API docs here - http://apidocs.brightlocal.com/) all the API doc examples are written in PHP, which I can't read all that great.
So first, to ask a specific question, how would I write this batch request in Ruby:
<?php
use BrightLocal\Api;
use BrightLocal\Batches\V4 as BatchApi;
$api = new Api('[INSERT_API_KEY]', '[INSERT_API_SECRET]');
$batchApi = new BatchApi($api);
$result = $batchApi->create();
if ($result['success']) {
$batchId = $result['batch-id'];
}
Also, any suggestions for how I can bring myself up to snuff on using API's in my rails apps?
Our docs do currently only show PHP examples - although we are planning to expand on that and Ruby is one of the languages we'll be looking to add.
A simple command line CURL request for the above PHP code would look like this:
curl --data "api-key=<YOUR API KEY HERE>" https://tools.brightlocal.com/seo-tools/api/v4/batch
and would return a response like this:
{"success":true,"batch-id":<RETURNED BATCH ID>}
All our API endpoints respond to either POST, PUT, GET or DELETE. It's also important to note that whenever data is posted with POST or PUT it's passed like "param1=value1&param2=value2" in the body of the request rather than JSON encoded.
I don't know Ruby at all I'm afraid but something like this might make the request you want:
params = {"api-key" => "<YOUR API KEY>"}
Net::HTTP::Post.new("https://tools.brightlocal.com/seo-tools/api/v4/batch").set_form_data(params)
I'm also implementing brightlocal into my Rails app. I'm using the HTTParty gem. this is what I have so far and am able to make successful calls
this is to obtain your batch id:
api_key = YOUR_API_KEY
secret_key = YOUR_SECRET_KEY
request_url = "https://tools.brightlocal.com/seo-tools/api/v4/batch?api-key=#{api_key}"
response = HTTParty.post(request_url)
if response.code == 201
batch_id = response['batch-id']
end
this is an example of running one job in the batch (the query parameters go inside the body):
rank_url = "https://tools.brightlocal.com/seo-tools/api/v4/rankings/search"
response = HTTParty.post(rank_url, {
:body => {
"api-key" => api_key,
"batch-id" => batch_id,
"search-engine" => "google",
"country" => "USA",
"search-term" => "restaurant"
}
})
I have not tested this next part, but theoretically, this is how you would deal with signatures and expirations
expires = Time.now.to_i + 1800
string_to_sign = "#{api_key}.#{expires}"
binary_signature = OpenSSL::HMAC.digest('sha1', string_to_sign, secret_key)
url_safe_signature = CGI::escape(Base64.encode64(binary_signature).chomp)
All that would be left is to use a PUT request to commit the batch, and a GET request to retrieve the data inside the batch.
EDIT: Figured out how to correctly get a passing signature for the jobs that require one. (this example is for local search rank checker http://apidocs.brightlocal.com/#local-search-rank-checker)
expires = Time.now.to_i + 1800
concat = api_key + expires.to_s
sig = OpenSSL::HMAC.digest('sha1', secret_key, concat)
sig = CGI::escape(Base64.encode64(sig).chomp)
local_rank = "https://tools.brightlocal.com/seo-tools/api/v2/lsrc/add?api-key=#{api_key}&sig=#{sig}&expires=#{expires}"
response = HTTParty.post(local_rank, {
:body => {
"name" => "pizza hut",
"search-terms" => "restaurant"
}
})
Since you are using Ruby and not PHP you will have to implement everything yourself. The example you give shows the user of a PHP wrapper created by BrightLocal (and it seems they only have it in PHP).
Basically you will have to make calls to the endpoints yourself and manage the data yourself instead of using their wrapper.

Mass ids request to Facebook Graph API

I am trying to get and array of more then 900 ids in this way, but I get an error message.
$url = 'https://graph.facebook.com/?ids=' . implode(',', $array);
$result = file_get_contents($url);
I think because the list is too big I got this error - if I shorten the list facebook gets me the JSON.
Is there any suggestion to get it in another fashion? (maybe batch call, or something like curl? )
The only other way is to implement a batch call with smaller API calls.
https://developers.facebook.com/docs/graph-api/making-multiple-requests

Google Play scraper

I'm trying to develop a Play Store reviews scraper in PHP and I need to make a POST request to this URL https://play.google.com/store/getreviews, and I saw the parameter post with firebug.
I am using Goutte library and here is my code:
require_once 'goutte.phar';
use Goutte\Client;
$client = new Client();
$params = Array(
"id" => "com.trello",
"pageNum" => 2 ,
"reviewSortOrder" => 2 ,
"reviewType" => 0,
"xhr" => 1
);
$crawler = $client->request('POST' , 'https://play.google.com/store/getreviews', $params);
The problem is that the request returns nothing. Is there anyone who already faced this problem and solved it?
I don't think this is possible. Google Play changed their review interface last year. They now have a "token" parameter which is missing here. I have worked before to try and work out what seeds this (see Google play review scraping changes) but I can't figure it out. After a number of attempts to hit that webservice with an incorrect request (presumably without the token) Google Play starts blocking your IP, that's why you'll be getting nothing back after a while (and won't be able to open Google Play in your browser either). If you find a solution, let me know!
This URL works for me, with the form-post data in your example.
https://play.google.com/store/getreviews?authuser=0

How to get user image with Twitter API 1.1?

In API 1.0, we can use users/profile_image/:screen_name
For example : http://api.twitter.com/1/users/profile_image/EA_FIFA_FRANCE
But, it doesn't work anymore in API 1.1.
Do you have a solution, please ?
You can also get the twitter profile image by calling this kind of url :
https://twitter.com/[screen_name]/profile_image?size=original
For instance : https://twitter.com/VancityReynolds/profile_image?size=original
Got the info from this post :
https://twittercommunity.com/t/how-to-get-user-image-original-size-with-api-1-1/10187/14
The user's profile image
Okay, so you want a user's profile image. You're going to need to take a look at the twitter REST API 1.1 docs. This is a list of all the different requests you can make to their API (don't worry, I'll get to how you actually do this later on).
There are multiple ways to get the user's profile image, but the most notable one is: users/show. According to the docs for this, the users/show method:
Returns a variety of information about the user specified by the required user_id or screen_name parameter. The author's most recent Tweet will be returned inline when possible.
Well, the user profile image must be in there somewhere, correct?
Let's have a look at a typical response to a request for this information, using the users/show url (we'll use my profile as an example).
I've cut off some from the bottom, because there is a lot of data to go through. Most importantly, you'll see what you require:
This is the profile_image_url key that you need to get access to.
So, how do you do all this? It's pretty simple, actually.
Authenticated Requests
As you rightly pointed out, as of June 11th 2013 you can't make unauthenticated requests, or any to the 1.0 API any more, because it has been retired. So OAuth is the way to make requests to the 1.1 API.
I wrote a stack overflow post with an aim to help all you guys make authenticated requests to the 1.1 API with little to no effort.
When you use it, you'll get back the response you see above. Follow the posts instructions, step-by-step, and you can get the library here (you only need to include one file in your project).
Basically, the previous post explains that you need to do the following:
Create a twitter developer account
Get yourself a set of unique keys from twitter (4 keys in total).
Set your application to have read/write access
Include TwitterApiExchange.php (the library)
Put your keys in a $settings array
Choose your URL and request method (Post/Get) from the docs (I put the link above!)
Make the request, that's it!
A practical example
I'm going to assume you followed the step-by-step instructions in the above post (containing pretty colour pictures). Here's the code you would use to get what you want.
// Require the library file, obviously
require_once('TwitterAPIExchange.php');
// Set up your settings with the keys you get from the dev site
$settings = array(
'oauth_access_token' => "YOUR_ACCESS_TOKEN",
'oauth_access_token_secret' => "YOUR_ACCESS_TOKEN_SECRET",
'consumer_key' => "YOUR_CONSUMER_KEY",
'consumer_secret' => "YOUR_CONSUMER_SECRET"
);
// Chooose the url you want from the docs, this is the users/show
$url = 'https://api.twitter.com/1.1/users/show.json';
// The request method, according to the docs, is GET, not POST
$requestMethod = 'GET';
// Set up your get string, we're using my screen name here
$getfield = '?screen_name=j7mbo';
// Create the object
$twitter = new TwitterAPIExchange($settings);
// Make the request and get the response into the $json variable
$json = $twitter->setGetfield($getfield)
->buildOauth($url, $requestMethod)
->performRequest();
// It's json, so decode it into an array
$result = json_decode($json);
// Access the profile_image_url element in the array
echo $result->profile_image_url;
That's pretty much it! Very simple. There's also users/lookup which effectively does the same thing, but you can:
Returns fully-hydrated user objects for up to 100 users per request, as specified by comma-separated values passed to the user_id and/or screen_name parameters.
If you ever need to get more than one user's details, use that, but as you only require one user's details, use users/show as above.
I hope that cleared things up a bit!
You say you want to use Twitter API 1.1 and yet you don't want to authenticate your requests.
Unauthenticated requests are not supported in API v1.1. So please adjust to the API change. See updates :
https://dev.twitter.com/blog/planning-for-api-v1-retirement
https://dev.twitter.com/docs/rate-limiting/1.1
You can get image from profile_image_url field of https://api.twitter.com/1.1/users/show.json request. Either a id or screen_name is required for this method. For example :
GET https://api.twitter.com/1.1/users/show.json?screen_name=rsarver
See details here https://dev.twitter.com/docs/api/1.1/get/users/show
I try the above methods to get the profile URL but it does not work for me. I think because Twitter changes API v1.1 to API v2.0.
I found a simple method to get a profile URL.
I use Twitter API v2 there User Lookup -> User by Username API part
Code Sample:
https://api.twitter.com/2/users/by/username/{user_name}?user.fields=profile_image_url
For Example:
https://api.twitter.com/2/users/by/username/TwitterDev?user.fields=profile_image_url
Of course, You should request with your Bearer Token then it properly work. For that, I recommend a platform it calls postman. It really helps for calling API.
Above example code return JSON like this:
{
"data": {
"name": "Twitter Dev",
"profile_image_url": "https://pbs.twimg.com/profile_images/1445764922474827784/W2zEPN7U_normal.jpg",
"username": "TwitterDev",
"id": "2244994945"
}
}
Additional:
If You want the Profile Image to be a higher size. Then you can put size in place of normal in the URL. For More Details read this one
Like This:
https://pbs.twimg.com/profile_images/1445764922474827784/W2zEPN7U_400x400.jpg
Give a vote to help more developers. 🍵
As the previous answers and comments point out:
Twitter API v1.0 is deprecated
Twitter API v1.1 requires OAuth
OP (#Steffi) doesn't want to authenticate
Pick any two; with all three it's a no-go. #Jimbo's answer is correct (and the proper way to do it), but excludes #3. Throwing out #1 means going back in time. But, we can throw out #2, and go directly to the source:
curl -s https://twitter.com/EA_FIFA_FRANCE |
sed -ne 's/^.*ProfileAvatar-image.*\(https:[^"]*\).*$/\1/p'
The sed command just says, find the line that contains "ProfileAvatar-image" and print the substring that looks like a quoted URL.
This is less stable than an authenticated API call, since Twitter may change their HTML at any time, but it's easier than dealing with OAuth, and no official rate limits!
The PHP translation should be straightforward.
try this
http://api.twitter.com/1/users/profile_image/{twitter_account}.xml?size=bigger
In API 1.1 the only way is to connect your application, retrieve the user by
https://dev.twitter.com/docs/api/1.1/get/users/show
and retrieve after his picture
profile_image_url
Hare is a very simple way to get Twitter Profile picture.
http://res.cloudinary.com/demo/image/twitter_name/w_300/{User_Name}.jpg
it's my Profile picutre:
Big: http://res.cloudinary.com/demo/image/twitter_name/w_300/avto_key.jpg
Small: http://res.cloudinary.com/demo/image/twitter_name/w_100/avto_key.jpg
you can regulate size by this part of URL - w_100, w_200, w_500 and etc.

Categories