I am working on Facebook ads api to get the account Campaign data.What I am doing here is I get list of all campaigns and doing forloop of each campaign get Campaign stat
$campaignSets = $account->getCampaigns(array(
CampaignFields::ID,
CampaignFields::NAME
));
foreach ($campaignSets as $campaign) {
$campaign = new Campaign($campaign->id);
$fields = array(
InsightsFields::CAMPAIGN_NAME,
InsightsFields::IMPRESSIONS,
InsightsFields::UNIQUE_CLICKS,
InsightsFields::REACH,
InsightsFields::SPEND,
InsightsFields::TOTAL_ACTIONS,
InsightsFields::TOTAL_ACTION_VALUE
);
$params = array(
'date_preset' => InsightsPresets::TODAY
);
$insights = $campaign->getInsights($fields, $params);
}
when executing above code I am getting error as (#17) User request limit reached.
Can anyone help me how to solve this kind of error?
Thanks,
Ronak Shah
You should consider generating a single report against the adaccount which returns insights for all of your campaigns, this should reduce the number of requests required significantly.
Cursor::setDefaultUseImplicitFetch(true);
$account = new AdAccount($account_id);
$fields = array(
InsightsFields::CAMPAIGN_NAME,
InsightsFields::CAMPAIGN_ID,
InsightsFields::IMPRESSIONS,
InsightsFields::UNIQUE_CLICKS,
InsightsFields::REACH,
InsightsFields::SPEND,
InsightsFields::TOTAL_ACTIONS,
InsightsFields::TOTAL_ACTION_VALUE,
);
$params = array(
'date_preset' => InsightsPresets::TODAY,
'level' => 'ad',
'limit' => 1000,
);
$insights = $account->getInsights($fields, $params);
foreach($insights as $i) {
echo $i->campaign_id.PHP_EOL;
}
If you run into API limits, your only option is to reduce calls. You can do this easily by delaying API calls. I assume you are already using a Cron Job, so implement a counter that stores the last campaign you have requested the data for. When the Cron Job runs again, request the data of the next 1-x campaign data (you have to test how many are possible per Cron Job call) and store the last one again.
Also, you should batch the API calls - it will not avoid limits, but it will be a lot faster. As fast as the slowest API call in the batch.
Add this to your code and you'll never have to worry about FB's Rate Limiting/User Limit Reached.
Your script will automatically sleep as soon as you approach the limit, and then pick up from where it left after the cool down. Enjoy :)
import logging
import requests as rq
#Function to find the string between two strings or characters
def find_between( s, first, last ):
try:
start = s.index( first ) + len( first )
end = s.index( last, start )
return s[start:end]
except ValueError:
return ""
#Function to check how close you are to the FB Rate Limit
def check_limit():
check=rq.get('https://graph.facebook.com/v3.3/act_'+account_number+'/insights?access_token='+my_access_token)
call=float(find_between(check.headers['x-business-use-case-usage'],'call_count":','}'))
cpu=float(find_between(check.headers['x-business-use-case-usage'],'total_cputime":','}'))
total=float(find_between(check.headers['x-business-use-case-usage'],'total_time":',','))
usage=max(call,cpu,total)
return usage
#Check if you reached 75% of the limit, if yes then back-off for 5 minutes (put this chunk in your loop, every 200-500 iterations)
if (check_limit()>75):
print('75% Rate Limit Reached. Cooling Time 5 Minutes.')
logging.debug('75% Rate Limit Reached. Cooling Time 5 Minutes.')
time.sleep(300)
Related
I'm working inside a Cake PHP 2 web application, I have a database table called jobs where data is stored, I have a Console command which runs on a cron every minute and when it runs it grabs data from my jobs table in a function called getJobsByQueuePriority and then does something.
The issue I'm facing is that I have multiple cron jobs that need to be ran every minute and need to run at the same time, when they run, they're both grabbing the same sets of data from the database table, how can I prevent this and ensure that if a column was already retrieved by one cron, the other cron picks a different row?
I Initially tried adding 'lock' => true to my queries as per the docs, but this isn't achieving the result I need as when logging data to a file both running crons are pulling the same database entry ID's.
I then tried using transactions, I put a begin before the queries and a commit afterwards, maybe this is what I need to use but am using it slightly wrong?
The function which performs the required query with my attempt of transactions is:
/**
* Get queues in order of priority
*/
public function getJobsByQueuePriority($maxWorkers = 0)
{
$jobs = [];
$queues = explode(',', $this->param('queue'));
// how many queues have been set for processing?
$queueCount = count($queues);
$this->QueueManagerJob = ClassRegistry::init('QueueManagerJob');
$this->QueueManagerJob->begin();
// let's first figure out how many jobs are in each of our queues,
// this is so that if a queue has no jobs then we can reassign
// how many jobs can be allocated based on our maximum worker
// count.
foreach ($queues as $queue) {
// count jobs in this queue
$jobCountInQueue = $this->QueueManagerJob->find('count', array(
'conditions' => array(
'QueueManagerJob.reserved_at' => null,
'QueueManagerJob.queue' => $queue
)
));
// if there's no jobs in the queue, subtract a queue
// from our queue count.
if ($jobCountInQueue <= 0) {
$queueCount = $queueCount - 1;
}
}
// just in case we end up on zero.
if ($queueCount <= 0) {
$queueCount = 1;
}
// the amount of jobs we should grab
$limit = round($maxWorkers / $queueCount);
// now let's get all of the jobs in each queue with our
// queue count limit.
foreach ($queues as $queue) {
$job = $this->QueueManagerJob->find('all', array(
'conditions' => array(
'QueueManagerJob.reserved_at' => null,
'QueueManagerJob.queue' => $queue
),
'order' => array(
'QueueManagerJob.available_at' => 'desc'
),
'limit' => $limit
));
// if there's no job for this queue
// skip to the next so that we don't add
// an empty item to our jobs array.
if (!$job) {
continue;
}
// add the job to the list of jobs
array_push($jobs, $job);
}
$this->QueueManagerJob->commit();
// return the jobs
return $jobs[0];
}
What am I missing or is there a small change I need to tweak in my function to prevent multiple crons picking the same entries?
Basically i have rate limit from my API provider for 50 simultaneous jobs. as an example lets say i have to run 500 jobs:
$jobs = $this->db->get('jobs')->result_array(); (loads all 500 jobs)
Bellow code loops api query for all 500 jobs.
foreach($jobs as $job)
{
//API call
}
each job has status parameter $job['status'], i want to do the following:
to avoid abuse of rate limit, i want to count rows with busy status
$busy = $this->db->get_where('jobs', ['status' => 'busy']->num_rows();
and keep on checking (looping) until $busy < 50
Final results
foreach($jobs as $job)
{
//API call
//Count busy jobs
//If busy >= 50 wait(50) and check again - (I need help with this part, no idea how to do it)
//If busy < 50 continue foreach loop
}
Hope all details are in place.
p.s.
I am using:
CodeIgniter3, PHP7.4
Edit:
To avoid confusion as mentioned in final result (//If busy >= 50 wait(50) and check again - (I need help with this part, no idea how to do it)
)
I am looking for a way to loop $busy SQL query until num_row() will be less than 50.
Found solution
$busy = $this->db->get_where('jobs', ['status' => 'busy']->num_rows();
while($busy >= 50 ) {
$busy = $this->db->get_where('jobs', ['status' => 'busy']->num_rows();
}
My code is based off the sample mentioned on this page:
use Google\Cloud\Logging\LoggingClient;
$filter = sprintf(
'resource.type="gae_app" severity="%s" logName="%s"',
strtoupper($level),
sprintf('projects/%s/logs/app', 'MY_PROJECT_ID'),
);
$logOptions = [
'pageSize' => 20,
'resultLimit' => 20,
'filter' => $filter,
];
$logging = new LoggingClient();
$logs = $logging->entries($logOptions);
foreach ($logs as $log) {
/* Do something with the logs */
}
This code is (at best) slow to complete, and (at worst) times out on the foreach loop with a DEADLINE_EXCEEDED error.
How can I fix this?
If your query does not match the first few logs it finds, Cloud Logging will attempt to search your entire logging history for the matching logs.
If there are too many logs to filter through, the search will time out with a DEADLINE_EXCEEDED message.
You can fix this by specifying a time frame to search from in your filter clause:
// Specify a time frame to search (e.g. last 5 minutes)
$fiveMinAgo = date(\DateTime::RFC3339, strtotime('-5 minutes'));
// Add the time frame constraint to the filter clause
$filter = sprintf(
'resource.type="gae_app" severity="%s" logName="%s" timestamp>="%s"',
strtoupper($level),
sprintf('projects/%s/logs/app', 'MY_PROJECT_ID'),
$fiveMinAgo
);
I am trying to get outbound sms logs against each individual phone number configured in twilio sub-account. I am using PHP programming language and looked into the api console:
https://www.twilio.com/docs/api/rest/usage-records
But according to my exploration, usage-record api only returns total usage information against my twilio account. i.e. 500 outbound sms against all the three subscribed numbers in my account say N1, N2 and N3. What I want here is to retrieve the number of sms sent and received individually against each number N1, N2 and N3. Please help me in this regard.
Thanks
I have already did like this.
$sub_acc = $client->accounts->get($sub_account->sid);
$numbers = $sub_acc->incoming_phone_numbers;
$phone_sms_count = array();
foreach ($numbers as $number) {
$sms_count = 0;
foreach ($sub_acc->messages->getIterator(0, 50, array(
"From" => $number->phone_number,
"DateSent" => $last_six_months_date)) as $sms)
{
$sms_count++;
}
$phone_sms_count[$number->phone_number] = $sms_count;
}
But it takes too much time to calculate sms counts for each number. I have configured 5 numbers in single subaccount and I have 10 subaccounts
this will break the limits.
Is there any way to get sms counts for each phone number without iterating messages?
I'm not sure why you received downvotes but to me the question is legit so I'll try to help.
You can use the Message API List Resource. Just filter the date range and from phone number. So to get records for last month (May 2016) just filter like:
// Set options
$from_number="XXXXXXXXXX";
$start_date="2016-05-01";
$end_date="2016-05-31";
// Build the option array
$get_iterator_options = array(
'from' => $from_number,
'DateSent' => $start_date,
'DateSent' => $end_date
);
// Make the call with our options
foreach ($client->account->message->getIterator(
0,
50,
$get_iterator_options
) as $message
) {
$msg_ids[] = $message->sid; //
}
echo count($msg_ids);
Twilio provides a List Resource for Messages, Calls, Recordings, etc..
Keep in mind this is almost certainly slower than what you were trying to do originally especially if there are a lot of messages.
You can create a subaccount for each subscribed number (documentation). You can then retrieve total usage per subaccount.
I am using https://stripe.com/docs/api?lang=php#list_charges to get List all Charges but here they specify
count optional — default is 10 A limit on the number of objects to be
returned. Count can range between 1 and 100 items.
and I have thousands of entries, now how can I get all. Though if I set count to 100 it returns 110 records.
You can use the offset argument.
Once you get the 100 transactions, then make another call by adding offset=100 in URL.
This will bring the next 100 transactions, then make offset=200 and so on.
Update:
offset parameter is partly deprecated: API changelog - 2015-09-23
$charges = \Stripe\Charge::all();
foreach ($charges->autoPagingIterator() as $charge) {
// Do something with $charge
}
Reference.
Yes I got it with offset we can get all records.
Here's a PHP example: \Stripe\Charge::all(array("limit" => 3, "offset" => 10));
A Ruby example:
Stripe::Charge.all(limit: 3, offset:3)
As good as the Stripe API docs are, they could be clearer on how to filter.
source: https://stripe.com/docs/api/php#list_charges, https://stripe.com/docs/api/ruby#list_charges
in case offset is deprecated
$result = [];
$created_at = strtotime($request->end_data);
//created_at should be today's date epoch. search google for epoch
$has_more = false;
$a = 0;
do{
print_r($a);
\Stripe\Stripe::setApiKey(env('STRIPE_SECRET'));
$temp = \Stripe\BalanceTransaction::all( array(
'limit' => 100,
'created' => array(
'lte' => $created_at,
)
));
$result = array_merge($temp->data,$result);
$created_at = $temp->data[99]->created_at;
//api returns a parameter has_more(boolean), which means there is more
//data or not so you can also put that in while condition, for ex.
// $has_more = $temp->has_more;
$a++;
}while($a < 5);
dd($result);
this worked for me i was able to get 500 records at once as $a < 5 the api hits 5 times and each time created parameter which is lte (less than equal) changes for each api request and return previous records than current request provide. also i am appending the result of each api call to another result array
Unfortunately you can't.
I can see where such a feature would be nice for accounting purposes or whatever, but it's generally a better user experience to implement some sort of paging when displaying copious amounts of data to the user.
If you need absolute control over how many records to display at a time, I would suggest setting up a webhook on the charge.succeeded event and store your charges locally.