Sending notification to 25k+ users causing problem on backend - php

There is currently 25k+ users we have in database. All users are subscribed to a common topic All.
I have two directories inside public_html. First is for API built in codeigniter. This API is used to provide data for all adnroid and iOS devices.
Second directory is for admin panel built in Laravel. We use it to upload data and also to send notification to firebase topics.
Both API and Admin Panel share same database.
If we send notification to topics which is not subscribed by much
users, there is no issue. But If I send notificaiton to a topic
which has much users It causes problems on our backend. The API
stops responding, or sometimes takes too long to respond.
Sometimes also admin panel stops responding too.
I am so confused because all the things are handled by firebase. I just make one API call.
Can anyone explain what's causing the problem?
Or any possible reason?
Update
use Kreait\Firebase\Messaging;
use Illuminate/Support/Http/Request;
trait UserTrait {
public function notify(Request $request, Messaging $messaging) {
$message_hi = array(
"notification_type" => $notification_type,
"notification_title" => $notification_title_hi,
"icon_image" => $icon_image,
"notification_description" => $request->notificationText_hi,
"image_url" => $request->image_url,
);
$message = array(
"notification_type" => $notification_type,
"notification_title" => $notification_title,
"icon_image" => $icon_image,
"notification_description" => $request->notificationText,
"image_url" => $request->image_url,
);
$commodityIdArray = $request->cId
//to send all
if($request->notification_type == 1) {
$messaging->sendAll([
['topic' => 'All', 'data' => $message],
['topic' => 'All_hi', 'data' => $message_hi],
]);
} else {
//to send to a fourite topic subscribed by some users
//Prepare Condition for both hindi and english users
$topic_condition = "";
$topic_condition_hi = "";
foreach($commodityIdArray as $topic) {
$topic_condition .="'".$topic."' in topics && ";
$topic_condition_hi.="'".$topic."_hi' in topics &&";
}
//send notification to hindi and english topics
$messaging->sendAll([
['condition' => substr($topic_condition, 0, -3), 'data' =>
$message],
['condition' => substr($topic_condition_hi, 0, -3), 'data' =>
$message_hi],
]);
}
}

Use Queue
You have to use a Queue which set your process in queue and when one process completes, the second one starts
you can also set number of retries of your process

Related

LARAVEL - Response from SAME url, showing different results

UPDATE 1
#lukepolo found out that the number 23842515550750742 randomly always goes to 23842515550750744 in JS. Any ideas why?
I am using Laravel for an API I am creating. I am using Forge to run the servers.
When I pull a URL for the interface in Chrome, Safari, FireFox, etc., it is returning an ID that has been increased by 2. When I pull that SAME url, with the same exact request in Postman, CRUD, Terminal via cURL, etc., it returns the CORRECT response.
Results from Postman:
accountid: 1334372826597482
campaign:"Facebook"
campaignid: 23842515550750742 <-- CORRECT
client_id: 72
id: 817
Results from Chrome:
accountid:1334372826597482
campaign:"Facebook"
campaignid:23842515550750744 <-- INCORRECT
client_id:72
id:817
I am using the Facebook Marketing API to save the information locally, so I can run massive queries in a much faster resposne than their API would give me. Here is the code for listing the campaigns with basic information:
$this->parseRequestData($request);
$campaigns = FacebookPerformanceOverview::whereIn('client_id', $this->sheet_ids)
->whereIn('accountid', $this->accountIds['facebook'])
->whereIn('campaignid', $this->campaigns)
->whereBetween('date', [$this->start_date, $this->end_date])
->groupBy('campaignid')
->get();
$response = [];
foreach ($campaigns AS $campaign) {
$data = FacebookPerformanceOverview::whereIn('client_id', $this->sheet_ids)
->whereIn('accountid', $this->accountIds['facebook'])
->where('campaignid', $campaign->campaignid)
->whereIn('adsetid', $this->adsets)
->whereBetween('date', [$this->start_date, $this->end_date]);
$response[] = array(
'company' => $campaign->company,
'campaign' => FacebookCampaignSettings::where('campaignid', $campaign->campaignid)->first(),
'impressions' => $data->sum('impressions'),
'clicks' => $data->sum('clicks'),
'cost' => $data->sum('cost'),
'reach' => $data->sum('reach'),
'frequency' => $data->avg('frequency'),
'uniqueclicks' => $data->sum('uniqueclicks'),
'actions' => $data->sum('actions'),
'offsiteconversions' => $data->sum('offsiteconversions'),
'leads' => $data->sum('actionsleadgen') + $this->calculateLeads(),
);
}
return response()->json($response, 200, [], JSON_NUMERIC_CHECK);
I am unable to give the URL, due to it containing sensitive information, but I am confused as to what is going on. I have cleared cache, different browsers, different computers, etc. Web browsers always return the ID in the network tab incorrectly. Any ideas?
Javascript not parsing large number in JSON correctly
Is the answer.

Google Analytics server-side tracking

Google Analytics, by just placing its sourcecode on my website, automatically tracks everything I used to need (pageviews, unique visitors).
But now, I need to track events, and the only way to do this is to do it server-side. Each time any users does an specific action i need to track, the server posts data to google to track the information, as explained here:
https://developers.google.com/analytics/devguides/collection/protocol/v1/devguide#event
And it does works amazingly perfect, but, since I realiced, I am now receiving a LOT of visits from Spain, doubling the visits from USA. And before I implemented the event tracking, Spain wasn't even part of the top 10 countries.
Today I have realiced that my servers are in Spain, and that may be causing the issue.
How can I track the event, without making it count as a pageview?
$url = 'http://www.google-analytics.com/collect';
$data = array('v' => '1', 'tid' => 'UA-HIDDEN-1', 'cid' => $_SERVER["REMOTE_ADDR"], 'ni' => '1', 't' => 'event', 'ec' => '', 'ea' => 'JUMP', 'el' => '');
$options = array(
'http' => array(
'header' => "Content-type: application/x-www-form-urlencoded\r\n",
'method' => 'POST',
'content' => http_build_query($data),
),
);
$context = stream_context_create($options);
$result = file_get_contents($url, false, $context);
Thank you very much!!
You are sending the IP adress as a client id, which is wrong. For one, the client id is supposed to be an UUID. Secondly, Analytics won't recognize that these events belong to an existing user.
You'd need to grab the existing client id for an existing user on the web page:
ga(function(tracker) {
var clientId = tracker.get('clientId');
});
and then send it back to the server and use it in your request (1). At the moment GA cannot assign correct geo information since the events do not belong to the session of the user who initiates the event (this quite possibly affects some other metrics, too).
(1) You might as well read the GA cookie in PHP, but Google recommends against it since the cookie format might change without notice. The script above will always return a correct client id even if the cookie format changes.
Updated: I have read a bit more documentation and while my answer seems still somewhat relevant it's probably wrong for the actual use case - Geo is determined by IP and the serverside script will still send the servers IP. So quite possibly (haven't done the science yet) this would look like one visitor with two devices instead of a single visitor.
Update 2: Apparently it is now possible to include the users IP adress as parameter, so this answer is possibly no longer relevant.
Here is a techopad presentation about mixing UA client- and serverside, maybe that helps.
An event in and of itself is not a pageview. See: Event Tracking
Is there a specific reason why you need to track events server side and pageviews from the normal ga.js client-side code?
You can easily track events from the client side, if you were unaware of that:
Click Link to Track Event
Assuming that you needed to keep events AND pageviews on the server side:
<?php
//Put SERVER_ADDR into a var
$request_ip = $_SERVER['REMOTE_ADDR'];
// Put any server IPs you need to filter out below in an array
$localhosts = array('127.0.0.1','192.168.15.1','10.1.10.1');
// Use this later
$url = 'http://www.google-analytics.com/collect';
Now, Figure out what to do with the REMOTE_ADDR check if its in our list above. then build an array of type to send GA (events, pageviews)
$actions = array();
// Note that the values are arbitrary and will let you do what you need.
if(in_array($request_ip)){
//Only track event, or track pageview differently, or track two events.
$handle_myServer = true;
$actions = ('event');
} else {
// Track everyone else
$handle_myServer = false;
$actions = ('event','pageview','mySpecialPageview','mySpecialEvent');
}
Finally We have built a list of events we can use in flow control with existing code for pageviews, user timing, events, etc. Be creative!
foreach($actions as $action){
$data = null; $options=null;
if($handle_myServer){
$someFlagForGA = 'RequestFromSpainServer';
}
if($action == 'event'){
$data = array('v' => '1'
, 'tid' => 'UA-HIDDEN-1',
,'cid' => $request_ip
,'ni' => '1'
, 't' => 'event'
, 'ec' => $someFlagForGA,
,'ea' => 'JUMP', 'el' => ''
);
} elseif($action == 'pageview'){
$data = array('v' => '1', 'tid' => 'UA-HIDDEN-1'
, 't' => 'pageview'
, 'dh'=> 'yourGAenabledDomainHere.com'
, 'dp'=> 'ViewedPage.html'
, 'dt'=> 'homepage'.' SERVER VISITED '.$someFlagForGA
);
} else {
// Do whatever else
}
// Would be better to do below with a single function
$options = array(
'http' => array(
'header' => "Content-type: application/x-www-form-urlencoded\r\n",
'method' => 'POST',
'content' => http_build_query($data),
) ,$data);
$context = stream_context_create($options);
$result = file_get_contents($url, false, $context) or die('Error!!');
}
?>

Zendesk php api create ticket without sending email to the user?

I am trying to create tickets on my Zendesk and that is working fine. However i do not want Zendesk to email the creator of the tickets (his or her email). Is this possible?
The idea is i have a contactForm widget on my site, i want the submits from this form to create tickets in my Zendesk.
Creating tickets is currently working using this code:
$zendesk = new zendesk(
$row->api_key,
$row->email_address,
$row->host_address,
$suffix = '.json',
$test = false
);
$arr = array(
"z_subject"=>"Offline Message",
"z_description"=> $r->contact_msg,
"z_recipient"=>$r->contact_email,
"z_name"=>$r->contact_name,
);
$create = json_encode(
array('ticket' => array(
'subject' => $arr['z_subject'],
'description' => $arr['z_description'],
'requester' => array('name' => $arr['z_name'],
'email' => $arr['z_requester']
))),
JSON_FORCE_OBJECT
);
$data = $zendesk->call("/tickets", $create, "POST");
Any ideas?
Totally possible! You need to add some conditions to the trigger "Notify requester of received request" in Zendesk - Trigger setting to prevent zendesk from sending email. For ex:
Ticket : Channel - Is Not - Webservice (API)
Ticket : Tags - Contains one of the following - "offline message"
You could use another API endpoint "Ticket Import" https://developer.zendesk.com/api-reference/ticketing/tickets/ticket_import/
It's do not send notifications

Posting to wall users use token

I have php code that posting in the wall of the user that use in my app. But in the post write
"post by _" (My app name).
Code:
$fql_info = $facebook->api($fql_query);
if($fql_info[0][offline_access] == 1)
{
$attachment = array(
'message' => "",
'name' => u("ghfhgfhfgh"),
'description' => u("fghgfhgfh"),
'link' => LINK,
'picture'=> "http://gfhgfhgfhfh"
);
$facebook->api('/'.$f[uid].'/feed', 'POST', $attachment);
$abc.=$f[uid]."\n";
echo $f[uid]."-".$fql_info[0][offline_access]."\n";
}
I want to know how I can to post but without my app name. User as poster itself, not the application has released it.
is it possible?
I guess it's not possible. The app name will appear in the post.
Sorry to say but i don't think this is possible . Facebook would not allow us to do that , it will always say Posted via : [Your APP name]

PayPal IPN, lots of sign ups, one payment, all in the wrong order

I've recently implemented a PayPal IPN into CodeIgniter2, using the PayPal Lib. I'm using the system for subscriptions.
I have a table in my database that records all IPN requests in the database.
For some reason, after every sign up the IPN requests aren't coming through properly. I tend to get one subscr_payment along with several subscr_signups, all with the same subscr_id. It's causing untolds amount of hassle within the system, for obvious reasons. What adds to this, is the fact that the IPN requests don't come in the correct order, sometimes I get the subscr_payment before the subscr_signup - making it impossible to track as there's no subscr_id from the sign up to link it to a user.
I've had a Google and can't find much on this, I seem to be a little bit of an anomaly. I'm wondering if it's something to do with the PayPal Lib I'm using, but I don't really want to have to do it outside of CodeIgniter, as I am doing a lot of processing. Below is the full IPN script.
class Paypal extends CI_Controller {
function _construct()
{
parent::_construct();
$this->load->library('paypal_lib');
}
function ipn()
{
$this->output->enable_profiler(TRUE);
$this->load->model('payments_model');
$this->load->model('paypal_model');
$this->load->model('users_model');
ob_start();
if ($this->paypal_lib->validate_ipn())
{
$paypal_id = $this->paypal_model->add_paypal_ipn($this->paypal_lib->ipn_data);
// Split the 'custom' field up, containing ID of temp user, ID of package and coupon
$custom = explode(';', $this->paypal_lib->ipn_data['custom']);
###
# subscription sign up
###
if($this->paypal_lib->ipn_data['txn_type'] == 'subscr_signup') {
// Activate user/move from temp > live
$this->users_model->move_temp($custom[0], $this->paypal_lib->ipn_data['subscr_id']);
} # end subscr_signup
###
# subscription payment
###
if($this->paypal_lib->ipn_data['txn_type'] == 'subscr_payment') {
// Grab the coupon info, if we have one
$discount = 1;
if(!empty($custom[2])){
$this->load->model('coupons_model');
$couponinfo = $this->coupons_model->get_coupon($custom[2]);
$discount = $couponinfo->discount;
}
// Grab the package info
$package = $this->packages_model->get_package($custom[1]);
$price = $package->monthly * $discount; // Calculate discount, 0.8 = 20% off
// Does the price calculated match the gross price? If not something fishy is going on, block it
if($price != $this->paypal_lib->ipn_data['mc_gross']){
mail(CONTACT_EMAIL, SITE_NAME.' failed payment attempt, possible hack', 'Price paid doesnt match price computed... paid: '.$this->paypal_lib->ipn_data['mc_gross'].' - price worked out: '.$price."\n\n".print_r($this->paypal_lib->ipn_data, true));
exit;
}
// Grab the user's details based on the subscr_id
$user = $this->users_model->get_user_by_subscr_id($this->paypal_lib->ipn_data['subscr_id']);
// Add payment to the payments table
$data = array(
'user_id' => $user->user_id,
'subscr_id' => $user->subscr_id,
'txn_id' => $this->paypal_lib->ipn_data['txn_id'],
'amount' => $this->paypal_lib->ipn_data['mc_gross'],
'package_id' => $custom[1],
'coupon' => (empty($custom[2]) ? '' : $custom[2])
);
$this->payments_model->add_payment($data);
// Set (forced) user as active, and update their current active package
$data1 = array(
'package_id' => $custom[1],
'active' => 1
);
$this->users_model->update_user($data1, $user->user_id);
} # end subscr_payment
###
# subscription failed/cancelled
###
if($this->paypal_lib->ipn_data['txn_type'] == 'subscr_cancel' || $this->paypal_lib->ipn_data['txn_type'] == 'subscr_failed') {
// Grab user
$user = $this->users_model->get_user_by_subscr_id($this->paypal_lib->ipn_data['subscr_id']);
// Make user inactive
$data = array('active' => 0);
$this->users_model->update_user($data, $user->user_id);
} # end subscr_cancel|subscr_failed
###
# subscription modified/payment changed
###
if($this->paypal_lib->ipn_data['txn_type'] == 'subscr_modify') {
// Grab the coupon info, if we have one
$discount = 1;
if(!empty($custom[2])){
$this->load->model('coupons_model');
$couponinfo = $this->coupons_model->get_coupon($custom[2]);
$discount = $couponinfo->discount;
}
// Grab the package info
$package = $this->packages_model->get_package($custom[1]);
$price = $package->monthly * $discount; // Calculate discount, 0.8 = 20% off
// Does the price calculated match the gross price? If not something fishy is going on, block it
if($price != $this->paypal_lib->ipn_data['mc_gross']){
mail(CONTACT_EMAIL, SITE_NAME.' failed payment attempt, possible hack', 'Price paid doesnt match price computed... paid: '.$this->paypal_lib->ipn_data['mc_gross'].' - price worked out: '.$price."\n\n".print_r($this->paypal_lib->ipn_data, true));
exit;
}
// Grab the user's details based on the subscr_id
$user = $this->users_model->get_user_by_subscr_id($this->paypal_lib->ipn_data['subscr_id']);
// Add payment to the payments table
$data = array(
'user_id' => $user->user_id,
'subscr_id' => $user->subscr_id,
'txn_id' => $this->paypal_lib->ipn_data['txn_id'],
'amount' => $this->paypal_lib->ipn_data['mc_gross'],
'package_id' => $custom[1],
'coupon' => (empty($custom[2]) ? '' : $custom[2])
);
$this->payments_model->add_payment($data);
// Set (forced) user as active, and update their current active package
$data1 = array(
'package_id' => $custom[1],
'active' => 1
);
$this->users_model->update_user($data1, $user->user_id);
} # end subscr_modify
}
}
Below is an example of the calls made to my IPN for each transaction (CSV).
paypal_id,txn_id,subscr_id,txn_type,created
1,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:19:43
2,9XM95194MM564230E,I-FMUK0B5KJWKA,subscr_payment,2011-02-03 16:19:45
3,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:19:57
4,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:20:19
6,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:21:03
7,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:22:25
8,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:25:08
10,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:30:33
12,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 16:41:16
14,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 17:02:42
16,NULL,I-FMUK0B5KJWKA,subscr_signup,2011-02-03 17:45:26
Consider this - PayPal is insert profanity. Now revisit the problem.
The chances are this isn't your fault, or CodeIgniter's or the Library's. PayPal is very bad at giving data in a uniform and timely manner, it is also slow and doesn't link data together very well.
My advice to you is save everything into an IPN table whenever a callback is made, even email yourself when ever an IPN call is made. Then work to try and figure out what PayPal is actually sending you, what you want and throw out the rest.
I think an IPN call is made even if the transaction has nothing to do with your web site. So if your Grandma sends you your Christmas money via PayPal it'll appear on the IPN callback.
Hope that helps a bit.
paypal isn't exactly easy to use but let me share 3 tips totackle the problems you are facing.
1) Create a table to store all IPN response from PayPal. Make sure you have a column called "raw" that stores EVERYTHING... do "json_encode($this->paypal_lib->ipn_data)". This will save you... since you can later write a script to pull out data from the raw column into it's own column down the road. This also helps with debugging.
2) For a start just pull out what is necessary out into columns of the ipn table so you can query easily... here's everything I've deem necessary for my use case which is probably similar to yours.
$this->payment_model->create_ipn(array(
'invoice' => $this->paypal_lib->ipn_data['invoice'],
'txn_type' => $this->paypal_lib->ipn_data['txn_id'],
'parent_txn_id' => $this->paypal_lib->ipn_data['parent_txn_id'],
'txn_type' => $this->paypal_lib->ipn_data['txn_type'],
'item_name' => $this->paypal_lib->ipn_data['item_name'],
'item_number' => $this->paypal_lib->ipn_data['item_number'],
'quantity' => $this->paypal_lib->ipn_data['quantity'],
'exchange_rate' => $this->paypal_lib->ipn_data['exchange_rate'],
'settle_amount' => $this->paypal_lib->ipn_data['settle_currency'],
'settle_amount' => $this->paypal_lib->ipn_data['settle_amount'],
'mc_currency' => $this->paypal_lib->ipn_data['mc_currency'],
'mc_fee' => $this->paypal_lib->ipn_data['mc_fee'],
'mc_gross' => $this->paypal_lib->ipn_data['mc_gross'],
'payment_date' => $this->paypal_lib->ipn_data['payment_date'],
'payment_status' => $this->paypal_lib->ipn_data['payment_status'],
'payment_type' => $this->paypal_lib->ipn_data['payment_type'],
'pending_reason' => $this->paypal_lib->ipn_data['pending_reason'],
'reason_code' => $this->paypal_lib->ipn_data['reason_code'],
'subscr_id' => $this->paypal_lib->ipn_data['subscr_id'],
'subscr_date' => $this->paypal_lib->ipn_data['subscr_date'] ? mdate('%Y-%m-%d %H:%i:%s', strtotime($this->paypal_lib->ipn_data['subscr_date'])) : NULL,
'subscr_effective' => $this->paypal_lib->ipn_data['subscr_effective'] ? mdate('%Y-%m-%d %H:%i:%s', strtotime($this->paypal_lib->ipn_data['subscr_effective'])) : NULL,
'period1' => $this->paypal_lib->ipn_data['period1'],
'period2' => $this->paypal_lib->ipn_data['period2'],
'period3' => $this->paypal_lib->ipn_data['period3'],
'amount1' => $this->paypal_lib->ipn_data['amount1'],
'amount2' => $this->paypal_lib->ipn_data['amount2'],
'amount3' => $this->paypal_lib->ipn_data['amount3'],
'mc_amount1' => $this->paypal_lib->ipn_data['mc_amount1'],
'mc_amount2' => $this->paypal_lib->ipn_data['mc_amount2'],
'mc_amount3' => $this->paypal_lib->ipn_data['mc_amount3'],
'recurring' => $this->paypal_lib->ipn_data['recurring'],
'reattempt' => $this->paypal_lib->ipn_data['reattempt'],
'retry_at' => $this->paypal_lib->ipn_data['retry_at'] ? mdate('%Y-%m-%d %H:%i:%s', strtotime($this->paypal_lib->ipn_data['retry_at'])) : NULL,
'recur_times' => $this->paypal_lib->ipn_data['recur_times'],
'payer_id' => $this->paypal_lib->ipn_data['payer_id'],
'payer_email' => $this->paypal_lib->ipn_data['payer_email'],
'payer_status' => $this->paypal_lib->ipn_data['payer_status'],
'payer_business_name' => $this->paypal_lib->ipn_data['payer_business_name'],
'ipn_track_id' => $this->paypal_lib->ipn_data['ipn_track_id'],
'raw' => json_encode($this->paypal_lib->ipn_data_arr),
'test_ipn' => $this->paypal_lib->ipn_data['test_ipn']
));
don't copy my code above since it's only meant to give you some rough ideas... if you do adapt my code do also ensure ipn_data function is like this (else you will get tons of errors)
function ipn_data($key)
{
return isset($this->fields[$key]) ? $this->fields[$key] : NULL;
}
to understand all the possible stuff they can send back this link is gold
https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=developer/e_howto_html_IPNandPDTVariables
but ^sigh^ don't trust it to be updated. i have found inconsistencies in what they doc says and what they sent back to me.
3) OK this i have to admit is another silly thing that paypal does - they don't give you an IPN date even tho' they don't guarantee the order in which it arrives at your server. For subscr_payment they give you payment_date... for subscr_signup they give you subscr_date... so what you need to do to get your IPN in the correct order is to have a column called ipn_date.
'ipn_date' => isset($this->paypal_lib->ipn_data['payment_date']) ?
mdate('%Y-%m-%d %H:%i:%s', strtotime($this->paypal_lib->ipn_data['payment_date'])) :
(isset($this->paypal_lib->ipn_data['subscr_date']) ?
mdate('%Y-%m-%d %H:%i:%s', strtotime($this->paypal_lib->ipn_data['subscr_date'])) : NULL),
now all is cool, you can "order by ipn_date" and I assure you everything will be in the correct order.
p.s. note the my first example code doesn't have this column, but it IS meant to be there. i'm just copying and pasting my development code to give you an idea.
What I do is ignore the signup ones and just process (create new user etc) on the actual payment transaction. And I wouldn't bother storing all those IPN trans. Have your IPN script send you an EMail on every one of them though, with an echo of all the fields posted. Then you'll have a record of them.

Categories