my site needs to keep track of followers of certain users
if i want to get the names of all the followers of specific user lets say this user have 1000 follower
i will be able to get the follower ids by https://api.twitter.com/1/followers/ids.json
1000 id
then to get the username of each id i will have to run https://api.twitter.com/1/users/lookup.json
for each id of those 1000 id but the rate limit of 350 makes this impossible
any solution , what i am doing wronge ?
ok i found the solutino actually in users/lookup as parameter of user_id i can send multiple ids saperated by commas
so in single API call i can send upto 100 ids which will mean in one hour i can get the lookup of 35000 users which is more than enough for most of the cases
Related
I use a notifications table and a subnotifications table, and I am also using queues so it runs in the background when a user posts something. When a user has 10 followers and they create a post, the notifications table gets a single entry which includes the post data for the notification, and the subnotifications table gets 10 entries (one subnotification per follower, each referring to the id of the notification, so we don't have to repeat the notification data 10 times, with a read_at to know if it was read or not by that follower).
This is quick and works great without any issues. However, when testing with 1 million followers, it takes about ~6 hours to insert the subnotifications for one post! This of course is not acceptable, as it is takes too long to insert 1 million subnotifications, one per follower. Imagine that same user posts 10 posts, that'll be like ~60 hours of inserting and 10 million subnotification rows.
I just want followers to know there is a new post if they didn't read it yet. Is there a better, more efficient way that scales?
UPDATE: Stuck with current approach see below...
If a follower $user has 100 leaders they follow (which they followed at different created_at timestamps of course in the followers table), what would the correct query be to know about leader new posts from the time the follower followed each leader? I get stuck at created_at with this pseudo code:
// Assume `leader_id` is a column in the notifications table
DB::table('notifications')
->whereIn('leader_id', $leaderIds)
->where(`created_at`, '>', $whatTimestampsGoHere)
->paginate(20);
There is 100 different timestamps and I am stuck on how to solve this one correctly and efficiently. Any ideas?
As stated in the comments, you can reduce the inserts, if you only insert to the child table i.e. subnotifications when the user reads it and not on creating it on the notification creation, which avoids that issue.
When trying to check if user has seen the notification, just check if they exist in subnotifications for the user in question and the notification.
Also as said, when fetching notifications to show to users fetch them from notifications but limit the notifications to the notifications created after the user started following so that new users don't get flooded with notifications.
I am trying to create an activity feed for each user in my application. I want to use Redis to store two lists per user. One for storing the ids of user's followers and one for storing the ids of the activities from every user he/she is following.
So what I want to do is when a user adds a post/creates an activity I want to push the id of that post/created activity to the list of every user whose id is in the user's followers list.
I am assuming I would have to loop through the 'followers list' to this, but I have no idea how to accomplish this in Redis.
I am using PHP as my server side language but I don't think it would make sense to do it there or is it ok to do that way?
I believe Redis has a much more efficient and faster way of doing this, especially if the 'followers list' contains 1000000s of records.
E.g
// to add followers to user with id of 1
lpush user.1.followers 4 90 3 48 8 2 45
// to get all user with id 1 followers
lrange user.1.followers 0 -1
I've had an idea to make my website viral: a competition where the users make points and they win prizes. One of the ways they can get points is the following: if they share their personal page (they made on my website) on facebook they get 1 point for each like they get.
Then i made a PHP script which periodically queries facebook to ask the like_count the URL got. It simply queries for the like_count column in the link_stat table and then it evaluates the score.
PROBLEM
I've found a trick: if i share my personal page URL 100 times and then i like all the 100 links i shared (on facebook), like_count goes to 100. Which is bad for my competition, because as soon as someone finds this out, he will easily win the competition.
What i want to get, instead of like_count, is the number of distinct people who liked that URL.
This means, if i share the URL 100 times and i like it 100 times, my girlfriend likes it 30 times and my grandmother likes it 10 times, instead of 140 the count should be 3.
This way the competition would be perfectly regular.
Any ideas?
Maybe you can do it with this https://developers.facebook.com/docs/reference/fql/like/ which says something like this:
SELECT ... FROM like WHERE object_id = ...
Basically it returns the people who liked certain object, which I assume would be 1 for you even if you like the same link 100 times?
Background Info :
I'm trying to retrieve images from people I follow, sort by latest time. It's like a twitter news feed where they show the latest feed by your friends.
Plans:
Currently there is only 1 item i need to keep in consideration, which is the images. In future i'm planning to analyse user's behavior and add in other images they might like into their feed, etc.
http://www.quora.com/What-are-best-practices-for-building-something-like-a-News-Feed
I personally feel that "Pull" Model, or Fan-out-on-load where i pull all info at real time would be worst than the push model. Because imagine i have 100 following, i would have to fetch and sort by time. (Let me know if i'm wrong eg, Read is 100x better than Write(Push Model)
The current design of the push model i have in mind is as follows
Table users_feed(ID, User_ID, Image_ID,datetime)
Option 1 : Store A list of Image_ID
Option 2 : Store one image ID and duplicate rows(More Rows of same User_ID but different Image_ID)
The plan is to limit each Row a user can have in this feed , which means , there would always be a max of 50 images. If they want more items beyond the 50 images in their news feed. They cant(I might code a alternative to store more so they can view more in future)
Question 1
Since when user following users add a item into their "collection" i have to push it into each of their follower's feed. Wont there be a problem in Write? 200 followers = 200 writes?
Question 2
Which method would be better for me keeping in consideration that i only have one type of data which is images. Feeds of images.
Question 3
If i choose to store the feed in advance(push method) how do i actually write it into all my friends?
Insert xxx into feeds whereIn (array of FriendsID)?
Any form of advice would be greatly appreciated. Thanks in advance!
I would recommend you to follow pull method over push method for the following reasons:
It gives to more freedom for extencibility in the future.
Less number of writes ( imagine 10M followers then there has to be
10M writes for just 1 post).
You can get all feed of a user simply by query similar to:
SELECT * FROM users_feed as a WHERE a.user_id in ( < //select all
user_ids of followers of loged in user// > )
(Syntax not followed as table
structure of followers is not known)
If I have 10 friends how do i get the latest status for each of them and then post a comment on all of these statuses?
how do you get the post ID?
$facebook->api('/'.$POST_ID.'/comments','post',array('message' => $comment));
I can't find a way to get friends statuses ordered by date from the Graph API, so you'll have to use FQL. This query will get you the 10 most recent status updates by your friends:
SELECT uid,status_id,time,message FROM status WHERE uid IN
(SELECT uid2 FROM friend WHERE uid1 = me()) ORDER BY time DESC limit 10
There is no guarantee that these 10 will come from different friends since there is no way to execute a UNIQUE query in FQL. If you want to ensure you're commenting on 10 unique friends' statuses, you'll have to request more results and keep track of whose posts you're commenting on in your script.
From here, you can loop through these results and set $POST_ID = $result['status_id'] then execute your API call above.
If this is for a public app you're building, you'll want to look at combining these 10 updates together into a single batch.request to keep from running into the API call limits.