Sorry for the long title and perhaps confusing half good now as we come. I'm asking advice or guidance on how I can get an RSS feed from a page that does not have RSS enabled by default. But that is not the problem itself. The problem is when on that page I am asked to enter a username and password. Well so otherwise would be the thing...
PROBLEM:
Get the RSS of a forum which does not have an RSS feed enabled and to see the 'news' we need to be logged.
POSSIBLE SOLUTIONS that come to mind:
There are several web sites which offer services in English to get RSS on pages where they are not. That's fine, but the problem is when these sites don't offer an option to login with a username and password to the web page where I want to get the info, so these types of sites are excluded.
I did not login via url and so put that url on web sites listed above (item 1) of the forum with the username and password variables directly from the url spec: www.forosinrss/login.php?usuario = me & password = your pff and I'm bounced the forum, telling me I'm not getting the correct data as we will be. Another problem is that the password is md5 encrypted, so I'm prevented from logging in with the URL (fk T_T).
Try using "SELECT * FROM DB Internet", or in other words, to use YQL. But it came out almost as much as they found no way to insert and log into user and password and also to generate a cookie for the forum is not happy I voted.
I need suggestions, recommendations, tips or complaints.
Download the page using something like cURL or fsockopen if you're feeling brave, then transform the page from html to rss using XSLT Stylesheets.
Once upon a time I wrote an app in PHP to do this with ok-ish results:
use curl to get the page and keep a copy
run a custom filter regular expression to select the bit of the page that actually matters (some sites have dynamic text like ads or just displaying the current date and time)
after a timeout, use curl to get the page again and run the same filter on it
run diff old_page, new_page and pipe the result into an rss template
The system worked ok but was fiddly filtering the page down to content that I wanted to get the feed from and it broke a lot because these kinds of sites are often hand edited so you can't guarantee any consistency.
Related
I have written a PHP based blog for the company i work for. Not using any frameworks. I am having trouble tracking users who come from my facebook page's posts to my blog (not wordpress).
I have created a shortlink url. Let's say it is sample.co and it redirects traffic to sample.com. Everything seems fine until here. The problem starts here.
I am adding all user's ip's, user agents. But if even i get 500 visits, my code adds somethig like 3.000 visits. Facebook stats and Analytics shows similar stats (~500 visits). I see that ip's added to MySQL are all different. It usually happens with Android users. I have read somewhere that Facebook sometimes renders to their users the actual URL when FB shows the post. I mean instead of the widget, Facebook shows the whole page. I am not quite sure about that to be honest.
To solve this problem, I have created and added an jquery script to my page and listened users' scroll event. It worked great. Not seeing too much traffic. But this time the problem is i am counting less users. Even I get 500 users from facebook and Analytics shows similar results, my script adds only 200-300 to MySQL.
Does anyone know a better way to track real traffic? Or do you aware of such problem?
Thanks
It should be filtered on the basis of user agent.
https://developers.facebook.com/docs/sharing/webmasters/crawler
how to detect search engine bots with php?
Identifying users through IP is a good idea, but if your IP keeps changing, it's a good idea to use cookies.
http://php.net/manual/en/function.uniqid.php
If the cookie does not exist, you should see it as a new user.
I have found the answer. The problem is called preview (prefetch). Here is the link:
https://www.facebook.com/business/help/1514372351922333
Simply, facebook preloads everything when FB shows the thumbnail to the visitor to speed up your page's load speed. They send X-Purpose: preview header. So you can simply check if HTTP_X_PURPOSE header's value is preview or not. If so, do not count it as a visitor.!
Here are more detailed descriptions:
http://inchoo.net/dev-talk/mitigating-facebook-x-fb-http-engine-liger/
http://inchoo.net/dev-talk/magento-website-hammering-facebook-liger/
My client has a Facebook page which has his latest posts and shows the reviews people have added.
I want to grab these from Facebook, and output them on his website all styled up nicely with my own markup and styles.
I have had this running in JavaScript with the access token pasted in the JS which is obviously not the way to go, and the token will expire. Now im trying to do the same with PHP. I've read & watched more clips than I care to mention, and I cant get my head around one particular part. Every demonstration i've read or watched requires the website user to login to facebook before this data is returned.
I cant expect ever user who goes on the site to do that. Am I missing something? I just want get the data and display it. No Adding, No Deleting, No Updating. Just display the same information on his website.
Sorry if am asking something stupid here, i'm a novice and for all my efforts I am struggling to understand. I kind of need it in laymans terms so to speak.
Thanks in advance.
I've seen there is some other discussion about this, however, none of the other threads I've read have targetted my situation.
For one, I do not need to get any Tweets from Twitter. I only need to send to Twitter.
For another, I don't need to have multiple accounts or different Twitter users. I can already ensure the page will only be seen by the one authorized user, so I can save any necessary login information in one configuration file and be done.
I am only posting occasionally from one web site, to one Twitter feed, and only one user will be logged onto the the web site that will send the tweet.
I am hoping to find a simple tutorial or script that can allow me to cut to the bare bones and just send tweets to my Twitter account.
However, all the tutorials and scripts I've seen so far have been too elaborate, handling logins, sessions, authenticating multiple users, and so on.
Is there any way I can get simple oAuth PHP code to accomplish what I'm describing?
This would help
http://code.google.com/p/dabr/source/browse/trunk/common/twitter.php
http://code.google.com/p/dabr/source/browse/trunk/common/OAuth.php
This should help or at least point you in the right direction.
https://dev.twitter.com/docs/twitter-libraries#php
You can use ZF component
get accessToken only once, keep it as serialized string anywhere
http://framework.zend.com/manual/en/zend.service.twitter.html
I'd like to remove as much as possible of my history with a certain facebook user via the Facebook API.
I want to be able to remove photo tags, wallposts, messages that I am tagged in, or associated with the certain user.
Is this possible, and if yes; how?
Blocking the user will go a long way towards removing most connections with the person in quesion. Go to Account->Privacy Settings->Manage Block Lists, or go to the users profile and look for the "Report/Block" link under their friends/family lists.
This FAQ about blocking describes a little about what happens when you block someone. As far as photos goes, it's a little unclear:
Photos: If the blocked person tagged you in any photos before the block, you’ll still be able to see those photos. You might also see photos of the blocked person if they are tagged in another friend’s album, but you won’t be able to click on their name to see their profile.
You do automatically get un-tagged from all their photos when you block someone. I'm guessing wall posts you're tagged in are removed as well, although I only tested with photo tags.
As far as removing their posts to your wall, as Igy pointed out you can't write an app that uses the API to delete posts that it didn't create. So, what I would do is write an app that gets all the posts on your wall and filters them, something like this pseudo-code:
posts = api('/me/feed')
for each post in posts:
if post.from.id == [blocked persons id]:
output post.id
Post ID's from the Graph API are in the format USERID_POSTID, so there are two ID's separated by an underscore. So for each ID in the output, go to https://www.facebook.com/permalink.php?story_fbid=[POSTID]&id=[USERID] and manually delete the post by selecting "Remove post" from the little gear menu on the top right of the post. Use the paging data from your API request, or until and since parameters (which are just UNIX timestamps, or any string accepted by the strtotime function), as described on http://developers.facebook.com/docs/reference/api/ under "Paging", to get all posts going back until the beginning of your wall.
If you are wanting to automate this process to make it available to end users and not just yourself, there are technically several ways you could go about it, although they may or may not violate (Terms of use)[http://www.facebook.com/terms.php]:
[3.2] You will not collect users' content or information, or otherwise access Facebook, using automated means (such as harvesting bots, robots, spiders, or scrapers) without our permission.
I'm sure there are ways of implementing such an app that doesn't violate these terms, for example providing a link to the post so that they can delete it themselves, rather than automatically deleting them.
http://suicidemachine.org/
I'd look into their code.
From their FAQ on if you can make your own:
Theoretically yes! Practically no (or let's say, not yet)! You'd need a Linux WebServer (apache2) with perl and python modules (php should be installed as well). Further, you'll need VNC-server and Java packages by Sun to launch selenium-remote applets. If you feel like contributing and can convince us with decent programming skills, please get in contact with us via email. We don't make the source code publicly available, since Facebook, Twitter, Myspace and LinkedIn would figure out how the suicidemachine is working in detail! So, please do not contact us, if you work for one of these companies!
I'm not even sure (personally) why you'd want to do this, surely you'd either want to block the other user or just remove them as a friend, but to answer your question:
It's not possible to delete most types of content via the API unless the App ID you're using was the original creator of that content (e.g an app for posting to your wall can delete the posts it made, but not posts made by other apps or on the Facebook.com interface)
Check the documentation at http://developers.facebook.com/docs/reference/api/ - you can definitely remove likes and comments of photos and wall posts - there may be other ways to do this (e.g retroactively changing your RSVP status to an event both users went to)
Here's the deal:
-Flair pretty much demands a Facebook app
-I'm working on learning the Facebook platform
Therefore, I've written a dinky little Facebook app to embed your Flair into a box on the side of your profile. If you're interested and on Facebook; beware that this is very much a work in progress, prone to change and has a whole bunch of rough edges.
However, I don't have an elegant way to say "this is MY Stackoverflow profile" from this dinky app. My current solution is best demonstrated this image. It is, frankly, idiotic; and there's nothing preventing impersonation.
Having no experience with OpenID, I'm wondering if there's some way (and some example of this way, please) to get a user to provide their Stackoverflow identify to a third party?
Behavior changed to that suggested by Noldorin's answer.
Request for a saner way to do this declined on UserVoice.
There's no way to get a StackOverflow ID from OpenID and to do the opposite would be possible but unnecessarily complicated (compared to your suggested method). Copying and pasting the user ID/profile URL isn't terribly inconvenient, in my mind.
However, I can suggest some sort of solution to the issue of impersonation. This may seem like a slightly silly method, but it's the simplest way of which I can think to insure that only the user themself can display the badge. Again, I don't think it's too inconvenient. (The more traditional method of using an email confirmation isn't possible, given that emails aren't public.)
Have your Facebook app generate a random (alphanumeric?) code. Something in the form A8IO45QW6T should do.
Hold this code on the server side of your Facebook app for a short time period (say, 5 minutes).
Instruct the user to edit their About Me information by adding the given code to the last line, then to return to the Facebook and confirm. Simple inspection of HTML for the profile page of the specified user would verify ownership. The user can then remove the verification code from their About Me text.
Nice idea creating this Facebook app, by the way. I may just give it a try!
Another solution that would allow for instant verification is hashing the email address and checking to see if it matches the user's Gravatar. Granted, not every user has provided an email address, but you can always perform this check first and if it fails you can fall back to the About Me section editing (or simply require that the user have an email address). This idea is not mine, so I'll let null explain:
Do you still have the source available? As I've gotten a solution to prevent other users from messing up your results. You just need to hash the email address used by the viewer and check for the gravatar image link on the page as outlined on the gravatar site. If the user id and image hash match, then record the change to the database, otherwise... just show results. I can make that change, but the source seems lost... :(