I am writing API for one service, and now I need to restrict some calls for some URL. Let's say that you are user and you choose that only stackoverflow.com can see your information, or that stackoverflow.com, facebook.com and google.com cannot see your information.
I have the system on the backend (white vs. black lists) but now I need to find a way to get the URL from the call.
Yes some people will be using curl and it will be easy, but a lot of people will simply use file_get_contents(), as well as this API has an image output so some people will use
<img src="http://domain.com/api/something.jpg" />
Any thoughts how to get the url, without having developers to actually encode the url into calling url?
Thanks
Check the IP address ($_SERVER['REMOTE_ADDR']) of the requesting client. You can whitelist IP addresses of the servers allowed to connect to your API. For the image output, you can use $_SERVER['HTTP_REFERER'] and ensure that it is from a domain you have whitelisted.
However, I am not recommending this implementation. You should make use of API keys. These are like passwords given to clients using your API. Only those with API keys can access your API. At the same time, when their servers change IP addresses, they will still be able to access your API.
Related
I know CURL & HTTP authentication to allow API calls only from certain requests.
I have something like this
www.mysite.com/list
www.mysite.com/api
list page calls the api page using jQuery Ajax post request and api page echos jSON response.
I am already checking HTTP referrer to verify Post requests come from certain origins but these requests can be forged.
I cannot use $_SERVER['remote_addr'] as this will check the client ip not the source ip of the API request.
As both the pages are hosted in the same web server and in the same hostname, its not possible to restrict based on hostname or IP address.
I am aware of some alternatives like
cURL with HTTP Authentication.
Encryption of jSON response and Decrypting before printing.
But is there any way to restrict these API pages to securely block access from outside the server or hostname?
Assuming you want to restrict access to "Joe Random user who is visiting your website with a web browser".
Not really.
There's no way to reliably determine that the client is a regular web browser. Anyone writing some other client to access your API can easily have it visit your website and get whatever tokens they need to prove that.
The best you can do is some sort of heuristic approach that looks for unusual traffic patterns (such as unusually high numbers of similar requests from one source, or ones which have very stable intervals between requests). You'll get false positives and negatives from that kind of approach though.
I may be wrong, but I think you are trying to invent csrf tokens. But with little modification:
You should store generated token in session/cookie before rendering /list. I am not sure whether jQuery send session/cookie with ajax request by default, but I think it should not be a big problem if not, to append it.
When you will process /api request you should validate cookie/session also for the token existance.
It would be useful if, when placing an order or submitting a form, we could reliably include the visitor's referrer with the request. This is not always simple because, for example, the user could have found us via Adwords, left the site and come back by typing in the address. In this case, we would want to know the Adwords campaign for this specific visitor. Google Events don't fit well with our workflow.
Other questions have tried to use Javascript and apparently that is not possible. I noticed that Google sets a _ga cookie which we can read - it gives us an ID that looks like this: GA1.2.123456789.1234567890. Can we use PHP to extract the referrer according to Google from that ID?
Not in realtime, no. It used to be possible with the previous version of GA which evaluated traffic sources clientside and stored the values in a cookie. However with Universal Analytics the processing is now done serverside and the information is not accessible in realtime (there is a realtime API but that'S for various reasons not feasible for getting data on individual users).
It is however possible to get the referrer data via javascript - you look at document.referer and (if the referrer domain is different from your own domain) store it into a cookie (since the question is tagged with PHP, you can do the same with $_SERVER['HTTP_REFERER']).
This will not exactly match Googles info (Google will show referrals from google properties, yahoo, bing etc. not as referrer but as organic search traffic) but should be close enough.
How do you retrieve the username of someone on Youtube who is using your application with PHP? I'd prefer to do this without the API but if needs be point me in the right direction because i don't know what to download , I've visited the google developers site / place already.
I found some code from 2009 but it was to retrieve usernames from a youtube video and it didn't work. I tried reading the contents of page with the fopen() and file_get_contents() but it always appears as if I'm not signed in whereas I am signed in via FireFox.
Don't know if this makes a difference but I was running this on EasyPHP via localhost
Thanks
You can't, by design. You do not have their session cookie server-side where you're using fread. Youtube determines whether you are or are not signed in based on that session cookie, and you do not have one on your server.
If you want their YouTube user name, use a client-side API in JavaScript or simply ask them for it with a text input field. It is impossible to find it server-side, as it should be. Your server has no business finding out my Youtube username without my express permission and interaction.
I need to execute a google analytics script on a site using curl. What are the reporting features that will be available to me for that curl request ?
To make my question more clearer, suppose i have a website www.abc.com which has analytics code on it and if i connect to www.abc.com from www.xyz.com using curl, in the analytics report of www.abc.com will the request i made using curl show ?
What are the parameters that wont show ? since there is no navigator there wouldnt be screen resolutions, color depth, or any other javascript based features. any other reporting features that wouldn't be available to me ?
I need the following to show up :
user agent (which i will be sending by setting header)
referrer (again which i will be sending by setting header)
source ip address and location (using proxies for different countries)
one issue that Im unsure of is Google's cookies and whether (since i might also be using proxies) these need to be cleared. The ultimate outcome is that I need to be able emulate site traffic as if it is coming from a variety of visitors....
IF the entire thing is not technically possible,is there any other way i can simulate diverse traffic into my google analytics account ?
So, the other answers are right: CURL doesn't execute JS, and there are some methods of automated requests that do.
Other methods to simulate diverse traffic to your account include:
Visit the site manually, and grab the __utm.gif request that Google Analytics generates, and manipulate its pieces so that you can CURL it in conjunction with curling the actual site, so that the GA pageviews are recorded. (ie, alter the hostname, pageview name, timestamp, etc.) You can find the meaning of the values of those parameters here.
Implement a server side GA on your target site.
Use a headless web engine to programmatically crawl sites. PhantomJS is a particularly user-friendly option.
Use a browser screenshot service like BrowserShots to get traffic from distributed locations to visit your site.
Use Amazon's Mechanical Turk to get people to visit the site. You could pay $0.01 per click, and get a large amount of diverse traffic from a large number of sources. (To verify, give them an arbitrary, simple task like asking them "What's the headline on this website?")
You can send events directly to Google Analytics using the Measurement Protocol, creating GET or POST requests directly with the tool of your choice.
See reference here:
https://developers.google.com/analytics/devguides/collection/protocol/v1/reference
No, because Google Analytics is based on Javascript and curl doesn't process HTML or Javascript.
Instead of curl, use a command-line tool that does JavaScript, like HTTPUnit (which includes Rhino). I have heard about WATIR too but never tried.
Those happen to be testing tools, but I guess you can use them to trigger Google Analytics too.
We have a webpage that we provide to partner companies via an iFrame. The iFrame contains several javascript files that make ajax requests to our server for data. The iFrame itself requires an API Key that is keyed to the domain of the partner. This prevents the iFrame from displaying if it is installed on a domain that isn't registered. However, it would be pretty easy to simply copy the contents and javascript files of the iFrame from a registered site and host them on a non-registered site.
Ideally we'd like to use the API key to restrict Ajax requests and prevent our server from providing the requested data for non-registered sites. However, it appears that the HTTP_REFERER server variable is not set for Ajax requests. How can we tell what site that the request is coming from? Is it possible? If not, how can we prevent unauthorized access?
Relying on HTTP_REFERER isn't the way to go. You want your client's website to use an API to contact your website over a secure link, and get a temporary session string, which is then used as part of the source url for the IFRAME, which is how google does it (not with referer.)
Make the url for the IFRAME valid for a limited time, after which you display a nice message about going back to the client's page to start over.
When the iframe is requested you can generate a unique ID on your server, then set that as a cookie on the client. Every AJAX request should contain that cookie. Only keep around the ID's for the last hour or so.
You can never rely on HTTP_REFERER because some proxy servers and firewalls will strip it out to preserve users' privacy.
The challenge is that the iframe is authorized to a specific domain, so my API Key is tied to that. I followed the following tutorial to generate my API keys.
https://ajax.dev.java.net/ajax/api-keys
Do you think that relying on the HTTP_REFERER variable will prevent users from accessing the iframe? Sun claims that Google uses this method for Google Maps API authentication.
Once the API key has been authenticated, then the cookie approach should do the trick, I think. Thanks!