I'm trying to create an application that will create a chart that will get the data by calling a financial web service. Users will interact with the chart by clicking on it. There could be millions of views on the application, so millions of requests on the web service. What's the best way to do this? Will I need to call the web service each time, a million times?
The way I see it you would be better off reading the data from the web service at fixed interval (every X seconds/minutes/hours depending if your data is real-time or not) and displaying the graph with that information. That way your number of requests to the web service will not vary and you can change one part of the equation without affecting the other.
Edit after first comment: my answer stands. You would be better off getting the data at a fixed interval and inform users that the data is supplied with a delay (the usual delay with financial activity data is 15 minutes). That way you know in advance the number of requests you will run and you greatly speed up the service to your users.
A word of warning: Yahoo Finance changed its API licensing in 2012 or 2011 and it is now forbidden to use data from the API for public applications without a commercial license.
Related
My web app is built with Angular, and php as an api server. I want to have a service which is limited by predefined time. For example a user gets 4 hours to run function consume() which consumes data from my server. This of course can't be handled client-side alone, because it will be bypassed.
What I'm currently thinking is some kind of credit system. for example, the user gets 240 credits, and consume() pulls data from the api every 1 minute which costs 1 credit. Is there a better way of doing it?
I couldn't find any hints about it on google because I don't even know what keywords to search for..
I run this website for my dad which pulls tweets from his twitter feed and displays them in an alternative format. Currently, the tweets are pulled using javascript so entirely client-side. Is the most efficient way of doing things? The website has next to no hit rate but I'm just interested in what would be the best way to scale it. Any advice would be great. I'm also thinking of including articles in the stream at some point. What would be the best way to implement that?
Twitter API requests are rate limited to 150 an hour. If your page is requested more than that, you will get an error from the Twitter API (an HTTP 400 error). Therefore, it is probably a better idea to request the tweets on the server and cache the response for a certain period of time. You could request the latest tweets up to 150 times an hour, and any time your page is requested it receives the cached tweets from your server side script, rather than calling the API directly.
From the Twitter docs:
Unauthenticated calls are permitted 150 requests per hour.
Unauthenticated calls are measured against the public facing IP of the
server or device making the request.
I recently did some work integrating with the Twitter API in exactly the same way you have. We ended up hitting the rate limit very quickly, even just while testing the app. That app does now cache tweets at the server, and updates the cache a few times every hour.
I would recommend using client-side to call the Twitter API. Avoid calling your server. The only downfall to using client-side js is that you cannot control whether or not the viewer will have js deactivated.
What kind of article did you want to include in the stream? Like blog posts directly on your website or external articles?
By pulling the tweets server side, you're routing all tweet traffic through your server. So, all your traffic is then coming from your server, potentially causing a decrease in the performance of your website.
If you don't do any magic stuff with those tweets that aren't possible client side, I should stick with your current solution. Nothing wrong with it and it scales tremendously (assuming that you don't outperform Twitter's servers of course ;))
Pulling your tweets from the client side is definitely better in terms of scalability. I don't understand what you are looking for in your second question about adding articles
I think if you can do them client side go for it! It pushes the bandwith usage to the browser. Less load on your server. I think it is scalable too. As long as your client can make a web request they can display your site! doesn't get any easier than that! Your server will never be a bottle neck to them!
If you can get articles through an api i would stick to the current setup keep everythign client side.
For really low demand stuff like that, it's really not going to matter a whole lot. If you have a large number of tasks per user then you might want to consider server side. If you have a large number of users, and only a few tasks (tweets to be pulled in or whatever) per user, client side AJAX is probably the way to go. As far as your including of articles, I'd probably go server side there because of the size of the data you'll be working with..
I am working on a API for my web application written in CodeIgniter. This is my first time writing a API.
What is the best way of imposing a API limit on the API?
Thanks for your time
Log the user's credentials (if he has to provide them) or his IP address, the request (optional) and a timestamp in a database.
Now, for every request, you delete records where the timestamp is more than an hour ago, check how many requests for that user are still in the table, and if that is more than your limit, deny the request.
Simple solution, keep in mind, though, there might be more performant solutions out there.
Pretty straight forward. If that doesn't answer your question, please provide more details.
I don't see how this is codeigniter related, for example.
You can use my REST_Controller to do basically all of this for you:
http://net.tutsplus.com/tutorials/php/working-with-restful-services-in-codeigniter-2/
I recently added in some key logging, request limiting features in so this can all be done through config.
One thing you can do is consider using an external service to impose API limits and provide API management functionality in general.
For example, my company, WebServius ( http://www.webservius.com ) provides a layer that sits in front of your API and can provide per-user throttling (e.g. requests per API key per hour), API-wide throttling (e.g. total requests per hour), adaptive throttling (where throttling limits decrease as API response time increases), etc, with other features coming soon (e.g. IP-address-based throttling). It also provides a page for user registration / issuing API keys, and many other useful features.
Of course, you may also want to look at our competitors, such as Mashery or Apigee.
I hope this is an appropriate question: I'm using the Netflix API and I'm wondering what the best way one would be able to automatically receive new data when presented (in this case, recently watched films when a Netflix user finishes watching one) The only way I can think of is spamming requests in intervals to query their feed. And would PHP be my best bet?
That's right, Netflix doesn't provide any push notifications through their API. You'll have to poll their feed periodically, but not too often: your consumer key is limited to a certain number of requests per second and requests per day.
I'm not exactly sure what you're trying to do, to know whether PHP would be the right choice. OAuth libraries are available for pretty much every major language, so it's up to you.
I want to display online NSE (National Stock Exchange) data in my php web application. Please help me how to do this.
You can use google api .. they provide real time data for NSE.
but the catch is google finance api is available for java only it was not available for c# last time i checked.
Not sure about whether finance api are there for PHP.
Find out if NSX has a web service that exposes the stock charts. If so, this will likely be a REST or a SOAP endpoint that can be queried for specific stocks.
Create a PHP component that makes a request to that endpoint, and caches the results for a certain period of time (so that your site visitors don't have to wait for the roundtrip every time a page is loaded).
NSE don't have any web service AFAIK, the only thing they export is previous day data. You might have to do some sort of data parsing on the NSE webpages.
If you want to take historical data from NSE website, you can always do that.
for example, following is the url to retrieve nifty index data.
[http://www.nseindia.com/content/indices/histdata/S&P%20CNX%20NIFTY01-02-2010-04-02-2010.csv]
observe date pattern in the url, it is from 01 FEB 2010 to 04 FEB 2010.
change dates as you need, call the url from your server. It is csv format, so any language can process it.
Regards.
http://www.kakup.com