I would like to implement a realtime dashboard like an index page of www.foursquare.com
I checked foursquare.com's index page with Chrome's developer tool and surprised that they don't use xhr to get those information approx. every 5 seconds.
Using ajax polling causes memory leak in some browsers and make a server busier.
Is there any way that I can implement a realtime dashboard efficiently with PHP and jQuery(AJAX)?
(Perhaps I need an extra server something like a push server?) :|
Foursquare's homepage loads 30 items (id=recent29, id=recent28, ...) but displays only 11 at once. So you will have a real-time feeling for about 90 seconds (after that, the same items reappear).
...
$('#recent'+toShow).slideDown(1000, move(i));
$('#recent'+i).slideUp(1000, move(i));
...
For some bidirectional client server communication, take a look at websockets, even though they are not universally supported yet, they eventually become a standard.
The WebSocket API is being standardized by the W3C, and the WebSocket protocol is being standardized by the IETF.
One method to get data pushed to a client is a loop like this:
Open AJAX request in browser.
Server waits with the request open until it either times out or new data is available.
Server returns the request with either new data or a timeout, and client immediately opens a new request.
You could use comet, APE is particularly easy to setup and configure:
http://www.ape-project.org/
Back-end is written in C so it's very fast.
Related
I'm making an app which accesses a database and updates the data every view seconds over a PHP script, the problem is that it currently always updates all data, I would like to know how to program something that dynamically updates data and decides what data to update and what not, so it basically keeps track of change somehow. So how would I best go along doing something like this ?
I think that there should be some where that this question has already be asked but I couldn't find it so maybe someone can show me a website where to look.
In general, you will need to user either XHR requests, web sockets, or HTTP/2 to solve this problem. Since HTTP/2 is not universally supported on the browser side, it may not work for you. Here is the outline of the solution:
Every few seconds, javascript you provide in the browser will need to poll the server for updates using an XHR request. You can use the returned data to update the screen with Javascript. If you only want to do some simple updates, like updating some numbers, you might use raw Javascript or jQuery. If your polling will result in complex screen updates or you want to move a lot of functionality into the client, you probably want to redo your client using one of the JavaScript frameworks like React or Angular.
Use web sockets (or HTTP/2) to create a persistent connection to the server, and have the server send updates to the clients as the data changes. This probably will require some code in the application to broadcast or multicast the updates. The client code would be similar to case 1, except that the client would not poll for updates.
The polling solution is easier to implement, and would be a good choice as long as you don't have too many clients sending polls at too high a rate - you can overwhelm the servers this way.
I'm building a prototype where on one page A I have a controller. Imagine a TV remote. On another page B, independent of form A -- it can be a different screen/computer -- with some elements X, Y, and Z that are going to be animated by the remote on page A.
My first idea would be:
Remote on page A saves an action wanted and sends JSON.
I create a listener on page B that reads the JSON to listen to what action to trigger.
I wonder if this the right direction.
It is just for a prototype so it doesn't have to be a production-perfect idea.
You can use web sockets for this.
Let's assume you're using 2 computers, both pointed at the website, as this technique could work in both scenarios.
If it's just a prototype, you could just simply have your page B poll the server every 5 seconds to look for updates that were submitted by page A.
However, in reality, for a production app with thousands of users, this could consume lots of bandwidth and put a heavy load on your server. To compensate for the load and bandwidth usage, you could increase the polling rate to 10 seconds, or 30 seconds, but in response to this change your users would experience delays while they wait for the browser to request an update from the server.
In a production app, many developers are turning to Comet as a solution. Comet is basically a term given by Alex Russell for a technique that involves using the request/response cycle to simulate server-push.
The general idea is that the browser makes a request to the server, and the server holds that connection open indefinitely. The connection remains open until another user posts an update to the server, in which case, the server then sends a response to the connected users.
The Dojo and Jetty team have demonstrations where they show that this technique can scale to 20,000 connections, when using continuations.
While I think you can carry out your experiment just fine with a database and/or some session variables, if your want to learn more about Comet on PHP, check out How to Implement Comet with PHP. Good luck!
UPDATE:
I also wanted to say that you definitely have the right idea for how to conceptually think about your message passing with JSON:
I create a listener on page B that reads the JSON to listen to what action to trigger.
I really like how you are thinking about passing a message that then tells the page what action to trigger. If you think about it, you could reuse your message passing concept to call other commands so that you avoid reinventing the wheel when a new command comes along that you need to call. Regardless of whether you poll, use Comet, or use WebSockets, it's a great idea to think about abstractions and generic, reusable data transports.
You could do this either with polling (having page B constantly poll for updates from the server) or use a server push technology like server sent events or websockets.
Yes, that would work. You could also just make it the same way you would make a vector line animation. Send the "commands" for movement to a server and record them (in a database, file, whatever) the client program can then request and redraw the movement smoothly any time and anywhere.
Using a cron job to execute Page B for every x unit time will make you check for any latest updated json (queried/returned output according your logic) from Page A. This way, you can use new updated json from page A and do your further task...
I am using jQuery's $.ajax method to retrieve some JSON from an API.
Each time the page is loaded, a call to the API is made, regardless if the user has has received this data before - which means when a large amount of users are on the page, the API limiting would come into effect.
My thought of how to deal with this would be firstly pushing the data to a database (pushing to a PHP script), and then checking the database to see if anything is cached, before going back to the API to get more up to date information if required.
Is this a viable method? What are the alternatives?
It just seems like jQuery is actually a hurdle, rather than doing it all in PHP to begin with, but as I'm learning the language, would like to use it as much as I can!
In order to help distinguish between opinions and recommended techniques, lets first break down your problem to make sure everyone understands your scenario.
Let's say we have two servers: 'Server A' and 'Server B'. Call 'Server A' our PHP web server and 'Server B' our API server. I'm assuming you don't have control over the API server which is why you are calling it separately and can't scale the API server in parallel to your demand. Lets say its some third party application like flickr or harvest or something... let's say this third party API server throttles requests per hour by your developer API key effectively limiting you to 150 requests per hour.
When one of your pages loads in the end-users browser, the source is coming from 'Server A' (our php server) and in the body of that page is some nice jQuery that performs an .ajax() call to 'Server B' our API server.
Now your developer API key only allows 150 requests per hour, while hypothetically you might see 1000 requests inside one hours to your 'Server A' PHP server. So how do we handle this discrepancy of loads, given the assumption that we can't simple scale up the API server (the best choice if possible).
Here are a few things you can do in this situation:
Just continue as normal, and when jQuery.ajax() returns a 503 service
unavailable error due to throttling (what most third party APIs do) tell your end user politely that
you are experiencing higher than normal traffic and to try again
later. This is not a bad idea to implement even if you also add in
some caching.
Assuming that data being retrieved by the API is cache-able, you
could run a proxy on your PHP server. This is particularly well
suited when the same ajax request would return the same response
repeatedly over time. (ex: maybe you are fetching some description
for an object, the same object request should return the same description response for
some period of time). This could be a PHP Pass through proxy or a
lower level proxy like SQUID caching proxy. In the case of a PHP Pass through proxy you would use the "save to DB or filesystem" strategy for caching and could even re-write the expires headers to suite your level of desired cache lifetime.
If you have established that the response data is cache-able, you can
allow the client side to also cache the ajax response using
cache:true. jQuery.ajax() actually defaults to having cache:true, so you simply need to not set cache to false for it to be caching responses.
If your response data is small, you might consider caching it client-side in a
cookie. But experience says that most users who clear their temporary
internet files will also clear their cookies. So maybe the built in caching with jQuery.ajax() is just as good?
HTML5 local storage for client-side caching is cool, but lacks wide-spread popular support.
If you control your user-base (such as in a corporate environment) you
may be able to mandate the use of an HTML5 compliant browser. Otherwise, you
will likely need a cookie based fallback or polyfill for browsers lacking
HTML5 local storage. In which case you might just reconsider other options above.
To summarize, you should be able to present the user with a friendly service unavailable message no matter what caching techniques you employ. In addition to this you may employ either or both server-side and client-side caching of your API response to reduce the impact. Server-side caching saves repeated requests to the same resource while client side caching saves repeated requests to the same resources by the same user and browser. Given the scenario described, I'd avoid Html5 LocalStorage because you'll need to support fallback/polyfill options which make the built in request caching just as effective in most scenarios.
As you can see jQuery won't really change this situation for you much either way vs calling the API server from PHP server-side. The same caching techniques could be applied if you performed the API calls in PHP on the server side vs performing the API calls via jQuery.ajax() on the client side. The same friendly service unavailable message should be implemented one way or another for when you are over capacity.
If I've misunderstood your question please feel free to leave a comment and clarify and/or edit your original question.
No, don't do it in PHP. Use HTML5 LocalStorage to cache the first request, then do your checking. If you must support older browsers, use a fallback (try these plugins).
I want to write a stream that automaticly adds new posts as they come in. I cant see how to do this with Ajax because i would have to request a script every second to keep it updated, and this would overload my server. Can anyone offer any solutions for this? Thanks :)
Overload your server? Are you sure? How many connections you planning on supporting? Make it every 2 seconds and you cut your traffic in half and most likely no one will notice.
Otherwise you're looking at some style of Comet, server side push, with persistent connections to the server.
Refer to:
Long-lived connections (asynchronous server push) with Apache/PHP/Javascript?
You can try XMPP if you want more thing in realtime but if you want only for above requirement then you can try out node.js
You may use the feature of streaming
https://github.com/mojolly/jquery.evented_ajax.js#readme
And actually or better stucture you ajax requests and use some pooling tecniques
http://dev.sencha.com/deploy/ext-4.0-beta2/examples/direct/direct.html
You want Comet long-polling. Comet is so named because like Ajax, it is a brand of detergent.
Comet takes advantage of the fact that browsers can have two open requests to the server. One of these requests is used to push data to the server, say in reaction to UI events happening in the browser.
The other connection is continually connected to the server in a 'long poll'. When the long-polling connection times out or is disconnected, it is automatically re-established by the client (browser).
When the server wants to push data to the client, it immediately writes this data to the waiting long-poll request and ends it, resulting in a more responsive experience than traditional setTimeout() polling with less overhead.
PHP is not ideal for this. Consider Nodejs.
What's the best method to use to notify javascript client's of changes that occur in a game asynchronously (i.e. moves made by other clients). As an example, assume a turn based board game. Should I just have the client poll the PHP backend every second or so for new moves, or is there a better way to send an asynchronous notification to the other clients in the same game? What is your best idea for how to distribute the notification between the backend instances for each client?
I'm currently planning on putting each move in an SQL database and then having each client poll the database for new moves every second, but this seems kludgy and inefficient...
Polling every second or so is one option, but you may want to consider long polling instead, to reduce the latency.
Quoting Comet Daily: The Long-Polling Technique:
The long-polling Comet technique is a technique that optimizes traditional polling to reduce latency.
Traditional polling sends an XMLHttpRequest to the server in fixed intervals. For example, open a new XMLHttpRequest every 15 seconds, receive an immediate response, and close the connection.
Long-polling sends a request to the server, but a response is not returned to the client until one is available. As soon as the connection is closed, either due to a response being received by the client or if a request times out, a new connection is initiated. The result is a significant reduction in latency because the server usually has a connection established when it is ready to return information to return to the client.
Unfortunately there are only a few options for this. All will involve having some sort of client on the server sending down notifications if you want to use push technology. Or else polling is the way to go.
But just as a tangent here is a solution I played with years ago: http://www.spsolutionscorp.com/blog/2007/03/14/RealTimeUpdatesToWebPageUsingMacromediaFlash.aspx
The gist of it was this: Sql Server with service broker turned on. .Net service on server that is using SqlDependency to listen for changes to query result (http://support.microsoft.com/kb/555893). Once change occurred the .net service looped through all connections that had been made from a flash client that I wrote about in the above blog article. Then the flash app could call out to the javascript on the page.
It was a crazy idea and I never put into productions. But depending on your use it might be worth looking into and it was pretty easy to get going.
But again this was a totally crazy idea and again one I never put into production so use at your own risk. Just thought I would share.
Polling can easily stack up and consume a lot of resources (eg: bandwidth), no matter how lightweight your requests and responses are.
http://ajaxpatterns.org/Periodic_Refresh
You might want to try and use Comet/HTTP Streaming
http://ajaxpatterns.org/HTTP_Streaming