I'm looking for the 'way to go' (i.e. the most efficient, most used, general accepted way) when it comes to the reloading of data from a web server to a front end. In the end application, I will have several output fields where data has to be written to, for example like this:
The data streams will be different from each other in the end application. The lines will have to be reloaded with fresh, up to date data from the server.
I have been thinking of using Ajax requests to update like every second, but there has to be an other way to do this. Ajax requests will cause a lot data traffic. Also, when using the Facebook chat, you don't have to wait every second, chats are received almost instantly. Yet I don't see any Ajax polling requests being made when I use the developer tools of Mozilla Firefox. This made me think if there would be a different way to do this.
I've looked into Node.js, but it appears that isn't possible with my host.
I have heard people talking about Ajax Push, is that what I should use? If so, can you give me a basic usage example?
If not, what would then be the way to go when having multiple data streams that have to be reloaded within a second?
Requirements are speed and low data traffic. It therefore wouldn't be an option to continuously poll the server, I think, because that would create an enormous overhead.
I don't think it's of any importance, but I'm using PHP5.3 in the back end and JavaScript with jQuery 1.9.1 in the front end.
This question has been asked a number of times, but in a slightly different ways. Here are a few references that are worth a read:
What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet?
Using comet with PHP?
Apache with Comet Support
Server Scalability - HTML 5 websockets vs Comet
How to implement event listening in PHP
In summary: if you are looking at building your solution using PHP on Apache then holding open persistent connections (HTTP long-polling or streaming) is going to use up resources very quickly (is highly inefficient). So, you would be better using a hosted solution (*disclaimer - I work for a hosted solution).
HTTP-Long polling and HTTP Streaming are solutions which have been superseded by Server-Sent Events and WebSockets. So, where possible (where the web client provides support) you should use one of these solutions before falling back to an HTTP-based solution. A good realtime web technology will automatically handle this for you.
Since your diagram shows you are subscribing to multiple data streams you should also consider a Publish/Subscribe solution that naturally fits with this. Again, a good realtime web tech solution will provide you with this.
Also see the realtime web technology guide.
I think what you are looking for is generally called Comet. The was this technique is often made to work is as follows:
The client (web browser) makes a request to the server for new data. This is not reloading the page, but rather is done in JavaScript
The server responds to the request when it has some data for the client. Again, this doesn't impact the UI since it isn't the page itself that's getting reloaded: the loaindg of data is done "in background" so to speak, in JavaScript code.
On the serve side, the request waits for new data, and returns the new data when available, or returns nothing if a timeout interval (defined on the server) is reached. This timeout is usually set to be lower than the browser HTTP timeout. The reason for this is so that the server can know whether a particular client got a particular piece of data. If the request is allowed to time out on the client side, the original request might be responded to by the server after the client has timed out, and the client will not get the data, even though the server thinks that it did.
The data is indeed usually transferred as JSON, but you can choose whatever encoding you'd like. See here for one example of how to do this. Goosh is another example of this technique, and so is Interactive Python Shell. The code for all is available.
On the PHP side you will want to create a page that will respond to these "background" JavaScript Comet requests. It could be the same page as the one that user loads, but let's say it is different, for ease of explanation. So the user loads index.php and the JavaScript Comet code calls getNewData.php to retrieve new data.
In your getNewData.php you will want to wait for your event and return the data then. You don't want to use polling for this, but there are PHP libraries that allow one to use various interprocess communication strategies to wait on events, see this question for instance. The high-level pseudocode for your getNewData.php would look as follows:
parse JSON request
Enter an efficient wait state (with timeout), waiting for your "new data is available" event
Did previous step time out?
Yes: send response indicating no data
No: send response with new data
Related
I'm looking for the 'way to go' (i.e. the most efficient, most used, general accepted way) when it comes to the reloading of data from a web server to a front end. In the end application, I will have several output fields where data has to be written to, for example like this:
The data streams will be different from each other in the end application. The lines will have to be reloaded with fresh, up to date data from the server.
I have been thinking of using Ajax requests to update like every second, but there has to be an other way to do this. Ajax requests will cause a lot data traffic. Also, when using the Facebook chat, you don't have to wait every second, chats are received almost instantly. Yet I don't see any Ajax polling requests being made when I use the developer tools of Mozilla Firefox. This made me think if there would be a different way to do this.
I've looked into Node.js, but it appears that isn't possible with my host.
I have heard people talking about Ajax Push, is that what I should use? If so, can you give me a basic usage example?
If not, what would then be the way to go when having multiple data streams that have to be reloaded within a second?
Requirements are speed and low data traffic. It therefore wouldn't be an option to continuously poll the server, I think, because that would create an enormous overhead.
I don't think it's of any importance, but I'm using PHP5.3 in the back end and JavaScript with jQuery 1.9.1 in the front end.
This question has been asked a number of times, but in a slightly different ways. Here are a few references that are worth a read:
What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet?
Using comet with PHP?
Apache with Comet Support
Server Scalability - HTML 5 websockets vs Comet
How to implement event listening in PHP
In summary: if you are looking at building your solution using PHP on Apache then holding open persistent connections (HTTP long-polling or streaming) is going to use up resources very quickly (is highly inefficient). So, you would be better using a hosted solution (*disclaimer - I work for a hosted solution).
HTTP-Long polling and HTTP Streaming are solutions which have been superseded by Server-Sent Events and WebSockets. So, where possible (where the web client provides support) you should use one of these solutions before falling back to an HTTP-based solution. A good realtime web technology will automatically handle this for you.
Since your diagram shows you are subscribing to multiple data streams you should also consider a Publish/Subscribe solution that naturally fits with this. Again, a good realtime web tech solution will provide you with this.
Also see the realtime web technology guide.
I think what you are looking for is generally called Comet. The was this technique is often made to work is as follows:
The client (web browser) makes a request to the server for new data. This is not reloading the page, but rather is done in JavaScript
The server responds to the request when it has some data for the client. Again, this doesn't impact the UI since it isn't the page itself that's getting reloaded: the loaindg of data is done "in background" so to speak, in JavaScript code.
On the serve side, the request waits for new data, and returns the new data when available, or returns nothing if a timeout interval (defined on the server) is reached. This timeout is usually set to be lower than the browser HTTP timeout. The reason for this is so that the server can know whether a particular client got a particular piece of data. If the request is allowed to time out on the client side, the original request might be responded to by the server after the client has timed out, and the client will not get the data, even though the server thinks that it did.
The data is indeed usually transferred as JSON, but you can choose whatever encoding you'd like. See here for one example of how to do this. Goosh is another example of this technique, and so is Interactive Python Shell. The code for all is available.
On the PHP side you will want to create a page that will respond to these "background" JavaScript Comet requests. It could be the same page as the one that user loads, but let's say it is different, for ease of explanation. So the user loads index.php and the JavaScript Comet code calls getNewData.php to retrieve new data.
In your getNewData.php you will want to wait for your event and return the data then. You don't want to use polling for this, but there are PHP libraries that allow one to use various interprocess communication strategies to wait on events, see this question for instance. The high-level pseudocode for your getNewData.php would look as follows:
parse JSON request
Enter an efficient wait state (with timeout), waiting for your "new data is available" event
Did previous step time out?
Yes: send response indicating no data
No: send response with new data
I am curently trying to make a chat application aimed at 1000-1500 users. What currently happens is once my webpage loads I make an ajax request every second to check if there is anything new in the database. I want to know if this is the standard practise or if there is a more efficient way to be notified by the server somehow when an insertion occurs.
Use WebSockets. Or at the very least AJAX polling. Firing a request every second from 1500 clients will most likely kill your server.
Look at http://socket.io/ if you are open to introduce something new to your stack. But there are PHP websocket solutions out there if you are limited to PHP.
Your approach is a standard method called Polling. Based on the number of clients this should be perfectly fine for a server with up-to-date hard-ware (do HEAD requests via AJAX that indicate the status via HTTP status code).
The other alternative - as pointed out by Jan - is called Pushing.
Pros: Involves a lot less requests to the server.
Cons: Requires technology that may or may not be provided by your client's browser.
In case you'll opt for the second approach, take a look into Server-Sent Events (W3C draft).
This specification defines an API for opening an HTTP connection for receiving push notifications from a server in the form of DOM events. The API is designed such that it can be extended to work with other push notification schemes such as Push SMS.
I've used a bit of AJAX with PHP for things like submitting forms and I've recently started looking into websockets. I followed this tutorial to understand the basics. From what I gather, websockets keep the connection open whereas AJAX opens and closes a request.
My question is do websockets provide any advantage over AJAX if you're just submitting forms or simple tasks like auto_complete (which there's a jQuery plugin for anyway)? Maybe the tutorial isn't the greatest, but it seems like there's a heck of a lot more code involved to get websockets to work(at least with PHP) than just a simple AJAX call (or using jQuery which bundles it). I've read in a few places that it's a bit quicker, but if I'm working on something that isn't receiving tons of requests, will it really make a difference? Correct me if I'm wrong, but not all browsers support websockets either, right?
Websockets have two advantages.
they have much less overhead, which results in a better network performance
they allow the server to send data which the client hasn't explicitely requested.
The second one is the most important advantage.
In AJAX, everything the server sends must be the response to a previous request by the client, and every request can only be answered once. But in many applications, especially multi-user applications, events happen on the server and these events must be pushed to the clients immediately. There are workarounds for that in AJAX, like delaying the answer to a request until there is something to report (long-polling), but these are quite dirty. That's why there are Websockets. With a websocket connection, the server can send messages to the clients when it wants and as often as it wants, without having to wait for a request from the client.
But unfortunately WebSockets also have disadvantages:
They aren't as well-supported by web development frameworks (yet!)
Not all web browsers support it (but most desktop browsers already do)
Many proxies and reverse-proxies can't relay websocket traffic (yet!)
Actually, AJAX and websockets are two different categories. AJAX is a concept, a technique. With AJAX you can perform (as the acronym stands for) asynchronous requests, so the browser doesn't need to reload/refresh the whole page. This is good for different things, e.g. checking form input.
Websockets are a protocol, technically quite the same as http, unless the connection will not be closed after transmition. This is good for things, where the webserver may need to contact the client (http can't do that), like a push service fore example (chat or mail client where you want to update the user interface even when the user does not refresh the page, or games). And it kills the http overhead as the whole thing has only to be done once in the beginning.
So, their for different purposes, even if they overlap. For your auto-completion I think it won't make a real difference in performance. And it even is a action/reaction thing, so the user types or submits (whatever) what can cause a request and the server responds.
Websockets is a powerful technology and could certainly cater for the limited use cases you;ve mentioned but there can be compatibility problems with older browsers and network intermediaries. In fact some folks even recommend having an HTTP fallback in case Websockets isn't supported.
Unless you have requirements that necessitate websockets, such as full duplex bi-directional communications for example, you might be better off using existing AJAX based solutions.
If you have requirements to push notifications to your user interface, Websockets can be a good idea, but if you are literally looking for form submission and auto-complete, then these problems have already been solved using Ajax.
I am looking to create a site that will allow users to create groups and then chat/post within those groups. However, when posts/chats are made within a group, I do not want users to have to reload the page to view these new posts/chats within that group. My question boils down to this: what is your best opinion of how to do this (languages, webservices, etc.)?
I know PHP, SQL, HTML, CSS well and know XML, Javascript, AJAX not so well (I have encountered them enough to read the code and know how they work, but I am by no means skilled or confident with them. I have feeling I will need to read a book on one/all of these to build the kind of site I described.)
Any and all input would be greatly appreciated.
The Web does a great job handling the typical request/response model where a client makes a request and a server responds with a resource. However, when it comes to applications where the server must send data to the client without that client requesting the data, this is where we must get creative.
There are a few different methods that one can use to facilitate a real time Web-based application.
Polling:
Polling involves the client periodically making requests to the server in order to receive updates. There are two main problems with this approach: First, there may not be any data for the server to push for a very long time. Hence, lots of bandwidth is potentially wasted continuing to poll the server for updates. Second, the polling rate determines how real-time the application feels. While a fast polling rate will make the updates appear sooner, it wastes bandwidth. Conversely, polling in longer intervals uses less bandwidth, but the downside is that updates don't appear as quickly.
In general, this is a very poor solution to use for a chat application in the year 2012.
Comet/Reverse AJAX:
Comet is a technique that has been successfully used in the last 5 years to take the concept of request/response and use a hack to simulate a real-time effect. The general idea behind Comet is that the client makes a request to the server, and the server holds the connection open indefinitely. The server waits until there is an update to send to the clients. Once the update is ready, the server sends the response, which simulates the server making the request to the client. Once the client receives the response, it opens a new connection, and the process repeats.
This technique has been shown to scale to over 20,000 simultaneous connections on some platforms when combined with Continuations, which ensure waiting threads are freed up for other tasks.
This not only saves bandwidth, but it makes the application feel extremely real time.
Websockets:
Websockets was introduced in HTML5 as a replacement for Comet, using the ws:// protocol instead of http. However, this has not yet been widely adopted by all browser vendors, and there may still be discussions regarding the specification for the protocol. It has many of the same benefits as Comet.
For more information on Comet, check out Comet and PHP and the challenges of Comet in PHP. For client side integration, check out the Dojo Cometd Library.
I would have said AJAX is the best method of doing this. Alternatively, create an iframe which reloads
I need to implement real-time page data update with php and jquery.
(I found www.ape-project.org/ but it seems site is down)
Is any other solutions?
Very TNX!
You might want to check out Comet:
Comet is a web application model in
which a long-held HTTP request allows
a web server to push data to a
browser, without the browser
explicitly requesting it.[1][2] Comet
is an umbrella term, encompassing
multiple techniques for achieving this
interaction. All these methods rely on
features included by default in
browsers, such as JavaScript, rather
than on non-default plugins. The Comet
approach differs from the original
model of the web, in which a browser
requests a complete web page at a
time.
http://en.wikipedia.org/wiki/Comet_%28programming%29
If you want to do streaming (sending multiple messages over a single long lived, low latency connection), you probably need a comet server. Check out http://cometdaily.com/maturity.html for details on a variety of server implementations (I am the maintainer of one of them - Meteor).
If you are happy to reconnect after each message is received, you can do without complicated servers and transports and just use long polling - where you make an ajax request and the server simply sleeps until it has something to send back. But you will end up with LOTS of connections hanging off your web server, so if you're using a conventional web server like Apache, make sure it's configured to handle that. By default Apache doesn't like having more than a few hundred concurrent connections.
There is lots of solutions to do this...
Depending on what is your data, how your data are organized and stored (mysql ?).
Your question is too open to have a real answer.