I'm making a web browser based multiplayer game. I've determined that websockets are the best way to handle communications given its realtime nature. The client uses a HTML5 canvas to render the game and websockets to communicate with the host.
I've elected to use PHP for hosting the game as it seems to be preferred by hosting providers. I haven't used PHP before but have done similar things with websockets in Java, but relying heavily on multithreading.
I've been looking at a few tutorials on php sockets with multiple clients; but most of them do things like fork off new processes for each client. Since I'll have a constantly running game loop I don't think this is suitable.
What I'm trying to achieve is a means of assigning ports to each client as they connect, listening for new clients, exchanging data with the current list of clients and running the game loop all together.
The places where I need help are:
How to find and assign ports to new clients, notify the client of that port, and clean it up when they disconnect.
How to do the above, and all other socket transactions, without blocking the game loop. It would be acceptable to accept messages from clients in partial chunks and only act upon a complete message.
Can anyone give me some technical advice on how to achieve these goals?
I don't think this all looks like too much to ask of PHP but correct me if I'm wrong!
Some pseudocode of what I'd ideally like to achieve server-side. None of the functions should block:
Array clients;
while(gamerunning)
{
CheckForNewClients();
GetStatusFromClients();
DoGameUpdate();
SendGameStateToClients();
}
[Update]
For anyone interested, I created a dedicated application supporting web sockets (specifically using Java, and 'TooTallNates' web socket library) rather than an actual web service as it seemed to make more sense, though incidentally it seems most web browsers have since slung web sockets in the bin due to security issues.
You really need to run a PHP daemon in order to do this effectively (and it NEEDS to be PHP 5.3). I wrote a fairly completely overview of using PHP for daemon processes. Whatever you chose, I would suggest you use an event based, run loop system.
I've designed a basic RunLoop library called LooPHP which could probably be helpful, especially if your going to be dealing with *_select. I'd be more than happy to answer any question you have about that.
EDIT:
In an event based system you don't simply while a list of commands, you react to a listener. For example...
Instead of doing:
while( 1 ) {
... /* listen, react */
} /* repeat */
Run loops work by registering listener (sockets, and other async event generators)
class ReactClass { ... }
$loop = new LooPHP_EventLoop( new ReactClass );
//add one time event
$loop->addEvent( function() {
print "This event was called 0.5 second after being added\n";
}, 0.5 /* in seconds */ );
//this creates a repeating event, this is called right away and repeats
$add_event = function() use ( $loop, &$add_event ) {
print "This event is REPEATEDLY called 0.1 every second\n";
$loop->addEvent( $add_event, 0.1 );
};
$add_event();
//start the loop processing, no events are processed until this is done
$loop->run(); //php doesn't leave this call until the daemon is done
exit(0); //cleanly exit
The above case is a very simple 1 source EventLoop and a manually add timed functions ( these can be added even from within a call to ReactClass).
In the application I'm working I needed to have both asynchronous event feed into the backend (via a socket) and then needed to have the ability to call functions arbitrary offset from the original event (for timed-out clients, etc).
If you'd like more examples, you can find them over at github.
I hope you find this useful.
I wouldn't suggest using PHP for this type of application. PHP doesn't officially support multithreading and running a PHP script for an undefined period of time (like a server) isn't really an advertised feature.
Of course you could try and make history :)
(please correct me if i'm mistaken)
Related
Hi I'm looking for a way to implement a coroutine in a php file. The idea is that I have long processes that need to be able to yield for potentially hours or days. So other php files will be calling functions in the same file as the coroutine to update something, then call a function like $coroutine.process() that causes the coroutine to continue from its last yield. This is to avoid having to use a large state machine.
I'm thinking that the coroutine php file will not actually be running when it's idle, but that when given processing time, will enter from the top and use something like a switch or goto to restart from the previous yield. Then when it reaches the next yield, the file will save its current state somewhere (like a session or database) and then exit.
Has anyone heard of this, or a metaphor similar to this? Bonus points for aggregating and managing multiple coroutines under one collection somehow, perhaps with support for a thread-like join so that flow continues in one place when they finish (a bit like Go).
UPDATE: php 5.5.0 has added support for generators and coroutines:
https://github.com/php/php-src/blob/php-5.5.0/NEWS
https://wiki.php.net/rfc/generators
I have not tried it yet, so perhaps someone can suggest a barebones example. I'm trying to convert a state machine into a coroutine. So for example a switch command inside of a for loop (whose flow is difficult to follow, and error prone as more states are added) converted to a cooperative thread where each decision point is easy to see in an orderly, linear flow that pauses for state changes at the yield keyword.
A concrete example of this is, imagine you are writing an elevator controller. Instead of determining whether to read the state of the buttons based on the elevator's state (STATE_RISING, STATE_LOWERING, STATE_WAITING, etc), you write one loop with sub-loops that run while the elevator is in each state. So while it's rising, it won't lower, and it won't read any buttons except the emergency button. This may not seem like a big deal, but in a complex state machine like a chat server, it can become almost impossible to update a state machine without introducing subtle bugs. Whereas the cooperative thread (coroutine) version has a plainly visibly flow that's easier to debug.
The Swoole Coroutine library provides go like coroutines for PHP. Each coroutine adds only 8K of ram per process. It provides a coroutine API with the basic functions expected (such as yield and resume), coro utilities such a coroutine iterator, as well as higher level coroutine builtins such as filesystem functions and networking (socket clients and servers, redis client and server, MySQL client, etc).
A second element to your question is the ability to have long lived coroutines - this likely isn't a good idea unless you are saving the state of the coro in a session and allowing the coro to end/close. Otherwise the request will have to live as long as the coroutine. If the service is being hosted by a long lived PHP script the scenario is easier and the coroutine will simply live until it is allowed to / forced to close.
Swoole performs comparibly to Node.js and Go based services, and is used in multiple production services that regularly host 500K+ TCP connections. It is a little known gem for PHP, largely because it is developed in China and most support and documentation is limited to Chinese speakers, although a small handful of individuals strive to help individuals that speak other languages.
One nice point for Swoole is that it's PHP classes wrap an expansive C/C++ api designed from to start to allow all of it's features to be used without PHP. The same source can easily be compiled as both a PHP extension and/or a standard library for both *NIX systems and Windows.
PHP does not support coroutines.
I would write a PHP extension with setcontext(), of course assuming you are targeting Unix platforms.
Here a StackOverflow question about getting started with PHP extensions: Getting Started with PHP Extension-Development.
Why setcontext()? It is a little known fact that setcontext() can be used for coroutines. Just swap the context when calling another coroutine.
I am writing a second answer because there seems to be a different approach to PHP coroutines.
With Comet HTTP responses are long-lived. Small <script> chunks are sent from time to time and the JavaScript is executed by the browser as they arrive. The response can pause for a long time waiting for an event. 2001 I wrote a small hobby chat server in Java exploiting this technique. I was abroad for half a year and was homesick and used this to chat with my parents and my friends at home.
The chat server showed me that it is possible that a HTTP request triggers other HTTP responses. This is somewhat like a coroutine. All the HTTP responses are waiting for an event and if the event applies for a response, it takes up processing and then goes sleeping again, after having triggered some other response.
You need a medium over which the PHP "processes" communicate with each other. A simple medium are files, but I think a database would be a better fit. My old chat server used a log file. Chat messages were appended to the log file and all chat processes were continually reading from the end of the log file in an endless loop. PHP supports sockets for direct communication, but this needs a different setup.
To get started, I propose these two functions:
function get_message() {
# Check medium. Return a message; or NULL if there are no messages waiting.
}
function send_message($message) {
# Write a message to the medium.
}
Your coroutines loop like this:
while (1) {
sleep(1); // go easy on the CPU
$message = get_message();
if ($message === NULL) continue;
# Your coroutine is now active. Act on the message.
# You can send send messages to other coroutines.
# You also can send <script> chunks to the browser, like this:
echo '<script type="text/javascript">';
echo '// Your JavaScript code';
echo '</script>';
flush();
# Yield
}
To yield use continue, because it restarts the while (1) loop waiting for messages. The coroutine also yields at the end of the loop.
You can give your coroutines IDs and/or devise a subscription model in which some coroutines listen to some messages but not all.
Edit:
Sadly PHP and Apache are not a very good fit for a scalable solution. Even if most of the time the coroutines don't do anything, they hog memory as processes, and Apache starts trashing memory if there are too many of them, maybe for a few thousand coroutines. Java is not very much better, but since my chat server was private, I didn't experience performance problems. There never were more than 10 users accessing it simultaneously.
Ningx, Node.js or Erlang have this solved in a better way.
I have a php based web application that captures certain events in a database table. It also features a visualization of those captured events: a html table listing the events which is controlled by ajax.
I would like to add an optional 'live' feature: after pressing a button ('switch on') all events captured from that moment on will be inserted into the already visible table. Three things have to happen: noticing the event, fetching the events data and inserting it into the table. To keep the server load inside sane limits I do not want to poll for new events with ajax request, instead I would prefer the long polling strategy.
The problem with this is obviously that when doing a long polling ajax call the servers counterpart has to monitor for an event. Since the events are registered by php scripts there is no easy way to notice that event without polling the database for changes again. This is because the capturing action runs in another process than the observing long polling request. I looked around to find a usable mechanism for such inter process communication as I know it from rich clients under linux. Indeed there are php extensions for semaphores, shared memory or even posix. However they all only exist under linux (or unix like) systems. Though not typically the application might be used under MS-Windows systems in rare cases.
So my simple question is: is there any means that is typically available on all (most) systems that can push such events to a php script servicing the long polling ajax request ? Something without polling a file or a database constantly, since I already have an event elsewhere ?
So, the initial caveats: without doing something "special", trying to do long polling with vanilla PHP will eat up resources until you kill your server.
Here is a good basic guide to basic PHP based long polling and some of the challenges associated with going the "simple" road:
How do I implement basic "Long Polling"?
As far as doing this really cross-platform (and simple enough to start), you may need to fall back to some sort of simple internal polling - but the goal should be to ensure that this action is much lower-cost than having the client poll.
One route would be to essentially treat it like you're caching database calls (which you are at this point), and go with some standard caching approaches. Everything from APC, to memcached, to polling a file, will all likely put less load on the server than having the client set up and tear down a connection every second. Have one process place data in the correct keys, and then poll them in your script on a regular basis.
Here is a pretty good overview of a variety of caching options that might be crossplatform enough for you:
http://simas.posterous.com/php-data-caching-techniques
Once you reach the limits of this approach, you'll probably have to move onto a different server architecture anyhow.
Js:
<script>
function cometConnect(){
$.ajax({
cache:false,
type:"post",
data:'ts='+1,
url: 'Controller/chatting',
async: true,
success: function (arr1) {
$(".page-header").append(arr1);
},
complete:function(){
cometConnect(true);
nerr=false;
},
dataType: "text"
});
}
cometConnect();
</script>
Php:
public function chatting()
{
while(true)
{
if(memcache_get(new_message))
return new_message;
sleep(0.5);
}
}
Is this a better solution than setting setInterval which connects to the PHP method which returns message if there is any every 1 second (1 sec increases +0.25 every 5 seconds let's say)?
If I used first solution, I could probably use sleep(0.5) it would give me messages instantly, because php loop is cheap, isn't?
So, what solution is better (more importantly, which takes less resources?). Because there are going to be hundreds of chats like this.
Plus, can first solution cause problems? Let's say I would reload a page or I would stop execution every 30 secs so I wouldn't get 502 Bad Gateway.
EDIT: I believe the second solution is better, so I am going to reimplement my site, but I am just curious if this can cause problems to the user or not? Can something not expected happen?
First problem I noticed is that you can't go to other page until there is at least one new message.
A chat is a one to many communication, while each one of the many can send messages and will receive messages from everybody else.
These two actions (sending, receiving) happen continuously. So this looks like an endless loop whereas the user can enter into (join the chat) and exit (leave the chat).
enter
send message
receive message
exit
So the loop looks like this (pseudo-code) on the client side:
while (userInChat)
{
if (userEnteredMessages)
{
userSendMessages(userEnteredMessages)
}
if (chatNewMessages)
{
displayMessages(chatNewMessages)
}
}
As you already note in your question, the problem is in implementing such a kind of chat for a website.
To implement such a "loop" for a website, you are first of all facing the situation that you don't want to have an actual loop here. As long as the user is in chat, it would run and run and run. So you want to distribute the execution of the loop over time.
To do this, you can convert it into a collection of event functions:
ChatClient
{
function onEnter()
{
}
function onUserInput(messages)
{
sendMessages = send(messages)
display(sendMessages)
}
function onReceive(messages)
{
display(messages)
}
function onExit()
{
}
}
It's now possible to trigger events instead of having a loop. Only left is the implementation to trigger these events over time, but for the moment this is not really interesting because it would be dependent to how the chat data exchange is actually implemented.
There always is a remote point where a chat client is (somehow) connected to to send it's own messages and to receive new messages from.
This is some sort of a stream of chat messages. Again this looks like a loop, but infact it's a stream. Like in the chat clients loop, at some point in time it hooks onto the stream and will send input (write) and receive output (read) from that stream.
This is already visible in the ChatClient pseudo code above, there is an event when the user inputs one or multiple messages which then will be send (written). And read messages will be available in the onReceive event function.
As the stream is data in order, there needs to be order. As this is all event based and multiple clients are available, this needs some dedicated handling. As order is relative, it will only work in it's context. The context could be the time (one message came before another message), but if the chat client has another clock as the server or another client, we can't use the existing clock as time-source for the order of messages, as it normally differs between computers in a WAN.
Instead you create your own time to line-up all messages. With a shared time across all clients and servers an ordered stream can be implemented. This can be easily done by just numbering the messages in a central place. Luckily your chat has a central place, the server.
The message stream starts with the first message and ends with the last one. So what you simply do is to give the first message the number 1 and then each new message will get the next higher number. Let's call it the message ID.
So still regardless which server technology you'll be using, the chat knows to type of messages: Messages with an ID and messages without an ID. This also represents the status of a message: either not part or part of the stream.
Not stream associated messages are those that the user has already entered but which have not been send to the server already. While the server receives the "free" messages, it can put them into the stream by assigning the ID:
function onUserInput(messages)
{
sendMessages = send(messages)
display(sendMessages)
}
As this pseudo code example shows, this is what is happening here. The onUserInput event get's messages that are not part of the stream yet. The sendMessages routine will return their streamed representation which are then displayed.
The display routine then is able to display messages in their stream order.
So still regardless how the client/server communication is implemented, with such a structure you can actually roughly handle a message based chat system and de-couple it from underlying technologies.
The only thing the server needs to do is to take the messages, gives each message an ID and return these IDs. The assignment of the ID is normally done when the server stores the messages into it's database. A good database takes care to number messages properly, so there is not much to do.
The other interaction is to read new messages from the server. To do this over network effectively, the client tells the server from which message on it likes to read from. The server will then pass the messages since that time (ID) to the client.
As this shows, from the "endless" loop in the beginning it's now turned into an event based system with remote calls. As remote calls are expensive, it is better to make them able to transfer much data with one connection. Part of that is already in the pseudo code as it's possible to send one or multiple messages to the server and to receive zero or more messages from the server at once.
The ideal implementation would be to have one connection to the server that allows to read and write messages to it in full-duplex. However no such technology exists yet in javascript. These things are under development with Websockets and Webstream APIs and the like but for the moment let's take things simple and look what we have: stateless HTTP requests, some PHP on the server and a MySQL database.
The message stream can be represented in a database table that has an auto-incrementing unique key for the ID and other fields to store the message.
The write transaction script will just connect to the database, insert the message(s) and return the IDs. That's a very common operation and it should be fast (mysql has a sort of memcache bridge which should make the store operation even more fast and convenient).
The read transaction script is equally simple, it will just read all messages with an ID higher than passed to it and return it to the client.
Keep these scripts as simple as possible and optimize the read/write time to the store, so they can execute fast and you're done even with chatting over plain HTTP.
Still your webserver and the overall internet connection might not be fast enough (although there is keep-alive).
However, HTTP should be good enough for the moment to test if you chat system is actually working without any loops, not client, nor server side.
It's also good to keep servers dead simple, because each client relies on them, so they should just do their work and that's it.
You can at any time change the server (or offer different type of servers) that can interact with your chat client by giving the chat client different implementations of the send and receive functions. E.g. I see in your question that you're using comet, this should work as well, it's probably easy to directly implement the server for comet.
If in the future websockets are more accessible (which might never be the case because of security considerations), you can offer another type of server for websockets as well. As long as the data-structure of the stream is intact, this will work with different type of servers next to each other. The database will take care of the congruency.
Hope this is helpful.
Just as an additional note: HTML5 offers something called Stream Updates with Server-Sent Events with an online demo and PHP/JS sources. The HTML 5 feature offers already an event object in javascript which could be used to create an exemplary chat client transport implementation.
I wrote a blog post about how I had to handle a similar problem (using node.js, but the principles apply).
http://j-query.blogspot.com/2011/11/strategies-for-scaling-real-time-web.html
My suggestion is, if it's going to be big either a) you need to cache like crazy on your web server layer, which probably means your AJAX call needs to have a timestamp on it or b) use something like socket.io, which is built for scaling real-time web apps and has built-in support for channels.
Infinite loops in php can and will use 100% of your CPU. Sleep functions will fix that problem. However, you probably don't want to have a separate HTTP process running all the time for every client that is connected to your server because you'll run out of connections. You could just have one php process that looks at all inbound messages and routes them to the right person as they come in. This process could be launched from a cron job once a minute. I've written this type of thing many times and it works like a charm. Note: Make sure you don't run the process if it's already running or you will run into multiprocessing problems (like getting double messages). In other words, you need to make the process thread safe.
If you want to get real time chatting, then you might want to take a look at StreamHub which opens a full duplex connection to the client's browser.
It's not a PHP or jQuery task now. Node.js!
There is socket.io, which means WebSockets.
I'll explain why node.js is better. I have a task to refresh on-page markers every, for example, 10 seconds. I've done it with the first method. When the persistent users count come to 200. Http server and php were in trouble. There were a lot of requests which was unnesessary.
Whats give you Node.js:
Creating separate rooms for chats (here)
Sends data, only for those who has updates (for example, if I do not have any new message my refresh will be blocked when there will be selection from database)
You run 1 query to the DB per 0.5 second, no matter how much users there are
Just look into Node.js and Socket.io. This solution help me with a great boost.
First off, ask yourself if it's necessary to update the chat frequently. What type of chats will be happening? Is it real-time? Simple Q&A? Tech support? Etc. In all but the real-time chat cases, you will be better off using a long polling JS-based design, because instantaneous responses are not that important. If this is for real-time chats, then you should consider a Gmail-like design whereby you keep an XHR open and push messages back to the client as they are received. If connection resources are a concern, you can get by using long polling with a very brief interval (ex. 5-10 seconds).
In my own system, I have a PHP page which displays all the goods I have sold through my online webshops, from the latest transaction to the first.
I would like this page to automatically update, whenever people buy something from me.
I could make AJAX call every 5 seconds to check the orders in my databases, but that seems 1980?
or is it that way people do it?
How can I go about pushing a notification to my php page whenever my php newOrder() function (lets call it that) is called?
You can achieve push within PHP but it won't be the most efficient solution because to achieve push you need to maintain long running connections between your client and your server (HTTP or WebSocket connections).
See:
Long Polling/HTTP Streaming General Questions
phpwebsocket
php-websocket on github
Ratchet
how to implement comet in PHP - frequently linked to guide
General best practice when building a realtime infrastructure has been to decouple the push solution from your web application (Note: node.js and socket.io has changed this a bit, but personally I still think it should be decoupled). But, assuming that the latter is still the best solution you would need to write/host/install this push solution. Decoupling also means that the technology doesn't have to be PHP but you can access/use it from PHP. Maybe a bit of overkill? And especially if you don't have too many users on your site?
For simplicity I would recommend looking at using a 3rd party hosted service. I work for one such company called Pusher. Using a service such as ours lets you remove the need to install and maintain the realtime part of your application. It also makes it really easy to add the push functionality you are looking for. All you need to do is add a few lines of PHP code to your existing app to trigger the push notifications and add a few lines of JavaScript to your front-end.
Resources:
Most commonly use PHP library for this: https://github.com/pusher/pusher-php-server
Quickstart guide
If you'd like to investigate the alternatives or some of the technologies I've mentioned above I'm maintaining a list of realtime technologies which you might also be interested in.
You could simulate a push effect by doing the following (pseudo code!)
// This would go on for 60-90 seconds, unless there is new data
for($i = 1; $i <= 60; $i++) {
// Check in the database what the last timestamp is
// Compare this to the timestamp that has been posted by the AJAX-call
// If there is something new, show the data and then exit
// If not, sleep for 1-3 seconds
}
Your javascript:
function pollForNewProducts() {
// Check if there is a timestamp
// Make an AJAX-request to the script that has the first code in it
// If there is a response (JSON?) than evaluate that
// If not, then run the script again
pollForNewProducts();
}
It is a simple yet effective way to let the server do all the hard work, instead of timeouts on the client side which will cause the browser to eat memory.
More about this on:
Simple “Long Polling” example code?
Client notification, should I use an AJAX Push or Poll?
PHP or Javascript poll code
ExtJS 3.0: Ext.Direct with PHP: Polling:ok and Pushing ?
Comet Programming: Using Ajax to Simulate Server Push
Apart from the excellent suggestion about nodejs ..if you want to still use php to achieve this, you want to look for the COMET method, not ajax.
Howto with php here : http://www.zeitoun.net/articles/comet%5Fand%5Fphp/start
I'm trying to make a theoretical web chat application with php and jquery, I've read about long polling and http streaming, and I managed to apply most principles introduced in the articles. However, there are 2 main things I still can't get my head around.
With Long Polling
How will the server know when an update have been sent? will it need to query the databse continually or is there a better way?
With HTTP Streaming
How do I check for the results during the Ajax connection is still active? I'm aware of jQuery's success function for ajax calls, but how do I check the data while the connection is still ongoing?
I'll appreciate any and all answers, thanks in advance.
Yeah, the Comet-like techniques usually blowing up the brain in the beginning -- just making you think in a different way. And another problem is there are not that much resources available for PHP, cuz everyone's doing their Comet in node.js, Python, Java, etc.
I'll try to answer your questions, hope it would shed some light on this topic for people.
How will the server know when an update have been sent? will it need to query the databse continually or is there a better way?
The answer is: in the most general case you should use a message queue (MQ). RabbitMQ or the Pub/Sub functionality built into the Redis store may be a good choices, though there are many competing solutions on the market available such as ZeroMQ, Beanstalkd, etc.
So instead of continuous querying your database, you can just subscribe for an MQ-event and just hang until someone else will publish a message you subscribed for and MQ will wake you up and send a message. The chat app is a very good use case to understand this functionality.
Also I have to mention that if you would search for Comet-chat implementations in other languages, you might notice simple ones not using MQ. So how do they exchange the information then? The thing is such solutions are usually implemented as standalone single-threaded asynchronous servers, so they can store all connections in a thread local array (or something similar), handle many connections in a single loop and just pick a one and notify when needed. Such asynchronous server implementations are a modern approach that fits Comet-technique really great. However you're most likely implementing your Comet on top of mod_php or FastCGI, in this case this simple approach is not an option for you and you should use MQ.
This could still be very useful to understand how to implement a standalone asynchronous Comet-server to handle many connections in a single thread. Recent versions of PHP support Libevent and Socket Streams, so it is possible to implement such kind of server in PHP as well. There's also an example available in PHP documentation.
How do I check for the results during the Ajax connection is still active? I'm aware of jQuery's success function for ajax calls, but how do I check the data while the connection is still ongoing?
If you're doing your long-running polls with a usual Ajax technique such as plain XHR, jQuery Ajax, etc. you don't have an easy way to transmit several responses in a single Ajax request. As you mentioned you only have 'success' handler to deal with the response in whole and not with its part. As a workaround people send only a single response per request and process it in a 'success' handler, after that they just open a new long-poll request. This is just how HTTP-protocol works.
Also should be mentioned that actually there are workaround to implement streaming-like functionality using various techniques using techniques such as infinitely long page in a hidden IFRAME or using multipart HTTP-responses. Both of those methods are certain drawbacks (the former one is considered unreliable and sometimes could produce unwanted browser behavior such as infinite loading indicator and the latter one leaks consistent and straightforward cross-browser support, however certain applications still are known to successfully rely on that mechanism falling back to long-polling when the browser can't properly handle multipart responses).
If you'd like to handle multiple responses per single request/connection in a reliable way you should consider using a more advanced technology such as WebSocket which is supported by the most current browsers or on any platform that supports raw sockets (such as Flash or if you develop for a mobile app for instance).
Could you please elaborate more on message queues?
Message Queue is a term that describes a standalone (or built-in) implementation of the Observer pattern (also known as 'Publish/Subscribe' or simply PubSub). If you develop a big application, having one is very useful -- it allows you to decouple different parts of your system, implement event-driven asynchronous design and make your life much easier, especially in a heterogeneous systems. It has many applications to the real-world systems, I'll mention just couple of them:
Task queues. Let's say we're writing our own YouTube and need to convert users' video files in the background. We should obviously have a webapp with the UI to upload a movie and some fixed number of worker processes to convert the video files (maybe we would even need a number of dedicated servers where our workers only will leave). Also we would probably have to write our workers in C to ensure better performance. All we have to do is just setup a message queue server to collect and deliver video-conversion tasks from the webapp to our workers. When the worker spawns it connects to the MQ and goes idle waiting for a new tasks. When someone uploads a video file the webapp connects to the MQ and publishes a message with a new job. Powerful MQs such as RabbitMQ can equally distribute tasks among number of workers connected, keep track of what tasks had been completed, ensure nothing will get lost and will provide fail-over and even admin UI to browse current tasks pending and stats.
Asynchronous behavior. Our Comet-chat is a good example. Obviously we don't want to periodically poll our database all time (what's the use of Comet then? -- Not big difference of doing periodical Ajax-requests). We would rather need someone to notify us when a new chat-message appears. And a message queue is that someone. Let's say we're using Redis key/value store -- this is a really great tool that provides PubSub implementation among its data store features. The simplest scenario may look like following:
After someone enters the chat room a new Ajax long poll request is being made.
Request handler on the server side issues the command to Redis to subscribe a 'newmessage' channel.
Once someone enters a message into his chat the server-side handler publishes a message into the Redis' 'newmessage' topic.
Once a message is published, Redis will immediately notify all those pending handlers which subscribed to that channel before.
Upon notification PHP-code that keeps long-poll request open, can return the request with a new chat message, so all users will be notified. They can read new messages from the database at that moment, or the messages may be transmitted directly inside message payload.
I hope my illustration is easy to understand, however message queues is a very broad topic, so refer to the resources mentioned above for further reading.
How do I check for the results during the Ajax connection is still active? I'm aware of jQuery's success function for ajax calls, but how do I check the data while the connection is still ongoing?
Actually, you can. I've provided a revised answer for the above but I don't know if it's still pending or has been ignored. Providing an update here so that the correct information is available.
If you keep the connection between the client and the server open it is possible to push updates through which are appended to the response. As each update comes in the XMLHttpRequest.onreadystatechange event is fired and the value of the XMLHttpRequest.readyState will be 3. This means that the XMLHttpRequest.responseText continues to grow.
You can see an example of this here:
http://www.leggetter.co.uk/stackoverflow/7213549/
To see the JS code simply view source. The PHP code is:
<?php
$updates = $_GET['updates'];
if(!$updates) {
$updates = 100;
}
header('Content-type: text/plain');
echo str_pad('PADDING', 2048, '|PADDING'); // initial buffer required
$sleep_time = 1;
$count = 0;
$update_suffix = 'Just keep streaming, streaming, streaming. Just keep streaming.';
while($count < 100) {
$message = $count . ' >> ' . $update_suffix;
echo($message);
flush();
$count = $count + 1;
sleep($sleep_time);
}
?>
In Gecko based browsers such as Firefox it's possible to completely replaces the responseText by using multipart/x-mixed-replace. I've not provided an example of this.
It doesn't look like it's possible to achieve the same sort of functionality using jQuery.ajax. The success callback does not fire whenever the onreadystatechange event is fired. This is surprising since the documentation states:
No onreadystatechange mechanism is provided, however, since success, error, complete and statusCode cover all conceivable requirements.
So the documentation is potentially wrong unless I'm misinterpreting it?
You can see an example that tries to use jQuery here:
http://www.leggetter.co.uk/stackoverflow/7213549/jquery.html
If you take a look at the network tab in either Firebug or Chrome Developer tools you'll see the file size of stream.php growing but the success callback still isn't fire.