I was thinking of implementing real time chat using a PHP backend, but I ran across this comment on a site discussing comet:
My understanding is that PHP is a
terrible language for Comet, because
Comet requires you to keep a
persistent connection open to each
browser client. Using mod_php this
means tying up an Apache child
full-time for each client which
doesn’t scale at all. The people I
know doing Comet stuff are mostly
using Twisted Python which is designed
to handle hundreds or thousands of
simultaneous connections.
Is this true? Or is it something that can be configured around?
Agreeing/expanding what has already been said, I don't think FastCGI will solve the problem.
Apache
Each request into Apache will use one worker thread until the request completes, which may be a long time for COMET requests.
This article on Ajaxian mentions using COMET on Apache, and that it is difficult. The problem isn't specific to PHP, and applies to any back-end CGI module you may want to use on Apache.
The suggested solution was to use the 'event' MPM module which changes the way requests are dispatched to worker threads.
This MPM tries to fix
the 'keep alive problem' in HTTP.
After a client completes the first
request, the client can keep the
connection open, and send further
requests using the same socket. This
can save signifigant overhead in
creating TCP connections. However,
Apache traditionally keeps an entire
child process/thread waiting for data
from the client, which brings its own
disadvantages. To solve this problem,
this MPM uses a dedicated thread to
handle both the Listening sockets, and
all sockets that are in a Keep Alive
state.
Unfortunately, that doesn't work either, because it will only 'snooze' after a request is complete, waiting for a new request from the client.
PHP
Now, considering the other side of the problem, even if you resolve the issue with holding up one thread per comet request, you will still need one PHP thread per request - this is why FastCGI won't help.
You need something like Continuations which allow the comet requests to be resumed when the event they are triggered by is observed. AFAIK, this isn't something that's possible in PHP. I've only seen it in Java - see the Apache Tomcat server.
Edit:
There's an article here about using a load balancer (HAProxy) to allow you to run both an apache server and a comet-enabled server (e.g. jetty, tomcat for Java) on port 80 of the same server.
You could use Nginx and JavaScript to implement a Comet based chat system that is very scalable with little memory or CPU utilization.
I have a very simple example here that can get you started. It covers compiling Nginx with the NHPM module and includes code for simple publisher/subscriber roles in jQuery, PHP, and Bash.
http://blog.jamieisaacs.com/2010/08/27/comet-with-nginx-and-jquery/
PHP
I found this funny little screencasts explaining simple comet. As a side note I really think this is going to kill your server on any real load. When just having a couple of users, I would say to just go for this solution. This solution is really simple to implement(screencasts only takes 5 minutes of your time :)). But as I was telling previously I don't think it is good for a lot of concurrent users(Guess you should benchmark it ;)) because:
It uses file I/O which is much slower then just getting data from memory. Like for example the functions filemtime(),
Second, but I don't think least PHP does not a have a decent thread model. PHP was not designed for this anyway because of the share nothing model. Like the slides says "Shared data is pushed down to the data-store layer" like for example MySQL.
Alternatives
I really think you should try the alternatives if you want to do any comet/long polling. You could use many languages like for example:
Java/JVM: Jetty continuations.
Python: Dustin's slosh.
Erlang: Popular language for comet/etc.
Lua, Ruby, C, Perl just to name a few.
Just performing a simple google search, will show you a lot alternatives also PHP(which I think on any big load will kill your server).
mod_php is not the only way to use PHP. You can use fastcgi. PHP must be compiled with --enable-fastcgi.
PHP as FastCGI: http://www.fastcgi.com/drupal/node/5?q=node/10
You may also try https://github.com/reactphp/react
React is a low-level library for event-driven programming in PHP. At its core is an event loop, on top of which it provides low-level utilities, such as: Streams abstraction, async dns resolver, network client/server, http client/server, interaction with processes. Third-party libraries can use these components to create async network clients/servers and more.
The event loop is based on the reactor pattern (hence the name) and strongly inspired by libraries such as EventMachine (Ruby), Twisted (Python) and Node.js (V8).
The introductory example shows a simple HTTP server listening on port 1337:
<?php
$i = 0;
$app = function ($request, $response) use (&$i) {
$i++;
$text = "This is request number $i.\n";
$headers = array('Content-Type' => 'text/plain');
$response->writeHead(200, $headers);
$response->end($text);
};
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$http = new React\Http\Server($socket);
$http->on('request', $app);
$socket->listen(1337);
$loop->run();
I'm having a similar issue. One option I'm finding interesting is to use an existing Comet server, like cometd-java or cometd-python, as the core message hub. Your PHP code is then just a client to the Comet server -- it can post or read messages from channels, just like other clients.
There's an interesting code snippet linked here: http://morglog.org/?p=22=1 that implements part of this method (although there are bits of debug code spread around, too).
I'm current implementing a scalable PHP Comet server using socket functions. It is called 'phet' ( [ph]p com[et] )
Project page: http://github.com/Tim-Smart/phet
Free free to join in on development. I have currently managed to get most of the server logic done, just need to finish off the client side stuff.
EDIT: Recently added 'Multi-threading' capabilities using the pcntl_fork method :)
You'll have a hard time implementing comet in PHP, just because of it's inherent single-threaded-ness.
Check out Websync On-Demand - the service lets you integrate PHP via server-side publishing, offloading the heavy concurrent connection stuff, and will let you create a real-time chat app in no time.
A new module just came out for the nginx web server that'll allow Comet with any language, including PHP.
http://www.igvita.com/2009/10/21/nginx-comet-low-latency-server-push/
You will have to create your own server in PHP. Using Apache/mod_php or even fastcgi will not scale at all. A few years old, but can get you started:
PHP-Comet-Server:
http://sourceforge.net/projects/comet/
I think this is more an issue that having a lot of apache threads running all the time is a problem. That will existing with any language if it works via apache in the same way as PHP (usually) does.
Related
I have a fairly big web application build with SproutCore and PHP as backend. What I want now is to use websockets to update my client application in real time.
From what I know, PHP is really bad to handle persistent connections. So I've been thinking that I could use Go to handle the websockets connections and call my PHP scripts each time a request is received (this package seem to make it possible).
So my first question is, do you guys think it's a good idea (and a viable idea, I haven't been able to find people doing so) or should I stick with PHP ?
If I use Go to handle the websockets connections I've also been thinking that I could progressively move away from PHP to only use Go (since it is a lot faster than PHP). If I do that, I will have to be able to call some Go package from PHP. Can this be done with the PHP exec function ? Is there a better way ? And again, is that a good idea ?
Go is a natural fit for websocket servers. I've built websocket servers in Go and have been extremely happy with how it all worked out. I have one service handling 300k users a month on a Go websocket server and it barely uses 1% CPU of an Amazon AWS micro instance. Couldn't be happier.
Websockets really need event driven frameworks like Go and Node.js in order to maximize server resources. Forked web processes like PHP consume far more resources than an event driven framework.
If you need to call Go from PHP at some point, I suggest using API calls. Although exec would work too.
It is an old question, but my two cents on this subject...
There is a very good php library that does exactly what you are asking for - websockets. It is called Ratchet. I would not use node.js (over hyped) or go when php can do exactly the same thing and it is the language that I am most comfortable with. In majority of cases, little gain in performance over websockets is not worth switching stack.
Other useful links if you plan on using Ratchet:
Backend - Ratchet Examples
Frontend - Autobahn WS
I was thinking of implementing real time chat using a PHP backend, but I ran across this comment on a site discussing comet:
My understanding is that PHP is a
terrible language for Comet, because
Comet requires you to keep a
persistent connection open to each
browser client. Using mod_php this
means tying up an Apache child
full-time for each client which
doesn’t scale at all. The people I
know doing Comet stuff are mostly
using Twisted Python which is designed
to handle hundreds or thousands of
simultaneous connections.
Is this true? Or is it something that can be configured around?
Agreeing/expanding what has already been said, I don't think FastCGI will solve the problem.
Apache
Each request into Apache will use one worker thread until the request completes, which may be a long time for COMET requests.
This article on Ajaxian mentions using COMET on Apache, and that it is difficult. The problem isn't specific to PHP, and applies to any back-end CGI module you may want to use on Apache.
The suggested solution was to use the 'event' MPM module which changes the way requests are dispatched to worker threads.
This MPM tries to fix
the 'keep alive problem' in HTTP.
After a client completes the first
request, the client can keep the
connection open, and send further
requests using the same socket. This
can save signifigant overhead in
creating TCP connections. However,
Apache traditionally keeps an entire
child process/thread waiting for data
from the client, which brings its own
disadvantages. To solve this problem,
this MPM uses a dedicated thread to
handle both the Listening sockets, and
all sockets that are in a Keep Alive
state.
Unfortunately, that doesn't work either, because it will only 'snooze' after a request is complete, waiting for a new request from the client.
PHP
Now, considering the other side of the problem, even if you resolve the issue with holding up one thread per comet request, you will still need one PHP thread per request - this is why FastCGI won't help.
You need something like Continuations which allow the comet requests to be resumed when the event they are triggered by is observed. AFAIK, this isn't something that's possible in PHP. I've only seen it in Java - see the Apache Tomcat server.
Edit:
There's an article here about using a load balancer (HAProxy) to allow you to run both an apache server and a comet-enabled server (e.g. jetty, tomcat for Java) on port 80 of the same server.
You could use Nginx and JavaScript to implement a Comet based chat system that is very scalable with little memory or CPU utilization.
I have a very simple example here that can get you started. It covers compiling Nginx with the NHPM module and includes code for simple publisher/subscriber roles in jQuery, PHP, and Bash.
http://blog.jamieisaacs.com/2010/08/27/comet-with-nginx-and-jquery/
PHP
I found this funny little screencasts explaining simple comet. As a side note I really think this is going to kill your server on any real load. When just having a couple of users, I would say to just go for this solution. This solution is really simple to implement(screencasts only takes 5 minutes of your time :)). But as I was telling previously I don't think it is good for a lot of concurrent users(Guess you should benchmark it ;)) because:
It uses file I/O which is much slower then just getting data from memory. Like for example the functions filemtime(),
Second, but I don't think least PHP does not a have a decent thread model. PHP was not designed for this anyway because of the share nothing model. Like the slides says "Shared data is pushed down to the data-store layer" like for example MySQL.
Alternatives
I really think you should try the alternatives if you want to do any comet/long polling. You could use many languages like for example:
Java/JVM: Jetty continuations.
Python: Dustin's slosh.
Erlang: Popular language for comet/etc.
Lua, Ruby, C, Perl just to name a few.
Just performing a simple google search, will show you a lot alternatives also PHP(which I think on any big load will kill your server).
mod_php is not the only way to use PHP. You can use fastcgi. PHP must be compiled with --enable-fastcgi.
PHP as FastCGI: http://www.fastcgi.com/drupal/node/5?q=node/10
You may also try https://github.com/reactphp/react
React is a low-level library for event-driven programming in PHP. At its core is an event loop, on top of which it provides low-level utilities, such as: Streams abstraction, async dns resolver, network client/server, http client/server, interaction with processes. Third-party libraries can use these components to create async network clients/servers and more.
The event loop is based on the reactor pattern (hence the name) and strongly inspired by libraries such as EventMachine (Ruby), Twisted (Python) and Node.js (V8).
The introductory example shows a simple HTTP server listening on port 1337:
<?php
$i = 0;
$app = function ($request, $response) use (&$i) {
$i++;
$text = "This is request number $i.\n";
$headers = array('Content-Type' => 'text/plain');
$response->writeHead(200, $headers);
$response->end($text);
};
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$http = new React\Http\Server($socket);
$http->on('request', $app);
$socket->listen(1337);
$loop->run();
I'm having a similar issue. One option I'm finding interesting is to use an existing Comet server, like cometd-java or cometd-python, as the core message hub. Your PHP code is then just a client to the Comet server -- it can post or read messages from channels, just like other clients.
There's an interesting code snippet linked here: http://morglog.org/?p=22=1 that implements part of this method (although there are bits of debug code spread around, too).
I'm current implementing a scalable PHP Comet server using socket functions. It is called 'phet' ( [ph]p com[et] )
Project page: http://github.com/Tim-Smart/phet
Free free to join in on development. I have currently managed to get most of the server logic done, just need to finish off the client side stuff.
EDIT: Recently added 'Multi-threading' capabilities using the pcntl_fork method :)
You'll have a hard time implementing comet in PHP, just because of it's inherent single-threaded-ness.
Check out Websync On-Demand - the service lets you integrate PHP via server-side publishing, offloading the heavy concurrent connection stuff, and will let you create a real-time chat app in no time.
A new module just came out for the nginx web server that'll allow Comet with any language, including PHP.
http://www.igvita.com/2009/10/21/nginx-comet-low-latency-server-push/
You will have to create your own server in PHP. Using Apache/mod_php or even fastcgi will not scale at all. A few years old, but can get you started:
PHP-Comet-Server:
http://sourceforge.net/projects/comet/
I think this is more an issue that having a lot of apache threads running all the time is a problem. That will existing with any language if it works via apache in the same way as PHP (usually) does.
I have seen it mentioned in various places around the internet that HTML5 websockets do not work well with PHP, that PHP by it's nature is just suitable for use with them. On the other hand, I see multiple tutorials on using PHP with websockets and Ive noticed some PHP websocket implementation such as http://code.google.com/p/phpwebsocket/
So does anyone have any definitive information on using websockets with PHP. Are they usable with PHP, what are the advantages/disadvantages of using them with PHP as opposed to Java or Python, and why have I read numerous people saying they don't work well together?
The problem is that WebSockets are designed for long running threads/processes which each maintain multiple event-driven connections, whereas PHP (and it's Apache cohort) was designed around the short-lived single process procedural paradigm (eg. max_execution_time is commonly set to 30 seconds, and the session is single threaded).
That's not to say that it's impossible to write a WebSockets server implementation in PHP. I'm aware of at least one project exists that has done exactly this (but note, even this example gets run from the command line, not through mod_php). But it is likely that the PHP implementation of WebSockets is incompatible with the setup of the cheap/shared hosting where PHP is most commonly used.
So while it's possible to it in PHP, you end up having to run a separate server process (from Apache) anyway, and if you're on the sort of hosting that allows separate server processes then it's easier to write WebSockets code in something which is designed for event-driven programming.
If you're not planning to serve tens of thousands of concurrent duplex connections then it's likely you'd be better off using a combination of AJAX and SSE with your PHP back-end.
I recently tried phpwebsocket and it doesn't work at this time (php 5.4 and chrome) the code refers to a secondkey in the handshake that doesn't exist in rev. 13 of the websocket protocol I don't have the time to read the RFC to understand what's the matter.
It's sure that this solution is more elegant and reactive than AJAX with long polling but websockets are not stable at this time I think it would be more interesting to wait that the w3c announce it stable.
I have been reading about websockets and also about socket.io.
I understand websockets are not support but enough browsers yet to be realistically used.
socket.io has browser support but appears to need node.js. With my hosting solution I have some space on a shared server, very limited control over the server and have access to php but there is no node.js.
I have read that people can get socket.io to work with PHP.
Question: seeing I have very limited server access, not control over ports, command line etc is it realistic that I could get socket.io working with PHP or should I just stick with the AJAX calls (I'm currently using)?
Note: I've developed a chat solution and it appears it would be much more light weight if it was running with a socket solution.
Also - is there good security with Sockets.io?
thx
Trying to make PHP talk to socket.io I suspect will be a big task.
Certainly it's not a great idea to run websockets via a pre-fork or threaded apache. Any event based server should be OK if you can handle the number of PHP processes. A better approach would be to write an event based server in php. Like this one
i researched a few websocket server implementations. i read, that there is a way to use node.js in production. using apache many years, i also searched for a module to handle websockets within apache, but found only half-hearted stuff. nothing which seemed solid.
i stumbled upon yaws, which is a websocket capable server written in erlang, which has the advantages of beeing the right language for a highly parallel application such as the chat thing, and also because websockets should be well integrated within the main server.
now, i'm developing a php chat server, which uses a framework called ratchet. -> google for 'ratchet php'. they got a good tutorial.
i start it via the commandline, and although for you it should be possible to also start the ratchet server via a browser call, i would not recommend that.
so in your constrained hosting environment, i would stick to ajax.
Is there a real solution for COMET AND PHP combination? Basically, I've come to a point that I need to update a user home page periodically whenever there is new data in the database. As far as I understand, I need to open a persistent connection between my server and my clients browsers to update the contents of their home page as soon as new info. available without dedicating a lot of resources but I had no luck finding anything clear about this issue. I read many articles suggests that PHP is not a good language to implement COMET. My web application is completely programmed in PHP and I don't want to learn another language but if I'm forced to, Would you suggest a good language to start with? Do you think that I can program an interface just to handle this issue?
Thanks in advance.
The times I've heard people say that PHP was not well suited for COMET (like you said yourself) was because of the way webservers and PHP work -- mostly, because there is one process per page, which means if you want 200 users connected to your server, you'll need 200 processes (which can quickly become a problem for a couple of hundred more users).
Maybe a solution to that problem would be to use nginx_http_push_module ?
I've not tried it (yet ?), but it might be just what we need...
I was working on a school project and ran into the exact same problem. Because each PHP process has so much memory overhead, it's impossible to support to many connections per box. It was at this point I decided to switch to using BOSH and XMPP. This is a rather new "wave" of technology but there is already quite a few libraries to help you on your way. I would suggest using Strophe and XMPPHP. Then your clients can connect to a BOSH server (I'm using Openfire) and that can scale to thousands of active connections per server.
You don't have to learn a new language to implement such a feature.
For example, you could use Quercus (Java implementation of PHP) and implement a server Comet application using the JVMs memory management model.
There are solutions you need:
almost COMET solution (uses php and one file written with perl):
http://translate.google.com/translate?js=y&prev=_t&hl=ru&ie=UTF-8&layout=1&eotf=1&u=http://dklab.ru/lib/dklab_multiplexor/&sl=ru&tl=en
exact COMET solution in php (this is what you want, I think):
http://translate.google.com/translate?hl=ru&sl=ru&tl=en&u=http://dklab.ru/lib/dklab_realplexor/
You would first need to understand what is a comet application like. The concept involved in building a comet application are explained at wiki at Comet (programming)
What you need to understand is that you can use any programming language to build a comet application as long as it follows the concepts explained at wiki
1.Ajax with long polling
2.Streaming
You can check some sample code at Simple “Long Polling” example code
Now coming to the problems -
1.You use ajax long polling then the browser(ajax request) would keep polling the server for data. This may eat up memory on the server or slow down the browser after some time.
Few Suggestions
JQuery PeriodicalUpdater (AJAX long polling/server polling)
Handling Long Polling
RobertFischer / JQuery-PeriodicalUpdater
What you need to check to implement this -
a) How often do you expect data to be updated on the server.
b) How much time the server side script would run to check, fetch and process data before sending it to the client side.
2.You can implement streaming by using the following -
How to implement COMET with PHP
Lightstreamer Dojo
Dojo Charting + Lightstreamer Comet Demo
Demo
Ajax Push Engine or The APE Project
What you need to check for this -
a) Will your hosting provider allow you to install these on hosting servers
b) Your RAM and Bandwidth utilization (You will need a dedicated server with package that gives you lots of RAM and Bandwidth)
It depends on what and how your requirements are. You will have to analyze and approach.
If what you are implementing is a small application you can go for Ajax Long polling given the fact that you analyzed and handled the negatives of this approach.
If you have a large application you can go for steaming.
Ajax with long polling is a easy solution, there are plugins in jquery and any other major js framework to help you do this.
Node.js seems like a pretty sweet solution for stuff like this. (Still a little gamey for production but cool all the same). PHP is a horrible environment for stuff like this, you have to change the way the server interacts with requests because you are no longer immediately responding. Python has a handful of servers like Twisted that are great for this because they let you be the server. No matter what language you write it in you've got to alter the typical request/response model. (Glassfish's Grizzly Comet server does this for Java as an example)
You should try Dmitry Koterov's Realplexor, which is a comet server, that provides Javascript and PHP APIs.
Readme.txt in english is provided in the package.