I've been "recruited" so to speak to help work on a web project that is currently written in PHP with an Apache server. We would like to integrate a real-time (or at least something very close to it) chat feature. Scalability is a definite concern, and this type of work is definitely not my typical.
Everything I've read about creating such a chat feature requires the use of "long-polling" so the servers don't get rapidly overloaded and, well, crash. PHP and Apache are not conducive to implementing such a feature, so I've explored some alternatives, like Twisted Python for example.
The website has roughly ~7,000 lines of PHP (i.e., it'd pretty difficult to just straight switch languages for the entire thing), so my question is how can I manage this situation as far as trying to integrate python and setting up a separate server? Or, is this a very bad way to do this? Are there other alternatives that would be better suited? (Sadly many of the PHP Comet, or even AJAX, solutions I've found don't scale in the slightest. Note, the Apache server is not necessarily required; however, any server used must work with PHP and Python etc, short of having separate servers.)
Thanks for the help!
I would use Tornado on the server to write the chat application. Client server communication can then be over websockets. If you use SockJS on the client side you can also support older browsers via long polling. There are plenty of example chat clients written using Tornado. It's very simple to get started and it is wonderfully scalable. A chat server like this can be serving thousands of clients without showing any appreciable CPU activity.
This is an example, possibly a bit over engineered https://github.com/diggidanne/websocket-chat/blob/master/server.py
Related
I have seen it mentioned in various places around the internet that HTML5 websockets do not work well with PHP, that PHP by it's nature is just suitable for use with them. On the other hand, I see multiple tutorials on using PHP with websockets and Ive noticed some PHP websocket implementation such as http://code.google.com/p/phpwebsocket/
So does anyone have any definitive information on using websockets with PHP. Are they usable with PHP, what are the advantages/disadvantages of using them with PHP as opposed to Java or Python, and why have I read numerous people saying they don't work well together?
The problem is that WebSockets are designed for long running threads/processes which each maintain multiple event-driven connections, whereas PHP (and it's Apache cohort) was designed around the short-lived single process procedural paradigm (eg. max_execution_time is commonly set to 30 seconds, and the session is single threaded).
That's not to say that it's impossible to write a WebSockets server implementation in PHP. I'm aware of at least one project exists that has done exactly this (but note, even this example gets run from the command line, not through mod_php). But it is likely that the PHP implementation of WebSockets is incompatible with the setup of the cheap/shared hosting where PHP is most commonly used.
So while it's possible to it in PHP, you end up having to run a separate server process (from Apache) anyway, and if you're on the sort of hosting that allows separate server processes then it's easier to write WebSockets code in something which is designed for event-driven programming.
If you're not planning to serve tens of thousands of concurrent duplex connections then it's likely you'd be better off using a combination of AJAX and SSE with your PHP back-end.
I recently tried phpwebsocket and it doesn't work at this time (php 5.4 and chrome) the code refers to a secondkey in the handshake that doesn't exist in rev. 13 of the websocket protocol I don't have the time to read the RFC to understand what's the matter.
It's sure that this solution is more elegant and reactive than AJAX with long polling but websockets are not stable at this time I think it would be more interesting to wait that the w3c announce it stable.
Only yesterday, I was asking a friend of mine how he would go about emulating direct communication channels between two clients through a web server, for the purpose of creating a chat application, but by using solely PHP/MySQL/JavaScript.
He told me that the best way to do this was by the use of SOCKETS, a term I had only heard of until then. This morning I started looking into it for the purpose of creating my chat application, but I'm quickly starting to believe that it's not as easy as I'd hoped.
So my question is this: if I don't have access to my own server (I have a domain hosted on a shared server that I also use for testing purposes), can I still use sockets to achieve my goal? If so, how exactly? (Please understand that I am completely new to the idea)
If not, what other way is there to accomplish the communication channels?
My only idea so far is to simply send periodic requests (AJAX) to the web server the application would be stored on and request any new messages, if any. But this does not seem very feasible.
Thanks in advance for your help!
I think what your friend is trying to get to is implementing Comet for your chat site.
Assuming he's getting you to use PHP sockets to act as a daemon, I highly doubt a shared hosting provider will let you do it.
You could try hanging the PHP script until there's data available. However, this will quickly consume resources on a CGI-based server since the PHP server can't tell if the client is still connected. I know this from experience.
For these kind of things, I highly recommend you get a dedicated server or VPS and write your backend in something like socket.io which automagically handles all your communication problems on both the client and server side. PHP, MYSQL and servers that fork to serve requests are usually the worst case scenarios for implementing Comet since they incur quite a bit of overhead and aren't scalable.
If you can't afford to run your own Comet server, then polling may be your only option. This will be the most resource intensive and least responsive.
I'd like to create an API for a project I'm working on, allowing developers to create desktop and mobile applications built around its functionality. One thing I've always wanted to learn how to do is create a stateless, push notification system, similar to Twitter's Streaming API.
Basically, I want to be able to notify users of any changes to the data in real time, or as close to it as possible. I know that this might be difficult on mobile devices, which is why mobile applications will probably be built to check for updates periodically, to save battery. However, desktop applications won't have that limitation. I'd like to avoid making the application ask the server if there is new information, and instead let the server tell the application that there is new data.
My programming language is PHP and my server is Apache. If I absolutely had to I could switch to Lighttpd or nginx, but that's an absolute last resort since it would require a lot of changes to all of my existing code.
I've read this article:
http://www.zeitoun.net/articles/comet_and_php/start
And tested it out, but unfortunately all that happens is my browser keeps attempting to load the page and never actually displays the time. I suspect this is because, for whatever reason, I've never been able to get output buffering to work on my server, unless I send 64kb (or more) of data. I heard that I had to disable gzip, which I did, and it still didn't work, so I don't know.
Have a look at some existing technologies to help you do this:
Tornado
Tornado is an open source version of the scalable, non-blocking web server and tools that power FriendFeed. The FriendFeed application is written using a web framework that looks a bit like web.py or Google's webapp, but with additional tools and optimizations to take advantage of the underlying non-blocking infrastructure.
Pusher
Pusher is a hosted API for quickly, easily and securely adding scalable realtime functionality via WebSockets to web and mobile apps.
Both are extremely fast and scalable, and I have setup both relatively easily.
Well you could do this in several ways, you could build in a poller at the client side, or you could use something like NodeJS. (http://nodejs.org/) or web sockets.
Yeah another good piece is
http://socket.io/
and
http://elephant.io/
some tuorials like this might also be useful.
http://www.phpbuilder.com/articles/application-architecture/optimization/creating-real-time-applications-with-php-and-websockets.html
Is there a real solution for COMET AND PHP combination? Basically, I've come to a point that I need to update a user home page periodically whenever there is new data in the database. As far as I understand, I need to open a persistent connection between my server and my clients browsers to update the contents of their home page as soon as new info. available without dedicating a lot of resources but I had no luck finding anything clear about this issue. I read many articles suggests that PHP is not a good language to implement COMET. My web application is completely programmed in PHP and I don't want to learn another language but if I'm forced to, Would you suggest a good language to start with? Do you think that I can program an interface just to handle this issue?
Thanks in advance.
The times I've heard people say that PHP was not well suited for COMET (like you said yourself) was because of the way webservers and PHP work -- mostly, because there is one process per page, which means if you want 200 users connected to your server, you'll need 200 processes (which can quickly become a problem for a couple of hundred more users).
Maybe a solution to that problem would be to use nginx_http_push_module ?
I've not tried it (yet ?), but it might be just what we need...
I was working on a school project and ran into the exact same problem. Because each PHP process has so much memory overhead, it's impossible to support to many connections per box. It was at this point I decided to switch to using BOSH and XMPP. This is a rather new "wave" of technology but there is already quite a few libraries to help you on your way. I would suggest using Strophe and XMPPHP. Then your clients can connect to a BOSH server (I'm using Openfire) and that can scale to thousands of active connections per server.
You don't have to learn a new language to implement such a feature.
For example, you could use Quercus (Java implementation of PHP) and implement a server Comet application using the JVMs memory management model.
There are solutions you need:
almost COMET solution (uses php and one file written with perl):
http://translate.google.com/translate?js=y&prev=_t&hl=ru&ie=UTF-8&layout=1&eotf=1&u=http://dklab.ru/lib/dklab_multiplexor/&sl=ru&tl=en
exact COMET solution in php (this is what you want, I think):
http://translate.google.com/translate?hl=ru&sl=ru&tl=en&u=http://dklab.ru/lib/dklab_realplexor/
You would first need to understand what is a comet application like. The concept involved in building a comet application are explained at wiki at Comet (programming)
What you need to understand is that you can use any programming language to build a comet application as long as it follows the concepts explained at wiki
1.Ajax with long polling
2.Streaming
You can check some sample code at Simple “Long Polling” example code
Now coming to the problems -
1.You use ajax long polling then the browser(ajax request) would keep polling the server for data. This may eat up memory on the server or slow down the browser after some time.
Few Suggestions
JQuery PeriodicalUpdater (AJAX long polling/server polling)
Handling Long Polling
RobertFischer / JQuery-PeriodicalUpdater
What you need to check to implement this -
a) How often do you expect data to be updated on the server.
b) How much time the server side script would run to check, fetch and process data before sending it to the client side.
2.You can implement streaming by using the following -
How to implement COMET with PHP
Lightstreamer Dojo
Dojo Charting + Lightstreamer Comet Demo
Demo
Ajax Push Engine or The APE Project
What you need to check for this -
a) Will your hosting provider allow you to install these on hosting servers
b) Your RAM and Bandwidth utilization (You will need a dedicated server with package that gives you lots of RAM and Bandwidth)
It depends on what and how your requirements are. You will have to analyze and approach.
If what you are implementing is a small application you can go for Ajax Long polling given the fact that you analyzed and handled the negatives of this approach.
If you have a large application you can go for steaming.
Ajax with long polling is a easy solution, there are plugins in jquery and any other major js framework to help you do this.
Node.js seems like a pretty sweet solution for stuff like this. (Still a little gamey for production but cool all the same). PHP is a horrible environment for stuff like this, you have to change the way the server interacts with requests because you are no longer immediately responding. Python has a handful of servers like Twisted that are great for this because they let you be the server. No matter what language you write it in you've got to alter the typical request/response model. (Glassfish's Grizzly Comet server does this for Java as an example)
You should try Dmitry Koterov's Realplexor, which is a comet server, that provides Javascript and PHP APIs.
Readme.txt in english is provided in the package.
I wanted to know if somebody could explain in somewhat simple terms how I could get
started with comet. I am on a shared host environment.
What exactly are my possibillity's?
From what I have read here sofar, is that php is not the best option., because it is
run as a one process per request instead off thread.
On another, they talk about sockets.
I have also read about facebook and that they run there own comet server written in erlang, but they still have a php website. So how exactly does that work then?
I have read alot off stuff now, but it is still a bit vague on what is actually possible
in my situation. Running a php script that is a client off a comet server?? That is not saying how it keeps a longlasting open connection! (just throwing in some comments from other posts)
thanks, Richard
You can use PHP to reproduce the Comet behavior. Like you said PHP isn't the best choice because of a lot of memory management issues that are still widely there.
However for small scale purposes (low-traffic) PHP will work just fine.
There's a lot of information on the web about Comet and how to use it with languages such as Python or Erlang (especially good because of its optimal concurrency behavior, but frankly I don't know much about it).
If you want to try out Python there's a good question/answer on SO:
Python Comet Server
A simple solution is to find a SaaS comet provider, such as the one we host at Frozen Mountain, WebSync On-Demand. Basically, it'll let you use our servers (running WebSync + IIS) to handle the long lived connections, but allow you to publish your messages via PHP, and pre-process your messages in PHP. If you're going to have < 10 simultaneous users, it's free, so for a small site it should do exactly what you need.