How can I create Wget/curl request to comet server
I have a php based server and embeded linux machine I have to create listener from client side but not based on browser and it must not use sockets.
I think you are referring to that one facet of Comet where a server-client connection remains open on an unfinished http request for a long(ish) period of time. I honestly don't think that it would be wise to do that with simple scripting of wget or curl, however it's completely possible to use libcURL (the library that cURL is built around) for such a purpose. Daniel Stenberg, author of cURL and libcURL himself said so:
AFAIK, COMET is just HTTP done in a slightly funny way but still plain HTTP
and libcurl should be able to do it fine. Even if COMET also is a what
wikipedia calls an "umbrella term" which seems to include a lot of different
approaches.
Just one thing (re the socket remark): do realize that "below" whatever solution you'll choose there will be a socket transporting the data, because that's how http gets over the wire.
Related
I was thinking of implementing real time chat using a PHP backend, but I ran across this comment on a site discussing comet:
My understanding is that PHP is a
terrible language for Comet, because
Comet requires you to keep a
persistent connection open to each
browser client. Using mod_php this
means tying up an Apache child
full-time for each client which
doesn’t scale at all. The people I
know doing Comet stuff are mostly
using Twisted Python which is designed
to handle hundreds or thousands of
simultaneous connections.
Is this true? Or is it something that can be configured around?
Agreeing/expanding what has already been said, I don't think FastCGI will solve the problem.
Apache
Each request into Apache will use one worker thread until the request completes, which may be a long time for COMET requests.
This article on Ajaxian mentions using COMET on Apache, and that it is difficult. The problem isn't specific to PHP, and applies to any back-end CGI module you may want to use on Apache.
The suggested solution was to use the 'event' MPM module which changes the way requests are dispatched to worker threads.
This MPM tries to fix
the 'keep alive problem' in HTTP.
After a client completes the first
request, the client can keep the
connection open, and send further
requests using the same socket. This
can save signifigant overhead in
creating TCP connections. However,
Apache traditionally keeps an entire
child process/thread waiting for data
from the client, which brings its own
disadvantages. To solve this problem,
this MPM uses a dedicated thread to
handle both the Listening sockets, and
all sockets that are in a Keep Alive
state.
Unfortunately, that doesn't work either, because it will only 'snooze' after a request is complete, waiting for a new request from the client.
PHP
Now, considering the other side of the problem, even if you resolve the issue with holding up one thread per comet request, you will still need one PHP thread per request - this is why FastCGI won't help.
You need something like Continuations which allow the comet requests to be resumed when the event they are triggered by is observed. AFAIK, this isn't something that's possible in PHP. I've only seen it in Java - see the Apache Tomcat server.
Edit:
There's an article here about using a load balancer (HAProxy) to allow you to run both an apache server and a comet-enabled server (e.g. jetty, tomcat for Java) on port 80 of the same server.
You could use Nginx and JavaScript to implement a Comet based chat system that is very scalable with little memory or CPU utilization.
I have a very simple example here that can get you started. It covers compiling Nginx with the NHPM module and includes code for simple publisher/subscriber roles in jQuery, PHP, and Bash.
http://blog.jamieisaacs.com/2010/08/27/comet-with-nginx-and-jquery/
PHP
I found this funny little screencasts explaining simple comet. As a side note I really think this is going to kill your server on any real load. When just having a couple of users, I would say to just go for this solution. This solution is really simple to implement(screencasts only takes 5 minutes of your time :)). But as I was telling previously I don't think it is good for a lot of concurrent users(Guess you should benchmark it ;)) because:
It uses file I/O which is much slower then just getting data from memory. Like for example the functions filemtime(),
Second, but I don't think least PHP does not a have a decent thread model. PHP was not designed for this anyway because of the share nothing model. Like the slides says "Shared data is pushed down to the data-store layer" like for example MySQL.
Alternatives
I really think you should try the alternatives if you want to do any comet/long polling. You could use many languages like for example:
Java/JVM: Jetty continuations.
Python: Dustin's slosh.
Erlang: Popular language for comet/etc.
Lua, Ruby, C, Perl just to name a few.
Just performing a simple google search, will show you a lot alternatives also PHP(which I think on any big load will kill your server).
mod_php is not the only way to use PHP. You can use fastcgi. PHP must be compiled with --enable-fastcgi.
PHP as FastCGI: http://www.fastcgi.com/drupal/node/5?q=node/10
You may also try https://github.com/reactphp/react
React is a low-level library for event-driven programming in PHP. At its core is an event loop, on top of which it provides low-level utilities, such as: Streams abstraction, async dns resolver, network client/server, http client/server, interaction with processes. Third-party libraries can use these components to create async network clients/servers and more.
The event loop is based on the reactor pattern (hence the name) and strongly inspired by libraries such as EventMachine (Ruby), Twisted (Python) and Node.js (V8).
The introductory example shows a simple HTTP server listening on port 1337:
<?php
$i = 0;
$app = function ($request, $response) use (&$i) {
$i++;
$text = "This is request number $i.\n";
$headers = array('Content-Type' => 'text/plain');
$response->writeHead(200, $headers);
$response->end($text);
};
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$http = new React\Http\Server($socket);
$http->on('request', $app);
$socket->listen(1337);
$loop->run();
I'm having a similar issue. One option I'm finding interesting is to use an existing Comet server, like cometd-java or cometd-python, as the core message hub. Your PHP code is then just a client to the Comet server -- it can post or read messages from channels, just like other clients.
There's an interesting code snippet linked here: http://morglog.org/?p=22=1 that implements part of this method (although there are bits of debug code spread around, too).
I'm current implementing a scalable PHP Comet server using socket functions. It is called 'phet' ( [ph]p com[et] )
Project page: http://github.com/Tim-Smart/phet
Free free to join in on development. I have currently managed to get most of the server logic done, just need to finish off the client side stuff.
EDIT: Recently added 'Multi-threading' capabilities using the pcntl_fork method :)
You'll have a hard time implementing comet in PHP, just because of it's inherent single-threaded-ness.
Check out Websync On-Demand - the service lets you integrate PHP via server-side publishing, offloading the heavy concurrent connection stuff, and will let you create a real-time chat app in no time.
A new module just came out for the nginx web server that'll allow Comet with any language, including PHP.
http://www.igvita.com/2009/10/21/nginx-comet-low-latency-server-push/
You will have to create your own server in PHP. Using Apache/mod_php or even fastcgi will not scale at all. A few years old, but can get you started:
PHP-Comet-Server:
http://sourceforge.net/projects/comet/
I think this is more an issue that having a lot of apache threads running all the time is a problem. That will existing with any language if it works via apache in the same way as PHP (usually) does.
I want to create a facebook-like notification system (the one who appears on the bottom-left side of the screen when someone comment on your post, for example).
The thing here is that the server needs to send you a notification when someone comments in the site in this exactly moment. I think this is called PUSH-System (sorry for my bad english).
I tried with node.JS but my dedicated server can't install it. Only if I buy a very expensive VPS plan.
So, is there a way using jQuery or something like that to make this Push notification system?
Thanks !!
, rodrigo.-
If you want a low latency, efficient solution you should use WebSockets. However you need to have fallbacks in place such as long-polling / short polling available if the browser doesn't support WebSockets.
The WebSocket Protocol provides a full-duplex (two way) connection between the server and client. Traditional HTTP is half-duplex (one way). This link will give you an overview of the benefits of using WebSockets vs HTTP: http://www.websocket.org/quantum.html.
You need to be aware that most modern browsers support WebSockets, but use different protocols. See here: What browsers support HTML5 WebSocket API?.
In addition to the references I posted in the comments above, implementing a Long Polling technique is a common solution to eliminate large amounts of normal polling. Here is what Wikipedia says about it:
Long polling is a variation of the traditional polling technique and allows emulation of an information push from a server to a client. With long polling, the client requests information from the server in a similar way to a normal poll. However, if the server does not have any information available for the client, instead of sending an empty response, the server holds the request and waits for some information to be available. Once the information becomes available (or after a suitable timeout), a complete response is sent to the client. The client will normally then immediately re-request information from the server, so that the server will almost always have an available waiting request that it can use to deliver data in response to an event. In a web/AJAX context, long polling is also known as Comet programming.
I need to implement real-time page data update with php and jquery.
(I found www.ape-project.org/ but it seems site is down)
Is any other solutions?
Very TNX!
You might want to check out Comet:
Comet is a web application model in
which a long-held HTTP request allows
a web server to push data to a
browser, without the browser
explicitly requesting it.[1][2] Comet
is an umbrella term, encompassing
multiple techniques for achieving this
interaction. All these methods rely on
features included by default in
browsers, such as JavaScript, rather
than on non-default plugins. The Comet
approach differs from the original
model of the web, in which a browser
requests a complete web page at a
time.
http://en.wikipedia.org/wiki/Comet_%28programming%29
If you want to do streaming (sending multiple messages over a single long lived, low latency connection), you probably need a comet server. Check out http://cometdaily.com/maturity.html for details on a variety of server implementations (I am the maintainer of one of them - Meteor).
If you are happy to reconnect after each message is received, you can do without complicated servers and transports and just use long polling - where you make an ajax request and the server simply sleeps until it has something to send back. But you will end up with LOTS of connections hanging off your web server, so if you're using a conventional web server like Apache, make sure it's configured to handle that. By default Apache doesn't like having more than a few hundred concurrent connections.
There is lots of solutions to do this...
Depending on what is your data, how your data are organized and stored (mysql ?).
Your question is too open to have a real answer.
Is it possible to download a single file with multiple parallel connections through php?
As far as I know not easily, no.
You would have to build a PHP implementation of whatever protocols or additions there are to the HTTP protocol to allow the download of files through multiple connections.
While that may not be impossible, I've never heard of such an implementation and even if one exists, it stands to reason that it would be a horrible drain on resources on server side to do this in PHP.
I recommend you look for solutions on the server side for this (i.e. Apache modules and settings for example).
Not through PHP alone. You may be able to do this using some client-side components (maybe even javascript) but I suspect that it would be a lot of work. Many companies that distribute software, instead of having the user download installers via HTTP, deliver a small executable with user interface that then downloads the file with all the optimizations possible. Maybe you could go this way?
You can use the PHP http extension. It has a "range" option for requests, and can wait for incoming request data in the background (open multiple socket connections, non-blocking functions calls). I've even seen callbacks mentioned somewhere there.
And you want to look into HttpRequestPool. Don't ask me for examples, though. If it's that important for your case, you have to write the processing logic yourself.
Or just googled:
Parallel HTTP requests in PHP using PECL HTTP classes [Answer: HttpRequestPool class]
I was thinking of implementing real time chat using a PHP backend, but I ran across this comment on a site discussing comet:
My understanding is that PHP is a
terrible language for Comet, because
Comet requires you to keep a
persistent connection open to each
browser client. Using mod_php this
means tying up an Apache child
full-time for each client which
doesn’t scale at all. The people I
know doing Comet stuff are mostly
using Twisted Python which is designed
to handle hundreds or thousands of
simultaneous connections.
Is this true? Or is it something that can be configured around?
Agreeing/expanding what has already been said, I don't think FastCGI will solve the problem.
Apache
Each request into Apache will use one worker thread until the request completes, which may be a long time for COMET requests.
This article on Ajaxian mentions using COMET on Apache, and that it is difficult. The problem isn't specific to PHP, and applies to any back-end CGI module you may want to use on Apache.
The suggested solution was to use the 'event' MPM module which changes the way requests are dispatched to worker threads.
This MPM tries to fix
the 'keep alive problem' in HTTP.
After a client completes the first
request, the client can keep the
connection open, and send further
requests using the same socket. This
can save signifigant overhead in
creating TCP connections. However,
Apache traditionally keeps an entire
child process/thread waiting for data
from the client, which brings its own
disadvantages. To solve this problem,
this MPM uses a dedicated thread to
handle both the Listening sockets, and
all sockets that are in a Keep Alive
state.
Unfortunately, that doesn't work either, because it will only 'snooze' after a request is complete, waiting for a new request from the client.
PHP
Now, considering the other side of the problem, even if you resolve the issue with holding up one thread per comet request, you will still need one PHP thread per request - this is why FastCGI won't help.
You need something like Continuations which allow the comet requests to be resumed when the event they are triggered by is observed. AFAIK, this isn't something that's possible in PHP. I've only seen it in Java - see the Apache Tomcat server.
Edit:
There's an article here about using a load balancer (HAProxy) to allow you to run both an apache server and a comet-enabled server (e.g. jetty, tomcat for Java) on port 80 of the same server.
You could use Nginx and JavaScript to implement a Comet based chat system that is very scalable with little memory or CPU utilization.
I have a very simple example here that can get you started. It covers compiling Nginx with the NHPM module and includes code for simple publisher/subscriber roles in jQuery, PHP, and Bash.
http://blog.jamieisaacs.com/2010/08/27/comet-with-nginx-and-jquery/
PHP
I found this funny little screencasts explaining simple comet. As a side note I really think this is going to kill your server on any real load. When just having a couple of users, I would say to just go for this solution. This solution is really simple to implement(screencasts only takes 5 minutes of your time :)). But as I was telling previously I don't think it is good for a lot of concurrent users(Guess you should benchmark it ;)) because:
It uses file I/O which is much slower then just getting data from memory. Like for example the functions filemtime(),
Second, but I don't think least PHP does not a have a decent thread model. PHP was not designed for this anyway because of the share nothing model. Like the slides says "Shared data is pushed down to the data-store layer" like for example MySQL.
Alternatives
I really think you should try the alternatives if you want to do any comet/long polling. You could use many languages like for example:
Java/JVM: Jetty continuations.
Python: Dustin's slosh.
Erlang: Popular language for comet/etc.
Lua, Ruby, C, Perl just to name a few.
Just performing a simple google search, will show you a lot alternatives also PHP(which I think on any big load will kill your server).
mod_php is not the only way to use PHP. You can use fastcgi. PHP must be compiled with --enable-fastcgi.
PHP as FastCGI: http://www.fastcgi.com/drupal/node/5?q=node/10
You may also try https://github.com/reactphp/react
React is a low-level library for event-driven programming in PHP. At its core is an event loop, on top of which it provides low-level utilities, such as: Streams abstraction, async dns resolver, network client/server, http client/server, interaction with processes. Third-party libraries can use these components to create async network clients/servers and more.
The event loop is based on the reactor pattern (hence the name) and strongly inspired by libraries such as EventMachine (Ruby), Twisted (Python) and Node.js (V8).
The introductory example shows a simple HTTP server listening on port 1337:
<?php
$i = 0;
$app = function ($request, $response) use (&$i) {
$i++;
$text = "This is request number $i.\n";
$headers = array('Content-Type' => 'text/plain');
$response->writeHead(200, $headers);
$response->end($text);
};
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$http = new React\Http\Server($socket);
$http->on('request', $app);
$socket->listen(1337);
$loop->run();
I'm having a similar issue. One option I'm finding interesting is to use an existing Comet server, like cometd-java or cometd-python, as the core message hub. Your PHP code is then just a client to the Comet server -- it can post or read messages from channels, just like other clients.
There's an interesting code snippet linked here: http://morglog.org/?p=22=1 that implements part of this method (although there are bits of debug code spread around, too).
I'm current implementing a scalable PHP Comet server using socket functions. It is called 'phet' ( [ph]p com[et] )
Project page: http://github.com/Tim-Smart/phet
Free free to join in on development. I have currently managed to get most of the server logic done, just need to finish off the client side stuff.
EDIT: Recently added 'Multi-threading' capabilities using the pcntl_fork method :)
You'll have a hard time implementing comet in PHP, just because of it's inherent single-threaded-ness.
Check out Websync On-Demand - the service lets you integrate PHP via server-side publishing, offloading the heavy concurrent connection stuff, and will let you create a real-time chat app in no time.
A new module just came out for the nginx web server that'll allow Comet with any language, including PHP.
http://www.igvita.com/2009/10/21/nginx-comet-low-latency-server-push/
You will have to create your own server in PHP. Using Apache/mod_php or even fastcgi will not scale at all. A few years old, but can get you started:
PHP-Comet-Server:
http://sourceforge.net/projects/comet/
I think this is more an issue that having a lot of apache threads running all the time is a problem. That will existing with any language if it works via apache in the same way as PHP (usually) does.