I have been looking at writing my own server using kqueue. I can do this with really no problems as long as I can control what kind of client will be accessing our system. Realistically, however, I would need to accept from standard web clients, including AJAX. I have been looking for examples of programs that use XMLHTTPRequest to connect to a custom server written in C. I have found nothing.
Can you help me?
Bruce
Ajax just means "Making an HTTP request using JavaScript". It isn't a client.
As far as the server is concerned, there is no difference between an Ajax request and any other HTTP request.
(Some libraries add an experimental HTTP header to state that the request was trigged by Ajax, but when you care about that on the server, it is almost always at the application level rather than the server level (i.e. your server side script not your HTTPD)).
Related
So I have a PHP script that listens for incoming webhooks. But: where do I put it, so that it permanently runs in the background and listens for those webhooks? I have done some PHP coding before, but never websites.
I admit this is a nooby question, but I seriously don't know and would appreciate any help.
A "webhook" is just a fancy term for some code which makes a HTTP request to a particular URL (usually in response to itself receiving an incoming HTTP request from somewhere, or at the time when some other internal event occurs within the application where the webhook is defined). Basically it's a conceptual technique which allows a degree of asynchronous communication between web applications in a controlled way.
So a PHP program which listens for webhook requests is really no different to any other PHP program which is accessible via a webserver - both are simply executed in a response to a HTTP request being received by the webserver. Whether that request comes from a browser, a "webhook" script, or any other kind of computer program is largely irrelevant.
Your webserver is already running in the background and doing the listening. Therefore you can just deploy your script in the normal way, to somewhere your webserver can make use of it to respond to requests to the URL you've provided to the webhook.
Generally the endpoints for webhooks are just regular PHP pages. The script doesn't need to be running and "listen" for webhook calls - the webhook call will send a request to a particular URL. The webserver will then execute the PHP script at that URL.
The location of the file depends on your webserver. On Debian with Apache, the default web root is /var/www/htdocs/. So if your site is www.jerm.com, putting your file at /var/www/htdocs/webhook.php would result in a URL of www.jerm.com/webhook.php
I'm looking for the 'way to go' (i.e. the most efficient, most used, general accepted way) when it comes to the reloading of data from a web server to a front end. In the end application, I will have several output fields where data has to be written to, for example like this:
The data streams will be different from each other in the end application. The lines will have to be reloaded with fresh, up to date data from the server.
I have been thinking of using Ajax requests to update like every second, but there has to be an other way to do this. Ajax requests will cause a lot data traffic. Also, when using the Facebook chat, you don't have to wait every second, chats are received almost instantly. Yet I don't see any Ajax polling requests being made when I use the developer tools of Mozilla Firefox. This made me think if there would be a different way to do this.
I've looked into Node.js, but it appears that isn't possible with my host.
I have heard people talking about Ajax Push, is that what I should use? If so, can you give me a basic usage example?
If not, what would then be the way to go when having multiple data streams that have to be reloaded within a second?
Requirements are speed and low data traffic. It therefore wouldn't be an option to continuously poll the server, I think, because that would create an enormous overhead.
I don't think it's of any importance, but I'm using PHP5.3 in the back end and JavaScript with jQuery 1.9.1 in the front end.
This question has been asked a number of times, but in a slightly different ways. Here are a few references that are worth a read:
What are Long-Polling, Websockets, Server-Sent Events (SSE) and Comet?
Using comet with PHP?
Apache with Comet Support
Server Scalability - HTML 5 websockets vs Comet
How to implement event listening in PHP
In summary: if you are looking at building your solution using PHP on Apache then holding open persistent connections (HTTP long-polling or streaming) is going to use up resources very quickly (is highly inefficient). So, you would be better using a hosted solution (*disclaimer - I work for a hosted solution).
HTTP-Long polling and HTTP Streaming are solutions which have been superseded by Server-Sent Events and WebSockets. So, where possible (where the web client provides support) you should use one of these solutions before falling back to an HTTP-based solution. A good realtime web technology will automatically handle this for you.
Since your diagram shows you are subscribing to multiple data streams you should also consider a Publish/Subscribe solution that naturally fits with this. Again, a good realtime web tech solution will provide you with this.
Also see the realtime web technology guide.
I think what you are looking for is generally called Comet. The was this technique is often made to work is as follows:
The client (web browser) makes a request to the server for new data. This is not reloading the page, but rather is done in JavaScript
The server responds to the request when it has some data for the client. Again, this doesn't impact the UI since it isn't the page itself that's getting reloaded: the loaindg of data is done "in background" so to speak, in JavaScript code.
On the serve side, the request waits for new data, and returns the new data when available, or returns nothing if a timeout interval (defined on the server) is reached. This timeout is usually set to be lower than the browser HTTP timeout. The reason for this is so that the server can know whether a particular client got a particular piece of data. If the request is allowed to time out on the client side, the original request might be responded to by the server after the client has timed out, and the client will not get the data, even though the server thinks that it did.
The data is indeed usually transferred as JSON, but you can choose whatever encoding you'd like. See here for one example of how to do this. Goosh is another example of this technique, and so is Interactive Python Shell. The code for all is available.
On the PHP side you will want to create a page that will respond to these "background" JavaScript Comet requests. It could be the same page as the one that user loads, but let's say it is different, for ease of explanation. So the user loads index.php and the JavaScript Comet code calls getNewData.php to retrieve new data.
In your getNewData.php you will want to wait for your event and return the data then. You don't want to use polling for this, but there are PHP libraries that allow one to use various interprocess communication strategies to wait on events, see this question for instance. The high-level pseudocode for your getNewData.php would look as follows:
parse JSON request
Enter an efficient wait state (with timeout), waiting for your "new data is available" event
Did previous step time out?
Yes: send response indicating no data
No: send response with new data
I am curently trying to make a chat application aimed at 1000-1500 users. What currently happens is once my webpage loads I make an ajax request every second to check if there is anything new in the database. I want to know if this is the standard practise or if there is a more efficient way to be notified by the server somehow when an insertion occurs.
Use WebSockets. Or at the very least AJAX polling. Firing a request every second from 1500 clients will most likely kill your server.
Look at http://socket.io/ if you are open to introduce something new to your stack. But there are PHP websocket solutions out there if you are limited to PHP.
Your approach is a standard method called Polling. Based on the number of clients this should be perfectly fine for a server with up-to-date hard-ware (do HEAD requests via AJAX that indicate the status via HTTP status code).
The other alternative - as pointed out by Jan - is called Pushing.
Pros: Involves a lot less requests to the server.
Cons: Requires technology that may or may not be provided by your client's browser.
In case you'll opt for the second approach, take a look into Server-Sent Events (W3C draft).
This specification defines an API for opening an HTTP connection for receiving push notifications from a server in the form of DOM events. The API is designed such that it can be extended to work with other push notification schemes such as Push SMS.
I have tried reading some articles, but I am not very clear on the concepts yet.
Would someone like to take a shot at explaining to me what these technologies are:
Long Polling
Server-Sent Events
Websockets
Comet
One thing that I came across every time was, the server keeps a connection open and pushes data to the client. How is the connection kept open, and how does the client get the pushed data? (How does the client use the data, maybe some code might help?)
Now, which one of them should I use for a real-time app. I have been hearing a lot about websockets (with socket.io [a node.js library]) but why not PHP?
In the examples below the client is the browser and the server is the webserver hosting the website.
Before you can understand these technologies, you have to understand classic HTTP web traffic first.
Regular HTTP:
A client requests a webpage from a server.
The server calculates the response
The server sends the response to the client.
Ajax Polling:
A client requests a webpage from a server using regular HTTP (see HTTP above).
The client receives the requested webpage and executes the JavaScript on the page which requests a file from the server at regular intervals (e.g. 0.5 seconds).
The server calculates each response and sends it back, just like normal HTTP traffic.
Ajax Long-Polling:
A client requests a webpage from a server using regular HTTP (see HTTP above).
The client receives the requested webpage and executes the JavaScript on the page which requests a file from the server.
The server does not immediately respond with the requested information but waits until there's new information available.
When there's new information available, the server responds with the new information.
The client receives the new information and immediately sends another request to the server, re-starting the process.
HTML5 Server Sent Events (SSE) / EventSource:
A client requests a webpage from a server using regular HTTP (see HTTP above).
The client receives the requested webpage and executes the JavaScript on the page which opens a connection to the server.
The server sends an event to the client when there's new information available.
Real-time traffic from server to client, mostly that's what you'll need
You'll want to use a server that has an event loop
Connections with servers from other domains are only possible with correct CORS settings
If you want to read more, I found these very useful: (article), (article), (article), (tutorial).
HTML5 Websockets:
A client requests a webpage from a server using regular http (see HTTP above).
The client receives the requested webpage and executes the JavaScript on the page which opens a connection with the server.
The server and the client can now send each other messages when new data (on either side) is available.
Real-time traffic from the server to the client and from the client to the server
You'll want to use a server that has an event loop
With WebSockets it is possible to connect with a server from another domain.
It is also possible to use a third party hosted websocket server, for example Pusher or others. This way you'll only have to implement the client side, which is very easy!
If you want to read more, I found these very useful: (article), (article) (tutorial).
Comet:
Comet is a collection of techniques prior to HTML5 which use streaming and long-polling to achieve real time applications. Read more on wikipedia or this article.
Now, which one of them should I use for a realtime app (that I need to
code). I have been hearing a lot about websockets (with socket.io [a
node.js library]) but why not PHP ?
You can use PHP with WebSockets, check out Ratchet.
Tieme put a lot of effort into his excellent answer, but I think the core of the OP's question is how these technologies relate to PHP rather than how each technology works.
PHP is the most used language in web development besides the obvious client side HTML, CSS, and Javascript. Yet PHP has 2 major issues when it comes to real-time applications:
PHP started as a very basic CGI. PHP has progressed very far since its early stage, but it happened in small steps. PHP already had many millions of users by the time it became the embed-able and flexible C library that it is today, most of whom were dependent on its earlier model of execution, so it hasn't yet made a solid attempt to escape the CGI model internally. Even the command line interface invokes the PHP library (libphp5.so on Linux, php5ts.dll on Windows, etc) as if it still a CGI processing a GET/POST request. It still executes code as if it just has to build a "page" and then end its life cycle. As a result, it has very little support for multi-thread or event-driven programming (within PHP userspace), making it currently unpractical for real-time, multi-user applications.
Note that PHP does have extensions to provide event loops (such as libevent) and threads (such as pthreads) in PHP userspace, but very, very, few of the applications use these.
PHP still has significant issues with garbage collection. Although these issues have been consistently improving (likely its greatest step to end the life cycle as described above), even the best attempts at creating long-running PHP applications require being restarted on a regular basis. This also makes it unpractical for real-time applications.
PHP 7 will be a great step to fix these issues as well, and seems very promising as a platform for real-time applications.
Polling
Basically, polling is a technique of requesting the information from the server after regular intervals. This connection happens by following HTTP protocol. There are two types of polling:
Short Polling
Long Polling
Short Polling
In short polling, the client requests information from the server. The server processes the request. If data is available for the request, server responds to the request with the required information. However, if the server has no data available for the client, server returns an empty response. In both the situation, the connection will be closed after returning the response. Clients keep issuing new requests even after server sends the empty responses. This mechanism increases the network cost on the server.
Long polling
In long polling, the clients can request information from the server with the expectation that the server may not respond immediately. When the server receives the request, if it has no fresh data for the client, rather than returning an empty response, the server keeps the request open and waits for data to arrive. When the server receives new data, it delivers the response to the client right away, completing the open request. The client can then send another request for new updates after getting the answer from the server. Long polling reduces costs by reducing the number of empty responses.
WebSocket
WebSocket is a protocol that provides two-way(bi-directional) communication channels over a single TCP connection. Websocket facilitates a persistent connection between a client and a server, allowing both parties to begin transferring data at any moment. The WebSocket handshake is the procedure through which the client creates a WebSocket connection. If the operation is successful, the server and client can send and receive data at any time. Mostly used in real-time web applications such as WhatsApp, Uber.
Server-sent event (SSE)
Unlike WebSockets, we can not issue requests from a client to a server using SSE since it's a one-way connection. When we require "near real-time" transmission from the server to the client, or if the server generates data in a loop, SSE is the ideal choice.
Comet
Comet is a web application design paradigm that describes a continuous, two-way interaction between a server and a web browser using native HTTP methods. Comet is an umbrella term. Ajax Push, HTTP Streaming, and HTTP Server Push are some of the HTTP mechanisms that may be used to provide this event-driven interaction.
You can easily use Node.JS in your web app only for real-time communication. Node.JS is really powerful when it's about WebSockets. Therefore "PHP Notifications via Node.js" would be a great concept.
See this example:
Creating a Real-Time Chat App with PHP and Node.js
I'm developing a simple chat web application based on the MSN protocol. The server communicates with the MSN server through a file resource returned from fsockopen (). The client accesses the server via XMLHttpRequest. The server initially logs in, and prints out the contact list (formatted in an HTML table) which the client receives through the responseText () of the XMLHttpRequest object.
Here's the problem. The file resource that is responsible for communication with the MSN server must be kept alive in order for all chat related functions to work (creating conversations, keeping track of offline/online state changes, etc). However in order for the XMLHttpRequest to complete, the PHP script must finish execution. Which means the client will get no response from the XMLHttpRequest while the chat session is in progress.
Whats worse is a file resource cannot be serialized, meaning I cannot simply store the chat session in a $_SESSION [] placeholder.
So, my question is, is there any possible way for me to 'transfer' a file resource from one file to another?
In most languages its not possible to pass file handles between applications - AFAIK most operating systems don't allow it either.
The solution is to keep the server process running as daemon - which means it needs to run outside of the webserver.
See
http://symcbean.blogspot.com/2010/02/php-and-long-running-processes.html
and
http://www.phpclasses.org/browse/package/5758.html
C.
A possible solution would be to have a PHP script on the server-side that just doesn't end ; this way, the resource corresponding to the fsockopen call would never be deleted, and the connection wouldn't be closed.
About this, you might want to search for the term "comet" ; the basic idea is to have a script that runs forever on the server-side, that sends updates to the client whenever it's necessary.
Instead of having the browser send an Ajax request every X seconds, you'd keep an openened connection between the client and the server -- just note that, unfortunatly, PHP is often said not to be the best tool for that job...
On stackoverflow : [php] comet
The resource can't survive the end of the request unless you create PHP extension that does it (like persistent MySQL connections do with mysql_pconnect() for example). However, you could use Comet technology and for example Bayeux protocol supported by Dojo toolkit among others, to talk to the server. That would require either standalone server or long-running request, in latter case ensure that PHP and webserver time limits would not kill that request for running too long.
Thanks everyone for the suggestions. Before I started this project I had considered using comet technology, but decided against (PHP/Apache don't seem to implement well). I've come up with a hacked together solution, not the most elegant but workable.
One PHP script is responsible for the MSN server communication, it will run as long as the user is active. It writes data to a file (email_out), as well as reads data from a file (email_in). Whenever the client sends a AJAX request a separate PHP script will write any POST data to the file (email_in) and will return any data from (email_out). Both scripts will not read/write data until they finally have access to the file (as there will be fighting for the file resource).
I don't know, suggestions? This is certainty not the most efficient means of doing things but it's really the only PHP/apache solution I could think of.