I have PHP server, in which I need to update client A about some client B action, at the moment it's done. How it is normally being accomplished ?
My current solution would be:
A leaves an open $.ajax(...).done(function myCallBack(){...}); call for this.
when A is in myCallBack(){...} it issues another $.ajax(...).done(function myCallBack(){...});. This way the communication with server stays open for receiving new info anytime
the PHP would have to continually check for inter-session communication file to transfer data to/from two concurrent sessions
Leaving $.ajax(...).done(function myCallBack(){...}); open (and spawning new ones all the time) is the way to do it ?
For inter-session communication - is there a way to signal events or something like that (instead of continuously monitoring some file [a waste of resources!]) ?
This is how I solved it:
Client A leaves an open ajax call: $.ajax(...).done(function myCallBack(){...});: on the PHP server side (session), A locks on a semaphore using sem_acquire($semaphore_A), and waits.
Client B removes semaphore $semaphore_A using sem_remove($semaphore_A), thus releasing A that returns to client JS callback myCallBack().
Of-course, it's very simplistic, and I use shared-memory (e.g. shm_attach()) to exchange status and data between processes.
For other developers stuck with this technique:
what took me so long to debug it, is the fact the I opened a session (session_start()) and didn't close it (session_ write_ close()) - which locked other processes and prevented any concurrency !
Client A would send data to the server, Client B needs to poll the server to ask for changes. When there are changes Client B would get them on it's next poll.
Related
I am buidling some webapp and have implemented long-polling (and a command queue in my db) so my server can send commands to my cleint asynchronously, etc. The commands are encoded into json and sent over ajax calls for the client to server, and via long-polling for the server to client way.
Everything was working just fine, until I included my "Authentication module" in the ajax.php file. This module wraps the session stuff and calls session_start().
The problem is that, my long polling routine can wait up to 21 seconds before comming back to the client. During this time, the server won't run anything from the same session. It's instead executed right after the long polling ajax call returned.
I understand there's probably a restriction of only 1 thread per session at a time, and that the requests are queued up.
Now here's the question : What is the best way to address this? Is there a setting to allow several threads per sessions (3 would be fine, in my case). Or should I just send tell the client what is his SessionID (i have some sessions table in my db, to track which user is connected to which session(s)). The client could then send it along with any ajax calls so authentication module could be bypassed.
On the later option, iam afraid it open's up a bunch of security problems because of eventual session spoofing. I would need to send a "random string" to each session, to make sure you can't spoof too easily, but even then, it's not perfect...
Thanks for your awnsers :)
Nicolas Gauthier
It's a well known issue/fact that PHP locks session files for the duration of their usage in order to prevent race conditions.
If you take a look at the PHP source code, (ext/session/mod_files.c) you can see that the ps_files_open function locks the session file, and ps_files_close unlocks it.
If you call session_start() right at the beginning of your long-running script, and do not explicitly close the session file, it will be locked until the script terminates, where PHP will release all file locks during script shutdown.
While you are not using the session, you should call session_write_close to flush the session data to disk, and release the lock so that your other "threads" can read the data.
I'm sure you can imagine what would happen if the file was not locked.
T1: Open Session
T2: Open Session
...
T2: Write Data
T1: Write Data
The data written by thread 2 will be completely overwritten by thread 1, and at the same time, any data that thread 1 wanted to write out, was not available to thread 2.
I've wrote a small chat system using jquery, php, and mysql; however, I'm looking for some kind of technology that will only update a if a new record is inserted into a row. I feel like using jquery ajax calls every second to retrieve new records is really overkill and strenuous on my server.
You are looking for a Comet solution: http://en.wikipedia.org/wiki/Comet_%28programming%29
The idea, as pdr noted, is to the javascript continuously open an async request with the server. The server holds it open, but does not send anything until it determines there is something to send. The request will timeout on the javascript side after 10-20 seconds, after which it should re-open the connection.
This uses a 'subscriber' based model, by which the server will send out the chat message or what have you, to all clients which are subscribed, all at once. This saves you many database requests, as the server is the one asking for the requests, not the individual clients.
What you want is long polling. Basically, you make an XHR, and the server and PHP holdes the request open until new data is ready to be sent back.
You need to configure Apache not to timeout in this circumstances, so do some substantial research. Basically, the PHP looks like so...
set_time_limit(0);
while (TRUE) {
$db->query('SELECT `message` FROM `messages` WHERE `new` = TRUE');
if ($db) {
echo json_encode($db->results());
exit;
}
sleep(1);
}
Then, you make an XHR for this page, and it will stay open until new data is ready. Then, on the complete callback, update your page's state and make a new XHR.
This is a lot more efficient than polling for updates continually using XHR.
Make sure you do a lot of research because I believe Apache is going to think things are wrong if a PHP script hasn't stopped after 30 seconds or so. :)
There are a couple routes I know of that you can take.
Long polling. This is where the browser opens a connection to the server and does nothing until the server responds. Once the server responds or times-out (sends an empty response to the browser), a new long-polling request is made.
When going this route, you should use a server that does not rely on using a new thread for each request.
Web sockets. Again, you'll want a server that can handle requests without spawning a new thread every request. With web sockets, a connection is kept open between the client and servier, and unlike Long polling, doesn't time out. However, this isn't well-supported yet.
I highly recommend checking out http://socket.io/
The point of Ajax is that it's asynchronous. Can you not just wait at the server until there's a worthwhile response to send?
With standard HTML/CSS/JS, that's pretty much the only way since the browser can make requests of the server, not vice versa. The AJAX call shouldn't have to be very big at all. A chat system, by definition, is going to require hitting the server a lot.
Do all comet style applications require a loop somewhere in the application on the serverside to detect updates/changes? If no, please could you explain how the logic behind a loopless comet style application would work?
This kind of application will always require a loop, you need to periodically check for new data etc. Of course you can make the "loop" non-blocking by using an even-loop based approach, but in the end there's still a loop somewhere.
Just think about it for a moment, how would you make it work without a loop? I sure can't imagine a way that doesn't utilize a loop somewhere.
Short answer is, no, not all require a loop on the serverside.
Instead you can use long-polling AJAX calls from the browser to request data,
at which the server simply responds with the data and the browser waits until the response is gotten before sending a new request.
The solution could be stream_set_blocking. Use any possible blocking resource to be suspended by OS and wait for appropriate interruption.
Client side:
Ajax call to endpoint script (timeout for ajax e.g. 30 seconds - after 30 seconds initiate another one - after 30 seconds you will get response from server - script execution time reached)
If you will get response during 30 seconds handle response (async) and open new connection (as in comet done - I saw it in cometD client)
Server setup:
Setup apache timeouts (between request and data sent to 30-31 second), this is so apache will allow you to wait so much
set apache to allow lot of child instances (concurrent users * 1.5), but you need to be sure that you have enough memory for this amount of apache instances (+ memory used by php children)
Script one:
execution_time = 28
set shutdown_function in order to send response (timeout, but formatted and understandable for ajax if You need it)
you need to open file, empty one
enable blocking mode using stream_set_blocking for file stream
try read from file and you will get suspended until other process will write to file or timeout be reached.
As soon as script gets content in file written from other process it will get back and will send response. (this will trigger another ajax call and another slept process)
Worst thing is that you need to think how to get multiple reader scripts reading from same bus (file) without disturbing each other.
Also there could be that timeout will be exactly at that time when message will be written into bus.
(hope that this solution is not as bad as my English)
I have done some google search on this topic and couldn't find the answer to my question.
What I want to achieve is the following:
the client make an asynchronous call to a function in the server
the server runs that function in the background (because that function is time consuming), and the client is not hanging in the meantime
the client constantly make a call to the server requesting the status of the background job
Can you please give me some advices on resolving my issue?
Thank you very much! ^-^
You are not specifying what language the asynchronous call is in, but I'm assuming PHP on both ends.
I think the most elegant way would be this:
HTML page loads, defines a random key for the operation (e.g. using rand() or an already available session ID [be careful though that the same user could be starting two operations])
HTML page makes Ajax call to PHP script to start_process.php
start_process.php executes exec /path/to/scriptname.php to start the process; see the User Contributed Notes on exec() on suggestions how to start a process in the background. Which one is the right for you, depends mainly on your OS.
long_process.php frequently writes its status into a status file, named after the random key that your Ajax page generated
HTML page makes frequent calls to show_status.php that reads out the status file, and returns the progress.
Have a google for long running php processes (be warned that there's a lot of bad advice out there on the topic - including the note referred to by Pekka - this will work on Microsoft but will fail in unpredicatable ways on anything else).
You could develop a service which responds to requests over a socket (your client would use fsockopen to connect) - some simple ways of acheiving this would be to use Aleksey Zapparov's Socket server (http://www.phpclasses.org/browse/package/5758.html) which handles requests coming in via a socket however since this runs as a single thread it may not be very appropriate for something which requiers a lot of processing. ALternatively, if you are using a non-Microsoft system then yo could hang your script off [x]inetd however, you'll need to do some clever stuff to prevent it terminating when the client disconnects.
To keep the thing running after your client disconnects then the PHP code must be running from the standalone PHP executable (not via the webserver) Spawn a process in a new process group (see posix_setsid() and pcntl_fork()). To enable the client to come back and check on progress, the easiest way to achieve this is to configure the server to write out its status to somewhere the client can read.
C.
Ajax call run method longRunningMethod() and get back an idendifier (e.g an id)
Server runs the method, and sets key in e.g. sharedmem
Client calls checkTask(id)
server lookup the key in sharedmem and check for ready status
[repeat 3 & 4 until 5 is finished]
longRunningMethod is finished and sets state to finished in sharedmem.
All Ajax calls are per definition asynchronous.
You could (although not a strictly necessary step) use AJAX to instantiate the call, and the script could then create a reference to the status of the background job in shared memory (or even a temporary entry in an SQL table, or even a temp file), in the form of a unique job id.
The script could then kick off your background process and immediately return the job ID to the client.
The client could then call the server repeatedly (via another AJAX interface, for example) to query the status of the job, e.g. "in progress", "complete".
If the background process to be executed is itself written in PHP (e.g. a command line PHP script) then you could pass the job id to it and it could provide meaningful progress updates back to the client (by writing to the same shared memory area, or database table).
If the process to executed it's not itself written in PHP then I suggest wrapping it in a command line PHP script so that it can monitor when the process being executed has finished running (and check the output to see if was successful) and update the status entry for that task appropriately.
Note: Using shared memory for this is best practice, but may not be available if you are using shared hosting, for example. Don't forget you want to have a means to clean up old status entries, so I would store "started_on"/"completed_on" timestamps values for each one, and have it delete entries for stale data (e.g. that have a completed_on timestamp of more than X minutes - and, ideally, that also checks for jobs that started some time ago but were never marked as completed and raises an alert about them).
I've a particularly long operation that is going to get run when a
user presses a button on an interface and I'm wondering what would be the best
way to indicate this back to the client.
The operation is populating a fact table for a number of years worth of data
which will take roughly 20 minutes so I'm not intending the interface to be
synchronous. Even though it is generating large quantities of data server side,
I'd still like everything to remain responsive since the data for the month the
user is currently viewing will be updated fairly quickly which isn't a problem.
I thought about setting a session variable after the operation has completed
and polling for that session variable. Is this a feasible way to do such a
thing? However, I'm particularly concerned
about the user navigating away/closing their browser and then all status
about the long running job is lost.
Would it be better to perhaps insert a record somewhere lodging the processing record when it has started and finished. Then create some other sort of interface so the user (or users) can monitor the jobs that are currently executing/finished/failed?
Has anyone any resources I could look at?
How'd you do it?
The server side portion of code should spawn or communicate with a process that lives outside the web server. Using web page code to run tasks that should be handled by a daemon is just sloppy work.
You can't expect them to hang around for 20 minutes. Even the most cooperative users in the world are bound to go off and do something else, forget, and close the window. Allowing such long connection times screws up any chance of a sensible HTTP timeout and leaves you open to trivial DOS too.
As Spencer suggests, use the first request to start a process which is independent of the http request, pass an id back in the AJAX response, store the id in the session or in a DB against that user, or whatever you want. The user can then do whatever they want and it won't interrupt the task. The id can be used to poll for status. If you save it to a DB, the user can log off, clear their cookies, and when they log back in you will still be able to retrieve the status of the task.
Session are not that realible, I would probably design some sort of tasks list. So I can keep records of tasks per user. With this design I will be able to show "done" tasks, to keep user aware.
Also I will move long operation out of the worker process. This is required because web-servers could be restrated.
And, yes, I will request status every dozens of seconds from server with ajax calls.
You can have JS timer that periodically pings your server to see if any jobs are done. If user goes away and comes back you restart the timer. When job is done you indicate that to the user so they can click on the link and open the report (I would not recommend forcefully load something though it can be done)
From my experience the best way to do this is saving on the server side which reports are running for each users, and their statuses. The client would then poll this status periodically.
Basically, instead of checkStatusOf(int session), have the client ask the server of getRunningJobsFor(int userId) returning all running jobs and statuses.