I've got a name lookup box that operates by your typical ajax requests. Here's the general flow of the Javascript that fires every time a letter is pressed:
If ajax request already open, then abort it.
If timeout already created, destroy it.
Set new timeout to run the following in half a second:
Send string to 'nameLookup.php' via ajax
Wait for response
Display results
The issue is that nameLookup.php is very resource heavy. In some cases up to 10,000 names are being pulled from an SQL database, decrypted, and compared against the string. Under normal circumstances requests can take anywhere from 5 to 60 seconds to return.
I completely understand that when you abort a request on the client side the server is still working on things and sends back the result. That it's just the client side that knows to ignore the response. But the server is getting so hung up on working on all of these requests.
So if you do:
Request 1
Abort Request 1
Request 2
Abort Request 2
Request 3
Wait for response from Request 3
The server is either not even working on Request 3 until it's finished with 1 and 2... or it's just so hung up on working on Request 1 and 2 that Request 3 is taking an extra long amount of time.
I need to know how to tell the server to stop working on Request 1 and 2 so I can free up resources for it to work on Request 3.
I'm using Javascript & jQuery on the client side. PHP/Apache and SQL on the server side.
Store a boolean value in the DB in a table, or in the session.
Have your resource intensive script check periodically that value to see if it should continue or not. If the DB says to stop, then your script cancels itself (by calling return; in the current function for example).
When you want to cancel, instead of calling abort();, make an AJAX request to set that value to false.
Next time the resource checks that value it will see that it has to stop.
Potential limitations:
1. Your script does not have a way of checking periodically the DB.
2. Based on how often the script checks the DB, it might take a few seconds to effectively kill the script.
I think there is something missing from this question. What are the triggers for doing the requests? You might be trying to solve the wrong problem.
Let me elaborate on that. If you lookup box is actually doing autocompletion of some kind and is doing a new search everytime the user presses a key, then you are going to have the issue you describe.
The solution in that case is not killing all the process. The solution lies in not starting them. So, you might make some decisions like not trying to search if there is only one character to search with - lets say we go for three. We then might say we want to wait until we can be reasonable sure the user has finished typing before sending off the request. Lets say we wait 1 second.
Now, someone looking for all the Paul's in you list of names will send off one search when they type 'pau' and pause for 1 second, instead of three searches for 'p' then 'pa' then 'pau'... so no need to kill anything.
I've come up with an awesome solution that I've tested and it's working beautifully. It's just a few lines of PHP code to put in whatever files are being resource intensive.
This solution utilizes the Process Identifier (PID) from the server. We can use two PHP function: posix_getpid() to get the current PID and posix_kill() to kill another PID. This also assumes that you already have called session_start() somewhere else.
Here's the code:
//if any existing PIDs to kill, go through each
if ($_SESSION['pid']) foreach ($_SESSION['pid'] as $i => $pid) {
//if posix_kill returns true, unset this PID from the session so we don't waste time killing it again
if(posix_kill($pid,0)) unset($_SESSION['pid'][$i]);
}
//now that all others are killed, we can store the current PID in the session
$_SESSION['pid'][]=posix_getpid();
//close the session now, otherwise the PID we just added won't actually be saved to the session until the process ends.
session_write_close();
Couple things to note:
posix_kill has two values. The first is the pid, and the second is supposed to be one of the signal constants from this list. Nothing there made any sense to me, other people seemed to have success just using 0, and when I use 0 it returns true. So whatever works!
calling session_write_close() before the resource intensive things start happening is crucial. Otherwise the new PID that has been saved to the session won't ACTUALLY be saved to the session until all of the page's processing is done. Which means the next process won't know to cancel the one that's still going on and taking forever.
Related
Requirement:
I need to run a background process (per a user request) that takes about 30 to 60 seconds to complete. I'd like to give the user some status feedback. Note: Toly is right, 'Background' is not required.
What's working:
The process prints about 20 status messages during this time and I retrieve them with a proc_open and listening on a read pipe using fgets. I can save those messages into a session var and using timestamps (to help debug) I can see that the session array is getting written to with these messages as the process progresses.
The Trouble:
My plan was to poll the server with ajax calls (every sec) to retrieve these session vars for display in the DOM. The bottleneck seems to be that the server cannot service the ajax request while it's still running the background process. Everything dumps out at once when the background process completes. From what I can tell, the issue is not with output buffering because using (debugging) timestamps saved with each process message shows the server is writing to the session var sequentially, so that's how I know the proc_open and pipe reads are working as I expect. The issue appears to be the server not being able to give the AJAX request it's JSON object until it is done with the process; or, probably more accurately, done with the loop that is reading the pipe.
Obvious Misconception:
I thought sending a process to the background (using &) might give me a solution here. Apparently I do not know the difference between a background process and a forked process. What benefit is gained - if any - by running a process in the background when doing so appears to make no difference to me in this scenario?
Possible Solutions:
I do not expect the user initiated process that runs this
process/scenario to be that heavy, but if there's something I can
build into this solution that would help a heavy load then I would
like to do that now.
Is this a multi-threading (pthreads) or a
multi-process (fork) solution?
Or, should I save a process id,
let go polling it with a while( .. fgets ..) statement and then
come back to the process after the server has serviced the ajax
request?
I suppose I could run fake status messages and then
response accurately when the results come back after completion.
The time to process the request is not dependent upon the user, so
my fake timing could be pretty accurate. However, I would like to
know what the solution would be to provide real-time feedback.
After google-ing one day for a technique to get the same behavior you are describing here I come up with an easy solution for my project.
A bit of important theory:
- session_start () and a set like $_SESSION["x"] = "y" will always lock the session file.
Case scenario:
- A - process.php - running through an ajax call
- B - get_session.php - a second ajax call;
The main problem is/was, that even if you set a $_SESSION inside a process that is being run through an AJAX it will always have to wait the for the session file to get unlocked and it will result into a sync between the two processes (A. + B.) - both finishing at the same time!
So, the easiest way to fix this matter and get a good result is by using session_write_close() after each set. E.g.:
%_SESSION["A"] = "B";
$_SESSION["x"] = "y";
session_write_close();
PS: Best approach is to have a customed set of functions to handle the sessions.
Sorry for the mark-up. I just created an stack account.
Why would you think that you need a background process? Also, where did you get the idea that you needed one?
A normal php script, with sufficient time out set, with flush() function used every step of the way will give you the output you need for your AJAX.
What's even easier, since you use sessions - AJAX request to a separate handler, that will just check what's in session, and if there is smth new - will return you the new part.
$_SESSION['progress'] = array();
inside process.php
$_SESSION['progress'][] = 'Done 5%';
// complete some commands
$_SESSION['progress'][] = 'Done 10%';
inside ajax.php
if(count($_SESSION['progress']) > $_GET['laststep']) {
// echo the new messages
}
inside your normal page
$.ajax('ajax.php', 'GET', 'laststep=1', success: function(data){ show(data);})
Something like that should work.
Part of the PHP web app I'm developing needs to do the following:
Make an AJAX request to a PHP script, which could potentially take from one second to one hour, and display the output on the page when finished.
Periodically update a loading bar on the web page, defined by a status variable in the long running PHP script.
Allow the long running PHP script to detect if the AJAX request is cancelled, so it can shut down properly and in a timely fashion.
My current solution:
client.php: Creates an AJAX request to request.php, followed by one request per second to status.php until the initial request is complete. Generates and passes along a unique identifier (uid) in case multiple instances of the app are running.
request.php: Each time progress is made, saves the current progress percentage to $_SESSION["progressBar"][uid]. (It must run session_start() and session_write_close() each time.) When finished, returns the data that client.php needs.
status.php: Runs session_start(), returns $_SESSION["progressBar"][uid], and runs session_write_close().
Where it falls short:
My solution fulfills my first two requirements. For the third, I would like to use connection_aborted() in request.php to know if the request is cancelled. BUT, the docs say:
PHP will not detect that the user has aborted the connection until an attempt is made to send information to the client. Simply using an echo statement does not guarantee that information is sent, see flush().
I could simply give meaningless output, but PHP must send a cookie every time I call session_start(). I want to use the same session, BUT the docs say:
When using session cookies, specifying an id for session_id() will always send a new cookie when session_start() is called, regardless of if the current session id is identical to the one being set.
My ideas for solutions, none of which I'm happy with:
A status database, or writing to temp files, or a task management system. This just seems more complicated than what I need!
A custom session handler. This is basically the same as the above solution.
Stream both progress data and result data in one request. This solves everything, but I would essentially be re-implementing AJAX. That can't be right.
Please tell me I'm missing something! Why doesn't PHP know immediately when a connection terminates? Why must PHP resend the cookie, even when it is exactly the same? An answer to any of these questions will be a big help!
My sincere thanks.
Why not set a second session variable, consisting of the unique request identifier and an access timestamp, from status.php.
If the client is closed it stops getting updates from status.php and the session variable stops being updated, which triggers a clean close in request.php if the variable isn't updated in a certain amount of time.
Problem
I have a long running import job which I start with an ajax request, it could take some minutes until the request is finished. While this first ajax request is running, I want to have a look at the server to know how far the import is gone, this second request will be done every 2 seconds or so.
When I use the Ext.Ajax method the requests seems to be chained - the first ajax request (import) runs until it is finished, just then the second (import update) is fired.
I saw that Ext.Ajax is singleton, so maybe thats the reason. So I tried to create my own Connection objects with Ext.create('Ext.data.Connection') but it doesn't work.
My current request chain is:
first request - start
first request - end
second request - start
second request - end
But it should be:
first request - start
second request - start
second request - end
...maybe more second requests
first request - end
Question
The browser should be able to handle multiple request, there must be a limitation inside ExtJS but I didn't find it?
Update 2011-10-16
Answer
The problem wasn't ExtJS - sorry! It was PHP, my first script works with the session and the second script tried to access the session as well. And because PHP sessions are file based, the session file was locked from the first request script and the second request script had to wait until the first release the session lock.
I solved this with this little piece of code I added to my import process (the first script) after every x row:
$id = session_id();
session_write_close();
sleep(1);
session_start($id);
So it stops and reloads the session and the other script was able to hook in and get the session information.
Singleton or non-singleton doesn't even change the way Ext.Ajax works. I think this could be due to the coding (did you wait for the calls to finish?)
Afaik, I never have this problem before when I do multiple calls. The only thing that is hogging the calls is the server (PHP), which doesn't support parallel processing and causes delays, and generate a pattern like this
Call 1 - start
Call 2 - start
Call 1 get processed in the server and Call 2 get queued up
Call 1 - finished
Call 2 get processed in server
Call 2 - finished
It could be disastrous if Call 1 requires more time to process than Call 2.
EDIT:
I have written this little demo just for you to feel how does it works. Check it out :) Spent me half an hour lol!
does anybody know how to update a mysql database if a user exits a browser or navigates away from a page? I have a value set to 1 and want it setting to 0 in such an event. I have been banging my head against the wall with this for weeks and any help would be massively appreciated, thanks. I'm using PHP. I have tried it on body unload but it does not do what I want.
Don't do that. Simply because you rely on the browser to send you a notification (through the beforeunload event) or some other mechanism. What will happen if the browser crashes, or the user put his computer to sleep/hibernate?
You may consider other options, something like a PING! method that the browser could send through Ajax calls every 1 min or so telling he's still alive. All you'd need to do is to create a MySQL procedure that would scan for any "alive" entry more than 1:30 min old (call that last_seen TIMESTAMP or something) and set it your "alive" to 0. This procedure could be called at random requests (pretty much like the PHP session cleanup mechanism).
This will ensure that the user is still there since you will get PING! requests periodically and when the user stops, you will be able to safely say that he won't PING! anymore (though it will cause a light delay, but an acceptable one). This will also ensure user security.
PHP can only change things when a page is ACCESSED. From the PHP perspective, as soon as the script ends, the transaction is over and it have nothing to do with that page anymore.
The only way to do this is keeping the connection open, the script running, and use something like this:
<?php
function shutdown()
{
// here you change the value on your db...
}
register_shutdown_function('shutdown');
// send the data to client
echo "anything";
flush();
// sleep for a lot of time... 99999 seconds...
sleep(99999);
?>
I am working in a tool in PHP that processes a lot of data and takes a while to finish. I would like to keep the user updated with what is going on and the current task processed.
What is in your opinion the best way to do it? I've got some ideas but can't decide for the most effective one:
The old way: execute a small part of the script and display a page to the user with a Meta Redirect or a JavaScript timer to send a request to continue the script (like /script.php?step=2).
Sending AJAX requests constantly to read a server file that PHP keeps updating through fwrite().
Same as above but PHP updates a field in the database instead of saving a file.
Does any of those sound good? Any ideas?
Thanks!
Rather than writing to a static file you fetch with AJAX or to an extra database field, why not have another PHP script that simply returns a completion percentage for the specified task. Your page can then update the progress via a very lightweight AJAX request to said PHP script.
As for implementing this "progress" script, I could offer more advice if I had more insight as to what you mean by "processes a lot of data". If you are writing to a file, your "progress" script could simply check the file size and return the percentage complete. For more complex tasks, you might assign benchmarks to particular processes and return an estimated percentage complete based on which process has completed last or is currently running.
UPDATE
This is one suggested method to "check the progress" of an active script which is simply waiting for a response from a request. I have a data mining application that I use a similar method for.
In your script that makes the request you're waiting for (the script you want to check the progress of), you can store (either in a file or a database, I use a database as I have hundreds of processes running at any time which all need to track their progress, and I have another script that allows me to monitor progress of these processes) a progress variable for the process. When the process begins, set this to 1. You can easily select an arbitrary number of 'checkpoints' the script will pass and calculate the percentage given the current checkpoint. For a large request, however, you might be more interested in knowing the approximate percent the request has completed. One possible solution would be to know the size of the returned content and set your status variable according to the percentage received at any moment. I.e. if you receive the request data in a loop, each iteration you could update the status. Or if you are downloading to a flat file you could poll the size of the file. This could be done less accurately with time (rather than file size) if you know the approximate time the request should take to complete and simply compare against the script's current execution time. Obviously neither of these are perfect solutions, but I hope they'll give you some insight into your options.
I suggest using the AJAX method, but not using a file or a database. You could probably use session values or something like that, that way you don't have to create a connection or open a file to do anything.
In the past, I've just written messages out to the page and used flush() to flush the output buffer. Very simple, but it may not work correctly on every web server or with every web browser (as they may do their own internal buffering).
Personally, I like your second option the best. Should be reliable and fairly simple to implement.
I like option 2 - using AJAX to read a status file that PHP writes to periodically. This opens up a lot of different presentation options. If you write a JSON object to the file, you can easily parse it and display things like a progress bar, status messages, etc...
A 'dirty' but quick-and-easy approach is to just echo out the status as the script runs along. So long as you don't have output buffering on, the browser will render the HTML as it receives it from the server (I know WordPress uses this technique for it's auto-upgrade).
But yes, a 'better' approach would be AJAX, though I wouldn't say there's anything wrong with 'breaking it up' use redirects.
Why not incorporate 1 & 2, where AJAX sends a request to script.php?step=1, checks response, writes to the browser, then goes back for more at script.php?step=2 and so on?
if you can do away with IE then use server sent events. its the ideal solution.