Problem:
I'm trying to see if I can have a back and forth between a program running on the server-side and JavaScript running on the client-side. All the outputs from the program are sent to JavaScript to be displayed to the user, and all the inputs from the user are sent from JavaScript to the program.
Having JavaScript receive the output and send the input is easily done with AJAX. The problem is that I do not know how to access an already running program on the server.
Attempt:
I tried to use PHP, but ran into some hurdles I couldn't leap over. Now, I can execute a program with PHP without any issue using proc_open. I can hook into the stdin and stdout streams, and I can get output from the program and send it input as well. But I can do this only once.
If the same PHP script is executed(?) again, I end up running the program again. So all I ever get out of multiple executions is whatever the program writes to stdout first, multiple times.
Right now, I use proc_open in the script which is supposed to only take care of input and output because I do not know how to access the stdout and stdin streams of an already running program. The way I see it, I need to maintain the state of my program in execution over multiple executions of the same PHP script; maintain the resource returned by proc_open and the pipes hooked into the stdin and stdout streams.
$_SESSION does NOT work. I cannot use it to maintain resources.
Is there a way to have such a back and forth with a program? Any help is really appreciated.
This sounds like a job for websockets
Try something like http://socketo.me/ or http://code.google.com/p/phpwebsocket/
I've always used Node for this type of thing, but from the above two links and a few others, it looks like there's options for PHP as well.
There may be a more efficient way to do it, but you could get the program to write it's output to a text file, and read the contents of that text file in with php. That way you'd have access to the full stream of data from the running program. There are issues with managing the size of the file, and handling requests from multiple clients, but it's a simple approach that might be good enough for your needs.
You are running the same program again, because it's the way PHP works. In your case client does a HTTP request and runs the script. Second request will run the script again. I'm not sure if continuous interaction is possible, so I would suggest making your script able to handle discrete transactions.
In order to figure different steps of the same "interaction", you will have to save data about previous ones in database. Basically, you need to give some unique hash to every client to identify them in your script, then it will know who does the request and will be able to differ consecutive requests from one user from requests of different users.
If your script is heavy and runs for a long time, consider making two script - one heavy and one for interaction (AJAX will query second one). In this case, second script will fill data into database and heavy script will simply fetch it from there.
Related
I'm trying to figure out the best way to send a response to an incoming API POST to our system, without returning.
We receive an XML post from another service, and consume it, and then our response to them is in XML also. What we are currently doing is digesting the incoming post, do some stuff on our end, then do a php return with the XML.
I would like to change this so that we can respond to their call with the XML, but then do some processing after the fact, without making some type of exec/background call.
What's the best way to send a response in PHP without returning? If we do an "echo" will that close the connection and allow us to process more afterwards without the other server "waiting?
Calling PHP's echo will not close the connection, in fact you can call echo multiple times in your PHP script and the output will be added to the response. The connection will only close when
The end of the script is reached
exit() or the alias die() are called
A fatal/parse error or an uncaught Exception occurs or your server runs out of memory
The maximum script execution time which you can set in php.ini is exceeded
Usually, the calling client code will also have some kind of timeout, so if your 'digesting' code could take longer and you want to take care about this as well as point 4 in the list, you can store the request data for later processing, for example in a database or serialized in files. Having successfully stored the data, you then have basically 2 options to go:
Option 1: Spawn a background PHP process
To spawn a background PHP process that will survive the livecycle of the calling script, use exec and nohup. The basic usage could look like this:
exec('RESOURCE_ID=123 nohup /path/to/your/php/executable your_script.php > /dev/null');
Within the first segment of the command, RESOURCE_ID=123, you can pass a unique identifier of the previously stored request data, maybe a database entry id or the storage filename, to the background script. Use getenv('RESOURCE_ID') in your background script to retrieve the variable.
[EDIT] > /dev/null for output redirection is crucial for running the process in the background, otherwise the parent script will wait for the output of the background process. I also suggest to write the output as well as error output to an actual file like &> my_script.out, this will have the same effect. You could also get the process id of the background process by appending & echo $! to the command, exec() will then return it.
After starting the background script, you can send your 'OK' response and exit the parent script.
Option 2: Cronjob for processing, as suggested by Jim Panse
The more complex your system grows, you probably need more control over the execution of your 'digesting' code. Perhaps you want to balance server load peaks, or restart failed tasks, or throttle malicious usage of your API. If you need this kind of control, you are better off with this option.
Since i guess you want your system-to-system communication synchronously there are multiple things you can consider.
Even though time consuming requests you usually still want a fast response.
To satisfy this you can't process the request immediately.
So, just save the request and process it later (give the client a 202 response back). Systems like queues are very popular to save time consuming jobs for running them later. Another time controlled script (cronjob) could then do polling and process the stacked messages/data.
If you want to provide the results to the client too, return them a unique resource id on the initial rest call and implement another resource with exactly this parameter as the input. If your system finished processing, the result will appear there.
Spawning a process from within another php script isn't very handy since it's very difficult to debug and error-prone.
I personally would't go for this solution.
I want to take the user's input, respond to them, and then update some stuff in the database. But the response doesn't need the stuff from the database, so I don't want the user to have to wait for it. Is there a way to say "I'm all done. Go ahead and send this to the user." but then keep executing more PHP code?
Take a look at exec() in php http://php.net/manual/en/function.exec.php. Just before the end of processing of your php page, you can fire a command with exec() like running a php script cli: http://php.net/manual/en/features.commandline.php (basically runs in the background.). Be sure to note (for exec()):
If a program is started with this function, in order for it to continue running in the background, the output of the program must be redirected to a file or another output stream. Failing to do so will cause PHP to hang until the execution of the program ends.
You can use flush, which will send the data, however it may not have the effect you're looking for (the page will still be loading).
http://us2.php.net/flush
It's used when you've got an admin type script that produces iterative output but might take a while to run.
I am working in a tool in PHP that processes a lot of data and takes a while to finish. I would like to keep the user updated with what is going on and the current task processed.
What is in your opinion the best way to do it? I've got some ideas but can't decide for the most effective one:
The old way: execute a small part of the script and display a page to the user with a Meta Redirect or a JavaScript timer to send a request to continue the script (like /script.php?step=2).
Sending AJAX requests constantly to read a server file that PHP keeps updating through fwrite().
Same as above but PHP updates a field in the database instead of saving a file.
Does any of those sound good? Any ideas?
Thanks!
Rather than writing to a static file you fetch with AJAX or to an extra database field, why not have another PHP script that simply returns a completion percentage for the specified task. Your page can then update the progress via a very lightweight AJAX request to said PHP script.
As for implementing this "progress" script, I could offer more advice if I had more insight as to what you mean by "processes a lot of data". If you are writing to a file, your "progress" script could simply check the file size and return the percentage complete. For more complex tasks, you might assign benchmarks to particular processes and return an estimated percentage complete based on which process has completed last or is currently running.
UPDATE
This is one suggested method to "check the progress" of an active script which is simply waiting for a response from a request. I have a data mining application that I use a similar method for.
In your script that makes the request you're waiting for (the script you want to check the progress of), you can store (either in a file or a database, I use a database as I have hundreds of processes running at any time which all need to track their progress, and I have another script that allows me to monitor progress of these processes) a progress variable for the process. When the process begins, set this to 1. You can easily select an arbitrary number of 'checkpoints' the script will pass and calculate the percentage given the current checkpoint. For a large request, however, you might be more interested in knowing the approximate percent the request has completed. One possible solution would be to know the size of the returned content and set your status variable according to the percentage received at any moment. I.e. if you receive the request data in a loop, each iteration you could update the status. Or if you are downloading to a flat file you could poll the size of the file. This could be done less accurately with time (rather than file size) if you know the approximate time the request should take to complete and simply compare against the script's current execution time. Obviously neither of these are perfect solutions, but I hope they'll give you some insight into your options.
I suggest using the AJAX method, but not using a file or a database. You could probably use session values or something like that, that way you don't have to create a connection or open a file to do anything.
In the past, I've just written messages out to the page and used flush() to flush the output buffer. Very simple, but it may not work correctly on every web server or with every web browser (as they may do their own internal buffering).
Personally, I like your second option the best. Should be reliable and fairly simple to implement.
I like option 2 - using AJAX to read a status file that PHP writes to periodically. This opens up a lot of different presentation options. If you write a JSON object to the file, you can easily parse it and display things like a progress bar, status messages, etc...
A 'dirty' but quick-and-easy approach is to just echo out the status as the script runs along. So long as you don't have output buffering on, the browser will render the HTML as it receives it from the server (I know WordPress uses this technique for it's auto-upgrade).
But yes, a 'better' approach would be AJAX, though I wouldn't say there's anything wrong with 'breaking it up' use redirects.
Why not incorporate 1 & 2, where AJAX sends a request to script.php?step=1, checks response, writes to the browser, then goes back for more at script.php?step=2 and so on?
if you can do away with IE then use server sent events. its the ideal solution.
My iPhone client app uploads a data to the server, which runs on PHP. There is a code to invoke a .exe program on the server side on PHP. The .exe program will take the uploaded data and run on a process on its own. That means the PHP execution will end without waiting for the .exe program to finish. After the .exe program finished processing the uploaded data and have an output, I want this output to be sent back to the iPhone.
Normally, if we call the .exe program to be run inside the php without making it a seperate process, we have to wait for the program to finish and we can send the output back to the iPhone client.
By running the .exe program as a seperate process, it is impossible to send the data back via PHP that invokes the .exe program. The question is, if we have the .exe program running on a seperate process rather than on the PHP script, what are the possible methods to send the output back to the iPhone client?
That's a need problem you've outlined. Let me explain a couple of ideas.
First of all, if you terminate the initial upload request, the only resonable way to check for it being done is to poll every few seconds from the iPhone. Send a request to "get-update.php" every 5 seconds to see if you have data.
By using $_SESSION, you should be able to store a token that will identify the data when it has finished processing.
Regarding the actually process, you may be able to accomplish that in a number of ways. One is to do a fairly standard double-fork, detaching the child process from the parent, so it will continue after the parent exits.
Another (recommended) would be to author a backend server process that would watch your database for requests, fetch them, process them, and update the database. So when the inital upload script actually uploads the data, have PHP put it in the database, store the record ID in $_SESSION, and return to the user.
The back end process will notice that there is a record to process, read the data, call the executable, and update the database with the result.
The get-update.php script will read $_SESSION for the record id, and check the database if the data has been processed (or what the status is).
If you do not have the ability to run a background process, and you are constrained to using PHP, you could do the double-fork magic, and fork of another PHP process to do the database read / exe / database update.
Feel free to comment with questions.
You need (a) a good way to pass the data to the program, and (b) a good way to get the data back.
I would say this is a perfect case for an AJAX snippet frequently polling data from, say, a text file the .exe writes its status in.
The upload script you call could return a unique identifier of some sort to the uploading client. Using that identifier, the client would poll the exe's status (e.g. "does the output file xyz already exist?") until it gets positive feedback.
You're going to have a hard time reconnecting with the iPhone once you've severed the connection. It may be out of coverage, it may have changed IP address, ......
Your best bet is to have the iPhone reconnect back to the server and poll for it's information.
You could do this by using Apple's Push Notification service, but that's probably overkill, unless you think the data processing is going to take a long time, and/or you want to update the app icon when the processing is done, even if the app isn't running.
Do you expect the user to just be patiently waiting for the result, or are they going to fire off the data, and check back later? If it's only going to take a couple of seconds, you could just have the iPhone app poll for the result after waiting a little while (while displaying a progress indicator).
I have a PHP script that takes about 10 minutes to complete.
I want to give the user some feedback as to the completion percent and need some ideas on how to do so.
My idea is to call the php page with jquery and the $.post command.
Is there a way to return information from the PHP script without ending the script?
For example, from my knowledge of this now, if I return the variable, the PHP script will stop running.
My idea is to split the script into multiple PHP files and have the .post run each after a return from the previous is given.
But this still will not give an accurate assessment of time left because each script will be a different size.
Any ideas on a way to do this?
Thanks!
You can echo and flush() output, but that's suboptimal and rather fragile solution.
For long operations it might be good idea to launch script in the background and store/updte script status in shared location.
e.g. you could lanuch script using fopen('http://… call, proc_open PHP CLI process or even just openg long-running script in an <iframe>.
You could store status in the database or in shared memory (using apc_store()).
This will let user to check status of the script at any time (by refreshing page, or using AJAX) and user won't lose track of the script if browser's connection times out.
It also lets you avoid starting same long script twice.