I've googled for this but didn't find any solution - is there a way to create a progessbar for adding/extracting files to/from zip archive in PHP?
Can I get some kind of status message which I can than get with an AJAX request and update the progress bar?
Thanks.
I am trying to do the same thing at the moment; it is mostly* complete (*see issues section below).
The basic concept I use is to have 2 files/processes:
Scheduler (starts task and can be called to get updates)
Task (actually completes the task of zipping)
The Scheduler will:
Create a unique update token and save to cache (APC)
Call the Task page using curl_multi_exec which is asynchronous, passing update_token
Return the token in JSON format
OR
Return the contents of the APC under the update_token (in my case this is a simple status array) as JSON
The Task will:
Update the APC with status, using the update token
Do the actual work :)
Client-side
You'll need some JavaScript to call the Scheduler, get the token in return, then call the Scheduler, passing update_token, to get updates, and then use these returned values to update the HTML.
** Potential pitfalls**
Sessions can be a problem. If you have the same session you will notice that your browser (or is this Apache?) waits for the first request in the session to complete before returning others. This is why I store in the APC.
Current Issues
The problem with the ZipArchive class is that it appears to all the grunt work in the ->close() method, whilst the addFile method appears to take little to no time to complete.
As a workaround you can close and then reopen the archive at specific byte or file intervals. This actually slows the process of zipping down a little, but in my case this is acceptable, as the visual progress bar is better than just waiting with no indication of what is happening.
Related
I have recently updated my site with the use of ajax calls to improve the end-user experience. Some calls are set to poll the db repeadedly, others are called at to alter the database upon user interaction ie. completing a task or cancelling a cart item.
Now I am getting server errors resulting from reaching my servers open file limit.
Here is an example of the sort of code I am using: (credit goes to every tutorial found on google...)
function checkForNewData() {
$.get('checkForNewData.php',false,function(data){
if(data.length){
$('#newData').html(data);
}
});
}
$(function(){
checkForNewData();
setInterval('checkForNewData()',10000);
});
I realize that by using "setInterval('checkForNewData()',10000);" that this means that file is loaded every 10000ms for every user that has this page open.
Here are my questions regarding my ignorance of ajax:
Does a unix server record each ajax call (of this manor) as a page load or open file?
If the page loads behind the scenes, do I have to close it?
Is there a better way to keep a site up-to-date than the repetitiously polling of my db.
Thanks for your time and assistants.
Does a unix server record each ajax call (of this manor) as a page load or open file?
Every-time a php file is run, it is logged. Executed PHP of any manner is recorded. That's why you can see errors in your error log if anything goes wrong during AJAX calls.
If the page loads behind the scenes, do I have to close it?
Which page? "checkForNewData.php"? No you don't. The AJAX call waits for the script to execute & finish and than gets the response.
Is there a better way to keep a site up-to-date than the repetitiously polling of my db?
Yes, there is. I would:
On the server
Use cache (maybe APC cache)
Run a DB check once every minute/ two minutes/ five minutes only
Store/ Update the results in an XML file
On the client
Get the timestamp of the most recent update on client-side page load
Get AJAX to check the timestamp (stored in the XML) of the last update
If timestamp of the AJAX response differs from the first-load timestamp, get new HTML from the XML file
Use AJAX headers or AJAX post-data to request a specific function (like asking for timestamp update vs getting HTML data).
Remember to use the correct flags for json_encode.
print_r(json_encode($html,JSON_HEX_QUOT|JSON_HEX_TAG|JSON_HEX_AMP|JSON_HEX_APOS));
Also remember to zip the data.
ob_start('ob_gzhandler');
It is a best practice to have as fewer DB calls as possible.
I need to create an event listener. I'm a novice so be kind :)
Basically I am on page1.php (php file); I want inside a loop to go check page2.xml (xml file) for some information which should be received at some point. Either check it all the time, or wait and every 5 minutes or so to see if some information has been received there. Either of them work for me.
If no info has been received after a few minutes, then I want to run again the loop (until it is received), otherwise, move forward and do something with my newly received information. This part I have no problem with, just the event listener itself. I couldn't find the function I should be using anywhere. :( I only need to check and retrieve the content of the xml file every so often.
I am not so sure how I should go about this if there isn't just a function which does this, but I couldn't find much when I searched for "event listener php".
Any help would be appreciated: reference to tutorials/sample code/even just telling me what keywords I should be looking for or what I need to learn first in order to do this.
Thanks!
Well, first you should understand the terminology you're using. PHP is not an event-driven language, it is a request-driven language. A request comes into the web-server, PHP parses it and a response is sent back to the requester. At no point are there events triggered that you can process or handle. You can implement your own "event system" but ultimately this is much more work than what your use-case entails.
Your best bet is likely utilizing AJAX and continuously making requests to your PHP script until you return the data that you are looking for. Ultimately you will need to learn about the XMLHttpRequest JavaScript object. After you understand how to make asynchronous requests utilizing JavaScript you can look at the setInterval() method for how to repeatedly make a request.
Once you can repeatedly make asynchronous requests it should be a relatively simple process of creating a webpage where you can trigger the AJAX requests to be sent.
There is no need for a loop in your PHP code. The loop is effectively done on the other end. Here's a textual workflow that you might follow:
Go to a site designed to trigger your AJAX calls and trigger them.
Make your async request to your PHP script.
Inside your PHP script open up the XML file and check for the necessary content.
Return a response in the form of a JSON object. One response can mean the data wasn't updated, the other response means the data was updated.
Parse the response, if the data was not updated repeat from step (2). If the data was updated continue to step (6).
Display a celebratory greeting that your data was updated or a notice that we are still waiting for the data to be updated. Perhaps you can have the number of tries as well, off to the side.
I did the following:
Automatically saved to database every time something new came in.
Then ran a php loop that every few minutes checked to see if there is something new in the database which fits the parameters of this new event (including that it happenned within the timeframe of the past few minutes). I used flush(); and then sleep(120); in the loop to get the loop to keep running every few minutes, until the new info came in in which case it will break(); or die();.
I did something like this writing an inbox parser in PHP. You're best option is to:
Code page1.php in which you just need to do 2 things: read XML from page2.xml and
if there is something "new" just execute the data-parsing
code.
Setup a Cron job (if you're under linux) to execute every 5 minutes or so (Cron command is something like: php /path/to/page1.php). In the same way, if you're running Windows you can setup a scheduled task and execute the same command. Be aware that the full path to your PHP installation should be in PATH environment variable.
I am working in a tool in PHP that processes a lot of data and takes a while to finish. I would like to keep the user updated with what is going on and the current task processed.
What is in your opinion the best way to do it? I've got some ideas but can't decide for the most effective one:
The old way: execute a small part of the script and display a page to the user with a Meta Redirect or a JavaScript timer to send a request to continue the script (like /script.php?step=2).
Sending AJAX requests constantly to read a server file that PHP keeps updating through fwrite().
Same as above but PHP updates a field in the database instead of saving a file.
Does any of those sound good? Any ideas?
Thanks!
Rather than writing to a static file you fetch with AJAX or to an extra database field, why not have another PHP script that simply returns a completion percentage for the specified task. Your page can then update the progress via a very lightweight AJAX request to said PHP script.
As for implementing this "progress" script, I could offer more advice if I had more insight as to what you mean by "processes a lot of data". If you are writing to a file, your "progress" script could simply check the file size and return the percentage complete. For more complex tasks, you might assign benchmarks to particular processes and return an estimated percentage complete based on which process has completed last or is currently running.
UPDATE
This is one suggested method to "check the progress" of an active script which is simply waiting for a response from a request. I have a data mining application that I use a similar method for.
In your script that makes the request you're waiting for (the script you want to check the progress of), you can store (either in a file or a database, I use a database as I have hundreds of processes running at any time which all need to track their progress, and I have another script that allows me to monitor progress of these processes) a progress variable for the process. When the process begins, set this to 1. You can easily select an arbitrary number of 'checkpoints' the script will pass and calculate the percentage given the current checkpoint. For a large request, however, you might be more interested in knowing the approximate percent the request has completed. One possible solution would be to know the size of the returned content and set your status variable according to the percentage received at any moment. I.e. if you receive the request data in a loop, each iteration you could update the status. Or if you are downloading to a flat file you could poll the size of the file. This could be done less accurately with time (rather than file size) if you know the approximate time the request should take to complete and simply compare against the script's current execution time. Obviously neither of these are perfect solutions, but I hope they'll give you some insight into your options.
I suggest using the AJAX method, but not using a file or a database. You could probably use session values or something like that, that way you don't have to create a connection or open a file to do anything.
In the past, I've just written messages out to the page and used flush() to flush the output buffer. Very simple, but it may not work correctly on every web server or with every web browser (as they may do their own internal buffering).
Personally, I like your second option the best. Should be reliable and fairly simple to implement.
I like option 2 - using AJAX to read a status file that PHP writes to periodically. This opens up a lot of different presentation options. If you write a JSON object to the file, you can easily parse it and display things like a progress bar, status messages, etc...
A 'dirty' but quick-and-easy approach is to just echo out the status as the script runs along. So long as you don't have output buffering on, the browser will render the HTML as it receives it from the server (I know WordPress uses this technique for it's auto-upgrade).
But yes, a 'better' approach would be AJAX, though I wouldn't say there's anything wrong with 'breaking it up' use redirects.
Why not incorporate 1 & 2, where AJAX sends a request to script.php?step=1, checks response, writes to the browser, then goes back for more at script.php?step=2 and so on?
if you can do away with IE then use server sent events. its the ideal solution.
I have done some google search on this topic and couldn't find the answer to my question.
What I want to achieve is the following:
the client make an asynchronous call to a function in the server
the server runs that function in the background (because that function is time consuming), and the client is not hanging in the meantime
the client constantly make a call to the server requesting the status of the background job
Can you please give me some advices on resolving my issue?
Thank you very much! ^-^
You are not specifying what language the asynchronous call is in, but I'm assuming PHP on both ends.
I think the most elegant way would be this:
HTML page loads, defines a random key for the operation (e.g. using rand() or an already available session ID [be careful though that the same user could be starting two operations])
HTML page makes Ajax call to PHP script to start_process.php
start_process.php executes exec /path/to/scriptname.php to start the process; see the User Contributed Notes on exec() on suggestions how to start a process in the background. Which one is the right for you, depends mainly on your OS.
long_process.php frequently writes its status into a status file, named after the random key that your Ajax page generated
HTML page makes frequent calls to show_status.php that reads out the status file, and returns the progress.
Have a google for long running php processes (be warned that there's a lot of bad advice out there on the topic - including the note referred to by Pekka - this will work on Microsoft but will fail in unpredicatable ways on anything else).
You could develop a service which responds to requests over a socket (your client would use fsockopen to connect) - some simple ways of acheiving this would be to use Aleksey Zapparov's Socket server (http://www.phpclasses.org/browse/package/5758.html) which handles requests coming in via a socket however since this runs as a single thread it may not be very appropriate for something which requiers a lot of processing. ALternatively, if you are using a non-Microsoft system then yo could hang your script off [x]inetd however, you'll need to do some clever stuff to prevent it terminating when the client disconnects.
To keep the thing running after your client disconnects then the PHP code must be running from the standalone PHP executable (not via the webserver) Spawn a process in a new process group (see posix_setsid() and pcntl_fork()). To enable the client to come back and check on progress, the easiest way to achieve this is to configure the server to write out its status to somewhere the client can read.
C.
Ajax call run method longRunningMethod() and get back an idendifier (e.g an id)
Server runs the method, and sets key in e.g. sharedmem
Client calls checkTask(id)
server lookup the key in sharedmem and check for ready status
[repeat 3 & 4 until 5 is finished]
longRunningMethod is finished and sets state to finished in sharedmem.
All Ajax calls are per definition asynchronous.
You could (although not a strictly necessary step) use AJAX to instantiate the call, and the script could then create a reference to the status of the background job in shared memory (or even a temporary entry in an SQL table, or even a temp file), in the form of a unique job id.
The script could then kick off your background process and immediately return the job ID to the client.
The client could then call the server repeatedly (via another AJAX interface, for example) to query the status of the job, e.g. "in progress", "complete".
If the background process to be executed is itself written in PHP (e.g. a command line PHP script) then you could pass the job id to it and it could provide meaningful progress updates back to the client (by writing to the same shared memory area, or database table).
If the process to executed it's not itself written in PHP then I suggest wrapping it in a command line PHP script so that it can monitor when the process being executed has finished running (and check the output to see if was successful) and update the status entry for that task appropriately.
Note: Using shared memory for this is best practice, but may not be available if you are using shared hosting, for example. Don't forget you want to have a means to clean up old status entries, so I would store "started_on"/"completed_on" timestamps values for each one, and have it delete entries for stale data (e.g. that have a completed_on timestamp of more than X minutes - and, ideally, that also checks for jobs that started some time ago but were never marked as completed and raises an alert about them).
This question is a followup to my previous one: Previous Questions.
So I setup my page to initiate an ajax call to initiate processing some records. And after each record it updates a row in another table to keep track of the status of this process. After that first ajax call is made, I have another start up. It's a Ajax.PeriodicalUpdater and it's set to hit a file which simply queries that row in the db and returns the status of the original process.
So this works perfectly fine... as long as the file that provides the status updates is outside my current app. If I put the file inside of my app, then it doesn't work right. If I watch firebug, the PeriodicalUpdater call doesn't get anything back until the original ajax call finishes, it just hangs out so it's as if the file is hung and not returning anything.
This whole app is running inside just a basic framework we are using. Nothing crazy, just handles routing, and basic template aspects etc... So all of these functions/files are inside this app and all these ajax calls are being routed through this.
What could be causing something like this?
Can this be due to the limit of concurrent connections supported by a browser to a particular domain?
This is caused by PHP session serialization. The session data is locked until the PHP process for each request has finished writing to it, so further requests in the same session will queue until the lock is released.
If your AJAX requests need access to session state, read out the information you need and then use session_write_close() as early in your code as possible to release those locks.