I have implemented a SAAS scenario with my Windows server: a user could upload a file in a website, then Fetch.exe is an application coded in C# hosted in the server, Fetch.exe takes the uploaded file as an input, executes and generates an output to download for the user.
So in my php, I use exec to wrap Fetch.exe:
exec("Fetch.exe " . $inputFile . " > " . $outputFile)
Uploading and executing (ie, Fetch.exe) may take more than several seconds, and I want to show the user that it is processing and everything is going fine.
I have found some threads that discuss how to show a progress bar for the uploading. Whereas, does anyone know what I could do to show the approximate progress of Fetch.exe? Do I have to split it into smaller applications and use several exec?
You could supply Fetch.exe with an randomly generated ID from php, for example from the uniqueid function. Fetch.exe will create a file called <uniqueid>.txt with a progress percentage. From the browser, you could call another script with that unique ID to get the contents of that .txt file. In order, it would be something like this:
User uploads the file to PHP
PHP:
handles the uploaded file
creates a uniqueID
starts Fetch.exe with the file and the uniqueID
returns a page with the uniqueID embedded
The following happens in parallel:
Fetch.exe creates a textfile called /progress/uniqueid.txt with the uniqueid as name. It logs the progress into it.
The browser does an AJAX call to http://example.com/progress/uniqueid.txt and shows the progress to the user
And finally, when the progress reaches 100% the browser downloads the file. The only thing you might want to add is the pruning of the progress files after a while. Say you delete all files older than 10 minutes every hour.
Your PHP program needs a way to know the state of the subprocess (the Fetch.exe application), so, Fetch.exe needs to send info about the processing state, the most natural way to do this is through the standard output (the standard output is the information that provides a program when you run it from cmd).
Knowing this, you can run and keep reading a subprocess output from php using popen().
And secod, you can use the PHP ob_flush() and flush() with the onmessage javascript event to establish the comunication from your client page with your running php script, here you can find a good tutorial on how do this.
Related
Like a Log-file is written by a php-script via fwrite($fp, ---HTML---),
I need to save an HTML DIV as png-file on the server.
The client-browser only start the php-script,
but without any client-interaction the png-file should be saved on the server.
Is there a way to do this?
All posts (over thousands) I have been reading are about html2canvas,
which is (as I understand) client-side operating.
I know the html-(html-div)-rendering normally does the browser.[=client-side]
But is there a way to do it in PHP on server-side ?
Reason:
Until now the procedure is
print the div via browser on paper twice
one for the costumer,
one to scan it in again to save it on the server as picture and throw it in the paperbasket.
By more than 500 times a day ...
By security reasons it need to be a saved picture on the server.
I'm working on an app that creates QR codes and renders them onto multiple graphics for a user.
The Problem:
I wrote a script to import users to create from a CSV. I'm needing to create over 100 users (each including the process above). Right now it takes roughly 1 minute to complete for each new user to complete the processing.. then spits out all my error/success messages at once.
My Question:
Rather than the browser slowly loading the result view (currently stays on a white page until complete) as my script is processing, is their a somewhat easy way to display the live progress and errors as they happen? Something like a progress bar updated as each user is created/fails. I'm guessing it will require AJAX?
When dealing with websites, remember the golden rule.
PHP MUST DIE.
Noobs assume this is people rubbishing PHP. It isn't. It's the HTTP request cycle.
Request In > PHP > Response Out > PHP process dies.
This is only the case when dealing with web servers and browsers, not CLI PHP. But the point is that you may end up getting Apache timeouts if your script takes as long as you say.
One solution could be to set up a cron that checks for a file and if it finds it, processes it, dumping a line number in a text file that your browser could check, which means you could fetch progress:
<?php
if (file_exists('/some/csv/to/process.csv')) {
// open file
// get row to work on
// process row
// update progress file with next line number
}
Meanwhile, you could set up a script that does this:
<?php
$progress = file_get_contents('/path/to/progress.txt');
header('Content-Type: application/json');
echo json_encode(['progress' => $progress]);
And then get the progress using AJAX inside a setInterval function:
$.get('/path/to/progress/json/page', function(data){
console.log(data);
});
Just an idea, may or may not suit you but give it a try!
I'm having problems sending an array to another PHP page. We send an array from one page to another to generate CSV file that has been transformed from XML. So we take a 800mb XML file and transform it down to a 20mb CSV file. There is a lot of information in it that we are removing and it runs for 30 minutes.
Anyway, we are periodically using a function to output the progress of the transformation in the browser with messages:
function outputResults($message) {
ob_start();
echo $message . "<br>";
ob_end_flush();
ob_flush();
}
$masterArray contains all the information in a associative array we have parsed from the XML.
The array ($masterArray) at the end we send from index.php to another php file called create_CSV_file.php
Originally we used include('create_CSV_file.php') within index.php , but due to the headers used in the CSV file, it was giving us the messages that
Warning: Cannot modify header information - headers already sent
. So we started looking at a solution of pushing the array as below.
echo "<a href='create_CSV_file.php?data=$masterArray'>**** Download CSV file ***</a>";
I keep getting the error message with the above echo :
Notice: Array to string conversion
What is the best method to be able to show echo statements from the server as it is running, then be able to download the result CSV at the end?
Ok, so first of all, using data in a url (GET) has some severe limitations. Older version of IE only supported 4096 byte urls. In addition, some proxies and other software impose their own limits.
I'm sure you've heard this before, but if not.... You should not be running a process that takes more than a couple of seconds (at most!) from a web server. They're not optimised for it. You definitely don't want to be passing megabytes of data to the client just so they can send it back to the server!
How about something like this...
User makes a web request (And uploads original data?) to the server
Server allocates an ID for the request (random? database?) and creates a file on disk using the ID as a name (tmp directory, or at least outside web root)
Server launches a new process (PHP?) to transform the data. As it runs, it can update the database with progress information
During this time, the user can check progress by making a sequence of AJAX requests (or just refreshing a page which shows latest status). Lots more control over appearance now
When the processing is complete, server-side process writes results to file, updates database to indicate completion.
Next time user checks status, redirect them to a PHP file that takes the ID and will read the file from disk / stream it to the user.
Benefits:
No long-running http requests
No data being passed back/forth to client in intermediate stage
Much more control over how users see progress
Depending on the tranformation you're applying / the detail stored in the database, you may be able to recover interrupted jobs (server failure)
It does have one downside which is that you need to clean up after yourself - the files you created on disk need to be deleted, however, you've got a complete audit of all files in the database and deleting anything over x days old would be trivial.
I am using PHP exec() for -convert Image Magick command, and i want to convert more than one page.
Given that it may take a few minutes, I need some kind of progress bar in order to monitor conversion.
Any ideas how I could done this?
well you can not really track the progress of a single conversion. but you could for exmaple do something like the following when you want to convert multiple documents:
the number of pages = 100%
current page number / number of pages * 100 = progress in percent
so after each processed page you can update the progress.
you can write the info in a file or in a database (linked to the user session for a multi-user software) and poll for the status with an ajax reqeust to a php script which reads this file/db.
A solution for this problem is message queues. I forked a code example of how to use Pheanstalkd (a PHP framework for beanstalkd) here
This example shows how to have a sender that puts jobs in queue, worker that pulls jobs from queue, and a watcher that watches jobs (this part essentially does what you are asking for).
You can download Pheanstalk from here
I have done a code to receive images from iphone to PHP Server and I need to resize these image and move to 4 folders.
Only then the json respose is giving to iphone. But it takes much time.
Requirement:
i want to move a file to the folder "folder1" then want to give the json response.
the resizing process should do from this "folder1" after giving json response.
How to run this resizing process in background.
Here is my code:
http://pastebin.com/qAcT1yi9
You could always send your php script to run in the background with a Linux command.
Example:
// using backticks to execute the Linux command but there are
// other alternatives
$cmd = `php runScriptInBackground.php &`;
echo $cmd;
First send/upload the images and send a response back, without doing the resize operation.
Then, if the upload was successful, let the browser issue another request and do the resizing. When this succeeds, send the message ‘resizing successful’ back.
A common solution to this problem is implementing a loading/processing message on hitting a specific event. Then - still being displayed - the action will continue to load on the background and the result page will finally be displayed when done.
Although the user must wait, I prefer this above display a result message when the actual result is not known. Unfortunately I'm not sure how this is done on iphone development.
if your building in objective c then you may just resize make a copy and resize it there and send the resized image to your php you could then display a spinner and json result back to the user and also if the is an error the user will still have the resized image to try again with... Also another thought I had was was to use push notification. I don't know what that code would look like but it's something to consider
you need some async javascript or an iframe in your page posting the image to your server and providing feedback to the user.
This means that the 'main' page would not change, but some visual information can be provided to the user.
You can display an animated gif loader or use JS setInterval to give the user the feeling that things are moving forward why waiting for the server to respond.
If the processing is split in more 1 parts, after each step the server could respond with an HTML page and a redirect: this would even work in an IFRAME without JS.
Each 'page' would perform one more step. But if the user closes the browser before all is done you would end with an unfinished task.
A DB, real background processing, and client side JS polling are a more robust alternative.
A full answer would be quite long and require way more details on your settings (apache CGI PHP? or mod_php? are you using an MVC model or framework, or are you writing a page-oriented website?).
If i had to write a full answer I would forget PHP and use Python and celery http://celeryproject.org/ ;-)
PS.
I just found out that a few related questions already existed:
PHP Background Processes
Asynchronous shell exec in PHP
You can do it realy in two times, first send de files and save on first server, after when the user request that you generate the necesary parts.
You will pass the costs from the file sender to the first request from that.