I have a script that checks the status of a few hundred webpages. However, the script takes about 2 minutes to load, and the screen is blank until the script has finished running. Then all the data is outputted at once.
I want to output data while the script is still running. Here's part of my script:
foreach ($urls as $url){
$headers = get_headers($url,true);
$status = $headers[0];
list($protocol, $code, $message) = explode(' ',$status,3);
echo '<br>'.$url.'<br>'.$code.'<br>';
}
http://php.net/manual/en/function.flush.php contains the answer you seek.
Be aware this may negatively hit your performance, but it sounds like you're more interested in seeing it's progress. :-)
This is a common function which I use throughout a lot of my scripts to see progress.
function flush_buffers() {
ob_end_flush();
ob_flush();
flush();
ob_start();
}
I can't remember the original source of this function, probably from php.net somewhere!
Hope it helps!
Related
I am trying to update information on front end while processing is in progress at back end. For this purpose I am using php flush function.
My code is
foreach ($csv_array as $row) {
$varint=$varint+1;
$curr_id = $row['Item'];
$toret['row_num']=$varint;
$toret['curr_url']=$curr_id;
echo json_encode($toret);
flush();
$this->scrape_one_id($curr_id);
$value['arrow_id']=$curr_id;
$this->ahm->insert_new_id($curr_id);
$this->ahm->insert_current($value);
}
I have tried it in many ways but it only sends back first echo only and sends rest after complete execution is finished. It works fine if scrape_one_id function is commented.
Some of the methods I have tried are:
foreach ($csv_array as $row) {
$varint=$varint+1;
$curr_id = $row['Item'];
$toret['row_num']=$varint;
$toret['curr_url']=$curr_id;
echo json_encode($toret);
if( ob_get_level() > 0 ) ob_flush();
$content = ob_get_clean();
flush();
if( ob_get_level() > 0 ) ob_clean();
$this->scrape_one_id($curr_id);
$value['arrow_id']=$curr_id;
$this->ahm->insert_new_id($curr_id);
$this->ahm->insert_current($value);
}
It also worked fine without scrape_one_id function
Any kind of help will be appreciated.
I'd try adding ob_flush() in there.
Edit: in your first code , just put ob_flush() after your first flush()
and see how that works.
http://php.net/manual/en/function.flush.php
flush() may not be able to override the buffering scheme of your web
server and it has no effect on any client-side buffering in the
browser. It also doesn't affect PHP's userspace output buffering
mechanism. This means you will have to call both ob_flush() and
flush() to flush the ob output buffers if you are using those.
I would like to point out that there is a function to replace ob_flush and flush. If you set ob_implicit_flush(true); at the top of the page it will automatically flush any echo or print you do in the rest of the script.
Note that you still need a minimum amount of data to come through the browser filter. I would advice using str_pad($text,4096); since this automatically lenghtens the text with spaces to 4 KB which is the minimum limit when using FireFox and linux.
You should read manual (caveats) and also comments there.
I hope this helps you.
I have a ob_start() and a corresponding ob_flush(). I would like to flush a portion of data and continue executing the rest. Using ob_flush() didn't help. Also if possible rest needs to happen without showing loading in browser.
EDIT:
I don't want to use ajax
I have done this in the past and this is how I solved it:
ob_start();
/*
* Generate your output here
*/
// Ignore connection-closing by the client/user
ignore_user_abort(true);
// Set your timelimit to a length long enough for your script to run,
// but not so long it will bog down your server in case multiple versions run
// or this script get's in an endless loop.
if (
!ini_get('safe_mode')
&& strpos(ini_get('disable_functions'), 'set_time_limit') === FALSE
){
set_time_limit(60);
}
// Get your output and send it to the client
$content = ob_get_contents(); // Get the content of the output buffer
ob_end_clean(); // Close current output buffer
$len = strlen($content); // Get the length
header('Connection: close'); // Tell the client to close connection
header("Content-Length: $len"); // Close connection after $len characters
echo $content; // Output content
flush(); // Force php-output-cache to flush to browser.
// See caveats below.
// Optional: kill all other output buffering
while (ob_get_level() > 0) {
ob_end_clean();
}
As I said in a couple of comments before, you should watch out for gzipping your content, since that will alter the length of your content, but not change the header about it. It also can buffer your output, so it won't get send to the client instantly.
You could try letting apache know to not gzip your content by using apache_setenv('no-gzip', '1');. But this will not work if you use rewrite-rules to go to your page, since then it will also modify those environment variables. At least, it did so for me.
See more caveats about flushing your content to the user in the manual.
ob_flush writes the buffer. In other words, ob_flush tells PHP to give Apache (or nginx/lighttpd/whatever) the output and then for PHP to forget about it. Once Apache has the output, it does whatever it wants with it. (In other words, after ob_flush it's out of your control whether or not it gets immediately written to the browser).
So, short answer: There's no guaranteed way to do that.
Just a guess, you're likely looking for AJAX. Whenever people are trying to manipulate when page content loads as you're doing, AJAX is almost always the correct path.
If you want to continue a task in the background, you can use ignore_user_abort, as detailed here, however, that is often not the optimal approach. You essentially lose control over that thread, and in my opinion, a web server thread is not where heavy processing belongs.
I would try to extract it out of the web facing stuff. This could mean a cron entry or just spawning a background process from inside of PHP (a process that though started from inside of script execution will not die with the script, and the script will not wait for it to finish before dying).
If you do go that route, it will mean that you can even make some kind of status system if necessary. Then you could monitor the execution and give the user periodic updates on the progress. (Technically you could make a status system with a ignore_user_abort-ed script too, but it doesn't seem as clean to me.)
this is my function
function bg_process($fn, $arr) {
$call = function($fn, $arr){
header('Connection: close');
header('Content-length: '.ob_get_length());
ob_flush();
flush();
call_user_func_array($fn, $arr);
};
register_shutdown_function($call, $fn, $arr);
}
wrap the function to be executed in the end, after php close the connection. and of course the browser will stop buffering.
function test() {
while (true) {
echo 'this text will never seen by user';
}
}
this is how to call the function
bg_process('test');
first argument is callable,
second argument is an array to be passed to 'test' function with an indexed array
Note : I don't use ob_start() at the beginning of the script.
I have an article explaining how this can be achieved using apache/mod_php on my blog here: http://codehackit.blogspot.com/2011/07/how-to-kill-http-connection-and.html Hope this helps, cheers
If you are using PHP-FPM:
ignore_user_abort(true);
fastcgi_finish_request();
Above two functions are the key factors which ignore_user_abort prevents error and fastcgi_finish_request closes client connection.
fastcgi_finish_request
This function flushes all response data to the client and finishes the request. This allows for time consuming tasks to be performed without leaving the connection to the client open.
not working on Apache.(PHP 5 >= 5.3.3, PHP 7)
Use:
header("Content-Length: $len");
..where $len is the length of the data to be flushed to the client.
I don't have the background to know when and where this is going to work, but I tried on a few browsers, and all returned instantly with:
<?PHP
header("Content-length:5");
echo "this is more than 5";
sleep(5);
?>
edit: Chrome, IE, and Opera showed this, while FireFox showed this is more than 5. All of them closed the request after that though.
I have a php script that uses cURL and takes about 10-15 minutes to execute. What it does, it parses about 1000 pages looking for specific matches and throughout the script I have diagnostic messages echo'ed out, like "Going to the next page", "Found a match", "Error loading page" ... The way it works now (and the way that it's normal) is it executes for like 10 minutes and only then spits out all my custom messages.
I would like to be able to display those messages as they happen, not when the script is done executing. I was thinking something like AJAX would do it, but am not sure how it would work. Any tips are greatly appreciated. Thanks.
So, this is a old post but I found a solution for this. As I also have to make the same thing, output when the script is still running. Not any answer from here helped.
First of all, I am using Win32 server(production) and XAMPP as local for tests. This example is just a proof of concept and can be modified as you please.
<?php
ob_implicit_flush(true);
for($i=1; $i<=10; $i++){
echo "$i ...<br>";
for($k = 0; $k < 40000; $k++) echo ' ';
sleep(1);
}
?>
So, we open output buffer as implicit. Then we make a demo loop to count from 1 to 10 and display the values as they are been processed. Second loop will fill in the browsers buffer. And finally to check if everything is working well we make a sleep for 1 second. Otherwise the script will run too fast and we could not know if we achieved the goal.
Hope this helps !
You could create a staging table.
The PHP script could, instead of echo'ing the message, store them into a database table (possibly memory table for performance).
You could then periodically poll a seperate PHP script using ajax, which would query the table, and return any new messages to the client.
Use flush to immediately send output to the browser, by flushing the output buffer.
echo "foo";
flush();
echo "bar";
flush();
Actually you're looking for something like flush and ob_flush, however bear in mind that there are a lot of factors that can prevent your output from being flush'd as it happens.
From the flush documentation you'll get:
Several servers, especially on Win32, will still buffer the output from your script until it terminates before transmitting the results to the browser.
Server modules for Apache like mod_gzip may do buffering of their own that will cause flush() to not result in data being sent immediately to the client.
I'm using the #ob_flush() after every echo. In this example PHP_EOL creates a new line after $string
function output($string){
echo $string.PHP_EOL;
#ob_flush();
}
Basically, have your script write HTML output to a temporary log file. Then use ajax to periodically update the end-user's browser with the temporary log file. jQuery will make quick work of this.
Ajax is the only guaranteed way to get it to work on all browsers. Here is a quote from PHP's flush page.
flush() may not be able to override
the buffering scheme of your web
server and it has no effect on any
client-side buffering in the browser.
It also doesn't affect PHP's userspace
output buffering mechanism. This means
you will have to call both ob_flush()
and flush() to flush the ob output
buffers if you are using those.
Sounds to be like you have output buffering turned on.
Calling ob_end_flush() will print what's currently in the buffer, and turn off the buffer for the rest of the script execution.
You can use the flush() function to send all the content of the buffer to the client. http://php.net/manual/fr/function.flush.php
You could use both flush and ob_flush, reminding to set the content type header:
<?php
header( 'Content-type: text/html; charset=utf-8' );
for( $i = 0 ; $i < 10 ; $i++ ){
echo $i . '<br>';
flush();
ob_flush();
sleep(1);
}
Source: dermeister note in php.net ob_flush page.
Tested on Firefox 42.0 and Chrome 46.0
One thing I have noticed with php, is that nothing is output to the screen until the script has stopped working. For the project I am working on I feed in a list of over 100 items and it performs a HTTP request for each item and when finished, shows a page with the status of each item, success failure etc.
What I want to know is if there is a way to output the results of each 'foreach' loop as they happen? So the user watching the screen sees the magic happening one line at a time or after say 5 lines.
I have only ever seen this done with Ajax type requests, is that what I should be looking to do instead maybe? Can anyone point me to a php function that does this or is it not possible?
It may be better to store all script output in a buffer then flush the buffer when required.
For example:
<?php
if (ob_get_level() == 0) ob_start();
$test = Array('one','two','three','four');
foreach ($test as $key=>$val)
{
echo $test;
ob_flush();
flush();
}
ob_end_flush();
?>
Make sure you have mod_gzip disabled!
flush() should do it, or you can look at all the output buffering functions
Use the flush() command
I use
flush(); #ob_flush();
after the output.
I am using flash to call a PHP page that needs to do a bit of processing. Is it possible to let PHP continue processing but show a response anyway so flash doesn't stall waiting?
My answer from here:
You can send Connection:Close headers,
which finishes the page for your user,
but enables you to execute things
"after page loads".
There is a simple way to ignore user
abort (see php manual too):
ignore_user_abort(true);
Use output control aka output buffering to do this. http://www.php.net/manual/en/function.ob-flush.php
You could try using flush()
As an example, try these two different pieces of code:
// without flush()
foreach ( range(1, 5) as $num ) {
echo "Beep $num<br>";
sleep(1);
}
// with flush()
foreach ( range(1, 5) as $num ) {
echo "Beep $num<br>";
flush();
sleep(1);
}
You can close the connection within a registered function within register_shutdown_function if you do not need to wait for the processing to be over to output content (i.e., if you do not need to output anything related to the outcome of the processing you wish to do).
See : http://www.php.net/manual/en/features.connection-handling.php#93441
The reason to put it in a register_shutdown_function is that even if the client aborts the connection, the processing will keep on going to the very end.