For some time (serveral nights..) I've been trying to get a time-expensive script to output simple dots so I know it's still processing the script. Basically it's a cronjob which is going to run nightly to update cache-keys in a memcache server.
No matter what I try I can't get PHP to output the current buffer. What I want is to send the echo'd dots while processing the script. What am I missing to get it to work? I've also tried the flush() function... and also to use ini_set("output_buffering", 1024);
At the moment this is my set up:
# clean all open buffers
while(ob_get_level() != 0)
{
ob_end_clean();
}
ob_start();
// Several loops, taking some minutes...
for( ..loopconditions ..){
echo ".";
ob_flush();
}
ob_end_clean()
Are you debugging this in a browser? If so, it may not be a problem with the output buffering at all.
Every browser has a buffer of it's own and ultimately it decides when to start flushing it.
Mentioned in the manual page for flush():
flush() ... has no effect on any client-side buffering in the browser.
Even the browser may buffer its input before displaying it. Netscape, for example, buffers text until it receives an end-of-line or the beginning of a tag, and it won't render tables until the tag of the outermost table is seen.
Some versions of Microsoft Internet Explorer will only start to display the page after they have received 256 bytes of output, so you may need to send extra whitespace before flushing to get those browsers to display the page.
IIRC, if you're using php-cgi, output buffering doesn't work.
In console, it is easier to use PEAR Console_ProgressBar. It will do it for you.
Example from the docs:
<?php
require_once 'Console/ProgressBar.php';
$bar = new Console_ProgressBar('[%bar%] %percent%', '=>', ' ', 80, 7);
//do some processing here
for ($i = 0; $i <= 7; $i++) {
$bar->update($i);
sleep(1);
}
echo "\n";
?>
[=======================================> ] 57.14%
More examples in:
http://pear.php.net/manual/en/package.console.console-progressbar.php#example-124
http://pear.php.net/manual/en/package.console.console-progressbar.php#example-125
Related
Is it possible, in PHP, to get all the generated HTML code, at the end of request processing?
What I want to achieve is to be able to retrieve (and, possibly, save/cache) the actual HTML that is about to be sent to users. I can do something similar in ASP.net with a Global.asax filter, that can access to low-level generated html code and modify/access it.
If needed, I can modify the web server settings and/or php interpreter settings (currently the web application runs on Apache+mod_php).
Use output buffering:
<?php
// Start buffering (no output delivered to the browser from now on)
ob_start();
// Generate the HTML
// ...
// Grab the buffer as a variable
$html_output = ob_get_contents();
// If you want to stop buffering and send the buffer to the browser
ob_end_flush();
// OR if you want to stop buffering and throw away the buffer
ob_end_clean();
Potential issues
There is a potential user impact as (depending on your web server) your page output is streamed to the user's browser as it's outputted (why you can start seeing really large pages before they've finished loading). But if you use the output buffer the user will only see the result after you've stopped buffering and outputted it.
Also, because you're buffering and not streaming your server will need to store what you're buffering which will use up additional memory (not a problem unless you're generating really large pages that exceed the memory limits of your PHP memory limit).
To avoid running out of memory you can chunk your buffering and write it to disc (or flush it to the user) at specific chunk sizes using a callback like this:
<?php
// The callback function each time we want to deal with a chunk of the buffer
$callback = function ($buffer, $flag) {
// Cache the next part of the buffer to file?
file_put_contents('page.cache', $buffer, FILE_APPEND & LOCK_EX);
// $flag contains which action is performing the callback.
// We could be ending due to the final flush and not because
// the buffer size limit was reached. PHP_OUTPUT_HANDLER_END
// means an ob_end_*() function has been called.
if ($flag == PHP_OUTPUT_HANDLER_END) {
// Do something different
}
// We could echo out this chunk if we want
echo $buffer;
// Whatever we return from this function is the new buffer
return '';
};
// Pass the buffer to $callback each time it reaches 1024 bytes
ob_start($callback, 1024)
// Generate the HTML
// ...
ob_end_clean();
I think what you would want to use is output buffering! At the start of your page use: ob_start();
At the end of the page you send to the client / browser using something like : ob_end_flush();
Before it is sent you can record that buffer to the db or text file
I have a PHP script which takes a while to process and I wanted to get the site to display it's progress while it is running.
I have been having problems displaying the contents of the output buffer while a PHP code is running. I have been trying the use of ob_implicit_flush(true), ob_end_flush() and ob_flush() with no success.
After looking for a fair bit at the PHP manual on php.net I have come across a comment by Lee saying
As of August 2012, all browsers seem to show an all-or-nothing
approach to buffering. In other words, while php is operating, no
content can be shown.
In particular this means that the following workarounds listed further
down here are ineffective:
1) ob_flush (), flush () in any combination with other output
buffering functions;
2) changes to php.ini involving setting output_buffer and/or
zlib.output_compression to 0 or Off;
3) setting Apache variables such as "no-gzip" either through
apache_setenv () or through entries in .htaccess.
So, until browsers begin to show buffered content again, the tips
listed here are moot.
see here
If he is correct, is there an alternative I can use to display the progress of the script running?
[edit]
Looks like Lee could be right. I have been doing some testing and here is an example of the problem
<?php
ob_implicit_flush(true);
ob_end_flush();
for ($i=0; $i<5; $i++) {
echo $i.'<br>';
sleep(1);
}
?>
Should display 0 to 4 with a second in between. What actually happens is that the site waits 5 seconds then displays 0 to 4 immediately. It is waiting for the script to complete then displays everything in one go.
I have had output_buffering = Off in my php.ini file and tried setting it to 1 and 4096 (4096 being 4KB) and it still does the same.
Check the buffer size and make an "empty" dummy of that size, for example...
$flushdummy="";
for ($i=0; $i<700; $i++) { $flushdummy=$flushdummy." "; }
$flushdummy=$flushdummy."\n";
then, when output is desired, add the dummy to the output, plus the flush:
echo "here I am<br />", $flushdummy;
flush();
#ob_flush();
I can't for the life of me get the PHP flush function to work properly, using WAMP. Here is some sample code, commented out are all of the different things I've tried:
//apache_setenv('no-gzip', 1); // returns error that apache_setenv does not exist
//ini_set('zlib.output_compression',0);
//ini_set('implicit_flush',1);
//ob_end_clean();
//for ($i = 0; $i < ob_get_level(); $i++) { ob_end_flush(); }
//ob_implicit_flush(1);
set_time_limit(0);
echo "<pre>";
for ($i = 0; $i < 100; ++$i) {
echo $i.' '.time().str_repeat(' ',256)."\n";
//ob_flush(); // returns error without output buffering enabled
flush();
usleep(100000);
}
It seems no matter what I do, I always get the results all together in one giant chunk.
Edit:
I've uploaded the same exact code to a server running on cPanel/linux, and it works perfect in all browsers. Why can't I get it to work properly on a localhost WAMP server??
flush() may not be able to override the buffering scheme of your web server and it has no effect on any client-side buffering in the browser. It also doesn't affect PHP's userspace output buffering mechanism. This means you will have to call both ob_flush() and flush() to flush the ob output buffers if you are using those.
Several servers, especially on Win32, will still buffer the output from your script until it terminates before transmitting the results to the browser.
Server modules for Apache like mod_gzip may do buffering of their own that will cause flush() to not result in data being sent immediately to the client.
Even the browser may buffer its input before displaying it. Netscape, for example, buffers text until it receives an end-of-line or the beginning of a tag, and it won't render tables until the tag of the outermost table is seen.
Some versions of Microsoft Internet Explorer will only start to display the page after they have received 256 bytes of output, so you may need to send extra whitespace before flushing to get those browsers to display the page.
php.net
Try using ob_flush() before flush();
I noticed that the php flush(); doesn't work in Firefox 4 beta 7, as it works in 3.6.12.
I recently installed firefox 4 beta 7, and the contents are not being flush immediately when flush() is called. It used to work fine in 3.6.12. Is there any thing else that could provide me with the flushing functionality.
I've tried
flush();
#ob_flush();
I also tried the following code at the top of the page.
#apache_setenv('no-gzip', 1);
#ini_set('zlib.output_compression', 0);
#ini_set('implicit_flush', 1);
for ($i = 0; $i < ob_get_level(); $i++) { ob_end_flush(); }
ob_implicit_flush(1);
By the way, I use php on XAMPP/Apache. Thanks.
I found that setting content type to text/plain works, but it just outputs plain text and not html content.
You're not seeing ghosts - I've experienced the same difference between FF3.6 and FF4.
Here's a work around: add an
echo str_repeat(" ", 1024);
before the output that needs to be flushed. You can put it for example in the <head>.
My theory is that FF4, like apparently IE and Safari, have a small buffer that needs to be filled before incremental rendering kicks in.
flush will function identically server-side regardless of the browser. If the client is displaying things differently, there's not a lot you can do server-side to fix it.
I have quite a long data mining script, and in parts of it I echo some information to the page (during a foreach loop, actually.)
However I am noticing that the information is being sent to the browse not immediately as I had hoped, but in 'segments'.
Is there some function I can use after my echo to send all the data to the browser immediately?
Thanks.
You probably want flush(). However, PHP may be using output buffering. There are a few ways that this can change things, but in a nutshell, you can flush(), then ob_flush().
You can try using flush() after each echo, but even that won't guarantee a write to the client depending on the web server you're running.
Yes, padding your output to 1024 bytes will cause most browsers to start displaying the content.
But we also learn from #nobody's answer to question "How to flush output after each `echo` call?" that the 1024 bytes browser buffering effect only happens when the browser has to guess the character encoding of the page, which can be prevented by sending the proper Content-Type header (eg. "Content-Type: text/html; charset=utf-8"), or by specifying the content charset through appropriate html meta tags. And it worked as well for me in all browsers.
So basically, all one need to do is:
header('Content-Type: text/html; charset=utf-8');
ob_implicit_flush(true);
With no requirement for extra padding or flushing, which is of great cosmetic benefit for the code! Of course, headers have to be sent before any content, and one also has to make sure no output buffering is going on.
Problem definitely solved for me! Please (+1) #nobody's answer on the other question as well if it works for you. If, although, one still encounters problems, I suggest checking out the answers to that other question for other specific situations that might presumely prevent implicit flushing from working correctly.
Note also that some browsers won't start displaying anything until the body of the response contains a certain amount of data - like 256 or 1024 bytes. I have seen applications before that pad data with a 1024 character long comment near the top of the page, before they do a flush. It's a bit of a hack, but necessary.
This applies to Internet Explorer and Safari IIRC.
So,
If it is the first flush, make sure you have output at least 1024 bytes sofar (not including HTTP headers).
Call flush()
If you can determine that there is output buffering in place, issue ob_flush()
I like to just use
while (ob_get_level()) ob_end_flush();
near the start of my script somewhere, and then just
flush();
whenever I want to flush. This assumes that you don't want any output buffering at all, even if it was set up before your script (such as in a PHP.ini or htaccess configuration).
You should be able to use something like this to force output to be sent immeadiately. Put it at the part of the code you want the output to be sent.
flush();
ob_flush();
Phew! I finally found the answer to Google Chrome's buffer issue! Thanks to boysmakesh for the push in the right direction. Here's the function I use:
function buffer_flush(){
echo str_pad('', 512);
echo '<!-- -->';
if(ob_get_length()){
#ob_flush();
#flush();
#ob_end_flush();
}
#ob_start();
}
And this is how I call it:
show_view('global', 'header'); // Echos the <html><head>... tags and
// includes JS and CSS.
show_view('global', 'splash_screen'); // Shows a loading image telling
// the user that everything's okay.
buffer_flush(); // Pretty obvious. At this point the loading view shows
// up on every browser i've tested (chrome, firefox,
// IE 7 & 8)
show_view('global', 'main'); // Has a loop that echos "Test $i<br>" 5
// times and calls buffer_flush() each time.
show_view('global', 'footer'); // End the html page and use JQuery to
// fade out the loading view.
To perfectly work this out in Google chrome,
try this:
$i = 0;
$padstr = str_pad("",512," ");
echo $padstr;
while ($i <= 4){
$padstr = str_pad("",512," ");
echo $padstr;
echo "boysmakesh <BR> ";
flush();
sleep(2);
$i = $i + 1;
}
Ee are sending 512 bytes before sending EACH echo. Don't forget to put <BR> at the end of content before you flush. Else it won't work in Chrome but works in IE.
The data we padding is browser dependent. For some browsers it's enough to have 256 bytes but some need 1024 bytes. For chrome it is 512.
ignore_user_abort(TRUE); // run script in background
set_time_limit(0); // run script forever
$interval=150000;
$i = 0;
if(
strpos($_SERVER["HTTP_USER_AGENT"], "Gecko") or
strpos($_SERVER["HTTP_USER_AGENT"], "WebKit")
){
# important to change browser into quirks mode
echo '<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">';
}
function buffer_flush(){
echo "\n\n<!-- Deal with browser-related buffering by sending some incompressible strings -->\n\n";
for ( $i = 0; $i < 5; $i++ )
echo "<!-- abcdefghijklmnopqrstuvwxyz1234567890aabbccddeeffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz11223344556677889900abacbcbdcdcededfefegfgfhghgihihjijikjkjlklkmlmlnmnmononpopoqpqprqrqsrsrtstsubcbcdcdedefefgfabcadefbghicjkldmnoepqrfstugvwxhyz1i234j567k890laabmbccnddeoeffpgghqhiirjjksklltmmnunoovppqwqrrxsstytuuzvvw0wxx1yyz2z113223434455666777889890091abc2def3ghi4jkl5mno6pqr7stu8vwx9yz11aab2bcc3dd4ee5ff6gg7hh8ii9j0jk1kl2lmm3nnoo4p5pq6qrr7ss8tt9uuvv0wwx1x2yyzz13aba4cbcb5dcdc6dedfef8egf9gfh0ghg1ihi2hji3jik4jkj5lkl6kml7mln8mnm9ono -->\n\n";
while ( ob_get_level() )
ob_end_flush();
if(ob_get_length()){
#ob_flush();
#flush();
#ob_end_flush();
}
#ob_start();
}
ob_start();
do{
if($i<10){
buffer_flush();
echo ". ";
buffer_flush();
usleep($interval);
} else {
echo sprintf("<pre>%s</pre>", print_r($_SERVER,true));
break;
}
$i++;
}while(true);
Running php 5.5 on IIS 7, IE 11 (win server) I found this worked as the opening lines of the file. Note putting the while statement before the header caused a header already written error.
header('Content-Type: text/html; charset=utf-8');
while (ob_get_level()) ob_end_flush();
ob_implicit_flush(true);
Further references to ob_flush() in the script caused a buffer does not exist error.
This worked fine when I was processing a file and sending sql statements to the browser, however when I hooked up the db (ms server 2008) I had no input returned till the script had completed.
this combination finally worked for me, based on thomasrutter's answer
while (ob_get_level()) ob_end_flush();
ob_implicit_flush(true);