Is it possible to know how many bytes sent to the client browser using php? My pages are created dynamically, so the size isn't fixed.
Using php's output buffering
// start output buffering
ob_start();
// create your page
// once the page is ready, measure the size of the output buffer
$length = ob_get_length();
// and emit the page, stop buffering and flush the buffer
ob_get_flush();
As usual with php, these functions are pretty well documented in the standard documentation, don't forget to read the user contributed notes.
You can see this in your webserver's access log file.
But you can also code some php to get an answer like this:
ob_start();
echo "your content"
$data = ob_get_contents();
$size = strlen($data);
see also: Measure string size in Bytes in php
Is it possible, in PHP, to get all the generated HTML code, at the end of request processing?
What I want to achieve is to be able to retrieve (and, possibly, save/cache) the actual HTML that is about to be sent to users. I can do something similar in ASP.net with a Global.asax filter, that can access to low-level generated html code and modify/access it.
If needed, I can modify the web server settings and/or php interpreter settings (currently the web application runs on Apache+mod_php).
Use output buffering:
<?php
// Start buffering (no output delivered to the browser from now on)
ob_start();
// Generate the HTML
// ...
// Grab the buffer as a variable
$html_output = ob_get_contents();
// If you want to stop buffering and send the buffer to the browser
ob_end_flush();
// OR if you want to stop buffering and throw away the buffer
ob_end_clean();
Potential issues
There is a potential user impact as (depending on your web server) your page output is streamed to the user's browser as it's outputted (why you can start seeing really large pages before they've finished loading). But if you use the output buffer the user will only see the result after you've stopped buffering and outputted it.
Also, because you're buffering and not streaming your server will need to store what you're buffering which will use up additional memory (not a problem unless you're generating really large pages that exceed the memory limits of your PHP memory limit).
To avoid running out of memory you can chunk your buffering and write it to disc (or flush it to the user) at specific chunk sizes using a callback like this:
<?php
// The callback function each time we want to deal with a chunk of the buffer
$callback = function ($buffer, $flag) {
// Cache the next part of the buffer to file?
file_put_contents('page.cache', $buffer, FILE_APPEND & LOCK_EX);
// $flag contains which action is performing the callback.
// We could be ending due to the final flush and not because
// the buffer size limit was reached. PHP_OUTPUT_HANDLER_END
// means an ob_end_*() function has been called.
if ($flag == PHP_OUTPUT_HANDLER_END) {
// Do something different
}
// We could echo out this chunk if we want
echo $buffer;
// Whatever we return from this function is the new buffer
return '';
};
// Pass the buffer to $callback each time it reaches 1024 bytes
ob_start($callback, 1024)
// Generate the HTML
// ...
ob_end_clean();
I think what you would want to use is output buffering! At the start of your page use: ob_start();
At the end of the page you send to the client / browser using something like : ob_end_flush();
Before it is sent you can record that buffer to the db or text file
For some time (serveral nights..) I've been trying to get a time-expensive script to output simple dots so I know it's still processing the script. Basically it's a cronjob which is going to run nightly to update cache-keys in a memcache server.
No matter what I try I can't get PHP to output the current buffer. What I want is to send the echo'd dots while processing the script. What am I missing to get it to work? I've also tried the flush() function... and also to use ini_set("output_buffering", 1024);
At the moment this is my set up:
# clean all open buffers
while(ob_get_level() != 0)
{
ob_end_clean();
}
ob_start();
// Several loops, taking some minutes...
for( ..loopconditions ..){
echo ".";
ob_flush();
}
ob_end_clean()
Are you debugging this in a browser? If so, it may not be a problem with the output buffering at all.
Every browser has a buffer of it's own and ultimately it decides when to start flushing it.
Mentioned in the manual page for flush():
flush() ... has no effect on any client-side buffering in the browser.
Even the browser may buffer its input before displaying it. Netscape, for example, buffers text until it receives an end-of-line or the beginning of a tag, and it won't render tables until the tag of the outermost table is seen.
Some versions of Microsoft Internet Explorer will only start to display the page after they have received 256 bytes of output, so you may need to send extra whitespace before flushing to get those browsers to display the page.
IIRC, if you're using php-cgi, output buffering doesn't work.
In console, it is easier to use PEAR Console_ProgressBar. It will do it for you.
Example from the docs:
<?php
require_once 'Console/ProgressBar.php';
$bar = new Console_ProgressBar('[%bar%] %percent%', '=>', ' ', 80, 7);
//do some processing here
for ($i = 0; $i <= 7; $i++) {
$bar->update($i);
sleep(1);
}
echo "\n";
?>
[=======================================> ] 57.14%
More examples in:
http://pear.php.net/manual/en/package.console.console-progressbar.php#example-124
http://pear.php.net/manual/en/package.console.console-progressbar.php#example-125
I want to copy the file http://searchr.us/Testing/web-search.phtml?search=SEARCHED+TEXT to
http://searchr.us/Testing/search/SEARCHED+TEXT.html
How do i do this?
NOTE:The source of http://searchr.us/Testing/search/SEARCHED+TEXT.html should be the same as http://searchr.us/Testing/web-search.phtml?search=SEARCHED+TEXT
Indirectly I'm just saving a query so that I can keep a track of them!
Hope i've get you right: you want to store output of any query into file. You can manage this with output buffering.
In the begining of script wright:
ob_start(); // start buffering
This turns buffering on.
At the eng of page wright something like this ($query is text of query):
$html = ob_get_contents();
ob_end_flush(); // this turns off buffering and sends buffered content to output
$fp = fopen("{$query}.html","w");
fwrite($fp,$html);
fclose($fp);
I have quite a long data mining script, and in parts of it I echo some information to the page (during a foreach loop, actually.)
However I am noticing that the information is being sent to the browse not immediately as I had hoped, but in 'segments'.
Is there some function I can use after my echo to send all the data to the browser immediately?
Thanks.
You probably want flush(). However, PHP may be using output buffering. There are a few ways that this can change things, but in a nutshell, you can flush(), then ob_flush().
You can try using flush() after each echo, but even that won't guarantee a write to the client depending on the web server you're running.
Yes, padding your output to 1024 bytes will cause most browsers to start displaying the content.
But we also learn from #nobody's answer to question "How to flush output after each `echo` call?" that the 1024 bytes browser buffering effect only happens when the browser has to guess the character encoding of the page, which can be prevented by sending the proper Content-Type header (eg. "Content-Type: text/html; charset=utf-8"), or by specifying the content charset through appropriate html meta tags. And it worked as well for me in all browsers.
So basically, all one need to do is:
header('Content-Type: text/html; charset=utf-8');
ob_implicit_flush(true);
With no requirement for extra padding or flushing, which is of great cosmetic benefit for the code! Of course, headers have to be sent before any content, and one also has to make sure no output buffering is going on.
Problem definitely solved for me! Please (+1) #nobody's answer on the other question as well if it works for you. If, although, one still encounters problems, I suggest checking out the answers to that other question for other specific situations that might presumely prevent implicit flushing from working correctly.
Note also that some browsers won't start displaying anything until the body of the response contains a certain amount of data - like 256 or 1024 bytes. I have seen applications before that pad data with a 1024 character long comment near the top of the page, before they do a flush. It's a bit of a hack, but necessary.
This applies to Internet Explorer and Safari IIRC.
So,
If it is the first flush, make sure you have output at least 1024 bytes sofar (not including HTTP headers).
Call flush()
If you can determine that there is output buffering in place, issue ob_flush()
I like to just use
while (ob_get_level()) ob_end_flush();
near the start of my script somewhere, and then just
flush();
whenever I want to flush. This assumes that you don't want any output buffering at all, even if it was set up before your script (such as in a PHP.ini or htaccess configuration).
You should be able to use something like this to force output to be sent immeadiately. Put it at the part of the code you want the output to be sent.
flush();
ob_flush();
Phew! I finally found the answer to Google Chrome's buffer issue! Thanks to boysmakesh for the push in the right direction. Here's the function I use:
function buffer_flush(){
echo str_pad('', 512);
echo '<!-- -->';
if(ob_get_length()){
#ob_flush();
#flush();
#ob_end_flush();
}
#ob_start();
}
And this is how I call it:
show_view('global', 'header'); // Echos the <html><head>... tags and
// includes JS and CSS.
show_view('global', 'splash_screen'); // Shows a loading image telling
// the user that everything's okay.
buffer_flush(); // Pretty obvious. At this point the loading view shows
// up on every browser i've tested (chrome, firefox,
// IE 7 & 8)
show_view('global', 'main'); // Has a loop that echos "Test $i<br>" 5
// times and calls buffer_flush() each time.
show_view('global', 'footer'); // End the html page and use JQuery to
// fade out the loading view.
To perfectly work this out in Google chrome,
try this:
$i = 0;
$padstr = str_pad("",512," ");
echo $padstr;
while ($i <= 4){
$padstr = str_pad("",512," ");
echo $padstr;
echo "boysmakesh <BR> ";
flush();
sleep(2);
$i = $i + 1;
}
Ee are sending 512 bytes before sending EACH echo. Don't forget to put <BR> at the end of content before you flush. Else it won't work in Chrome but works in IE.
The data we padding is browser dependent. For some browsers it's enough to have 256 bytes but some need 1024 bytes. For chrome it is 512.
ignore_user_abort(TRUE); // run script in background
set_time_limit(0); // run script forever
$interval=150000;
$i = 0;
if(
strpos($_SERVER["HTTP_USER_AGENT"], "Gecko") or
strpos($_SERVER["HTTP_USER_AGENT"], "WebKit")
){
# important to change browser into quirks mode
echo '<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">';
}
function buffer_flush(){
echo "\n\n<!-- Deal with browser-related buffering by sending some incompressible strings -->\n\n";
for ( $i = 0; $i < 5; $i++ )
echo "<!-- abcdefghijklmnopqrstuvwxyz1234567890aabbccddeeffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz11223344556677889900abacbcbdcdcededfefegfgfhghgihihjijikjkjlklkmlmlnmnmononpopoqpqprqrqsrsrtstsubcbcdcdedefefgfabcadefbghicjkldmnoepqrfstugvwxhyz1i234j567k890laabmbccnddeoeffpgghqhiirjjksklltmmnunoovppqwqrrxsstytuuzvvw0wxx1yyz2z113223434455666777889890091abc2def3ghi4jkl5mno6pqr7stu8vwx9yz11aab2bcc3dd4ee5ff6gg7hh8ii9j0jk1kl2lmm3nnoo4p5pq6qrr7ss8tt9uuvv0wwx1x2yyzz13aba4cbcb5dcdc6dedfef8egf9gfh0ghg1ihi2hji3jik4jkj5lkl6kml7mln8mnm9ono -->\n\n";
while ( ob_get_level() )
ob_end_flush();
if(ob_get_length()){
#ob_flush();
#flush();
#ob_end_flush();
}
#ob_start();
}
ob_start();
do{
if($i<10){
buffer_flush();
echo ". ";
buffer_flush();
usleep($interval);
} else {
echo sprintf("<pre>%s</pre>", print_r($_SERVER,true));
break;
}
$i++;
}while(true);
Running php 5.5 on IIS 7, IE 11 (win server) I found this worked as the opening lines of the file. Note putting the while statement before the header caused a header already written error.
header('Content-Type: text/html; charset=utf-8');
while (ob_get_level()) ob_end_flush();
ob_implicit_flush(true);
Further references to ob_flush() in the script caused a buffer does not exist error.
This worked fine when I was processing a file and sending sql statements to the browser, however when I hooked up the db (ms server 2008) I had no input returned till the script had completed.
this combination finally worked for me, based on thomasrutter's answer
while (ob_get_level()) ob_end_flush();
ob_implicit_flush(true);