Disabling Output Buffer with Apache and PHP-FPM via mod_proxy - php

Question

Outputting content as soon as PHP generates it is fine when using Apache with PHP as a module as you can simply disable output_buffering in PHP and use flush() or implicit_flush(1). This is what I previously used and it worked fine.
I'm running into an issue since having switched to PHP-FPM wherein I cannot get Apache (2.4) to output PHP's content until the entire script has completed. I still have output_buffering off and flushing in place but that's not enough. Apache isn't using mod_gzip (and that would have affected both the PHP module as well anyway).
Nginx has an option to disable proxy_buffering which, from reading other people's comments fixes this, but I cannot find any way of doing this in Apache.
Here's how PHP is currently being called within Apache:
<FilesMatch \.php$>
SetHandler "proxy:unix:/run/php-fpm/php-fpm.sock|fcgi://localhost/"
</FilesMatch>
<Proxy fcgi://localhost/ enablereuse=on retry=0 timeout=7200 max=500 flushpackets=on>
</Proxy>
The Apache documentation mentions flushpackets (used above) which appears to be what is needed, but then it also goes on to say that it only applies to AJS for now, not all proxied content so it won't do anything in this case.
Echoing enough whitespace to fill the buffer may work, but it's a messy workaround which is far from ideal.
In short: Does anyone know the correct way of having Apache send PHP content as soon as it's echo'd rather than waiting until script completion?

Answer 1

I successfully disabled output buffering by rewriting your Proxy section (based on this answer):
<FilesMatch \.php$>
SetHandler "proxy:unix:/run/php-fpm/php-fpm.sock|fcgi://localhost"
</FilesMatch>
<Proxy fcgi://localhost>
ProxySet enablereuse=on flushpackets=on
</Proxy>

Answer 2

Reposting the answer I just posted to a very similar question here: How to disable buffering with apache2 and mod_proxy_fcgi?
A few notes, since I just spent the past few hours experimenting to find the answer to this question:
It's not possible to entirely disable output buffering when using mod_proxy/mod_proxy_fcgi, however, you can still have responses streamed in chunks.
It seems, based on my experimentation, that chunks have to be at least 4096 bytes before the output will be flushed to the browser.
You can disable output buffering with the mod_fastcgi or mod_fcgi module, but those mods aren't as popular/widely used with Apache 2.4.
If you have mod_deflate enabled and don't set SetEnv no-gzip 1 for the virtualhost/directory/etc. that's streaming data, then gzip will not allow the buffer to flush until the request is complete.
I was testing things out to see how to best use Drupal 8's new BigPipe functionality for streaming requests to the client, and I posted some more notes in this GitHub issue.

Answer 3

In my environment (Apache 2.4, php-fpm) it worked when turning off compression and padding the output to output_buffering, see script:
header('Content-Encoding: none;');
$padSize = ini_get('output_buffering');
for($i=0;$i<10;$i++) {
echo str_pad("$i<br>", $padSize);
flush();
sleep(1);
}

Answer 4

https://www.php.net/manual/en/function.fastcgi-finish-request.php was what saved my sanity. I tried all kinds of hacks and techniques to get Apache and php-fpm (7.4) to display progress in a browser for a long-running process, including Server-Sent Events, writing progress to a text file and polling it with xhr, flush()ing like crazy, etc. Nothing worked until I did something like this (in my MVC action-controller)
public function longRunningProcessAction()
{
$path = \realpath('./data/progress.sqlite');
$db = new \PDO("sqlite:$path");
$db->setAttribute(\PDO::ATTR_ERRMODE, \PDO::ERRMODE_EXCEPTION);
$stmt = $db->prepare("UPDATE progress SET status = :status");
$stmt->execute([':status' => "starting"]);
header("content-type: application/json");
echo json_encode(['status'=>'started']);
// this here is critical ...
session_write_close();
fastcgi_finish_request();
// otherwise it will NOT work
for ($i = 0; $i <= 150; $i++) {
usleep(250*1000);
$stmt->execute([':status' => "$i of 150"]);
// this also works
file_put_contents('./data/progress.txt',"$i of 150");
}
$stmt->execute([':status' => "done"]);
}
// and...
public function progressAction()
{
$path = \realpath('./data/progress.sqlite');
$db = new \PDO("sqlite:$path");
$query = 'SELECT status FROM progress';
$stmt = $db->query($query);
// and this is working as well..
$text = file_get_contents('./data/progress.txt');
return new JsonModel(['status'=>$stmt->fetchColumn(),'text'=>$text]);
}
and then some Javascript (jQuery)
var check_progress = function() {
$.get("/my/job/progress").then(r=>{
$("#progress").text(r.status);
if (r.status === "done") { return; }
window.setTimeout(check_progress,300);
});
$.post("/long/running/process",data).then(check_progress);
Voilà!

Answer 5

A hack to make PHP FPM with Apache 2.4 mod_proxy work:
call ob_end_clean() at the beginning of your PHP script
call flush() at least 21 times to flush your output instead of calling it once; always send at least one character between calling flush()
using ob_end_clean() without ob_start() doesn't make sense to me, but it seems to help - and it returns true (=success!)