Stream very large Blob from MySQL to PHP and create a file - php

Question

We have a MySQL database that has some very large files stored in blob fields, such as some videos that are over 700MB. File sizes range from .5 MB PDFs to JPEGS, etc...
I'm trying to use PHP to retrieve these columns and create a file on the server that will be later offered up as a download.
I'm currently using the following method:
$dsn = "mysql:host=$host;dbname=$db;charset=$charset";
$options = [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
PDO::ATTR_EMULATE_PREPARES => false,
PDO::MYSQL_ATTR_MAX_BUFFER_SIZE => 1024*1024*500
];
try {
$pdo = new PDO($dsn, $user, $pass, $options);
} catch (\PDOException $e) {
throw new \PDOException($e->getMessage(), (int)$e->getCode());
}
$stmt = $pdo->prepare('SELECT a.TITLE, a.ATTVERSION, a.ATTACHMENTID, a.CONTENTTYPE, a.FILESIZE, d.DATA FROM ATTACHMENTS a
LEFT JOIN ATTACHMENTDATA d ON d.ATTACHMENTID=a.ATTACHMENTID
WHERE a.ATTACHMENTID= ?');
$stmt->execute([$fileid]);
$file = $stmt->fetch();
file_put_contents($storage_dir . "/" . $filename, $file['DATA']);
This works for smaller files (note I'm setting the buffer size to 500MB), but larger files get truncated and corrupted.
I next tried the LOB and unbuffered query approach:
$dsn = "mysql:host=$host;dbname=$db;charset=$charset";
$options = [
PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION,
PDO::ATTR_EMULATE_PREPARES => false,
PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false
];
try {
$pdo = new PDO($dsn, $user, $pass, $options);
} catch (\PDOException $e) {
throw new \PDOException($e->getMessage(), (int)$e->getCode());
}
$stmt = $pdo->prepare('SELECT a.TITLE, a.ATTVERSION, a.ATTACHMENTID, a.CONTENTTYPE, a.FILESIZE, d.DATA FROM ATTACHMENTS a
LEFT JOIN ATTACHMENTDATA d ON d.ATTACHMENTID=a.ATTACHMENTID
WHERE a.ATTACHMENTID= ?');
$stmt->execute([$fileid]);
$stmt->bindColumn(1, $title, PDO::PARAM_STR, 256 );
$stmt->bindColumn(2, $attversion, PDO::PARAM_INT);
$stmt->bindColumn(3, $attid, PDO::PARAM_INT);
$stmt->bindColumn(4, $contenttype, PDO::PARAM_STR, 256);
$stmt->bindColumn(5, $filesize, PDO::PARAM_INT);
$stmt->bindColumn(6, $data, PDO::PARAM_LOB);
$stmt->fetch(PDO::FETCH_BOUND);
file_put_contents($storage_dir . "/" . $filename, $data)
With this option, I only get 1MB files, and it seems to be ignoring the MYSQL_ATTR_USE_BUFFERED_QUERY => false and since I'm not setting a buffer size, it's defaulting to 1MB. Can anyone offer any advice or see anything glaring? MySQL is on a different server, but I'd be open to doing this another way as well and calling a bash script or something via PHP.

Answer 1

This was initially done on php 5.4.16, which seemed to not support the unbuffered stream. After upgrading to php 8.0.11, I now have this working, with the exception of increasing PHP memory using ini_set.