Suppose my server has 4GB of ram and i uploaded a file having size 5GB. How can i download that file using gridfs. Following site states that http://www.php.net/manual/en/mongogridfsfile.getbytes.php
If your file is bigger than memory than its a problem but doesn't tells a solution for that.
Can anyone have any solution for this.
i use this demo code to access a file.
<?php
// Connect to Mongo and set DB and Collection
$mongo = new Mongo();
$db = $mongo->myfiles;
// GridFS
$gridFS = $db->getGridFS();
// Find image to stream
$file = $gridFS->findOne("win.tar");
// Stream image to browser
header("Content-Description: File Transfer");
header("Content-Type: application/octet-stream");
header("Content-Disposition: attachment; filename=\"win.tar\"");
echo $file->getBytes();
?>
As of version 1.3.0 of the PHP Driver you can access the GridFS Files as a PHP stream, using $MongoGridFSFile->getResource().
Using that method you can iteratively read the data and print it out, avoiding the memory limitation on your server.
just split a source file by chunk and save meta information about each of these chunks in mongodb, each of your chunk will be ordinary file in gridfs
after that, you have a meta layer with meta data about the source file
also you must solved problems reverse downloaded file from gridfs and compound source file from chunks
size of this chunk you may select based on your network speed and width limitation, this chunks and chunk in gridfs is different
Related
I have a PHP site that currently pulls images off an Azure Blob, writes it to disk using file_put_contents, then imagick reads the file from the disk using readImageFile. I would rather this live in memory than be wrote to disk, then read from disk. How can I accomplish this? When I try to ReadImageBlob, I get the below error:
Warning: Imagick::readimageblob() expects parameter 1 to be string, resource given in <file> <line>
Below is a snippet of my code (This is just testing code, not production):
// Get Data from Azure Storage Blob
$blob = $blobClient->getBlob($containerName, $documentPath);
// Get TIF file from Blob and convert to PDF
$im = new imagick();
$im->readImageBlob($blob->getContentStream());
$im->setImageFormat('pdf');
// Echo as PDF
header('Content-Type: application/pdf');
echo $im;
You should be able to use stream_get_contents to read the string from the stream that you have. Example:
$im->readImageBlob(stream_get_contents($blob->getContentStream()));
First post. I'm working on a project for a client where they have pdf files uploaded to a file structure (LAMP Stack) but the files have no extensions on them. Under the assumption that those files have to be PDF how would I get the browsers to understand that, and open them accordingly? Obviously with adding the file extensions this would suddenly work but I can't change the way their system works, it would result in too many changes and they are on a tight deadline. As for saving a temporary copy somewhere, I could do that, but I was hoping for a better solution. Is there a way to suggest to the browsers that they open a file a certain way?
Any thoughts guys/gals?
You just set the application type and file name in the headers, like so:
// This points to the file in question, note that it doesn't
// care whether it has an extension on the name or not.
$filePathOnDisk = '/path/to/your/pdffile';
// You can make this whatever you like, it doesn't have to
// be the same as the file name on the disk! This is the name of the file your end
// user will see when they are asked if they want to save. open, etc in the browser.
$fileName = 'file.pdf';
$data = file_get_contents($filePathOnDisk);
header("Content-type: application/pdf");
header("Content-disposition: attachment;filename=$fileName");
echo $data;
See PHP: stream remote pdf to client browser and Proper MIME media type for PDF files for reference as well.
Tested
You can use the following which will prompt the user to save the (PDF) file on their computer.
Notice the different file names.
One is the file that will be uploaded/prompted to the user download_example.pdf, while the other is the file without an extension as set in readfile('example');
<?php
header('Content-type: application/pdf');
header('Content-Disposition: attachment; filename="download_example.pdf"');
readfile('example');
?>
I have been trying to read an audio file from mongoDB which i have stored using GridFS. I could download the file in the system and play from it but I wanted to stream those audio/video files from the DB itself and play it in the browser. Is there anyway to do that without downloading the file to the system? Any help would be good.
The PHP GridFS support has a MongoGridFSFile::getResource() function that allows you to get the stream as a resource - which doesn't load the whole file in memory. Combined with fread/echo or stream_copy_to_stream you can prevent the whole file from being loaded in memory. With stream_copy_to_stream, you can simply copy the GridFSFile stream's resource to the STDOUT stream:
<?php
$m = new MongoClient;
$images = $m->my_db->getGridFS('images');
$image = $images->findOne('mongo.png');
header('Content-type: image/png;');
$stream = $image->getResource();
stream_copy_to_stream( $stream, STDOUT );
?>
Alternatively, you can use fseek() on the returned $stream resource to only send back parts of the stream to the client. Combined with HTTP Range requests, you can do this pretty efficiently.
If the other recipe fails, for example with NginX and php-fpm, because STDOUT is not available in fpm, you can use
fpassthru($stream);
instead of
stream_copy_to_stream( $stream, STDOUT );
So a complete solution looks like:
function img($nr)
{
$mongo = new MongoClient();
$img = $mongo->ai->getGridFS('img')->findOne(array('metadata.nr'=>$nr));
if (!$img)
err("not found");
header('X-Accel-Buffering: no');
header("Content-type: ".$img->file["contentType"]);
header("Content-length: ".$img->getSize());
fpassthru($img->getResource());
exit(0);
}
FYI:
In this example:
File is not acccessed by the filename, instead it is accessed by a number stored in the metadata. Hint: You can set an unique index to ensure, that no number can be used twice.
Content-Type is read from GridFS, too, so you do not need to hardcode this.
NginX caching is switched off to enable streaming.
This way you can even handle other things like video or html pages. If you want to enable NginX caching, perhaps only output X-Accel-Buffering on bigger sizes.
I want to allow the user to download a file up to 1GB in size, but according to my code only a file of 113MB can be downloaded...
header('Content-type: application/zip');
//open/save dialog box
header('Content-Disposition: attachment; filename="check.zip"');
//read from server and write to buffer
readfile('check.zip');
Can anyone tell me how to download a larger file?
I'm going to guess from what you've said that you're getting an "out of memory" error.
In that case, perhaps this note from the documentation might be of interest:
Note:
readfile() will not present any memory issues, even when sending large files, on its own. If you encounter an out of memory error ensure that output buffering is off with ob_get_level().
So, check ob_get_level() and call ob_end_flush() if necessary to stop output buffering.
Alternatively, you could do something like this:
$f = fopen("check.zip","rb");
while(!feof($f)) {
echo fgets($f);
flush();
}
Another option is this:
header("Location: check.zip");
This will redirect the browser to the check.zip file. Since it's a download, the existing page won't be affected. You can even output the rest of a page to say something like "Your download will begin momentarily" to the user.
Either read and echo the file a chunk at a time, or use something like mod_sendfile to make it Not Your Problem.
Increase your file write buffer chunk size to maximum of file. That will decrease the utilization of resources and your download works fine.
Edit:
Use HTML5 webworkers to download large. Webworkers works in background so you can able to download large files.
I just recently asked and solved a question pertaining to uploading .PDF files that are greater than 2 MB into a MySQL database as BLOBS. I had to change some settings in my php.ini file and MySQLs maximum packet setting. However, fixing this issue has led me to discover a new issue with my script.
Now since I can upload files to my BLOB database I attempted to download the file for testing purposes. Much to my dismay when I went to open the .PDF file I received the following error: Failed to load document (error 3) 'file:///tmp/test-13.pdf'. Upon further investigation I found out that the file being downloaded, test.pdf, was only 1 MB, a little less than half of its supposed size in the database of a little more than 2 MB. This is obviously the reason for the error.
The following piece of code is the part of my script I am using for downloading files from the database. It is is at the very top of of script and works Flawlessly for files that are less than 1 MB.
foreach($_REQUEST as $key => $value)
{
if ($value == 'Open')
{
header();
session_start();
$dbh = new PDO('mysql:host='.$_SESSION['OpsDBServer'].'.ops.tns.its.psu.edu;
dbname='.$_SESSION['OpsDB'], $_SESSION['yoM'], $_SESSION['aMa']);
$id = $key;
$sqlDownload = "SELECT name, type, content, size FROM upload WHERE
id='".$id."'";
$result = $dbh->query($sqlDownload);
$download = $result->fetchAll();
$type = $download[0]['type'];
$size = $download[0]['size'];
$name = $download[0]['name'];
$content = $download[0]['content'];
header("Content-type: $type");
header("Content-Disposition: inline; filename=$name");
header("Content-length: $size");
header("Cache-Control: maxage=1");
header("Pragma: public");
echo $content;
exit;
}
}
I am thinking that maybe I have some header statements wrong? I am very confused about what to do. I have searched through php.ini and I have found no settings that I think need to changed and my maximum packet setting for MySQL is 4 MB so a 2 MB should download.
Thanks for any help.
According to (http://dev.mysql.com/doc/refman/5.0/en/blob.html):
The maximum size of
a BLOB or TEXT object is determined by
its type, but the largest value you
actually can transmit between the
client and server is determined by the
amount of available memory and the
size of the communications buffers.
You can change the message buffer size
by changing the value of the
max_allowed_packet variable, but you
must do so for both the server and
your client program.
According to (http://dev.mysql.com/doc/refman/5.0/en/server-parameters.html) the default value for max_allowed_packet is 1048576.
I actually fixed the issue. I changed all of the values that were recommended here in php.ini and my.cnf but I also needed to change a setting for PDO.
I changed:
PDO::MYSQL_ATTR_MAX_BUFFER_SIZE (integer)
Maximum buffer size. Defaults to 1 MiB.
This has to be set when the PDO object is created to work though. All is good now.