I am implementing S3 file uploads and downloads using pre-signed urls. I have one s3 bucket (versioning enabled) and one AWS user but I want to track the history of each file in terms of which of my application users modified the file.
I have a versioned S3 bucket and my thought is that I can append metadata to every file to identify my application user and possibly other data too.
Here is my code:
public function presignedUpload(Request $request)
{
$this->validate($request, [
'name' => 'string|required'
]);
$s3 = Storage::disk('s3');
$client = $s3->getDriver()->getAdapter()->getClient();
$expiry = "+10 minutes";
$options = ['user-data' => 'user-meta-value'];
$cmd = $client->getCommand('PutObject', [
'Bucket' => \Config::get('filesystems.disks.s3.bucket'),
'Key' => 'path/to/file/' . $request->name,
'ACL' => 'public-read',
], $options);
$request = $client->createPresignedRequest($cmd, $expiry);
$presignedUrl = (string)$request->getUri();
return response()->json(['url' => $presignedUrl], 201);
}
I deduced that I can pass an $options array from the FilesystemAdapter::class. This code does upload the file, but the metadata in AWS looks empty.
Is my $options array in the wrong format?
Appreciate any help on this.
The selected answer doesn't work anymore after Laravel 9. Flysystem recommends you to use S3 SDK instead of Storage driver.
https://github.com/thephpleague/flysystem/issues/1423
Use the S3 SDK instead to create a resigned upload URL.
$uploadDisk = 's3';
$s3Client = new S3Client([
'credentials' => [
'key' => config('filesystems.disks.' . $uploadDisk . '.key'),
'secret' => config('filesystems.disks.' . $uploadDisk . '.secret'),
],
'region' => config('filesystems.disks.' . $uploadDisk . '.region'),
'version' => 'latest',
]);
$s3Bucket = config('filesystems.disks.' . $uploadDisk . '.bucket');
$s3Key = \Illuminate\Support\Str::uuid(); // your file name
$s3Options = [];
$command = $s3Client->getCommand('PutObject', [
'Bucket' => $s3Bucket,
'Key' => $s3Key,
'MetaData' => $s3Options,
]);
$request = $s3Client->createPresignedRequest($command, '+20 minutes');
$url = (string) $request->getUri();
Metadata can be uploaded like this:
public function presignedUpload(Request $request)
{
$this->validate($request, [
'name' => 'string|required'
]);
$s3 = Storage::disk('s3');
$client = $s3->getDriver()->getAdapter()->getClient();
$expiry = "+10 minutes";
$options = ['user-data' => 'user-meta-value'];
$cmd = $client->getCommand('PutObject', [
'Bucket' => \Config::get('filesystems.disks.s3.bucket'),
'Key' => 'path/to/file/' . $request->name,
'ACL' => 'public-read',
'Metadata' => $options,
]);
$request = $client->createPresignedRequest($cmd, $expiry);
$presignedUrl = (string)$request->getUri();
return response()->json(['url' => $presignedUrl], 201);
}
Related
I can't figure it out what's going on with this. The requirement is to upload large files up to 25GB to storage on AWS S3. We have a laravel app running with no problems for a couple of years. So, I just add a controller to do the job with the MultipartUploader tool from AWS...
Files up to 64mb are uploading with no problem
Files larger than that, return a 500 0 error, and I find nothing on the log.
I've change the upload_max_filesize & post_max_size to 4G.
Here's the code...
$s3Client = new S3Client([
'region' => 'us-east-1',
'version' => 'latest',
'credentials' => $credentials
]);
$source = fopen( $file['fileName']->getRealPath() , 'r');
$storageClass = 'STANDARD_IA';
$chunkSize = 100 * 1024 * 1024; // 100MB
if(!isset($results['backup']['uploadId'])) {
$response = $s3Client->createMultipartUpload([
'Bucket' => $bucket,
'Key' => $path,
'StorageClass' => $storageClass,
'Tagging' => '',
'ServerSideEncryption' => 'AES256',
'ContentType' => $file['fileName']->getMimeType(),
]);
$results['backup']['uploadId'] = $response['UploadId'];
$results['backup']['partNumber'] = 1;
}
//Reading parts already uploaded
for($i = 1; $i < $results['backup']['partNumber']; $i++) {
set_time_limit(0);
if(!feof($source)) fread($source, $chunkSize);
}
// Uploading next parts
while(!feof($source)) {
do {
try {
set_time_limit(0);
$uploadSuccess = $s3Client->uploadPart([
'Bucket' => $bucket,
'Key' => $path,
'UploadId' => $results['backup']['uploadId'],
'PartNumber' => $results['backup']['partNumber'],
'Body' => fread($source, $chunkSize),
]);
$results['uploadFile ' . $key] = ['status' => 'success', 'result' => $fileName ];
} catch (MultipartUploadException $e) {
rewind($source);
$uploader = new MultipartUploader($s3Client, $source, [
'state' => $e->getState(),
]);
$results['uploadFile ' . $key] = ['status' => 'error', 'result' => $e->getMessage() . "\n" ];
}
} while (!isset($uploadSuccess));
$results['backup']['parts'][] = [
'PartNumber' => $results['backup']['partNumber'],
'ETag' => $uploadSuccess['ETag'],
];
$results['backup']['partNumber']++;
}
fclose($source);
$uploadSuccess = $s3Client->completeMultipartUpload([
'Bucket' => $bucket,
'Key' => $path,
'UploadId' => $results['backup']['uploadId'],
'MultipartUpload' => [
'Parts' => $results['backup']['parts'],
],
]);
unset($results['backup']);
return $results;
You can set Maximum allowed content length in Request Filtering rules.
The default maximum upload is 30mb, you can change it as needed.
I have managed to upload large files to s3 using multiPart Upload, but I can't download them again using the getObject function. Is there another way I can achieve this?
Here my code:
$keyname= 'key';
$bucket = 'bucketname';
$fileName = 'filename.txt';
$result = $s3->getObject([
'Bucket' => $bucket,
'Key' => $keyname
]);
var_dump($fileName);
$result['ContentDisposition'] = 'attachment; filename="'.$fileName.'"';
$result['fileName'] = $result['ContentDisposition'];
header("Content-Type: {$result['ContentType']}");
header("Content-Disposition: {$result['ContentDisposition']}");
header("Content-Length: {$result['ContentLength']}");
echo $result['Body'];
Thanks For the help. This is my solution:
$keyname= 'key';
$bucket = 'bucketname';
$fileName = 'filename.txt';
#create S3 Client
$s3 = new S3Client([
'version' => 'latest',
'region' => 'eu-central-1',
'credentials' => [
]
]);
$cmd = $s3->getCommand('GetObject', [
'Bucket' => $bucket,
'Key' => $keyname,
'ResponseContentDisposition' => 'attachment; filename="'.$fileName.'"'
]);
$request = $s3->createPresignedRequest($cmd, '+15 min');
$presignedUrl = (string)$request->getUri();
echo $presignedUrl;
after this, I download it in my frontend with an a tag via js
you can create a Presigned connection with S3 like this
$keyname= 'key';
$bucket = 'bucketname';
$fileName = 'filename.txt';
$command = $s3->getCommand('GetObject', array(
'Bucket' => $bucket,
'Key' => $keyname
'ResponseContentDisposition' => 'attachment; filename="'.$fileName.'"'
));
$signedUrl = $command->createPresignedUrl('+15 minutes');
header('Location: '.$signedUrl);
I try to upload an image to s3 using Laravel but I receive a runtime error. Using Laravel 5.8, PHP7 and API REST with Postman I send by body base64
I receive an image base64 and I must to upload to s3 and get the request URL.
public function store(Request $request)
{
$s3Client = new S3Client([
'region' => 'us-east-2',
'version' => 'latest',
'credentials' => [
'key' => $key,
'secret' => $secret
]
]);
$base64_str = substr($input['base64'], strpos($input['base64'], ",") + 1);
$image = base64_decode($base64_str);
$result = $s3Client->putObject([
'Bucket' => 's3-galgun',
'Key' => 'saraza.jpg',
'SourceFile' => $image
]);
return $this->sendResponse($result['ObjectURL'], 'message.', 'ObjectURL');
}
Says:
RuntimeException: Unable to open u�Z�f�{��zڱ��� .......
The SourceFile parameter is leading to the path of file to upload to S3, not the binary
You can use Body parameter to replace the SourceFile, or saving the file to local temporary and get the path for SourceFile
Like this:
public function store(Request $request)
{
$s3Client = new S3Client([
'region' => 'us-east-2',
'version' => 'latest',
'credentials' => [
'key' => $key,
'secret' => $secret
]
]);
$base64_str = substr($input['base64'], strpos($input['base64'], ",") + 1);
$image = base64_decode($base64_str);
Storage::disk('local')->put("/temp/saraza.jpg", $image);
$result = $s3Client->putObject([
'Bucket' => 's3-galgun',
'Key' => 'saraza.jpg',
'SourceFile' => Storage::disk('local')->path('/temp/saraza.jpg')
]);
Storage::delete('/temp/saraza.jpg');
return $this->sendResponse($result['ObjectURL'], 'message.', 'ObjectURL');
}
And, if you're using S3 with Laravel, you should consider the S3 filesystem driver instead of access the S3Client manually in your controller
To do this, add the S3 driver composer require league/flysystem-aws-s3-v3, put your S3 IAM settings in .env or config\filesystems.php
Then update the default filesystem in config\filesystems, or indicate the disk driver when using the Storage Storage::disk('s3')
Detail see document here
Instead of SourceFile you have to use Body. SourceFile is a path to a file, but you do not have a file, you have a base64 encoded source of img. That is why you need to use Body which can be a string. More here: https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3-2006-03-01.html#putobject
Fixed version:
public function store(Request $request)
{
$s3Client = new S3Client([
'region' => 'us-east-2',
'version' => 'latest',
'credentials' => [
'key' => $key,
'secret' => $secret
]
]);
$base64_str = substr($input['base64'], strpos($input['base64'], ",") + 1);
$image = base64_decode($base64_str);
$result = $s3Client->putObject([
'Bucket' => 's3-galgun',
'Key' => 'saraza.jpg',
'Body' => $image
]);
return $this->sendResponse($result['ObjectURL'], 'message.', 'ObjectURL');
}
A very simple way to uploads Any file in AWS-S3 Storage.
First, check your ENV setting.
AWS_ACCESS_KEY_ID=your key
AWS_SECRET_ACCESS_KEY= your access key
AWS_DEFAULT_REGION=ap-south-1
AWS_BUCKET=your bucket name
AWS_URL=Your URL
The second FileStorage.php
's3' => [
'driver' => 's3',
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'region' => env('AWS_DEFAULT_REGION'),
'bucket' => env('AWS_BUCKET'),
'url' => env('AWS_URL'),
//'visibility' => 'public', // do not use this line for security purpose. try to make bucket private.
],
Now come on main Code.
Upload Binary File from HTML Form.
$fileName = 'sh_'.mt_rand(11111,9999).".".$imageFile->clientExtension();;
$s3path = "/uploads/".$this::$SchoolCode."/";
Storage::disk('s3')->put($s3path, file_get_contents($req->file('userDoc')));
Upload Base64 File
For Public Bucket or if you want to keep file Public
$binary_data = base64_decode($file);
Storage::disk('s3')->put($s3Path, $binary_data, 'public');
For Private Bucket or if you want to keep file Private
$binary_data = base64_decode($file);
Storage::disk('s3')->put($s3Path, $binary_data);
I Recommend you keep your file private... that is a more secure way and safe. for this, you have to use PreSign in URL to access that file.
For Pre sign-In URL check this post. How access image in s3 bucket using pre-signed url
I have used vlaim\fileupload\FileUpload; and yii\web\UploadedFile;
$image = UploadedFile::getInstance($model, 'flag');
$model->flag = new FileUpload(FileUpload::S_S3, [
'version' => 'latest',
'region' => 'us-west-2',
'credentials' => [
'key' => 'KEY',
'secret' => 'SECRET'
],
'bucket' => 'mybucket/uploads/flags/'.$model->code
]);
$uploader = $model->flag;
$model->flag = $uploader->uploadFromFile($image)->path;
In db i'm saving the path. How to customize the url?
Now my url looks like https://s3-us-west-2.amazonaws.com/mybucket%2Fuploads%2Fflags%2Fus/uploads%5C9f%5C7e%5Cc093ad5a.png
I need the url like https://mybucket.s3.amazonaws.com/uploads/flags/us.png
S3 does not have the concept of folders, It is an object store, with key/value pairs. They key for your file would be uploads/flags/us.png
with the PHP SDK it's easy to set the key of the object.
$USAGE = "\n" .
"To run this example, supply the name of an S3 bucket and a file to\n" .
"upload to it.\n" .
"\n" .
"Ex: php PutObject.php <bucketname> <filename>\n";
if (count($argv) <= 2){
echo $USAGE;
exit();
}
$bucket = $argv[1];
$file_Path = $argv[2];
$key = basename($argv[2]);
try{
//Create a S3Client
$s3Client = new S3Client([
'region' => 'us-west-2',
'version' => '2006-03-01'
]);
$result = $s3Client->putObject([
'Bucket' => $bucket,
'Key' => $key,
'SourceFile' => $file_Path,
]);
} catch (S3Exception $e) {
echo $e->getMessage() . "\n";
}
yii2 i think you need to set setFsUrl()
http://www.yiiframework.com/extension/yii2-file-upload/#hh8
setFsUrl(string $url)
(Only for Local mode)
Sets url. For example, if you set path to 'http://static.example.com' file after uploading will have URL http://static.example.com/path/to/your/file
Default to /
php $uploader->setFsPath('http://pathtoyoursite.com');
I am trying to upload error file in AWSS3 but it shows error like "The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint: "test9011960909.s3.amazonaws.com"."
i also specified 'region' => 'us-east-1' but still same error occurs.
it is working when i specify
'Bucket' => $this->bucket,
but i wanted to upload file in sub-folder of main bucket
'Bucket' => $this->bucket . "/" . $PrefixFolderPath,
i already applied approved answer from AWS S3: The bucket you are attempting to access must be addressed using the specified endpoint
but still getting same error, i am using php
Code :
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
class AWSS3Factory {
private $bucket;
private $keyname;
public function __construct() {
$this->bucket = AWSS3_BucketName;
$this->keyname = AWSS3_AccessKey;
// Instantiate the client.
}
public function UploadFile($FullFilePath,$PrefixFolderPath="") {
try {
$s3 = S3Client::factory(array(
'credentials' => array(
'key' => MYKEY,
'secret' => MYSECKEY,
'region' => 'eu-west-1',
)
));
// Upload data.
$result = $s3->putObject(array(
'Bucket' => $this->bucket . "/" . $PrefixFolderPath,
'Key' => $this->keyname,
'SourceFile' => $FullFilePath,
'StorageClass' => 'REDUCED_REDUNDANCY'
));
return true;
// Print the URL to the object.
//echo $result['ObjectURL'] . "\n";
} catch (S3Exception $e) {
echo $e->getMessage() . "\n";
}
}
}
You must create s3 instance in another way, like this:
$s3 = S3Client::factory([
'region' => '',
'credentials' => ['key' => '***', 'secret' => '***'],
'version' => 'latest',
]);
You must add $PrefixFolderPath not to 'Bucket' but to 'Key':
$result = $s3->putObject(array(
'Bucket' => $this->bucket,
'Key' => $PrefixFolderPath . "/" . $this->keyname,
'SourceFile' => $FullFilePath,
'StorageClass' => 'REDUCED_REDUNDANCY'
));