For some reason public-read is not being applied when I'm uploading a folder to an S3 bucket. (IE, public can not access the files)
The files upload fine, but they are all set to private. Tried everything I can think of. Feels like I'm missing something basic.
Was using this guide:
https://blogs.aws.amazon.com/php/post/Tx2W9JAA7RXVOXA/Syncing-Data-with-Amazon-S3
Here is my code:
require '../vendor/autoload.php';
use Aws\S3\S3Client;
$client = S3Client::factory(array(
'version' => '2006-03-01',
'region' => 'ap-southeast-2',
'credentials' => array(
'key' => 'MYKEY',
'secret' => 'MYSECRET',
)
));
$dir = 'assets';
$bucket = 'gittestbucket';
$keyPrefix = 'assets';
$options = array(
'params' => array('ACL' => 'public-read'),
'concurrency' => 20,
'debug' => true
);
$UploadAWS = $client->uploadDirectory($dir, $bucket, $keyPrefix, $options);
var_dump($UploadAWS);
My IAM user policy (also has a group of list all buckets):
{
"Statement": [
{
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::gittestbucket",
"arn:aws:s3:::gittestbucket/*",
]
}
]
}
Any help much appreciated. Cheers
I struggled with this a while back.
Try changing you upload statement to this one bellow
$UploadAWS = $client->uploadDirectory($dir, $bucket, $keyPrefix, array(
'concurrency' => 20,
'debug' => true,
'before' => function (\Aws\Command $command) {
$command['ACL'] = strpos($command['Key'], 'CONFIDENTIAL') === false
? 'public-read'
: 'private';
}
));
AWS is shocking sometimes for its documentation as it changes so much
Related
I am implementing S3 file uploads and downloads using pre-signed urls. I have one s3 bucket (versioning enabled) and one AWS user but I want to track the history of each file in terms of which of my application users modified the file.
I have a versioned S3 bucket and my thought is that I can append metadata to every file to identify my application user and possibly other data too.
Here is my code:
public function presignedUpload(Request $request)
{
$this->validate($request, [
'name' => 'string|required'
]);
$s3 = Storage::disk('s3');
$client = $s3->getDriver()->getAdapter()->getClient();
$expiry = "+10 minutes";
$options = ['user-data' => 'user-meta-value'];
$cmd = $client->getCommand('PutObject', [
'Bucket' => \Config::get('filesystems.disks.s3.bucket'),
'Key' => 'path/to/file/' . $request->name,
'ACL' => 'public-read',
], $options);
$request = $client->createPresignedRequest($cmd, $expiry);
$presignedUrl = (string)$request->getUri();
return response()->json(['url' => $presignedUrl], 201);
}
I deduced that I can pass an $options array from the FilesystemAdapter::class. This code does upload the file, but the metadata in AWS looks empty.
Is my $options array in the wrong format?
Appreciate any help on this.
The selected answer doesn't work anymore after Laravel 9. Flysystem recommends you to use S3 SDK instead of Storage driver.
https://github.com/thephpleague/flysystem/issues/1423
Use the S3 SDK instead to create a resigned upload URL.
$uploadDisk = 's3';
$s3Client = new S3Client([
'credentials' => [
'key' => config('filesystems.disks.' . $uploadDisk . '.key'),
'secret' => config('filesystems.disks.' . $uploadDisk . '.secret'),
],
'region' => config('filesystems.disks.' . $uploadDisk . '.region'),
'version' => 'latest',
]);
$s3Bucket = config('filesystems.disks.' . $uploadDisk . '.bucket');
$s3Key = \Illuminate\Support\Str::uuid(); // your file name
$s3Options = [];
$command = $s3Client->getCommand('PutObject', [
'Bucket' => $s3Bucket,
'Key' => $s3Key,
'MetaData' => $s3Options,
]);
$request = $s3Client->createPresignedRequest($command, '+20 minutes');
$url = (string) $request->getUri();
Metadata can be uploaded like this:
public function presignedUpload(Request $request)
{
$this->validate($request, [
'name' => 'string|required'
]);
$s3 = Storage::disk('s3');
$client = $s3->getDriver()->getAdapter()->getClient();
$expiry = "+10 minutes";
$options = ['user-data' => 'user-meta-value'];
$cmd = $client->getCommand('PutObject', [
'Bucket' => \Config::get('filesystems.disks.s3.bucket'),
'Key' => 'path/to/file/' . $request->name,
'ACL' => 'public-read',
'Metadata' => $options,
]);
$request = $client->createPresignedRequest($cmd, $expiry);
$presignedUrl = (string)$request->getUri();
return response()->json(['url' => $presignedUrl], 201);
}
I'm setting up Google Cloud Storage bucket CORS configuration using PHP API, but it doesn't seem to work
I read the document given in : https://googleapis.github.io/google-cloud-php/#/docs/google-cloud/v0.96.0/storage/bucket
Here's my Laravel source code:
use Google\Cloud\Core\ServiceBuilder;
...
$projectId = 'myProjectId';
$bucketName = 'myBucketName';
$gcloud = new ServiceBuilder([
'keyFilePath' => 'resources/google-credentials.json',
'projectId' => $projectId
]);
$storage = $gcloud->storage();
$bucket = $storage->bucket($bucketName);
//change bucket configuration
$result = $bucket->update([
'cors' => [
'maxAgeSeconds' => 3600,
'method' => [
"GET","HEAD"
],
"origin" => [
"*"
],
"responseHeader" => [
"Content-Type"
]
]
]);
//print nothing and bucket doesn't changed
dd($bucket->info()['cors']);
After execute this code, the bucket CORS configuration doesn't changed
(My boss don't want me to use gsutil shell command to deal with this)
You're very close! CORS accepts a list, so you'll just need to make a slight modification:
$result = $bucket->update([
'cors' => [
[
'maxAgeSeconds' => 3600,
'method' => [
"GET","HEAD"
],
"origin" => [
"*"
],
"responseHeader" => [
"Content-Type"
]
]
]
]);
Let me know if it helps :).
The only thing I needed to change was when I config disks in laravel, using this code in config/filesystems.php when adding a disk for google:
'google' => [
'driver' => 's3',
'key' => 'xxx',
'secret' => 'xxx',
'bucket' => 'qrnotesfiles',
'base_url'=>'https://storage.googleapis.com'
]
Here is the code example fist get file contents from request:
$file = $request->file('avatar')
second save it into storage:
Storage::disk('google')->put('avatars/' , $file);
I try for the first time to use the PHP AWS SDK ("aws/aws-sdk-php": "^3.19") to use S3.
I created a bucket : 'myfirstbucket-jeremyc'
I created a policy :
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::myfirstbucket-jeremyc/*"
]
}
]
}
I applied the policy to a group and then created a user 's3-myfirstbucket-jeremyc' in this group.
My PHP code is :
<?php
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
error_reporting(E_ALL);
require(__DIR__ . '/vendor/autoload.php');
$s3Client = S3Client::factory([
'credentials' => [
'key' => $_SERVER['AWS_S3_CLIENT_KEY'],
'secret' => $_SERVER['AWS_S3_CLIENT_SECRET']
],
'region' => 'eu-west-1',
'version' => 'latest',
'scheme' => 'http'
]);
$result = $s3Client->putObject(array(
'Bucket' => 'myfirstbucket-jeremyc',
'Key' => 'text.txt',
'Body' => 'Hello, world!',
'ACL' => 'public-read'
));
But i get this error :
Error executing "PutObject" on
"http://s3-eu-west-1.amazonaws.com/myfirstbucket-jeremyc/text.txt";
AWS HTTP error: Client error: PUT
http://s3-eu-west-1.amazonaws.com/myfirstbucket-jeremyc/text.txt
resulted in a 403 Forbidden response
Do you know where i'm wrong ?
Thanks in advance !
You're setting the ACL for the new object but you haven't allowed s3:PutObjectAcl.
I know there is no concept of folders in S3, it uses a flat file structure. However, i will use the term "folder" for the sake of simplicity.
Preconditions:
An s3 bucket called foo
The folder foo has been made public using the AWS Management Console
Apache
PHP 5
Standard AWS SDK
The problem:
It's possible to upload a folder using the AWS PHP SDK. However, the folder is then only accessible by the user that uploaded the folder and not public readable as i would like it to be.
Procedure:
$sharedConfig = [
'region' => 'us-east-1',
'version' => 'latest',
'visibility' => 'public',
'credentials' => [
'key' => 'xxxxxx',
'secret' => 'xxxxxx',
],
];
// Create an SDK class used to share configuration across clients.
$sdk = new Aws\Sdk($sharedConfig);
// Create an Amazon S3 client using the shared configuration data.
$client = $sdk->createS3();
$client->uploadDirectory("foo", "bucket", "foo", array(
'params' => array('ACL' => 'public-read'),
'concurrency' => 20,
'debug' => true
));
Success Criteria:
I would be able to access a file in the uploaded folder using a "static" link. Fx:
https://s3.amazonaws.com/bucket/foo/001.jpg
I fixed it by using a defined "Before Execute" function.
$result = $client->uploadDirectory("foo", "bucket", "foo", array(
'concurrency' => 20,
'debug' => true,
'before' => function (\Aws\Command $command) {
$command['ACL'] = strpos($command['Key'], 'CONFIDENTIAL') === false
? 'public-read'
: 'private';
}
));
Use can use this:
$s3->uploadDirectory('images', 'bucket', 'prefix',
['params' => array('ACL' => 'public-read')]
);
I am working on AWS EC2 Ubuntu Machine and trying to fetch image from AWS S3 but following error has been shown to me every time.
<Error>
<Code>InvalidArgument</Code>
<Message>
Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4.
</Message>
<ArgumentName>Authorization</ArgumentName>
<ArgumentValue>null</ArgumentValue>
<RequestId>7C8B4BF1CE2FDC9E</RequestId>
<HostId>
/L5kjuOET4XFgGter2eFHX+aRSvVm/7VVmIBqQE/oMLeQZ1ditSMZuHPOlsMaKi8hYRnGilTqZY=
</HostId>
</Error>
Here is my bucket policy
{
"Version": "2012-10-17",
"Id": "Policy1441213815928",
"Statement": [
{
"Sid": "Stmt1441213813464",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mytest.sample/*"
}
]
}
Here is the code
require 'aws-autoloader.php';
$credentials = new Aws\Credentials\Credentials('key', 'key');
$bucketName = "mytest.sample";
$s3 = new Aws\S3\S3Client([
'signature' => 'v4',
'version' => 'latest',
'region' => 'ap-southeast-1',
'credentials' => $credentials,
'http' => [
'verify' => '/home/ubuntu/cacert.pem'
],
'Statement' => [
'Action ' => "*",
],
]);
$result = $s3->getObject(array(
'Bucket' => $bucketName,
'Key' => 'about_us.jpg',
));
Html
<img src="<?php echo $result['#metadata']['effectiveUri']; ?>" />
Edit for Michael - sqlbot : here I am using default KMS.
try {
$result = $this->Amazon->S3->putObject(array(
'Bucket' => 'mytest.sample',
'ACL' => 'authenticated-read',
'Key' => $newfilename,
'ServerSideEncryption' => 'aws:kms',
'SourceFile' => $filepath,
'ContentType' => mime_content_type($filepath),
'debug' => [
'logfn' => function ($msg) {
echo $msg . "\n";
},
'stream_size' => 0,
'scrub_auth' => true,
'http' => true,
],
));
} catch (S3Exception $e) {
echo $e->getMessage() . "\n";
}
let me know if you need more.
PHP sdk v2
the Credentials package is Aws\Common\Credentials
to create an S3Client you need a factory
Try something like this
use Aws\S3\S3Client;
use Aws\Common\Credentials\Credentials;
$credentials = new Credentials('YOUR_ACCESS_KEY', 'YOUR_SECRET_KEY');
// Instantiate the S3 client with your AWS credentials
$s3Client = S3Client::factory(array(
'signature' => 'v4',
'region' => 'ap-southeast-1',
'credentials' => $credentials,
.....
]);
)
If that does not work you might try to declare explicitly a SignatureV4 object
use Aws\S3\S3Client;
use Aws\Common\Credentials\Credentials;
use Aws\Common\Signature\SignatureV4;
$credentials = new Credentials('YOUR_ACCESS_KEY', 'YOUR_SECRET_KEY');
// Instantiate the S3 client with your AWS credentials
$s3Client = S3Client::factory(array(
'signature' => new SignatureV4(),
'region' => 'ap-southeast-1',
'credentials' => $credentials,
.....
]);
)
In case you upgrade to sdk v3
You need to have signature_version (instead of signature) as parameter when you declare your s3 client
Statement does not appear to be a valid parameter (http://docs.aws.amazon.com/aws-sdk-php/v3/guide/guide/configuration.html#signature-version)
if issue you can turn on debug param to get more output
This would look like this
$s3 = new Aws\S3\S3Client([
'signature_version' => 'v4',
'version' => 'latest',
'region' => 'ap-southeast-1',
'credentials' => $credentials,
'http' => [
'verify' => '/home/ubuntu/cacert.pem'
],
'debug' => true
]);
see here for the full list of available parameter
I have also face this issue with aws:kms encyrption key, I suggest that if you wanted to use kms key then you have to create your kms key in IAM section of AWS Console. I love to recommended AES256 server side encryption, here S3 automatically Encrypted your data while putting and decryption while getting object. Please go through below link:
S3 Server Side encryption with AES256
My Solution is change this line 'ServerSideEncryption' => 'aws:kms' with 'ServerSideEncryption' => 'AES256'
try {
$result = $this->Amazon->S3->putObject(array(
'Bucket' => 'mytest.sample',
'ACL' => 'authenticated-read',
'Key' => $newfilename,
'ServerSideEncryption' => 'AES256',
'SourceFile' => $filepath,
'ContentType' => mime_content_type($filepath),
'debug' => [
'logfn' => function ($msg) {
echo $msg . "\n";
},
'stream_size' => 0,
'scrub_auth' => true,
'http' => true,
],
));
} catch (S3Exception $e) {
echo $e->getMessage() . "\n";
}
Please also update your bucket policy with below json, it will prevent you to upload object with out AES256 encryption
{
"Sid": "DenyUnEncryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::yourbucketname/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "AES256"
}
}
}