I know there is no concept of folders in S3, it uses a flat file structure. However, i will use the term "folder" for the sake of simplicity.
Preconditions:
An s3 bucket called foo
The folder foo has been made public using the AWS Management Console
Apache
PHP 5
Standard AWS SDK
The problem:
It's possible to upload a folder using the AWS PHP SDK. However, the folder is then only accessible by the user that uploaded the folder and not public readable as i would like it to be.
Procedure:
$sharedConfig = [
'region' => 'us-east-1',
'version' => 'latest',
'visibility' => 'public',
'credentials' => [
'key' => 'xxxxxx',
'secret' => 'xxxxxx',
],
];
// Create an SDK class used to share configuration across clients.
$sdk = new Aws\Sdk($sharedConfig);
// Create an Amazon S3 client using the shared configuration data.
$client = $sdk->createS3();
$client->uploadDirectory("foo", "bucket", "foo", array(
'params' => array('ACL' => 'public-read'),
'concurrency' => 20,
'debug' => true
));
Success Criteria:
I would be able to access a file in the uploaded folder using a "static" link. Fx:
https://s3.amazonaws.com/bucket/foo/001.jpg
I fixed it by using a defined "Before Execute" function.
$result = $client->uploadDirectory("foo", "bucket", "foo", array(
'concurrency' => 20,
'debug' => true,
'before' => function (\Aws\Command $command) {
$command['ACL'] = strpos($command['Key'], 'CONFIDENTIAL') === false
? 'public-read'
: 'private';
}
));
Use can use this:
$s3->uploadDirectory('images', 'bucket', 'prefix',
['params' => array('ACL' => 'public-read')]
);
Related
I have found the documentation for it here. I have PHP SDK installed. Now when I go through the documents there is not so much in detail about the PHP one. I have the following questions:
Here how can I specify the $client
$result = $client->createDatabase([
'DatabaseName' => '<string>', // REQUIRED
'KmsKeyId' => '<string>',
'Tags' => [
[
'Key' => '<string>', // REQUIRED
'Value' => '<string>', // REQUIRED
],
// ...
],
]);
Is there any good documents or videos regarding the timestream in PHP from where I can get some help.
There are two client classes. One for writing and one for reading.
TimestreamWriteClient
https://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.TimestreamWrite.TimestreamWriteClient.html
and
TimestreamQueryClient
https://docs.aws.amazon.com/aws-sdk-php/v3/api/class-Aws.TimestreamQuery.TimestreamQueryClient.html
You can use the function createTimestreamQuery and createTimestreamWrite on the $sdk object to instantiate those classes.
A sample Timestream client and query below.
// Create client
$client = new \Aws\TimestreamQuery\TimestreamQueryClient([
'version' => 'latest',
'region' => AWS_REGION, /* eg: eu-west-1 */
'endpoint' => AWS_TIMESTREAM_ENDPOINT, /* eg: https://query-cell3.timestream.eu-west-1.amazonaws.com */
'credentials' => new \Aws\Credentials\Credentials(AWS_KEY, AWS_SECRET)
]);
// Perform a basic query with the client
$client->query([
'QueryString' => 'select * from "db_timestream"."tbl_usage_logs"',
'ValidateOnly' => true,
]);
If you receive endpoint warning, such as "The endpoint required for this service is currently unable to be retrieved"
You can find endpoint using AWS CLI command,
aws timestream-query describe-endpoints --region eu-west-1
Sample response:
{
"Endpoints": [
{
"Address": "query-cell3.timestream.eu-west-1.amazonaws.com",
"CachePeriodInMinutes": 1440
}
]
}
One can create TimestreamWriteClient and write records in a similar way.
The documentation seems sparse and a bit misleading, to me anyhow.
This is how I got it going for a write client (assuming SDK is installed).
//Create the client
$client = new \Aws\TimestreamWrite\TimestreamWriteClient([
'version' => 'latest',
'region' => 'eu-west-1',
'credentials' => new \Aws\Credentials\Credentials('***KEY***', '***SECRET***')
]);
Note that the 'endpoint' is not specified, as I've seen in some examples. There seems to be some misleading documentation of what the endpoint should be for any given region. The SDK does some magic and creates a suitable endpoint; providing a specific endpoint didn't work for me.
$result = $client->writeRecords(
[
'DatabaseName' => 'testDB',
'TableName' => 'history',
'Records' =>
[
[
'Dimensions' => [
[
'DimensionValueType' => 'VARCHAR',
'Name' => 'Server',
'Value' => 'VM01',
],
],
'MeasureName' => 'CPU_utilization',
'MeasureValue' => '1.21',
'MeasureValueType' => 'DOUBLE',
'Time' => strval(time()),
'TimeUnit' => 'SECONDS',
]
]
]
);
This seems to be the minimum set of things needed to write a record to Timestream successfully. The code above writes one record, with one dimension, in this case, a 'Name' of a 'Server', recording its CPU utilization at time().
Note:
Time is required, although the documentation suggested it is optional.
Time has to be a String.
I try to upload an image to s3 using Laravel but I receive a runtime error. Using Laravel 5.8, PHP7 and API REST with Postman I send by body base64
I receive an image base64 and I must to upload to s3 and get the request URL.
public function store(Request $request)
{
$s3Client = new S3Client([
'region' => 'us-east-2',
'version' => 'latest',
'credentials' => [
'key' => $key,
'secret' => $secret
]
]);
$base64_str = substr($input['base64'], strpos($input['base64'], ",") + 1);
$image = base64_decode($base64_str);
$result = $s3Client->putObject([
'Bucket' => 's3-galgun',
'Key' => 'saraza.jpg',
'SourceFile' => $image
]);
return $this->sendResponse($result['ObjectURL'], 'message.', 'ObjectURL');
}
Says:
RuntimeException: Unable to open u�Z�f�{��zڱ��� .......
The SourceFile parameter is leading to the path of file to upload to S3, not the binary
You can use Body parameter to replace the SourceFile, or saving the file to local temporary and get the path for SourceFile
Like this:
public function store(Request $request)
{
$s3Client = new S3Client([
'region' => 'us-east-2',
'version' => 'latest',
'credentials' => [
'key' => $key,
'secret' => $secret
]
]);
$base64_str = substr($input['base64'], strpos($input['base64'], ",") + 1);
$image = base64_decode($base64_str);
Storage::disk('local')->put("/temp/saraza.jpg", $image);
$result = $s3Client->putObject([
'Bucket' => 's3-galgun',
'Key' => 'saraza.jpg',
'SourceFile' => Storage::disk('local')->path('/temp/saraza.jpg')
]);
Storage::delete('/temp/saraza.jpg');
return $this->sendResponse($result['ObjectURL'], 'message.', 'ObjectURL');
}
And, if you're using S3 with Laravel, you should consider the S3 filesystem driver instead of access the S3Client manually in your controller
To do this, add the S3 driver composer require league/flysystem-aws-s3-v3, put your S3 IAM settings in .env or config\filesystems.php
Then update the default filesystem in config\filesystems, or indicate the disk driver when using the Storage Storage::disk('s3')
Detail see document here
Instead of SourceFile you have to use Body. SourceFile is a path to a file, but you do not have a file, you have a base64 encoded source of img. That is why you need to use Body which can be a string. More here: https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3-2006-03-01.html#putobject
Fixed version:
public function store(Request $request)
{
$s3Client = new S3Client([
'region' => 'us-east-2',
'version' => 'latest',
'credentials' => [
'key' => $key,
'secret' => $secret
]
]);
$base64_str = substr($input['base64'], strpos($input['base64'], ",") + 1);
$image = base64_decode($base64_str);
$result = $s3Client->putObject([
'Bucket' => 's3-galgun',
'Key' => 'saraza.jpg',
'Body' => $image
]);
return $this->sendResponse($result['ObjectURL'], 'message.', 'ObjectURL');
}
A very simple way to uploads Any file in AWS-S3 Storage.
First, check your ENV setting.
AWS_ACCESS_KEY_ID=your key
AWS_SECRET_ACCESS_KEY= your access key
AWS_DEFAULT_REGION=ap-south-1
AWS_BUCKET=your bucket name
AWS_URL=Your URL
The second FileStorage.php
's3' => [
'driver' => 's3',
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('AWS_SECRET_ACCESS_KEY'),
'region' => env('AWS_DEFAULT_REGION'),
'bucket' => env('AWS_BUCKET'),
'url' => env('AWS_URL'),
//'visibility' => 'public', // do not use this line for security purpose. try to make bucket private.
],
Now come on main Code.
Upload Binary File from HTML Form.
$fileName = 'sh_'.mt_rand(11111,9999).".".$imageFile->clientExtension();;
$s3path = "/uploads/".$this::$SchoolCode."/";
Storage::disk('s3')->put($s3path, file_get_contents($req->file('userDoc')));
Upload Base64 File
For Public Bucket or if you want to keep file Public
$binary_data = base64_decode($file);
Storage::disk('s3')->put($s3Path, $binary_data, 'public');
For Private Bucket or if you want to keep file Private
$binary_data = base64_decode($file);
Storage::disk('s3')->put($s3Path, $binary_data);
I Recommend you keep your file private... that is a more secure way and safe. for this, you have to use PreSign in URL to access that file.
For Pre sign-In URL check this post. How access image in s3 bucket using pre-signed url
I'm trying to create signed urls from cloudfront with aws-sdk-php
I have created both Distributions WEB and RTMP
and this is the code i used to do that
this is start.php
<?php
require 'vendor/autoload.php';
use Aws\S3\S3Client;
use Aws\CloudFront\CloudFrontClient;
$config = require('config.php');
// S3
$client = new Aws\S3\S3Client([
'version' => 'latest',
'region' => 'us-east-2',
]);
// CloudFront
$cloudfront = CloudFrontClient::factory([
'version' => 'latest',
'region' => 'us-east-2',
]);
and this is config.php
<?php
return [
's3'=>[
'key' => 'XXXXXXXXXXXXXXXXXXXXXXXXXX',
'secret' => 'XXXXXXXXXXXXXXXXXXXXXXXXXX',
'bucket' => 'hdamovies',
'region' => 'us-east-2',
],
'cloudFront' => [
'url' => 'https://d2t7o0s69hxjwd.cloudfront.net',
],
];
and this is index.php
<?php
require 'config/start.php';
$video = 'XXXXXXXXXXX.mp4';
$expiry = new DateTime( '+1 hour' );
$url = $cloudfront->getSignedUrl([
'private_key' => 'pk-XXXXXXXXXXXXXXXXXXXXX.pem',
'key_pair_id' => 'XXXXXXXXXXXXXXXXXXXXX',
'url' => "{$config['cloudFront']['url']}/{$video}",
'expires' => strtotime('+10 minutes'),
]);
echo "Downlod";
When i click on the link i get that error
<Error>
<Code>KMS.UnrecognizedClientException</Code>
<Message>No account found for the given parameters</Message>
<RequestId>0F0A772FE67F0503</RequestId>
<HostId>juuIQZKHb1pbmiVkP7NVaKSODFYmBtj3T9AfDNZuXslhb++LcBsw9GNjpT0FG8MxgeQGqbVo+bo=</HostId></Error>
What is the problem here and how can i solve that?
CloudFront does not support downloading objects that were stored, encrypted, in S3 using KMS Keys, apparently because the CloudFront Origin Access Identity is not an IAM user, so it's not possible to authorize it to have the necessary access to KMS.
https://forums.aws.amazon.com/thread.jspa?threadID=268390
I had this issue and had it resolved after setting up the correctly Identities. However, I had a lot of issues with the error even after setting things up correctly. This was because I was attempting to download a file that was originally uploaded when the bucket was KMS encrypted, then later when I changed it to SSE-S3, it still was throwing a KMS error.
After reuploading the file, it seemed to work without any issues. Hope this helps someone else
I used the filesystems.php file to configure my S3.
When I try to put content in the bucket I receive this error:
Encountered a permanent redirect while requesting https://s3-us-west-2.amazonaws.com/MYBUCKET... Are you sure you are using the correct region for this bucket?
I then try to acces the url and I get this message:
The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.
On the same page I get:
<Endpoint>s3.amazonaws.com</Endpoint>
Then how could I remove the region from the URL Laravel generates?
You can create a customer service provider like that:
use Illuminate\Support\ServiceProvider;
use Aws\S3\S3Client;
use League\Flysystem\AwsS3v3\AwsS3Adapter;
use League\Flysystem\Filesystem;
use Aws\Laravel\AwsServiceProvider;
use Storage;
class AwsS3ServiceProvider extends ServiceProvider
{
/**
* Perform post-registration booting of services.
*
* #return void
*/
public function boot()
{
Storage::extend('s3', function($app, $config) {
$client = new S3Client([
'credentials' => [
'key' => $config['key'],
'secret' => $config['secret'],
],
'region' => $config['region'],
'version' => $config['version'],
'endpoint' => $config['endpoint'],
'ua_append' => [
'L5MOD/' . AwsServiceProvider::VERSION,
],
]);
return new Filesystem(new AwsS3Adapter($client, $config['bucket_name']));
});
}
/**
* Register bindings in the container.
*
* #return void
*/
public function register()
{
//
}
}
And add endpoint variable into config/filesystems.php as well:
's3' => [
'driver' => 's3',
'key' => env('AWS_KEY'),
'secret' => env('AWS_SECRET'),
'region' => env('AWS_REGION'),
'version' => 'latest',
'endpoint' => env('AWS_ENDPOINT'),
'bucket_name' => env('AWS_BUCKET_NAME')
]
Look at docs to get details about how to extend Storage facade.
The simplest approach is to create a new disk that uses the S3 driver in config/filesystems.php. You don't need to create a service provider - the S3 driver will pick up the endpoint from the disk config if supplied.
'spaces' => [
'driver' => 's3',
'key' => env('DO_SPACES_KEY'),
'secret' => env('DO_SPACES_SECRET'),
'endpoint' => 'https://nyc3.digitaloceanspaces.com',
'region' => 'nyc3',
'bucket' => env('DO_SPACES_BUCKET'),
],
Set the DO_SPACES_KEY, DO_SPACES_SECRET and DO_SPACES_BUCKET environment variables to the appropriate values.
Source: https://laracasts.com/discuss/channels/laravel/custom-file-driver-digital-ocean-spaces?page=1
You can use this package https://github.com/aws/aws-sdk-php-laravel
In this package you can specify your settings in config file as desired.
The specific bucket is not created in AWS, so please create a bucket in AWS, and use same bucket at filesystem config, the issue will be resolved.
For some reason public-read is not being applied when I'm uploading a folder to an S3 bucket. (IE, public can not access the files)
The files upload fine, but they are all set to private. Tried everything I can think of. Feels like I'm missing something basic.
Was using this guide:
https://blogs.aws.amazon.com/php/post/Tx2W9JAA7RXVOXA/Syncing-Data-with-Amazon-S3
Here is my code:
require '../vendor/autoload.php';
use Aws\S3\S3Client;
$client = S3Client::factory(array(
'version' => '2006-03-01',
'region' => 'ap-southeast-2',
'credentials' => array(
'key' => 'MYKEY',
'secret' => 'MYSECRET',
)
));
$dir = 'assets';
$bucket = 'gittestbucket';
$keyPrefix = 'assets';
$options = array(
'params' => array('ACL' => 'public-read'),
'concurrency' => 20,
'debug' => true
);
$UploadAWS = $client->uploadDirectory($dir, $bucket, $keyPrefix, $options);
var_dump($UploadAWS);
My IAM user policy (also has a group of list all buckets):
{
"Statement": [
{
"Action": "s3:*",
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::gittestbucket",
"arn:aws:s3:::gittestbucket/*",
]
}
]
}
Any help much appreciated. Cheers
I struggled with this a while back.
Try changing you upload statement to this one bellow
$UploadAWS = $client->uploadDirectory($dir, $bucket, $keyPrefix, array(
'concurrency' => 20,
'debug' => true,
'before' => function (\Aws\Command $command) {
$command['ACL'] = strpos($command['Key'], 'CONFIDENTIAL') === false
? 'public-read'
: 'private';
}
));
AWS is shocking sometimes for its documentation as it changes so much