putObject works well for me. But when I do a getObjectURL s3 responds with an Access Denied. Looked into different ways to set the policy, etc. Still no luck. Any help appreciated!
putObject code:
$this->load->library('aws');
$temporary_png_path = tempnam(sys_get_temp_dir(), 'sig') . $upload_data['file_ext'];
$document_s3_key = 'profile_pictures/'.$user->id.'/profile_'.$user->id.$upload_data['file_ext'];
$resource = fopen($upload_data['full_path'], 'r');
try{
$aws_object=$this->aws->putObject(array(
'Bucket' => 'tbucket',
'Key' => $document_s3_key,
'SourceFile' => $resource,
'Body' => $resource,
'ContentType' => 'image/jpeg'
));
}
catch (Exception $e)
{
$error = "Something went wrong saving your file.\n".$e;
echo $error;
}
getObjectURL code:
$this->load->library('aws');
$profile_image = $this->aws->getObjectUrl('tbucket', 'profile_pictures/'.$user->id.'/'.'profile_'.$user->id.'.jpeg');
$this->data['image'] = ($profile_image == '')? "/public/images/avatar-large.jpg" : $profile_image;
AWS Permissions:
Authenticated Users => All Permissions enabled
Bucket Policy:
{
"Version": "2008-10-17",
"Id": "Policynumber",
"Statement": [
{
"Sid": "Stmtnumber",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::number:user/t_master"
},
"Action": [
"s3:GetObjectAcl",
"s3:GetObjectTorrent",
"s3:GetObjectVersion",
"s3:GetObjectVersionAcl"
],
"Resource": "arn:aws:s3:::tbucket/*"
}
]
}
Related
Using bref as PHP serverless framework.
Problem:
Unable to upload the dynamically generated file to S3 bucket from Lambda using PHP
Code:
serverless.yml
service: lambdaToS3
provider:
name: aws
region: ap-southeast-2
runtime: provided.al2
#stage: prod
profile: default
iam:
role:
statements:
- Effect: Allow
Action:
- 's3:GetObject'
- 's3:PutObject'
- 's3:GetObjectAcl'
- 's3:PutObjectAcl'
Resource:
- 'arn:aws:s3:::*/*'
plugins:
- ./vendor/bref/bref
functions:
lambdaToS3:
handler: index.php
description: ''
layers:
- ${bref:layer.php-74}
events:
- httpApi: '*'
# Exclude files from deployment
package:
patterns:
- '!tests/**'
- '!tmp/**'
index.php
comments are output when executing a file using HTTP GateWay
In this file, I am generating a simple text file in /tmp/ folder and trying to upload it to the S3 bucket with two different attributes, SourceFile or Body. None of them working.
<?php declare(strict_types=1);
ini_set('display_errors', "1");
ini_set('display_startup_errors', "1");
error_reporting(E_ALL);
require __DIR__ . '/vendor/autoload.php';
use Aws\S3\S3Client;
use Bref\Logger\StderrLogger;
$logger = new StderrLogger();
$s3client = new S3Client([
'scheme' => 'http',
'version' => '2006-03-01',
'region' => 'ap-southeast-2',
'output' => 'JSON'
]);
$bucket = 'bucketname';
$file = "/tmp/newfile.txt";
$myfile = fopen($file, "w") or die("Unable to open file!");
$txt = "John Doe\n";
fwrite($myfile, $txt);
$txt = "Jane Doe\n";
fwrite($myfile, $txt);
fclose($myfile);
echo "<pre>"; print_r("File Exist: ". file_exists($file)); echo "</pre>"; // 1
echo "<pre>"; print_r(" ================ "); echo "</pre>";
echo "<pre>"; print_r(file_get_contents($file)); echo "</pre>"; // we can get content using this method.
echo "<pre>"; print_r(" ================ "); echo "</pre>";
// Put on S3 using SourceFile
$result = $s3client->putObject([
'Bucket' => $bucket,
'Key' => "AWS_LAMBDA_S3.txt",
'SourceFile' => $file,
'ACL' => 'public-read',
]);
echo "<pre>"; print_r("File Upload: " . json_encode($result)); echo "</pre>";
// Put on S3 using Body
$myfile = fopen($file, "rb");
$result = $s3client->putObject([
'Bucket' => $bucket,
'Key' => "AWS_LAMBDA_S3_BODY.txt",
'Body' => $myfile,
'ACL' => 'public-read',
]);
echo "<pre>"; print_r("File Upload: " . json_encode($result)); echo "</pre>";
exit;
Regarding policy:
Once the function deploy to AWS Lambda, we got a following policy for the role
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:CreateLogGroup"
],
"Resource": "arn:aws:logs:ap-southeast-2:xxxx:log-group:/aws/lambda/lambdaToS3-dev*:*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObjectAcl",
"s3:PutObjectAcl"
],
"Resource": "*"
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"sqs:DeleteMessage",
"sqs:ReceiveMessage",
"sqs:GetQueueAttributes",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:ap-southeast-2:xxxx:log-group:/aws/lambda/lambdaToS3-dev*:*:*",
"arn:aws:sqs:ap-southeast-2:xxxx:xxxx"
]
}
]
}
The above solution will work.
The problem was with VPC:
If your lambda function is inside VPC, then you need to enable the endpoint of S3 from the VPC dashboard.
You can refer to any video in any other programming language showing how to deploy the lambda function in VPC and access S3.
I am trying to set up wav -> mp3 conversion through AWS, I've followed documentation closely but I can't find anything about the problem I am facing so hoping someone can help here. Here is my code to start a conversion on AWS:
$job_settings = '{
"TimecodeConfig": {
"Source": "ZEROBASED"
},
"OutputGroups": [
{
"Name": "File Group",
"Outputs": [
{
"ContainerSettings": {
"Container": "RAW"
},
"AudioDescriptions": [
{
"AudioTypeControl": "FOLLOW_INPUT",
"AudioSourceName": "Audio Selector 1",
"CodecSettings": {
"Codec": "MP3",
"Mp3Settings": {
"Bitrate": 192000,
"Channels": 2,
"RateControlMode": "CBR",
"SampleRate": 48000
}
},
"LanguageCodeControl": "FOLLOW_INPUT"
}
]
}
],
"OutputGroupSettings": {
"Type": "FILE_GROUP_SETTINGS",
"FileGroupSettings": {
"Destination": "{DESTINATION}"
}
}
}
],
"AdAvailOffset": 0,
"Inputs": [
{
"AudioSelectors": {
"Audio Selector 1": {
"Tracks": [
1
],
"Offset": 0,
"DefaultSelection": "DEFAULT",
"SelectorType": "TRACK",
"ProgramSelection": 1
}
},
"FilterEnable": "AUTO",
"PsiControl": "USE_PSI",
"FilterStrength": 0,
"DeblockFilter": "DISABLED",
"DenoiseFilter": "DISABLED",
"InputScanType": "AUTO",
"TimecodeSource": "ZEROBASED",
"FileInput": "{INPUT}"
}
]
}';
//Job starts here
$job_settings = json_decode($job_settings, true);
$convert_client = new MediaConvertClient(array(
'version' => '2017-08-29',
'region' => $this->login_details->region,
'credentials' => $this->credentials
));
try {
$res = $convert_client->describeEndpoints([]);
} catch(AwsException $e) {
//echo $e->getMessage();
return null;
}
//print_r($res);
$single_endpoint = $res['Endpoints'][0]['Url'];
$convert_client = new MediaConvertClient(array(
'version' => '2017-08-29',
'region' => $this->login_details->region,
'credentials' => $this->credentials,
'endpoint' => $single_endpoint
));
$res = $client->createJob(array(
"Role" => "arn:aws:iam::{$this->login_details->account_number}:role/MediaConvert_Default_Role",
"Settings" => $job_settings,
"Queue" => "arn:aws:mediaconvert:{$this->login_details->region}:{$this->login_details->account_number}:queues/Default"
));
I grabbed the JSON by creating a conversion job on AWS Console and copying the JSON of the job, the destination and input tags are replaced in the json before decoding to assoc array however I am getting an error of the following:
Error executing "CreateJob" on "aws_url"; AWS HTTP error: Client error: POST aws_url resulted in a 400 Bad Request response: { "errorType": "BadRequestException", "httpStatus" : 400, "requestId" : "-----------", (truncated...) BadRequestException (client): /outputGroups/0/outputs/0/audioDescriptions/0/codecSettings: Should match exactly one schema defined in "oneOf" | /outputGroups/0/outputs/0/audioDescriptions/0/codecSettings: Should have at least 2 properties | /outputGroups/0/outputs/0/audioDescriptions/0/codecSettings/codec: Should be equal to one of the allowed values in ["PASSTHROUGH","OPUS","VORBIS"]
This is using aws-3.93.3 PHP SDK. Any idea why this happens? Of course the job runs perfectly fine if ran through console.
Actually I just found what I suspected was the case; this version of the library does not yet support MP3 ingestion as outlined here:
https://github.com/aws/aws-sdk-php/blob/master/CHANGELOG.md
Updating to the newest version solved this issue.
In development environment, below code works perfectly to upload a file to AWS S3 Frankfurt region with credentials set up as mentioned in
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html
PHP
require("aws.phar");
use Aws\S3\S3Client;
use Aws\S3\Exception\S3Exception;
$bucket = 'my bucket';
$filepath = 'currentfilepath.jpeg';
$filename = 'newfilename.jpeg';
$s3 = S3Client::factory(array(
'key' => 'XXX',
'secret' => 'YYY',
'region' => 'eu-west-2',
'version' => 'latest'
));
try {
$result = $s3->putObject(array(
'Bucket' => $bucket,
'Key' => $filename,
'SourceFile' => $filepath,
'ACL' => 'public-read'
));
var_dump($result);
} catch (S3Exception $e) {
echo $e->getMessage() . "\n";
}
However in deployed environment, I get Access denied error.
Error executing "PutObject" on
"https://s3.eu-west-2.amazonaws.com/mybucket/newfilename.jpeg";
AWS HTTP error: Client error: 403 AccessDenied (client): Access Denied
The only difference from development and deployed is that I do not have a credential file like in dev environment. I wonder how to manage credentials in deployed environment to make this work?
Reza Mousavi's answer in my view should help.
But I got this fixed by adding AmazonS3FullAccess to aws-elasticbeanstalk-ec2-role. This is done from the
IAM control panel >> Roles >>
Choose aws-elasticbeanstalk-ec2-role >> Attach Policies:AmazonS3FullAccess
Thanks
You should create the new API key/secret and assign the right IAM for access to your S3 bucket and publish the new credentials like the document you mentioned before:
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/setup-credentials.html
Also, if your EC2 has a public IP address, you can grant access by the change in your S3 bucket policy:
{
"Version": "2012-10-17",
"Id": "Policy1462808223348",
"Statement": [
{
"Sid": "Stmt1462808220978",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::714656454815:role/ecsInstanceRole"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket-name/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": "YOUR-PUBLIC-IP/32"
}
}
}
]
}
I feel a bit stupid to ask this, but is there anything special required to upload something via the current PHP SDK to S3? I can upload via the cli with the same credentials, but when I try the SDK it fails.
Here the code:
<?php
require "awssdk_v3/aws-autoloader.php";
use Aws\S3\S3Client;
function s3_upload($file, $name) {
$s3 = S3Client::factory(
array(
'key' => getenv('AWS_ACCESS_KEY_ID'),
'secret' => getenv('AWS_SECRET_ACCESS_KEY'),
'version' => "2006-03-01",
'region' => getenv('AWS_REGION')
)
);
$result = $s3->putObject(
array(
'Bucket' => getenv('AWS_BUCKET'),
'Key' => $name,
'SourceFile' => $file,
'ContentType' => mime_content_type($file),
'ACL' => 'public-read'
)
);
return true;
}
I call it like this
s3_upload($_FILES['avatarfile']['tmp_name'], "avatar_2.jpg");
The user I use has this policy attached:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1480066717000",
"Effect": "Allow",
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::my-bucket/*"
]
},
{
"Sid": "Stmt1480066765000",
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::my-bucket"
]
}
]
}
As mentioned I was able to upload a file from the CLI using that users credentials. The region is Frankfurt, so I specified eu-central-1, correct?
The error I get starts like this:
Fatal error: Uncaught exception 'Aws\S3\Exception\S3Exception' with message 'Error executing "PutObject" on "https://my-bucket.s3.eu-central-1.amazonaws.com/avatar_2.jpg"; AWS HTTP error: Client error: `PUT https://my-bucket.s3.eu-central-1.amazonaws.com/avatar_2.jpg` resulted in a `403 Forbidden` response
I found the problem thanks to this answer. I'm trying to set the ACL 'public-read', but haven't granted myself s3:PutObjectAcl, just s3:PutObject. Changing either fixes the problem.
Thanks anyway for the help.
Under IAM Policy, add S3:PutObjectAcl in permission if you set S3:PutObject for action.
I ran into this same error message, and it turns out my S3 buckets were created in the wrong region.
I need to call my Amazon Lambda function, from php, but I am getting strange rights error when I have AWSLambdaFullAccess rights.
My code:
$client = LambdaClient::factory(array(
'key' => 'AKI...G',
'secret' => 'VXD...YOse',
'region' => 'us-west-2'
));
$result = $client->invokeAsync(array(
'FunctionName' => 'arn:aws:lambda:us-west-2:180...52:function:fe...st',
'InvokeArgs' => json_encode($array),
));
This is the error:
User: arn:aws:iam::69...67:user/developer is not authorized to
perform: lambda:InvokeFunction on resource:
arn:aws:lambda:us-west-2:180...52:function:fe...st
Any ideas? Thanks
You should give your PHP code the permission to call your Lambda function (otherwise everybody will be able to call your code...).
You need to create a role with the right permission and make sure that your PHP code assumes that role (for examples with Cognito or EC2 roles).
The role should include something like:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1234567890",
"Effect": "Allow",
"Action": [
"lambda:InvokeFunction"
],
"Resource": [
"arn:aws:lambda:us-west-2:<YOUR-ACOUNT-ID>:function:<YOUR-FUNCTION-NAME>"
]
}
]
}