I am storing some customer PDFs in S3 for multiple parties to either view in the browser or download. The trouble is I can only get a single file in S3 to either always download or always view in the browser.
I could just upload the same file twice with each having its own ContentDisposition, but that seems wasteful when ideally it could be as simple as adding something like ?ContentDisposition=inline to the public bucket URL.
My Question: How can dynamically set a ContentDisposition for a single S3 file?
For context, my current code looks something like this:
$s3_object = array(
'ContentDisposition' => sprintf('attachment; filename="%s"', addslashes($basename)),
'ACL' => 'public-read',
'ContentType' => 'pdf',
'StorageClass' => 'REDUCED_REDUNDANCY',
'Bucket' => 'sample',
'Key' => static::build_file_path($path, $filename, $extension),
'Body' => $binary_content,
);
$result = $s3_client->putObject($s3_object);
Also, I did try to search for this elsewhere in SO, but most people seem to just be looking for one or the other, so I didn't find any SO answers that showed how to do this.
I ended up stumbling across the definitive answer for this today (over a month later) while looking at other S3 documentation. Going to the GetObject docs for the S3 API and under the section labeled "Overriding Response Header Values" we find the following:
Note: You must sign the request, either using an Authorization header or a presigned URL, when using these parameters. They cannot be used with an unsigned (anonymous) request.
response-content-language
response-expires
response-cache-control
response-content-disposition
response-content-encoding
This answer's how to dynamically change any S3 object's content-disposition in the URL. However, at least for me, this is an imperfect solution because my intended use case was to store the URL for years as part of an invoicing archive, but signed URLs are only valid for a maximum of 1 week.
I could technically also try to find a way to make the Authorization header work for me or just query the S3 API to get a new signed URL every time I want to link to it, but that has other security, performance, and ROI implications for me that make it not worth it.
I've been working with some example code from Amazon to get a script to upload an object to a bucket in version 3 of the php sdk for aws. I can get the object to upload to a bucket, but I'm trying to add a tag to this new object during this PutObject method call. I've worked through a few examples that I found, but nothing has worked for me. Here is my php code so far:
$cmd = $s3Client->getCommand('PutObject', [
'Bucket' => $config['s3BucketName'],
'Key' => 'file_upload_direct.mp4',
'Tagging' => 'status=notProcessed',
]);
The Tagging property doesn't get applied and doesn't give any error when the form is sent. I've seen a few ways of adding tags to uploads, but none of those have worked for me. I'm trying to avoid using the PutObjectTagging method since that seems to be extra work if I'm able to define the tag in the PutObject method. I'm not sure if the issue is trying to use the PutObject method in the getCommand or not, but as far as I can tell you should be able to pass the normal parameters as an array like this. Has anyone been able to get this to work, or is there a different way I should be trying to accomplish this?
it is better later than never, right?
The problem here is that AWS Docs are poor on emphasising important details like:
Note: Not all operation parameters are supported when using pre-signed URLs. Certain parameters, such as SSECustomerKey, ACL, Expires, ContentLength, or Tagging must be provided as headers when sending a request.
You can find these in API Docs and easy to overlook.
This means you prepare data to create a signature with this:
$cmd = $s3Client->getCommand('PutObject', [
'Bucket' => $config['s3BucketName'],
'Key' => 'file_upload_direct.mp4',
'Tagging' => 'status=notProcessed',
]);
Then when you get the URL back from
$s3Client->createPresignedRequest($cmd, '+5 minutes');
And you PUT to that URL, you have to also send HTTP Header X-Amz-Tagging: status=notProcessed
Another important thing to keep in mind is that, if you are PUTting from the frontend, your bucket must have CORS policies properly set up and allow headers like x-amz-tagging.
I am trying to make a PutObject presigned request using the AWS S3 PHP SDK.
I have gotten the request to work but now I only want to allow my users to be able to only upload video files.I have tried a lot of combinations and searched a lot but I could not get it to work.
Here is the sample code I use:
$cmd = $this->s3client->getCommand('PutObject', [
'Bucket' => 'myBucket',
'Key' => 'inputs/' . $movie->getId(),
'ACL' => 'private',
'Conditions' => ['Starts-With', '$Content-Type', 'video/'], // I have tried other combinations but it seems to not work
]);
$request = $this->s3client->createPresignedRequest($cmd, '+30 minutes');
$movie->setSignedUrl((string)$request->getUri());
The signed url generated does never include the Content-Type in the X-Amz-SignedHeaders query parameter, only the host is included.
The putObject() request has no documented Conditions key.
You appear to be confusing S3's PUT upload interface with the pre-signed POST capability, which supports policy document conditions like ['Starts-With', '$Content-Type', 'video/'],
PUT does not support "starts with". It requires the exact Content-Type and the key for this (which should result in the header appearing in the X-Amz-SignedHeaders query string parameter) is simply ContentType. It goes in the outer parameters array, just like Bucket and Key.
But if you want to support multiple content types without knowing the specific type in advance, you need to use POST uploads.
I have searched on the web for over two days now, and probably have looked through most of the online documented scenarios and workarounds, but nothing worked for me so far.
I am on AWS SDK for PHP V2.8.7 running on PHP 5.3.
I am trying to connect to my Amazon S3 bucket with the following code:
// Create a `Aws` object using a configuration file
$aws = Aws::factory('config.php');
// Get the client from the service locator by namespace
$s3Client = $aws->get('s3');
$bucket = "xxx";
$keyname = "xxx";
try {
$result = $s3Client->putObject(array(
'Bucket' => $bucket,
'Key' => $keyname,
'Body' => 'Hello World!'
));
$file_error = false;
} catch (Exception $e) {
$file_error = true;
echo $e->getMessage();
die();
}
My config.php file is as follows:
return [
// Bootstrap the configuration file with AWS specific features
'includes' => ['_aws'],
'services' => [
// All AWS clients extend from 'default_settings'. Here we are
// overriding 'default_settings' with our default credentials and
// providing a default region setting.
'default_settings' => [
'params' => [
'credentials' => [
'key' => 'key',
'secret' => 'secret'
]
]
]
]
];
It is producing the following error:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
I've already checked my access key and secret at least 20 times, generated new ones, used different methods to pass in the information (i.e. profile and including credentials in code) but nothing is working at the moment.
After two days of debugging, I finally discovered the problem...
The key I was assigning to the object started with a period i.e. ..\images\ABC.jpg, and this caused the error to occur.
I wish the API provides more meaningful and relevant error message, alas, I hope this will help someone else out there!
I get this error with the wrong credentials. I think there were invisible characters when I pasted it originally.
I had the same error in nodejs. But adding signatureVersion in s3 constructor helped me:
const s3 = new AWS.S3({
apiVersion: '2006-03-01',
signatureVersion: 'v4',
});
I had the same problem when tried to copy an object with some UTF8 characters. Below is a JS example:
var s3 = new AWS.S3();
s3.copyObject({
Bucket: 'somebucket',
CopySource: 'path/to/Weird_file_name_ðÓpíu.jpg',
Key: 'destination/key.jpg',
ACL: 'authenticated-read'
}, cb);
Solved by encoding the CopySource with encodeURIComponent()
This error seems to occur mostly if there is a space before or after your secret key
My AccessKey had some special characters in that were not properly escaped.
I didn't check for special characters when I did the copy/paste of the keys. Tripped me up for a few mins.
A simple backslash fixed it. Example (not my real access key obviously):
secretAccessKey: 'Gk/JCK77STMU6VWGrVYa1rmZiq+Mn98OdpJRNV614tM'
becomes
secretAccessKey: 'Gk\/JCK77STMU6VWGrVYa1rmZiq\+Mn98OdpJRNV614tM'
For Python set - signature_version s3v4
s3 = boto3.client(
's3',
aws_access_key_id='AKIAIO5FODNN7EXAMPLE',
aws_secret_access_key='ABCDEF+c2L7yXeGvUyrPgYsDnWRRC1AYEXAMPLE',
config=Config(signature_version='s3v4')
)
I've just encountered this and, I'm a little embarrassed to say, it was because I was using an HTTP POST request instead of PUT.
Despite my embarrassment, I thought I'd share in case it saves somebody an hour of head scratching.
Actually in Java i was getting same error.After spending 4 hours to debug it what i found that that the problem was in meta data in S3 Objects as there was space while sitting cache controls in s3 files.This space was allowed in 1.6.* version but in 1.11.* it is disallowed and thus was throwing the signature mismatch error
In a previous version of the aws-php-sdk, prior to the deprecation of the S3Client::factory() method, you were allowed to place part of the file path, or Key as it is called in the S3Client->putObject() parameters, on the bucket parameter. I had a file manager in production use, using the v2 SDK. Since the factory method still worked, I did not revisit this module after updating to ~3.70.0. Today I spent the better part of two hours debugging why I had started receiving this error, and it ended up being due to the parameters I was passing (which used to work):
$s3Client = new S3Client([
'profile' => 'default',
'region' => 'us-east-1',
'version' => '2006-03-01'
]);
$result = $s3Client->putObject([
'Bucket' => 'awesomecatpictures/catsinhats',
'Key' => 'whitecats/white_cat_in_hat1.png',
'SourceFile' => '/tmp/asdf1234'
]);
I had to move the catsinhats portion of my bucket/key path to the Key parameter, like so:
$s3Client = new S3Client([
'profile' => 'default',
'region' => 'us-east-1',
'version' => '2006-03-01'
]);
$result = $s3Client->putObject([
'Bucket' => 'awesomecatpictures',
'Key' => 'catsinhats/whitecats/white_cat_in_hat1.png',
'SourceFile' => '/tmp/asdf1234'
]);
What I believe is happening is that the Bucket name is now being URL Encoded. After further inspection of the exact message I was receiving from the SDK, I found this:
Error executing PutObject on https://s3.amazonaws.com/awesomecatpictures%2Fcatsinhats/whitecats/white_cat_in_hat1.png
AWS HTTP error: Client error:
PUT https://s3.amazonaws.com/awesomecatpictures%2Fcatsinhats/whitecats/white_cat_in_hat1.png resulted in a 403 Forbidden
This shows that the / I provided to my Bucket parameter has been through urlencode() and is now %2F.
The way the Signature works is fairly complicated, but the issue boils down to the bucket and key are used to generate the encrypted signature. If they do not match exactly on both the calling client, and within AWS, then the request will be denied with a 403. The error message does actually point out the issue:
The request signature we calculated does not match the signature you
provided. Check your key and signing method.
So, my Key was wrong because my Bucket was wrong.
In my case I was using s3.getSignedUrl('getObject') when I needed to be using s3.getSignedUrl('putObject') (because I'm using a PUT to upload my file), which is why the signatures didn't match.
For me I used axios and by deafult it sends header
content-type: application/x-www-form-urlencoded
so i change to send:
content-type: application/octet-stream
and also had to add this Content-Type to AWS signature
const params = {
Bucket: bucket,
Key: key,
Expires: expires,
ContentType: 'application/octet-stream'
}
const s3 = new AWS.S3()
s3.getSignedUrl('putObject', params)
I had the same issue, the problem I had was I imported the wrong environment variable, which means that my secret key for AWS was wrong. Based on reading all the answers, I would verify that all your access ID and secret key is right and there are no additional characters or anything.
If none of the other mentioned solution works for you , then try using
aws configure
this command will open a set of options asking for keys, region and output format.
Hope this helps!
Another possible issue might be that the meta values contain non US-ASCII characters. For me it helped to UrlEncode the values when adding them to the putRequest:
request.Metadata.Add(AmzMetaPrefix + "artist", HttpUtility.UrlEncode(song.Artist));
request.Metadata.Add(AmzMetaPrefix + "title", HttpUtility.UrlEncode(song.Title));
In my case I parsed an S3 url into its components.
For example:
Url: s3://bucket-name/path/to/file
Was parsed into:
Bucket: bucket-name
Path: /path/to/file
Having the path part containing a leading '/' failed the request.
I had the same issue. I had the default method, PUT set to define the pre-signed URL but was trying to perform a GET. The error was due to method mismatch.
When I gave the wrong secret key which is of value "secret" knowingly, it gave this error. I was expecting some valid error message details like "authentication failed" or something
I just experienced this uploading an image to S3 using the AWS SDK with React Native. It turned out to be caused by the ContentEncoding parameter.
Removing that parameter "fixed" the issue.
generating a fresh access key worked for me.
Most of the time it happens because of the wrong key (AWS_SECRET_ACCESS_KEY). Please cross verify your AWS_SECRET_ACCESS_KEY. Hope it will work...
After debugging and spending a lot of time, in my case, the issue was with the access_key_id and secret_access_key, just double check your credentials or generate new one if possible and make sure you are passing the credentials in params.
This issue happened to me because I was accidentally assigning the value of the ACCESS_KEY_ID to SECRET_ACCESS_KEY_ID. Once this was fixed everything worked fine.
Like others, I also had the similar issue but in java sdk v1. For me, below 2 fixes helped me.
My key to object looked like this /path/to/obj/. In this, i first removed the / in the beginning.
Further, point 1 alone did not solve the issue. I upgraded my sdk version from 1.9.x to 1.11.x
After applying both the fixes, it worked. So my suggestion is not slog it out. If nothing else is working, just try upgrading the lib.
I had a similar error, but for me it seemed to be caused by re-using an IAM user to work with S3 in two different Elastic Beanstalk environments. I treated the symptom by creating an identically permissioned IAM user for each environment and that made the error go away.
I don't know if anyone came to this issue while trying to test the outputted URL in browser but if you are using Postman and try to copy the generated url of AWS from the RAW tab, because of escaping backslashes you are going to get the above error.
Use the Pretty tab to copy and paste the url to see if it actually works.
I run into this issue recently and this solution solved my issue. It's for testing purposes to see if you actually retrieve the data through the url.
This answer is a reference to those who try to generate a download, temporary link from AWS or generally generate a URL from AWS to use.
I was getting this error in our shared environment where the SDK was being used, but using the same key/secret and the aws cli, it worked fine. The build system script had a space after the key and secret and session keys, which the code read in as well. So the fix for me was to adjust the build script to remove the spaces after the variables being used.
Just adding this for anyone who might miss that frustrating invisible space at the end of their creds.
In my case the bucketname was wrong, it included the first part of the key (bucketxxx/keyxxx) - there was nothing wrong with the signature.
In my case (python) it failed because I had these two lines of code in the file, inherited from an older code
http.client.HTTPConnection._http_vsn = 10
http.client.HTTPConnection._http_vsn_str = 'HTTP/1.0'
I encountered this in a Docker image, with a non-AWS S3 endpoint, when using the latest awscli version available to Debian stretch, i.e. version 1.11.13.
Upgrading to CLI version 1.16.84 resolved the issue.
To install the latest version of the CLI with a Dockerfile based on a Debian stretch image, instead of:
RUN apt-get update
RUN apt-get install -y awscli
RUN aws --version
Use:
RUN apt-get update
RUN apt-get install -y python-pip
RUN pip install awscli
RUN aws --version
I'm trying to use the AWS SDK for PHP to programatically upload a file to a bucket that's set to be a static website in the S3 Console.
The bucket is named foo.ourdomain.com and is hosted in eu-west. I'm using the following code to try and test if I can upload a file:
$client = \Aws\S3\S3Client::factory(array('key' => bla, 'secret' => bla));
$client->upload('foo.ourdomain.com', 'test.txt', 'hello world', 'public-read');
This is pretty much like it is in the examples, however, I received the following exception:
PHP Fatal error: Uncaught Aws\S3\Exception\PermanentRedirectException: AWS Error Code: PermanentRedirect, Status Code: 301, AWS Request ID: -, AWS Error Type: client, AWS Error Message: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint: "foo.ourdomain.com.s3.amazonaws.com"., User-Agent: aws-sdk-php2/2.4.8 Guzzle/3.7.4 curl/7.22.0 PHP/5.3.10-1ubuntu3.8
At this point I was surprised as there's no mention of this in the manual for the S3 SDK. But okay, I found a method setEndpoint and adjusted the code to:
$client = \Aws\S3\S3Client::factory(array('key' => bla, 'secret' => bla));
$client->setEndpoint('foo.ourdomain.com.s3.amazonaws.com');
$client->upload('foo.ourdomain.com', 'test.txt', 'hello world', 'public-read');
I assumed that'd work, but I'm getting the exact same error. I've doublechecked and the endpoint mentioned in the exception byte-for-byte matches the one I'm setting in the second line.
I've also tried using foo.ourdomain.com.s3-website-eu-west-1.amazonaws.com as the endpoint (this is the host our CNAME points to as per the S3 consoles instructions). Didn't work either.
I must be missing something, but I can't find it anywhere. Perhaps buckets set to 'static website' behave differently in a way which is not currently supported by the SDK? If so, I can't find mention of it in the docs nor in the management console.
Got it. The solution was to change the initialisation of the client to:
$client = \Aws\S3\S3Client::factory(array(
'key' => bla,
'secret' => bla,
'region' => 'eu-west-1'
));
I.e. rather than specify an endpoint I needed to explicitly set the region in the options array. I guess the example code happens to use whatever the default region is.
If you don't want to initialize the client to a specific region and/or you'll need to work with different regions, I have been successful in using the getBucketLocation/setRegion set of calls as follows:
// Bucket location is fetched
$m_bucketLocation = $I_s3->getBucketLocation(array(
'Bucket' => $s_backupBucket,
));
// Bucket location is specified before operation is made
$I_s3->setRegion($m_bucketLocation['Location']);
I have one extra call, but solved my issue without the need to intervene on the factory.