I am trying to upload a file using a signed url but I keep getting key too long error. running this via curl on cli works fine.
$result = "hl7/1/2df8c3ef-8eca-4e19-aae0-41b074f37c8e.hl7?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=CREDENTIALS%2F20211206%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20211206T144541Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQoJb3JpZ2luX2VjELX%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSJGMEQCIH%2BBvsVggdM0wLpK0Kd7gGVopCVYrDNxrHoJWdJZs%2Bl7AiBwFw1i3LNWsoDsK6SZzz8%2BPdHxYh%2Bn6T4DbHt5Ya5w8Sr6AwiO%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAIaDDc4MTcyNzU3MTE3NyIMK4bkPUK4yBIM3U65Ks4DQAkLuCkrZ%2F7tcoN7ItKGxmSBM0TMYbGq7dhsS1SWSxqffm%2BOM9QzEROrQRpt%2Bm%2BUQ3R5o7vz7Y4x5lxhkvIOyzXxVUqNznCBnWEYwEdK8aLPlZLiFkh2wznL2SeTLZsYO4LD3WXoY3AiVZZYZroNkJcUicNlectdsuoFWov5DIJ7k6PyAoEaRFk00Afa9W8NX5pN4NevvgclgqpceqeLD7L%2FBHBje13F68aJ6sv%2F%2FLeWZ4pvSR5ySgWaUaQ1aHHIsNVhuuZCgqC3R8wlN2AGRQ4h5M%2Byb6Tq6pFJwgI2CeP%2BgAFmMUh0feyJDROLq88i%2BRTSIEsUlH8GQGDg%2FSwIPBlL3Fg8TxDSzL8wa8Quk2h%2BgpNalvdRmIUR6iYem4DoNUxkihU%2F77g0TEOVLIl%2Fk20freTIWSeR8ZNCmIZOhUvm9Ea7TQl88B3jWq3oVWWG3xsKkEvCZ%2FwdnQAI20fUp8EtrnODtjjNNanmgWSafHURLTAm2HjtNbnKri%2B2mQmGQoHo2s2xX%2Fi6CYuEZI5yJBYuUSFZHhSum0h%2Bja643YDOp14TmYO38xFcIThxKDqaEMBjdRo15WvVDFcXSMsUdXLElXvOPMfHBoMdT4meMMiTuI0GOqYBT%2FMubfy97zprO%2FNP8xyb2u8qfDYeO0P7xBXdNFrbfVgX%2Fdr6%2Fhh8WSEz2K9PojxLN3NLGnfiMaUaUpev2kHcThP%2BuMkb%2FurH9Pl9QCFKsgKnhylrIKC0bQ%2B4TaEbJoRwh%2BRQUZGqaJfaPrMR5tAyYZ5kN4w1PXLRjpqi2U27B3qpGfu3jruEtOsuHtxM57tFQgyUHy3J%2FOdV7YVzYX%2B8cHSHkx2ndA%3D%3D&X-Amz-Signature=e653b4bbc96cba74979a087b3bc0f14ea6ff9fcd8eff0bd28234d897833240d1&X-Amz-SignedHeaders=host&x-id=PutObject";
$client = new S3Client( [
'version' => '2006-03-01',
'region' => 'eu-west-1',
'validate' => false
] );
$params = [
'Bucket' => env('RESULTS_UPLOAD_SERVICE_BUCKET'),
'Key' => $result,
'Body' => file_get_contents($fileData)
];
$response = $client->putObject($params);
Here is my curl command which works
curl --request PUT --upload-file 20211008192601_60461040005062.HL7 "https://domain.s3.eu-west-1.amazonaws.com/hl7/1/2df8c3ef-8eca-4e19-aae0-41b074f37c8e.hl7?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=ASIA3MAUYDTUUN444IX6%2F20211206%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Date=20211206T144541Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQoJb3JpZ2luX2VjELX%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaCWV1LXdlc3QtMSJGMEQCIH%2BBvsVggdM0wLpK0Kd7gGVopCVYrDNxrHoJWdJZs%2Bl7AiBwFw1i3LNWsoDsK6SZzz8%2BPdHxYh%2Bn6T4DbHt5Ya5w8Sr6AwiO%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAIaDDc4MTcyNzU3MTE3NyIMK4bkPUK4yBIM3U65Ks4DQAkLuCkrZ%2F7tcoN7ItKGxmSBM0TMYbGq7dhsS1SWSxqffm%2BOM9QzEROrQRpt%2Bm%2BUQ3R5o7vz7Y4x5lxhkvIOyzXxVUqNznCBnWEYwEdK8aLPlZLiFkh2wznL2SeTLZsYO4LD3WXoY3AiVZZYZroNkJcUicNlectdsuoFWov5DIJ7k6PyAoEaRFk00Afa9W8NX5pN4NevvgclgqpceqeLD7L%2FBHBje13F68aJ6sv%2F%2FLeWZ4pvSR5ySgWaUaQ1aHHIsNVhuuZCgqC3R8wlN2AGRQ4h5M%2Byb6Tq6pFJwgI2CeP%2BgAFmMUh0feyJDROLq88i%2BRTSIEsUlH8GQGDg%2FSwIPBlL3Fg8TxDSzL8wa8Quk2h%2BgpNalvdRmIUR6iYem4DoNUxkihU%2F77g0TEOVLIl%2Fk20freTIWSeR8ZNCmIZOhUvm9Ea7TQl88B3jWq3oVWWG3xsKkEvCZ%2FwdnQAI20fUp8EtrnODtjjNNanmgWSafHURLTAm2HjtNbnKri%2B2mQmGQoHo2s2xX%2Fi6CYuEZI5yJBYuUSFZHhSum0h%2Bja643YDOp14TmYO38xFcIThxKDqaEMBjdRo15WvVDFcXSMsUdXLElXvOPMfHBoMdT4meMMiTuI0GOqYBT%2FMubfy97zprO%2FNP8xyb2u8qfDYeO0P7xBXdNFrbfVgX%2Fdr6%2Fhh8WSEz2K9PojxLN3NLGnfiMaUaUpev2kHcThP%2BuMkb%2FurH9Pl9QCFKsgKnhylrIKC0bQ%2B4TaEbJoRwh%2BRQUZGqaJfaPrMR5tAyYZ5kN4w1PXLRjpqi2U27B3qpGfu3jruEtOsuHtxM57tFQgyUHy3J%2FOdV7YVzYX%2B8cHSHkx2ndA%3D%3D&X-Amz-Signature=e653b4bbc96cba74979a087b3bc0f14ea6ff9fcd8eff0bd28234d897833240d1&X-Amz-SignedHeaders=host&x-id=PutObject"
it errors with
<?xml version=\"1.0\" encoding=\"UTF-8\"?>
<Error><Code>KeyTooLongError</Code><Message>Your key is too long</Message><Size>1 (truncated...)
at /var/www/snapshot/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php:113)
[stacktrace]
You are receiving that error because you are passing pre-signed url result in to put request as key, That put request method should actually be used to upload directly with php sdk.
It seems like you want to upload file to a pre-signed url, Once you generate pre-signed url you will need to do a HTTP PUT request, you can do it directly from browser that is the usual way why we generate pre-signed url that expires after certain time so you don't expose your keys to public.
If you are using PHP directly you could simply use your keys to upload using above code or if you still want to upload to a pre-signed url with php see this link
How do I upload a file to an AWS Presigned URL with PHP Curl?
Related
I am trying to make a PutObject presigned request using the AWS S3 PHP SDK.
I have gotten the request to work but now I only want to allow my users to be able to only upload video files.I have tried a lot of combinations and searched a lot but I could not get it to work.
Here is the sample code I use:
$cmd = $this->s3client->getCommand('PutObject', [
'Bucket' => 'myBucket',
'Key' => 'inputs/' . $movie->getId(),
'ACL' => 'private',
'Conditions' => ['Starts-With', '$Content-Type', 'video/'], // I have tried other combinations but it seems to not work
]);
$request = $this->s3client->createPresignedRequest($cmd, '+30 minutes');
$movie->setSignedUrl((string)$request->getUri());
The signed url generated does never include the Content-Type in the X-Amz-SignedHeaders query parameter, only the host is included.
The putObject() request has no documented Conditions key.
You appear to be confusing S3's PUT upload interface with the pre-signed POST capability, which supports policy document conditions like ['Starts-With', '$Content-Type', 'video/'],
PUT does not support "starts with". It requires the exact Content-Type and the key for this (which should result in the header appearing in the X-Amz-SignedHeaders query string parameter) is simply ContentType. It goes in the outer parameters array, just like Bucket and Key.
But if you want to support multiple content types without knowing the specific type in advance, you need to use POST uploads.
I am trying to generate a URL that other system can use to upload a file to my S3 bucket. I looked over the documentation and similar issues, however, I cannot find the right solution.
I have tried creating the PreSigned URL multiple ways ($this->s3 is a reference so S3Client):
1:
$signedUrl = $this->s3->getCommand(
'PutObject',
array(
'Bucket' => $this->bucket,
'Key' => 'recording_test.mp3',
'Body' => ''
)
)->createPresignedUrl('+2 hours');
2:
$command = $this->s3->getCommand('PutObject', array('Bucket' => $this->bucket, 'Key' => 'recording_test3.mp3', 'Body' => ''));
$request = $command->prepare();
$signedUrl = $this->s3->createPresignedUrl($request, '+2 hours');
When trying to simply access the URL I get following error:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
I have also tried it this way:
$key = 'recording_test2.mp3';
$url = $this->bucket.'/'.$key;
$request = $this->s3->put($url);
$signedUrl = $this->s3->createPresignedUrl($request, '+2 hours');
However, this generates a URL in format of s3.amazonaws.com/bucket/key..., which yields an error about invalid endpoint, and that endpoint in format bucket.s3.amazonaws.com/key... should be used. Manually changing the URL gives the same invalid signature error as per above.
I cannot see what am I doing wrong or any other way to generate the PreSigned URL?
Thanks for help
When creating your signed URL, you should not use the Body key. The signed URL generated includes a signature of the body to be uploaded. Then, when you try upload a file using the URL, it can't match the empty string you are using to the content of the file.
Also, if you are using Curl to put the file, I recommend using \CURLFile, and remember to use curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PUT"); for your PUT request.
I am using s3 direct uploads along with a database to store the URLS of the files (along with other data like who uploaded etc).
To allow direct upload to s3, I'm creating a presigned URL like :
$s3 = App::make('aws')->createClient('s3', [
'credentials' => [
'key' => 'AAAAAAAAAAAAAAA',
'secret' => 'YYYYYYYYYYYYYYYYYYYY',
]
]);
$command = $s3->getCommand('PutObject', [
'#use_accelerate_endpoint'=>true,
'Bucket' => 'remdev-experimental',
'Key' => "newest newest.txt",
'Metadata' => array(
'foo' => "test",
)
]);
return response()->json(((string)$s3->createPresignedRequest($command, '+1 minutes')->getUri()));
Now, after the file from the client has finished uploading , I want my server to know about it. So I will require the client to send me a request , notifying about the fact that he has finished uploading. For this, I think the simplest(and also secure) way is to just allow the client to send back the signed URL that he just sent back.
Is there a way to parse the URL ?
I am interested in getting the object key , and more importantly , I want to verify that the URL has not been tampered with (meaning, the signature in the URL should match the rest of the contents). How can I do this in php sdk ?
The signed URL is the file's URL with the signature information in the query data. So a signed request for bucket: remdev-experimental file: abc.txt looks like https://s3.amazonaws.com/remdev-experimental/abc.txt?X-Amz-Date=date&X-Amz-Expires=60&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Signature=signature&X-Amz-Credential=SOMEID/20160703/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=Host&x-amz-security-token=some-long-token so all you need to do is get the URL's Path (/remdev-experimental/abc.txt and take everything after the 2nd slash.
Also you should be aware that you can have S3 redirect the browser to a URL using success_action_redirect in an HTTP post policy
Lastly you can have S3 trigger a notification to your server (via SQS, SNS, or Lambda) whenever a file is uploaded.
Currently I am generating the authenticated URL as :
$this->s3Client = new S3Client($this->options);
$cmd = $this->s3Client->getCommand('GetObject', [
'Bucket' => $bucket_name,
'Key' => $object_key
]);
$request = $this->s3Client->createPresignedRequest($cmd, PRESIGNED_URL_VALIDITY);
$presignedUrl = (string) $request->getUri();
Is there a way I could generate the query parameters in the authenticated URL on my own an not using createPresignedRequest.
I need to separately create :
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=
&X-Amz-Date=20130721T201207Z
&X-Amz-Expires=86400
&X-Amz-SignedHeaders=host
&X-Amz-Signature=<signature-value>
and the append it to the bucket file path to make it into the presigned request.
It's not exactly clear to me what you mean by "generate the query parameters in the authenticated URL on my own". AWS uses OpenSSL to deal with generating signatures and building signed URLs.
You could create signed URLs entirely w/o PHP, for e.g. on a Linux command line using the openssl command. Check this for a bash example: https://github.com/gdbtek/aws-tools/blob/master/sign_s3_url.bash.
You can also use PHP w/o the S3Client to generate signed URLs using the PHP OpenSSL extension and utilising openssl_* functions.
I'm trying to use the AWS SDK for PHP to programatically upload a file to a bucket that's set to be a static website in the S3 Console.
The bucket is named foo.ourdomain.com and is hosted in eu-west. I'm using the following code to try and test if I can upload a file:
$client = \Aws\S3\S3Client::factory(array('key' => bla, 'secret' => bla));
$client->upload('foo.ourdomain.com', 'test.txt', 'hello world', 'public-read');
This is pretty much like it is in the examples, however, I received the following exception:
PHP Fatal error: Uncaught Aws\S3\Exception\PermanentRedirectException: AWS Error Code: PermanentRedirect, Status Code: 301, AWS Request ID: -, AWS Error Type: client, AWS Error Message: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint: "foo.ourdomain.com.s3.amazonaws.com"., User-Agent: aws-sdk-php2/2.4.8 Guzzle/3.7.4 curl/7.22.0 PHP/5.3.10-1ubuntu3.8
At this point I was surprised as there's no mention of this in the manual for the S3 SDK. But okay, I found a method setEndpoint and adjusted the code to:
$client = \Aws\S3\S3Client::factory(array('key' => bla, 'secret' => bla));
$client->setEndpoint('foo.ourdomain.com.s3.amazonaws.com');
$client->upload('foo.ourdomain.com', 'test.txt', 'hello world', 'public-read');
I assumed that'd work, but I'm getting the exact same error. I've doublechecked and the endpoint mentioned in the exception byte-for-byte matches the one I'm setting in the second line.
I've also tried using foo.ourdomain.com.s3-website-eu-west-1.amazonaws.com as the endpoint (this is the host our CNAME points to as per the S3 consoles instructions). Didn't work either.
I must be missing something, but I can't find it anywhere. Perhaps buckets set to 'static website' behave differently in a way which is not currently supported by the SDK? If so, I can't find mention of it in the docs nor in the management console.
Got it. The solution was to change the initialisation of the client to:
$client = \Aws\S3\S3Client::factory(array(
'key' => bla,
'secret' => bla,
'region' => 'eu-west-1'
));
I.e. rather than specify an endpoint I needed to explicitly set the region in the options array. I guess the example code happens to use whatever the default region is.
If you don't want to initialize the client to a specific region and/or you'll need to work with different regions, I have been successful in using the getBucketLocation/setRegion set of calls as follows:
// Bucket location is fetched
$m_bucketLocation = $I_s3->getBucketLocation(array(
'Bucket' => $s_backupBucket,
));
// Bucket location is specified before operation is made
$I_s3->setRegion($m_bucketLocation['Location']);
I have one extra call, but solved my issue without the need to intervene on the factory.