I am having trouble getting Cloudfront videos to play when using a signed url. If I do NOT require a signed URL, everything works fine. Here is the code that signs the url:
function rsa_sha1_sign($policy, $private_key_filename)
{
$signature = "";
// load the private key
$fp = fopen($private_key_filename, "r");
$priv_key = fread($fp, 8192);
fclose($fp);
//echo $priv_key;
$pkeyid = openssl_get_privatekey($priv_key);
// compute signature
openssl_sign($policy, $signature, $pkeyid);
// free the key from memory
openssl_free_key($pkeyid);
//echo $signature;
return $signature;
}
function url_safe_base64_encode($value)
{
$encoded = base64_encode($value);
// replace unsafe characters +, = and / with
// the safe characters -, _ and ~
return str_replace(
array('+', '=', '/'),
array('-', '_', '~'),
$encoded);
}
// No restriction
$keyPairId = "KEYPAIRID-DIST-NOT-REQUIRING-SIGNEDURL";
$download_url = "http://URL-DIST-NOT-REQUIRING-SIGNEDURL.cloudfront.net/myvideo.mp4";
//This is just a flag to aid in switching between the 2 testing distributions
if($restrict) {
$key_pair_id = "KEYPAIRID-DIST-REQUIRING-SIGNEDURL"";
$download_url = "http://URL-DIST-REQUIRING-SIGNEDURL.cloudfront.net/myvideo.mp4";
}
$DateLessThan = time() + (24*7*60*60);
$policy = '{"Statement":[{"Resource":"'.$download_url.'","Condition":{"DateLessThan":{"AWS:EpochTime":'.$DateLessThan.'}}}]}';
$private_key_file = "/path/to/privatekey.pem";
$signature = rsa_sha1_sign($policy, $private_key_file);
$signature = url_safe_base64_encode($signature);
$final_url = $download_url.'?Policy='.url_safe_base64_encode($policy).'&Signature='.$signature.'&Key-Pair-Id='.$key_pair_id;
echo $final_url;
In the above, if I use the Cloudfront distribution that requires a signed URL (by passing in $restrict=1) then I get an error, "Video not found". In console I see that the GET request for the video was canceled (Status Text: cancelled... weirdly I see this twice). If I use the Distribution that doe NOT require a signed URL everything works fine and the video loads correctly.
What am I missing? The distributions are identical except for the requiring of the signed URL and they both use the same Amazon S3 bucket source for the video.
The player is flowplayer(HTML5) but since it works fine without the signed url I would assume the player isn't the problem.
Please see my answer here: Amazon S3 signed url not working with flowplayer
Hopefully that will help.
In my case, I needed to remove the "mp4:" prefix before signing the url, and then add it back on again.
Related
I'm using Google Cloud Storages Signed URLs feature with the below code. File names without spaces work fine, but those with spaces give this error:
The request signature we calculated does not match the signature you provided.
I'm passing the file name via ajax to the below code, which then returns the link to the user. I've tried a combination of rawurlencode, urlencode & urldecode but can't seem to work it out.
Thanks in advance
$bucket = 'savoy_storage';
if (isset($_GET['file']))
{
$file = urldecode($_GET['file']);
$link = storageURL( $bucket, $file);
echo $link;
}
//https://groups.google.com/forum/#!topic/google-api-php-client/kvN-yoyzf_w
function storageURL( $bucketName, $file) {
$expiry = time() + 3600;
$accessId = '1234#developer.gserviceaccount.com';
$stringPolicy = "GET\n\n\n".$expiry."\n/".$bucketName."/".$file;
$fp = fopen('key.pem', 'r');
$priv_key = fread($fp, 8192);
fclose($fp);
$pkeyid = openssl_get_privatekey($priv_key,"password");
if (openssl_sign( $stringPolicy, $signature, $pkeyid, 'sha256' )) {
$signature = urlencode( base64_encode( $signature ) );
return 'https://'.$bucketName.'.commondatastorage.googleapis.com/'.$file.'?
GoogleAccessId='.$accessId.'&Expires='.$expiry.'&Signature='.$signature;
}
}
If it helps here is the returned XML from the link:
<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>
The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.
</Message>
<StringToSign>
GET 1391312980 /savoy_storage/August%20promotions.xlsx
</StringToSign>
</Error>
I've written a PHP script that generates a signed CloudFront URL for RTMP with use in Flowplayer that's working just fine, but when I use the same signature generation method to create a download URL I get an AccessDenied XML file from Amazon. I've tried just about everything and I'm at my wits end. Anyone know why the signature would work for RTMP streaming, but that same signature generation method would fail for a download?
$keyPairId = 'APK...';
$privateKey = '/var/www/certs/pk-APK....pem';
$rtmp = false;
$distribution = 'd2m...';
// Get extension.
$extension = substr($this->getFilename(), strrpos($this->getFilename(), '.') + 1);
$fileName = substr($this->getFilename(), 0, strrpos($this->getFilename(), '.'));
$expires = strtotime(gmdate('Y-m-d H:i:s', strtotime('+3 hours')));
$json = '{"Statement":[{"Resource":"' . $fileName . '","Condition"{"DateLessThan":{"AWS:EpochTime":' . $expires . '}}}]}';
// read cloudfront private key pair
$fp = fopen($privateKey, 'r');
$priv_key = fread($fp, 8192);
fclose($fp);
// create the private key
$key = openssl_get_privatekey($priv_key);
// sign the policy with the private key
// depending on your php version you might have to use
// openssl_sign($json, $signed_policy, $key, OPENSSL_ALGO_SHA1)
openssl_sign($json, $signed_policy, $key);
openssl_free_key($key);
// create url safe signed policy
$base64_signed_policy = base64_encode($signed_policy);
$signature = str_replace(array('+', '=', '/'), array('-', '_', '~'), $base64_signed_policy);
// construct the url
$urlParams = urlencode($this->getFilename()) . '?Expires=' . $expires .'&Signature=' . $signature . '&Key-Pair-Id=' . $keyPairId;
$keyPairId;
if ($rtmp) {
$url = ( ($this->getExtension() != 'flv') ? $this->getExtension() . ':' : '' ) . $urlParams;
} else {
$url = 'https://' . $distribution . '.cloudfront.net/' . $urlParams;
}
First of all, signed RTMP urls are made differently than regular urls
RTMP distributions: Include only the stream name. For example, if the
full URL for a streaming video is:
rtmp://s5c39gqb8ow64r.cloudfront.net/videos/mp3_name.mp3
then use the following value for Resource:
videos/mp3_name
Regular signed urls contain the entire path.
Secondly, cloudfront RTMP distributions only distribute streaming media over RTMP. You said that you wanted a download url, so using an RTMP distribution will not enable you to download the file.
You probably want to create a cloudfront web distribution and link it to the same bucket, then generate a signed url using the web distribution, and access it that way.
I just wanted to ask if anyone could give me a tip to get private and secure files on S3 accessible only to logged in users or when the business logic wants them to be accessible. Here is the scenario...
A PHP Web application to enter billing details which generates a PDF
invoice on the fly and uploads it to the S3 bucket. (It actually
doesn't always generate and upload - only when the user wants to print or
download it - the code generates a pdf - uploads it to S3 and gets
back the url or the file accordingly)
Now, the file is accessible to anyone who has the url to the file on S3 bucket. I wanted to limit the file access in such a way that people who are not even logged in can get the file only after lets say answering a secret question etc.
Is generating a signed url which is valid for a small time the only answer here or am I looking at other possibilities also? Also, can we generate signed url's directly from S3 or do we need to do it through cloudfront?
Do suggest me a direction to research further... Thanks!
This might help? Amazon CloudFront Private Content
I would assume you will need to setup a VPC to ensure your user's credentials are verified and valid to keep the content "private".
This post https://css-tricks.com/snippets/php/generate-expiring-amazon-s3-link/ can help you. Don't forget to check out the comments as well.
*** EDIT
Generate Expiring Amazon S3 Link
You don't have to make files on Amazon S3 public (they aren't by default). But you can generate special keys to allow access to private files. These keys are passed through the URL and can be made to expire.
<?php
if( !function_exists( 'el_crypto_hmacSHA1' ) ) {
function el_crypto_hmacSHA1($key, $data, $blocksize = 64) {
if (strlen($key) > $blocksize) $key = pack('H*', sha1($key));
$key = str_pad($key, $blocksize, chr(0x00));
$ipad = str_repeat(chr(0x36), $blocksize);
$opad = str_repeat(chr(0x5c), $blocksize);
$hmac = pack( 'H*', sha1(
($key ^ $opad) . pack( 'H*', sha1(
($key ^ $ipad) . $data
))
));
return base64_encode($hmac);
}
}
if(!function_exists('el_s3_getTemporaryLink')){
function el_s3_getTemporaryLink($accessKey, $secretKey, $bucket, $path, $expires = 5) {
// Calculate expiry time
$expires = time() + intval(floatval($expires) * 60);
// Fix the path; encode and sanitize
$path = str_replace('%2F', '/', rawurlencode($path = ltrim($path, '/')));
// Path for signature starts with the bucket
$signpath = '/'. $bucket .'/'. $path;
// S3 friendly string to sign
$signsz = implode("\n", $pieces = array('GET', null, null, $expires, $signpath));
// Calculate the hash
$signature = el_crypto_hmacSHA1($secretKey, $signsz);
// Glue the URL ...
$url = sprintf('http://%s.s3.amazonaws.com/%s', $bucket, $path);
// ... to the query string ...
$qs = http_build_query($pieces = array(
'AWSAccessKeyId' => $accessKey,
'Expires' => $expires,
'Signature' => $signature,
));
// ... and return the URL!
return $url.'?'.$qs;
}
}
Usage
<?php
echo el_s3_getTemporaryLink('your-access-key', 'your-secret-key', 'bucket-name', '/path/to/file.mov');
We have a large number of videos/audo/media hosted on a custom domain on S3 and have created a set of functions in order to sign the URLs and allow them to be both streamable and downloadable. The problem is that the signed URL of course never works. The error is:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
Of course if we take the bytecode returned from this page and input it into the Amazon S3 Signature Tester and grab the bytecode from there it works just fine. Even if the string to sign from our function as well as the decoded byte code in the Signature Tester are identical, it never works.
It's called via a small block of PHP code:
$headers = createS3parameters($expiry, $file_type);
$request = preg_replace("/^.*?:\/\/.*\//", "/", $bucketurl);
$signature = signRequest($request, $expiry, $s3secret, $headers, "GET", $file_type);
$signed_request = "$bucketurl?AWSAccessKeyId=$s3key&Expires=$expiry&$headers&Signature=$signature";
This is the function which actually signs it.
function signRequest($request, $expiration, $s3secret, $headers = '', $type = 'GET', $content_type = 'default')
{
if ($expiration == 0 || $expiration == null)
{
$expiration = time() + 315576000; // 10 years (never)
}
if (strcmp($content_type, 'default') == 0)
{
$content_type = "";
}
// S3 tester spits out this format
/*$string = "$type\n".
"\n\n".
"$expiration\n".
"/mybucket$request?$headers";*/
$string = "$type\n".
"\n".
"$content_type\n".
"$expiration\n".
"$headers\n".
"$request";
// must be in UTF8 format
$string = utf8_encode(trim($string));
// encode to binary hash using sha1. require S3 bucket secret key
$hash = hash_hmac("sha1",$string, $s3secret,false);
// sha1 hash must be base64 encoded
$hash = base64_encode($hash);
// base64 encoded sha1 hash must be urlencoded
$signature = rawurlencode($hash);
return $signature;
}
Which then creates a URL such as:
http://mybucket.s3.amazonaws.com/My_Boring_Video.wmv?AWSAccessKeyId=AKIAIEXAMPLE6GA3WYQ&Expires=1344160808&response-content-type=application/force-download&response-expires=1344160808&Signature=OTIxOTI0YjNjMTA1NjMyNmJjYTk0MGE2YWJkMmI5OWQ3MGM2ZGY0MQ%3D%3D
Which unfortunately doesn't work. Is there an obvious problem here I've been staring at far too long to properly figure out?
UPDATE: Specs are specs, but are only help if they match actual practice.
Amazon's S3 specs say the signature should be formed as the following:
Signature = URL-Encode( Base64( HMAC-SHA1( YourSecretAccessKeyID, UTF-8-Encoding-Of( StringToSign ) ) ) );
StringToSign = HTTP-VERB + "\n" +
Content-MD5 + "\n" +
Content-Type + "\n" +
Expires + "\n" +
CanonicalizedAmzHeaders +
CanonicalizedResource;
HOWEVER the actual request needed looks like this:
StringToSign = HTTP-VERB + "\n" +
"\n" +
"\n" +
Expires + "\n" +
Bucket + CanonicalizedResource + "?" + CanonicalizedAmzHeaders;
Strangely enough, in PHP you also can't seem to do this:
$string = "$type\n".
"\n".
"\n".
"$expiration\n".
"/$bucket$request?$headers";
It changes the signature and ends up rejected so must all be on a single line. I haven't gone as far as checking whether this is a bug in our particular version of PHP or just a general PHP bug (or maybe intended functionality??!). You must also include the name of your bucket even if you are using the vanity URLs available such as mybucket.s3.amazonaws.com or mybucket.mydomain.com. The documentation doesn't specify what you can or can't do and I made the assumption that since we are using an S3 based vanity URL it (S3) would pick up on the domain name and translate it to the bucket.
I ended up changing my function to be the following:
function signRequest($bucket, $request, $expiration, $s3secret, $headers = '', $type = 'GET', $content_type = 'default')
{
if ($expiration == 0 || $expiration == null)
{
$expiration = time() + 315576000; // 10 years (never)
}
if (strcmp($content_type, 'default') == 0)
{
$content_type = "";
}
$headers = trim($headers, '&');
// This is the spec:
/*$string = "$type\n".
"\n".
"$content_type\n".
"$expiration\n".
"$headers\n".
"$bucket$request";*/
// but it will only work as this
$string = "$type\n\n\n$expiration\n/$bucket$request?$headers";
// this could be a single line of code but left otherwise for readability
// must be in UTF8 format
$string = utf8_encode(trim($string));
// encode to binary hash using sha1. require S3 bucket secret key
$hash = hash_hmac("sha1",$string, $s3secret,true);
// sha1 hash must be base64 encoded
$hash = base64_encode($hash);
// base64 encoded sha1 hash must be urlencoded
$signature = urlencode($hash);
return $signature;
}
Hopefully someone else finds this useful as well.
UPDATE (20180109): Adding in the function that calls this (aka let's make this blatently simple).
It helps understanding of what to pass to the signRequest() function.
private function genS3QueryString($bucketurl)
{
$file_type = 'application/force-download';
$expiry = '1831014000'; //Sun, Jan 09 2028 0700 UTC
$headers = '&response-content-type='.$file_type.'&response-expires='.$expiry;
$bucket = preg_replace("/^.*?:\/\/(.*)\.s3\.amazonaws\.com\/.*/", "$1", $bucketurl);
$request = preg_replace("/^.*?:\/\/.*\//", "/", $bucketurl);
$signature = $this->signRequest($bucket, $request, $expiry, S3_SECRET_KEY, $headers, 'GET', $file_type);
$signed_request = '?AWSAccessKeyId='.S3_KEY.'&Expires='.$expiry.$headers.'&Signature='.$signature;
return $signed_request;
}
You'll note however the first function only generates the encrypted signature section required to attach to a GET request for AWS. As per the docs (see the link far above) more is required for the request, as is formed from the second function just above.
The expiry time I believe can be a rolling expiry time if you so desire but should be sufficiently in the future that the asset becomes outdated and replaced long before the actual expiry time. An arbitrary time was chosen to sufficiently outlast the current version of the website.
The second function only requires the bucket url of the protected asset. The desired response type could be added to the call or simply changed so that asset (in this case just non-displayable) documents are downloaded as a different type.
The signed_request that is then returned must then be appended back onto the bucketurl for a working URI to request from a protected S3 asset.
This is the function pulled out of an old WP plugin for returning a signed Amazon S3 URL, but I can't get it to work! When I visit the signed URL it returns, I get this:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
function s3Url($text) {
$AWS_S3_KEY = 'KEY';
$AWS_S3_SECRET = 'SECRET';
$tag_pattern = '/(\[S3 bucket\=(.*?)\ text\=(.*?)\](.*?)\[\/S3\])/i';
define("AWS_S3_KEY", $AWS_S3_KEY); // replace this with your AWS S3 key
define("AWS_S3_SECRET", $AWS_S3_SECRET); // replace this with your secret key.
$expires = time()+get_option('expire_seconds');
if (preg_match_all ($tag_pattern, $text, $matches)) {
for ($m=0; $m<count($matches[0]); $m++) {
$bucket = $matches[2][$m];
$link_text = $matches[3][$m];
$resource = $matches[4][$m];
$string_to_sign = "GET\n\n\n$expires\n/".str_replace(".s3.amazonaws.com","",$bucket)."/$resource";
//$string_to_sign = "GET\n\n\n{$expires}\n/{$bucket}/{$resource}";
$signature = urlencode(base64_encode((hash_hmac("sha1", utf8_encode($string_to_sign), AWS_S3_SECRET, TRUE))));
$authentication_params = "AWSAccessKeyId=".AWS_S3_KEY;
$authentication_params.= "&Expires={$expires}";
$authentication_params.= "&Signature={$signature}";
$tag_pattern_match = "/(\[S3 bucket\=(.*?)\ text\={$link_text}\]{$resource}\[\/S3\])/i";
if(strlen($link_text) == 0)
{
$link = "http://{$bucket}/{$resource}?{$authentication_params}";
}
else
{
$link = "<a href='http://{$bucket}/{$resource}?{$authentication_params}'>{$link_text}</a>";
}
$text = preg_replace($tag_pattern_match,$link,$text);
}
}
return $text;
}
The example provided in the Amazon AWS PHP SDK: sdk-latest\sdk-1.3.5\sdk-1.3.5\_samples\cli-s3_get_urls_for_uploads.php the following code works quite well:
/* Execute our queue of batched requests. This may take a few seconds to a
few minutes depending on the size of the files and how fast your upload
speeds are. */
$file_upload_response = $s3->batch()->send();
/* Since a batch of requests will return multiple responses, let's
make sure they ALL came back successfully using `areOK()` (singular
responses use `isOK()`). */
if ($file_upload_response->areOK())
{
// Loop through the individual filenames
foreach ($individual_filenames as $filename)
{
/* Display a URL for each of the files we uploaded. Since uploads default to
private (you can choose to override this setting when uploading), we'll
pre-authenticate the file URL for the next 5 minutes. */
echo $s3->get_object_url($bucket, $filename, '5 minutes') . PHP_EOL . PHP_EOL;
}
}