Force download on GCS via App Engine using Signed URL - php

I get my file via:
require_once 'google/appengine/api/cloud_storage/CloudStorageTools.php';
use google\appengine\api\cloud_storage\CloudStorageTools;
$public_link = CloudStorageTools::getPublicUrl("gs://bucket/file.pdf", false);
If I go to $public_link in the browser, it shows the PDF inside the browser. I am trying to figure out how I can force the download of this file.
Google App Engine only has a 60 second timeout so I'm afraid the serve function wont work via GAE. Does anyone have any suggestions?
--
EDIT
Andrei Volga's previous answer in this post suggests I use a Signed URL with a response-content-distribution header.
So far, I am able to create a signed URL that successfully shows the file but I am not able to generate a signed url that has any sort of header at all aka create a signed URL that will force the download instead of just showing it.
This is what I have so far, most of which is courtesy of mloureiro.
function googleBuildConfigurationString($method, $expiration, $file, array $options = [])
{
$allowedMethods = ['GET', 'HEAD', 'PUT', 'DELETE'];
// initialize
$method = strtoupper($method);
$contentType = $options['Content_Type'];
$contentMd5 = $options['Content_MD5'] ? base64_encode($options['Content_MD5']) : '';
$headers = $options['Canonicalized_Extension_Headers'] ? $options['Canonicalized_Extension_Headers'] . PHP_EOL : '';
$file = $file ? $file : $options['Canonicalized_Resource'];
// validate
if(array_search($method, $allowedMethods) === false)
{
throw new RuntimeException("Method '{$method}' is not allowed");
}
if(!$expiration)
{
throw new RuntimeException("An expiration date should be provided.");
}
return <<<TXT
{$method}
{$contentMd5}
{$contentType}
{$expiration}
{$headers}{$file}
TXT;
}
function googleSignString($p12FilePath, $string)
{
$certs = [];
if (!openssl_pkcs12_read(file_get_contents($p12FilePath), $certs, 'notasecret'))
{
echo "Unable to parse the p12 file. OpenSSL error: " . openssl_error_string(); exit();
}
$RSAPrivateKey = openssl_pkey_get_private($certs["pkey"]);
$signed = '';
if(!openssl_sign( $string, $signed, $RSAPrivateKey, 'sha256' ))
{
error_log( 'openssl_sign failed!' );
$signed = 'failed';
}
else $signed = base64_encode($signed);
return $signed;
}
function googleBuildSignedUrl($serviceEmail, $file, $expiration, $signature)
{
return "http://storage.googleapis.com{$file}" . "?GoogleAccessId={$serviceEmail}" . "&Expires={$expiration}" . "&Signature=" . urlencode($signature);
}
$serviceEmail = '<EMAIL>';
$p12FilePath = '../../path/to/cert.p12';
$expiration = (new DateTime())->modify('+3hours')->getTimestamp();
$bucket = 'bucket';
$fileToGet = 'picture.jpg';
$file = "/{$bucket}/{$fileToGet}";
$string = googleBuildConfigurationString('GET', $expiration, $file, array("Canonicalized_Extension_Headers" => ''));
$signedString = googleSignString($p12FilePath, $string);
$signedUrl = googleBuildSignedUrl($serviceEmail, $file, $expiration, $signedString);
echo $signedUrl;

For small files you can use serve option instead of public URL with save-as option set to true. See documentation.
For large files you can use a Signed URL with response-content-disposition parameter.

You can add and additional query string only.
https://cloud.google.com/storage/docs/xml-api/reference-headers#responsecontentdisposition
response-content-disposition
A query string parameter that allows content-disposition to be overridden for authenticated GET requests.
Valid Values URL-encoded header to return instead of the content-disposition of the underlying object.
Example
?response-content-disposition=attachment%3B%20filename%3D%22foo%22

Related

Openssl_pkcs7_sign(): error opening file

it's my first time doing signing of cert using openssl. Keep hitting the above error and tried realpath() and appending file:// but still can't get openssl to sign the profile. I don't understand how this works. Any insights would be appreciated.
Edit1: I'm not sure which file is the problematic one. The error messages wasn't specific enough. Is there a way to tell?
Code and screenshots below:
function signProfile()
{
$filename = "./template.mobileconfig";
$filename = realpath($filename);
$outFilename = $filename . ".tmp";
$pkey = dirname(__FILE__) . "/PteKey.key";
$pkey = realpath($pkey);
$certFile = dirname(__FILE__) . "/CertToSign.crt";
$certFile = realpath($certFile);
// try signing the plain XML profile
if (openssl_pkcs7_sign($filename, $outFilename, 'file://'.$certFile, array('file://'.$pkey, ""), array(), 0, ""))
{
// get the data back from the filesystem
$signedString = file_get_contents($outFilename);
// trim the fat
$trimmedString = preg_replace('/(.+\n)+\n/', '', $signedString, 1);
// convert to binary (DER)
$decodedString = base64_decode($trimmedString);
// write the file back to the filesystem (using the filename originally given)
$fh = fopen($filename, 'w');
fwrite($fh, $decodedString);
fclose($fh);
// delete the temporary file
unlink($outFilename);
return TRUE;
}
else
{
return FALSE;
}
}
Remove unwanted fields if not used in
openssl_pkcs7_sign($mobileConfig, $tmpMobileConfig, $certFile, array($pkey, ""), array());
Make sure file paths are correctly supplied.
require_once('variables.php'); //stores abs path to $tmpMobileConfig/$pteKeyPath/$CertToSignPath
$prepend = "file://";
$mobileConfig = realpath("./template.mobileconfig");
$pkey = $prepend . $pteKeyPath;
$pkey = str_replace('\\', '/', $pkey);
$certFile = $prepend . $CertToSignPath;
$certFile = str_replace('\\', '/', $certFile);
$isSignedCert = openssl_pkcs7_sign($mobileConfig, $tmpMobileConfig, $certFile, array($pkey, ""), array());

CloudFront URL signature works fine for RTMP, why won't it work for download URL?

I've written a PHP script that generates a signed CloudFront URL for RTMP with use in Flowplayer that's working just fine, but when I use the same signature generation method to create a download URL I get an AccessDenied XML file from Amazon. I've tried just about everything and I'm at my wits end. Anyone know why the signature would work for RTMP streaming, but that same signature generation method would fail for a download?
$keyPairId = 'APK...';
$privateKey = '/var/www/certs/pk-APK....pem';
$rtmp = false;
$distribution = 'd2m...';
// Get extension.
$extension = substr($this->getFilename(), strrpos($this->getFilename(), '.') + 1);
$fileName = substr($this->getFilename(), 0, strrpos($this->getFilename(), '.'));
$expires = strtotime(gmdate('Y-m-d H:i:s', strtotime('+3 hours')));
$json = '{"Statement":[{"Resource":"' . $fileName . '","Condition"{"DateLessThan":{"AWS:EpochTime":' . $expires . '}}}]}';
// read cloudfront private key pair
$fp = fopen($privateKey, 'r');
$priv_key = fread($fp, 8192);
fclose($fp);
// create the private key
$key = openssl_get_privatekey($priv_key);
// sign the policy with the private key
// depending on your php version you might have to use
// openssl_sign($json, $signed_policy, $key, OPENSSL_ALGO_SHA1)
openssl_sign($json, $signed_policy, $key);
openssl_free_key($key);
// create url safe signed policy
$base64_signed_policy = base64_encode($signed_policy);
$signature = str_replace(array('+', '=', '/'), array('-', '_', '~'), $base64_signed_policy);
// construct the url
$urlParams = urlencode($this->getFilename()) . '?Expires=' . $expires .'&Signature=' . $signature . '&Key-Pair-Id=' . $keyPairId;
$keyPairId;
if ($rtmp) {
$url = ( ($this->getExtension() != 'flv') ? $this->getExtension() . ':' : '' ) . $urlParams;
} else {
$url = 'https://' . $distribution . '.cloudfront.net/' . $urlParams;
}
First of all, signed RTMP urls are made differently than regular urls
RTMP distributions: Include only the stream name. For example, if the
full URL for a streaming video is:
rtmp://s5c39gqb8ow64r.cloudfront.net/videos/mp3_name.mp3
then use the following value for Resource:
videos/mp3_name
Regular signed urls contain the entire path.
Secondly, cloudfront RTMP distributions only distribute streaming media over RTMP. You said that you wanted a download url, so using an RTMP distribution will not enable you to download the file.
You probably want to create a cloudfront web distribution and link it to the same bucket, then generate a signed url using the web distribution, and access it that way.

Google Storage REST PUT With Signed URLs

I'm trying to upload the base64 data of an image directly through javascript to Google Storage using signed URLs as authentication, which is apparently possible to do.
According to developers.google.com/storage/docs/reference-methods#putobject there are only six headers that need to be set for this to work. Also for the header 'Authorization' I'm attempting to use the last option here:
developers.google.com/storage/docs/reference-headers#authorization
Which is 'A signature' developers.google.com/storage/docs/authentication#service_accounts
The only thing I want to use PHP for is to get the signature. Here is what I have been trying to get working with no success.
PHP & JS page/code
<?php
$theDate = Date(DATE_RFC822);
function signedURL( $filename, $bucket, $method = 'PUT' ) {
$signature = "";
$duration = 30;
$emailID = "980000000000-ytyertyr#developer.gserviceaccount.com";
$certs = array();
$priv_key = file_get_contents("9999999999999999999999999999-privatekey.p12");
if (!openssl_pkcs12_read($priv_key, $certs, 'notasecret')) { echo "Unable to parse the p12 file. OpenSSL error: " . openssl_error_string(); exit(); }
$expires = time() + $duration;
$to_sign = ( $method . "\n\n\n" . $expires . "\n" . "/" . $bucket . "/" . $filename );
$RSAPrivateKey = openssl_pkey_get_private($certs["pkey"]);
if (!openssl_sign( $to_sign, $signature, $RSAPrivateKey, 'sha256' ))
{
error_log( 'openssl_sign failed!' );
$signature = 'failed';
} else {
$signature = urlencode( base64_encode( $signature ) );
}
return (
'http://storage.googleapis.com/' . $bucket . '/' . $filename . '?GoogleAccessId=' . $emailID . '&Expires=' . $expires . '&Signature=' . $signature
);
openssl_free_key($RSAPrivateKey);
}
?>
<script>
var base64img = 'data:image/png;base64,AAABAAIAICA....snip...A';
var xhr = new XMLHttpRequest();
//PUT test - PUT status "(Canceled)" - OPTION status 200 (OK)
xhr.open("PUT", "<?php echo signedURL('test.png', 'mybucket'); ?>");
//xhr.setRequestHeader("Content-type", "image/png");
xhr.setRequestHeader("x-goog-acl", "public-read"); //try to set public read on file
xhr.setRequestHeader("Content-Length", base64img.length); // Chrome throws error (Refused to set unsafe header "Content-Length" )
xhr.send( base64img );
//GET test.txt temp file - working and returning 200 status (signing must be working ?)
/*
xhr.open("GET", "<?php echo signedURL('test.txt', 'mybucket', 'GET'); ?>");
xhr.send();
*/
//
</script>
Cors xml (seems to be fine) - I've set a wildcard only while testing and a low cache/maxage time
<?xml version="1.0" ?>
<CorsConfig>
<Cors>
<Origins>
<Origin>*</Origin>
</Origins>
<Methods>
<Method>GET</Method>
<Method>HEAD</Method>
<Method>OPTIONS</Method>
<Method>PUT</Method>
</Methods>
<ResponseHeaders>
<ResponseHeader>accept-encoding</ResponseHeader>
<ResponseHeader>cache-control</ResponseHeader>
<ResponseHeader>content-length</ResponseHeader>
<ResponseHeader>content-type</ResponseHeader>
<ResponseHeader>expect</ResponseHeader>
<ResponseHeader>if-modified-since</ResponseHeader>
<ResponseHeader>origin</ResponseHeader>
<ResponseHeader>range</ResponseHeader>
<ResponseHeader>referer</ResponseHeader>
<ResponseHeader>x-goog-acl</ResponseHeader>
<ResponseHeader>x-goog-api-version</ResponseHeader>
</ResponseHeaders>
<MaxAgeSec>900</MaxAgeSec>
</Cors>
</CorsConfig>
I've tested the GET method on a file and get a 200 status back now (\n\n - fix)
Update:
Looking in Firefox it does return a 403, unlike Chrome.
So the following lines are weird, as the conflate signed URLs with OAuth and PUT with POST:
# This looks like a PUT to signed URL
xhr.open("PUT", '<?php echo signedURL('imgfile.png','PUT',30,'mybucketname'); ?>', true);
# But multipart requires POST
xhr.setRequestHeader("Content-type", "multipart/form-data; boundary="+boundary);
# And here's a second form of authorization
xhr.setRequestHeader("Authorization", "OAuth <?php echo $signature; ?>");
multipart/form-data uploads require POST verb and are intended for html forms: Google Cloud Storage : PUT Object vs POST Object to upload file.?.
As long as you are sending a custom headers in an XMLHttpRequest I would recommend using PUT with either OAuth credentials:
xhr.open("PUT", "https://storage.googleapis.com/mybucketname/imgfile.png");
xhr.setRequestHeader("Authorization", "OAuth Bearer 1234567abcdefg");
xhr.setRequestHeader("Content-Length", raw_img_bytes.length);
xhr.send(raw_img_bytes);
or a signed url:
xhr.open("PUT", "https://storage.googleapis.com/mybucketname/imgfile.png?" +
"GoogleAccessId=1234567890123#developer.gserviceaccount.com&" +
"Expires=136891473&" +
"Signature=BClz9e...WvPcwN%2BmWBPqwg...sQI8IQi1493mw%3D");
xhr.setRequestHeader("Content-Length", raw_img_bytes.length);
xhr.send(raw_img_bytres);
I gess your Content-Type is something known (like Content-Type:video/mp4 for instance)? Try to upload a file with not known extention. For me, PUT is working in this case, not when Content-Type is not empty...
I don't understand why...

Amazon S3 Signed URL in PHP

This is the function pulled out of an old WP plugin for returning a signed Amazon S3 URL, but I can't get it to work! When I visit the signed URL it returns, I get this:
The request signature we calculated does not match the signature you provided. Check your key and signing method.
function s3Url($text) {
$AWS_S3_KEY = 'KEY';
$AWS_S3_SECRET = 'SECRET';
$tag_pattern = '/(\[S3 bucket\=(.*?)\ text\=(.*?)\](.*?)\[\/S3\])/i';
define("AWS_S3_KEY", $AWS_S3_KEY); // replace this with your AWS S3 key
define("AWS_S3_SECRET", $AWS_S3_SECRET); // replace this with your secret key.
$expires = time()+get_option('expire_seconds');
if (preg_match_all ($tag_pattern, $text, $matches)) {
for ($m=0; $m<count($matches[0]); $m++) {
$bucket = $matches[2][$m];
$link_text = $matches[3][$m];
$resource = $matches[4][$m];
$string_to_sign = "GET\n\n\n$expires\n/".str_replace(".s3.amazonaws.com","",$bucket)."/$resource";
//$string_to_sign = "GET\n\n\n{$expires}\n/{$bucket}/{$resource}";
$signature = urlencode(base64_encode((hash_hmac("sha1", utf8_encode($string_to_sign), AWS_S3_SECRET, TRUE))));
$authentication_params = "AWSAccessKeyId=".AWS_S3_KEY;
$authentication_params.= "&Expires={$expires}";
$authentication_params.= "&Signature={$signature}";
$tag_pattern_match = "/(\[S3 bucket\=(.*?)\ text\={$link_text}\]{$resource}\[\/S3\])/i";
if(strlen($link_text) == 0)
{
$link = "http://{$bucket}/{$resource}?{$authentication_params}";
}
else
{
$link = "<a href='http://{$bucket}/{$resource}?{$authentication_params}'>{$link_text}</a>";
}
$text = preg_replace($tag_pattern_match,$link,$text);
}
}
return $text;
}
The example provided in the Amazon AWS PHP SDK: sdk-latest\sdk-1.3.5\sdk-1.3.5\_samples\cli-s3_get_urls_for_uploads.php the following code works quite well:
/* Execute our queue of batched requests. This may take a few seconds to a
few minutes depending on the size of the files and how fast your upload
speeds are. */
$file_upload_response = $s3->batch()->send();
/* Since a batch of requests will return multiple responses, let's
make sure they ALL came back successfully using `areOK()` (singular
responses use `isOK()`). */
if ($file_upload_response->areOK())
{
// Loop through the individual filenames
foreach ($individual_filenames as $filename)
{
/* Display a URL for each of the files we uploaded. Since uploads default to
private (you can choose to override this setting when uploading), we'll
pre-authenticate the file URL for the next 5 minutes. */
echo $s3->get_object_url($bucket, $filename, '5 minutes') . PHP_EOL . PHP_EOL;
}
}

Download rapidshare file using rapidshare api in php

I am trying to download a rapidshare file using its "download" subroutine as a free user. The following is the code that I use to get response from the subroutine.
function rs_download($params)
{
$url = "http://api.rapidshare.com/cgi-bin/rsapi.cgi?sub=download&fileid=".$params['fileid']."&filename=".$params['filename'];
$reply = #file_get_contents($url);
if(!$reply)
{
return false;
}
$result_arr = array();
$result_keys = array(0=> 'hostname', 1=>'dlauth', 2=>'countdown_time', 3=>'md5hex');
if( preg_match("/DL:(.*)/", $reply, $reply_matches) )
{
$reply_altered = $reply_matches[1];
}
else
{
return false;
}
foreach( explode(',', $reply_altered) as $index => $value )
{
$result_arr[ $result_keys[$index] ] = $value;
}
return $result_arr;
}
For instance; trying to download this...
http://rapidshare.com/files/440817141/AutoRun__live-down.com_Champ.rar
I pass the fileid(440817141) and filename(AutoRun__live-down.com_Champ.rar) to rs_download(...) and I get a response just as rapidshare's api doc says.
The rapidshare api doc (see "sub=download") says call the server hostname with the download authentication string but I couldn't figure out what form the url should take.
Any suggestions?, I tried
$download_url = "http://$the-hostname/$the-dlauth-string/files/$fileid/$filename"
and a couple other variations of the above, nothing worked.
I use curl to download the file, like the following;
$cr = curl_init();
$fp = fopen ("d:/downloaded_files/file1.rar", "w");
// set curl options
$curl_options = array(
CURLOPT_URL => $download_url
,CURLOPT_FILE => $fp
,CURLOPT_HEADER => false
,CURLOPT_CONNECTTIMEOUT => 0
,CURLOPT_FOLLOWLOCATION => true
);
curl_setopt_array($cr, $curl_options);
curl_exec($cr);
curl_close($cr);
fclose($fp);
The above curl code doesn't seem to work, nothing gets downloaded. Probably its the download url that is incorrect.
Also tried this format for the download url:
"http://rs$serverid$shorthost.rapidshare.com/files/$fileid/$filename"
With this curl writes a file entry but that is all it does(writes a 0/1 kb file).
Here is the code that I use to get the serverid, shorthost, among a few other values from rapidshare.
function rs_checkfile($params)
{
$url = "http://api.rapidshare.com/cgi-bin/rsapi.cgi?sub=checkfiles_v1&files=".$params['fileids']."&filenames=".$params['filenames'];
// the response from rapishare would a string something like:
// 440817141,AutoRun__live-down.com_Champ.rar,47768,20,1,l3,0
$reply = #file_get_contents($url);
if(!$reply)
{
return false;
}
$result_arr = array();
$result_keys = array(0=> 'file_id', 1=>'file_name', 2=>'file_size', 3=>'server_id', 4=>'file_status', 5=>'short_host'
, 6=>'md5');
foreach( explode(',', $reply) as $index => $value )
{
$result_arr[ $result_keys[$index] ] = $value;
}
return $result_arr;
}
rs_checkfile(...) takes comma seperated fileids and filenames(no commas if calling for a single file)
Thanks in advance for any suggestions.
You start by requesting ?sub=download&fileid=X&filename=Y, and it returns $hostname,$dlauth,$countdown,$md5hex.. since you're a free user you have to delay for $countdown seconds, and then call ?sub=download&fileid=X&filename=Y&dlauth=Z to perform the download.
There's a working implementation in python here that would probably answer any of your other questions.

Categories