I am using S3 to accept direct uploads from the user to S3. Therefore I will be using pre-signed urls.
After successful upload, AWS Lambda will make sure that the file upload is an image, and then the client will tell my server that he has finished uploading.
Then my server will check if that file exists in S3 (if Lambda detects an invalid image, it deletes it). If it does, then the rest of the application logic will follow.
However, there is a loophole in this mechanism. A user can use the same url to upload a malicious file after telling my server that he has finished uploading (and initially passing a valid file).
Lambda will still delete the file, but now my server will think that a file exists whereas it actually does not.
Is there any way to generate a one-time upload pre-signed url, or is it possible to forcefully invalidate a url that was generated but has not yet expired?
A pre-signed URL expires at a set date/time. It is not possible to create a one-time use pre-signed URL.
It is also not possible to invalidate a pre-signed URL. However, the pre-signed URL uses permissions from the Access Key that is referenced by the pre-signed URL. If permissions are removed from the User linked to the Access Key, then the pre-signed URL will not work.
Turning this into an answer...
Once a file is uploaded, have Lambda move it (using the Copy Object API), i.e. from uploads/123.png to received/123.png or something similar.
If a malicious user attempts to re-use the signed URL, it'll go to uploads/123.png. Worst-case, Lambda checks it again and rejects the new file. Since your server's looking in received/ instead of uploads/ for files to process, we've rendered things safe.
Related
I'm using the AWS S3 bucket in a very simple way.
There's a bucket, <somebucket1>
There's a folder, <somebucket1>/sitevideos
And video files in it, <somebucket1>/sitevideos/video.mp4
I use this bucket so playback using HTML5 video (<video></video>) is more optimised and doesn't lag compared to just calling the video from the same server of the website (which is ridiculous).
The video files are encrypted, but they are set to be read-only to Public.
Now, my worries are, because they are public, people can download them from the S3 bucket instead of playing them on the website.
The Question
Is there a way to play a video file in S3 bucket, on an HTML video from a remote website, but will refuse downloads of the file if they are accessed directly via the S3 path?
If there are tutorials for this, I'd appreciate it. If this is already on the S3 documentation, I apologise for the laziness, please show me the link. I also heard that you can set them the permission to private, but they can still play on a remote server (although I haven't made that work).
Cheers & many thanks
A Bucket Policy can be configured to Restrict Access to a Specific HTTP Referrer.
For example, if a web page includes an image on the page, then the HTTP request for that object will include a referer. (I presume this would work for a video, too.)
However, this is not very good security, since the HTTP request can be easily manipulated to include the referer (eg in a web scraper).
A more secure method would be to use a Pre-Signed URL. This is a specially-constructed URL that grants time-limited access to a private Amazon S3 object.
When rendering the web page, your app would determine whether the user is permitted to access the object. If so, it would construct the pre-signed URL using AWS credentials. The URL can then be included in the standard HTML tags (eg <img src='...'>). The user will be able to access the object until the expiry time. If they shared the URL with somebody else (eg in a Tweet), other people would also be able to access the object until the expiry time.
By the way, Amazon CloudFront can also serve video content using various video protocols. It also supports pre-signed URLs (and also signed cookies).
I am, for the first time, implementing file uploads using S3 (in this case specifically user profile avatar images) using Flysystem. I'm currently at the point where I have created an S3 bucket, and a user can upload an image, which is then visible online in the bucket console.
I now need the ability to display those images when requested (i.e. viewing that user's profile). I assumed that the process for this would be to generate the URL (e.g https://s3.my-region.amazonaws.com/my-bucket/my-filename.jpeg) and use that as the src of an image tag however to do this, the file (or bucket) must be marked as public. This seemed reasonable to me because the files within are not really private. When updating the bucket to public status however you are presented with a message stating;
We highly recommend that you never grant any kind of public access to your S3 bucket.
Is there a different, or more secure, way to achieve direct image linking like this that a newcomer to AWS is not seeing?
The warning is there because many people unintentionally make information public. However, if you are happy for these particular files to be accessed by anyone on the Internet at any time, then you can certainly make the individual objects public or create an Amazon S3 bucket policy to make a particular path public.
The alternative method to granting access is to create an S3 Pre-Signed URL, which is a time-limited URL that grants access to a private object.
Your application would be responsible for verifying that the user should be given access to a particular object. It would then generate the URL, supplying a duration for the access. Your application can then insert the URL into the src field and the image would appear as normal. However, once the duration has passed, it will no longer be accessible.
This is typically used when providing access to private files -- similar to how DropBox gives access to a private file without making the file itself public.
I would recommend to put cloudfront at the front (no pun intended) of the static assets, this way it would serve it all over their datacenters and not just the region you uploaded it, and I think this would charge you less because it does not use bandwidth from your S3 bucket.
This way you give cloudfront permissions to your S3 bucket and there is no need to set files public in your bucket manually. Google how to set IAM user for cloudfront and S3 to get you set up.
I need to use s3 to store content from users and govern access, in a social network type of service.
So far this is what I have thought of doing:
Client tells LAMP server he wants to upload a file
LAMP authenticates, and generates a presigned url for s3 where user can upload. It also creates an encrypted version of that key using private key. Then it adds this key, along with the user who started it in a mysql table(along with when it was started)
LAMP then sends the key and the digital signature from 2. to client.
Client then uploads the file to s3
After finishing, he tells LAMP that that file was completed. It sends in the key and the digital signature.
LAMP makes sure both the key and the signature match. If they do, LAMP knows that the client is honest about the key being given to him(and he has not randomly generated it)
LAMP then checks s3 to make sure that the file with that key exists, if it does,then delete the row which was added in 2.
After asking around, I was told that it is not possible for s3 itself to verify that a file is a 'valid' file of a certain type(I want to enforce only images )
So I decided to use aws lambda to verify it(if its a 'wrong' file, just delete it) . Just after the file is uploaded to s3, lambda can be fired.
However, it is possible that BEFORE lambda finishes checking the file,step 7 above gets executed. This means my server will think file is valid.
Is there any way to make an s3 upload+lambda execution atomic ?
Any suggestions are welcome
I need service that allow to upload large files to S3, only for authenticated client.
So,
1. I need to check if is client has an access (for example HMAC or any unique key)
2. Check file format (extension/MIME-type, it must be only MUSIC)
3. If everything is ok - upload it to S3
Looks very simple. But, I don't want to store file at the service. I want to stream it directly to S3 after all checkings. Or, if auth. data is invalid, request must be aborted and file wouldn't e send.
Can you advise what technology I should use or right way for searching?
I have a bucket with files in it in AS3. I have access to the PHP API and a server that can send requests to Amazon on command.
What I want to do is grant access to a file in my bucket using an HTTP GET/POST request. From what I understand using this function:
get_object_url ( $bucket, $filename, $preauth, $opt )
I can make the file publicly accessible for the $preauth amount of time at a given URL. I don't want to do that, I want the file to be privately available at a URL with required POST or GET credentials (deciding who can access the file would be based on a database containing application 'users' and their permissions). I understand the security implications of passing any kind of credentials over GET or POST on a non-HTTPS connection.
Is this possible? I could just download the file from AS3 to my server for the extent of the transaction then do all the controls on my own box, but that's an expensive solution (two file downloads instead of one, when my server shouldn't have had to do a download at all) to a seemingly easy problem.
The short answer is no.
You could look at Amazons IAM for some more ways to secure the content especially in conjunction with Cloudfront but essentially there is no way to provide access to content by passing along a username and password.
Of course, if you are already authenticating users on your site, then you can only supply the signed url to those users. The url only has to be valid at the time the user initiates the download and not for the entire duration of the download.
Also, if you intend to use your server as a proxy between S3 and the user you'll be removing a lot of the benefits of using S3 in the first place. But you could use EC2 as the server to remove the extra cost you mentioned - transfers between S3 and EC2 are free.