I have an image upload functionality on my website - I upload images directly to s3 bucket. What I do is I made that bucket public, I fetch images on my website and use public links of images from the s3 bucket to show them as html img tags.
Users of my website will take these images, possibly as links, and use them on their websites. I can see several problems here:
links of images are direct links to aws
the whole bucket is public
there will be a lot of unpredictable reads
Its a better idea to use cdn here (probably cloudfront). How to integrate cdn into this process though?
Can I upload images directly into cdn, without storing them in s3? Why would I need s3 if I have a cdn?
Any suggestions? Thanks a lot!
S3 is still required, since Cloudfront uses S3 as its origin. S3 remains the data store, but Cloudfront caches these objects at the edge.
Check out this walkthrough on adding a Cloudfront distribution to cache your static files.
This will allow you to use your own url for the static content, as well as restrict direct access to S3 via Origin Access Identity
As already answered, cloudfront needs an 'origin' and often types that origin is S3, so you can't use just cloudfront - the images need to exist somewhere.
One precaution you could do (not clear from your question if you are - you said the links are 'directy to aws'), is to serve all of these images under your own domain name - either directly from s3, or with cloudfront - doing that now will ensure that if you do ever need to switch where you store the images, you customers won't need to change their links and the process would be seamless for them.
Related
I'm using the AWS S3 bucket in a very simple way.
There's a bucket, <somebucket1>
There's a folder, <somebucket1>/sitevideos
And video files in it, <somebucket1>/sitevideos/video.mp4
I use this bucket so playback using HTML5 video (<video></video>) is more optimised and doesn't lag compared to just calling the video from the same server of the website (which is ridiculous).
The video files are encrypted, but they are set to be read-only to Public.
Now, my worries are, because they are public, people can download them from the S3 bucket instead of playing them on the website.
The Question
Is there a way to play a video file in S3 bucket, on an HTML video from a remote website, but will refuse downloads of the file if they are accessed directly via the S3 path?
If there are tutorials for this, I'd appreciate it. If this is already on the S3 documentation, I apologise for the laziness, please show me the link. I also heard that you can set them the permission to private, but they can still play on a remote server (although I haven't made that work).
Cheers & many thanks
A Bucket Policy can be configured to Restrict Access to a Specific HTTP Referrer.
For example, if a web page includes an image on the page, then the HTTP request for that object will include a referer. (I presume this would work for a video, too.)
However, this is not very good security, since the HTTP request can be easily manipulated to include the referer (eg in a web scraper).
A more secure method would be to use a Pre-Signed URL. This is a specially-constructed URL that grants time-limited access to a private Amazon S3 object.
When rendering the web page, your app would determine whether the user is permitted to access the object. If so, it would construct the pre-signed URL using AWS credentials. The URL can then be included in the standard HTML tags (eg <img src='...'>). The user will be able to access the object until the expiry time. If they shared the URL with somebody else (eg in a Tweet), other people would also be able to access the object until the expiry time.
By the way, Amazon CloudFront can also serve video content using various video protocols. It also supports pre-signed URLs (and also signed cookies).
I am, for the first time, implementing file uploads using S3 (in this case specifically user profile avatar images) using Flysystem. I'm currently at the point where I have created an S3 bucket, and a user can upload an image, which is then visible online in the bucket console.
I now need the ability to display those images when requested (i.e. viewing that user's profile). I assumed that the process for this would be to generate the URL (e.g https://s3.my-region.amazonaws.com/my-bucket/my-filename.jpeg) and use that as the src of an image tag however to do this, the file (or bucket) must be marked as public. This seemed reasonable to me because the files within are not really private. When updating the bucket to public status however you are presented with a message stating;
We highly recommend that you never grant any kind of public access to your S3 bucket.
Is there a different, or more secure, way to achieve direct image linking like this that a newcomer to AWS is not seeing?
The warning is there because many people unintentionally make information public. However, if you are happy for these particular files to be accessed by anyone on the Internet at any time, then you can certainly make the individual objects public or create an Amazon S3 bucket policy to make a particular path public.
The alternative method to granting access is to create an S3 Pre-Signed URL, which is a time-limited URL that grants access to a private object.
Your application would be responsible for verifying that the user should be given access to a particular object. It would then generate the URL, supplying a duration for the access. Your application can then insert the URL into the src field and the image would appear as normal. However, once the duration has passed, it will no longer be accessible.
This is typically used when providing access to private files -- similar to how DropBox gives access to a private file without making the file itself public.
I would recommend to put cloudfront at the front (no pun intended) of the static assets, this way it would serve it all over their datacenters and not just the region you uploaded it, and I think this would charge you less because it does not use bandwidth from your S3 bucket.
This way you give cloudfront permissions to your S3 bucket and there is no need to set files public in your bucket manually. Google how to set IAM user for cloudfront and S3 to get you set up.
The client has a site running on PHP deployed over AWS infrastructure. I have access to AWS console of a client.
There are some images in his site that is loading from cloudfront (which I understand is mapped to an S3 bucket). I need to update these images, but I do not know which S3 bucket they are in, since client has a lot of S3 buckets configured. How do I figure this one from the console?
First, you will need to find the CloudFront distribution that is serving the content. Find the distribution with the same cloudfront.net URL that you're using to access the images.
Then, look at the behaviors and origins of the distribution to determine when it is going to each origin, and which origin is serving the path. This will then tell you which Amazon S3 bucket is being used.
also to add on John's answer; after you have changed the image in origin S3 bucket; you will have to invalidate the cache for those images in CloudFront .. otherwise even though origin has changed u will keep seeing old images
We have six appication server(under LVS) which send request randomly to all servers.
Around 8 years back we used to store images in databases.
Pros : Can be accessed from all the application server
Cons : Slow
Around 5 years back, we shifted images to store as a file on one of the six application server with the help of nginx rules that make sure all image read/write request go to single server.
Pros : Fast
Cons : All images read/write request go to single server.
Question: Is there any better images to solve the following issue:
1. Can be accessed from all application server.
2. Fast access
Note : we move images to common image server after some time.
We don not move instantly as we dont want to reply on that server and also it will increase user upload time.
You can leverage the power of Content Delivery Networks (CDN) and storage buckets which is provided by services like AWS.
Upload all images to a single server say an AWS S3 bucket https://aws.amazon.com/s3/ which will help you to get all images from a central server and will be accessible from all application servers.
You can then link your S3 bucket with a CDN service like AWS's Cloudfront or some of the free services like Cloudflare.
https://aws.amazon.com/cloudfront/
Read more about how to use S3 for PHP here:
https://devcenter.heroku.com/articles/s3
http://docs.aws.amazon.com/AmazonS3/latest/dev/RetrieveObjSingleOpPHP.html
Read more about linking an S3 bucket to cloudfront here:
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/MigrateS3ToCloudFront.html
So AWS's S3 will provide you globally accessible images and Cloudfront CSN will provide you amazing speeds.
I have a (php) website where teachers upload recordings of their class, and the students can log in and then play back the recording.
I want to make these videos more secure. Currently, the videos are stored on my server, and anyone with the url can download them. So, (1) I want to store them somewhere that can't be downloaded just using a url. And second, I need to stop them from right-clicking and saving the video as it is being played.
I'm trying to work this out with s3 but not getting it...
Is this possible? Does it need to use a special player? Does streaming the video help (can any video be streamed)?
I appreciate the help, I've spent many hours researching this and just getting more confused as I go along!
There are a couple of options you may wish to use.
1. Amazon CloudFront RTMP Distribution
Amazon CloudFront is a Content Distribution Network that caches content closer to users worldwide, in over 60 locations. It also has the ability to service Real-Time Media Playback (RTMP) protocols. This means that your web page could present a media player (eg JW Player, Flowplayer, or Adobe Flash) and CloudFront can serve the content.
See: Working with RTMP Distributions
CloudFront Distributions can also service private content. Your application can generate a URL that provides content for a limited period of time. The content is served via a media protocol, so the entire file cannot be easily downloaded.
See: Serving Private Content through CloudFront
2. Amazon S3 Pre-Signed URLs
By default, all objects in Amazon S3 are private. You can then add permissions so that people can access your objects. This can be done via:
Access Control List permissions on individual objects
A Bucket Policy (as per yours above)
IAM Users and Groups
A Pre-Signed URL
A Pre-Signed URL can be used to grant access to S3 objects as a way of "overriding" access controls. A normally private object can be accessed via a URL by appending an expiry time and signature. This is a great way to serve private content without requiring a web server.
Similar to the above example with CloudFront, your application can generate a URL that provides access to S3 content for a limited time period. Once the period expires, the Pre-Signed URL will no longer function. However, during the active period, people would be able to download the entire file, so this might not be advisable for video content you wish to protect.