So as we all know Cloudfront now supports uploading to S3 via the edge points.
However, I'm not really sure how to do this? I know it'll not be fully featured (i.e. not support authorize headers and part uploads) but I'm keen to do straight uploads.
I'm working in PHP, though it doesn't appear this is supported on the API yet as a method. Doing something rather simpler looks to require various authorisation milestones.
Has anyone found the best way to do this yet or just some suggestions for the best way as I'm trying CURL POST and other such things to no avail.
You should probably read more about the browser-based POST uploads feature of Amazon S3 on the AWS docs. Doing an POST upload to Amazon S3 requires you to send a special JSON policy doc along with your POST request and upload. The S3 PostObject class in the AWS SDK for PHP is helpful for generating this policy, based on your provided options, as well as generating other form element values you need to include with your form/request.
Though I haven't tried it yet, you probably just need to swap out the S3 bucket endpoint for your CloudFront distribution endpoint to do the upload via the CloudFront edge location. Also, make sure your distribution is configured to accept POST requests. To get a more official answer to your question, I'd ask the Amazon CloudFront team on the CloudFront forum.
You should be able to do this now, I would give it another try. Be sure to use the S3 bucket's DNS name when signing your request, not your CloudFront distribution's.
Related
I'm stuck wondering what the best solution is to handling large file uploads and sending them to a third-party API. Any pointers on what would be a good solution would be very welcome. Thank you in advance.
The end goal is to send video files to this API - https://docs.bunny.net/reference/manage-videos#video_uploadvideo. The complication is that the files are often large - up to 5GB in size.
I have an existing website built in PHP7 that runs on a LAMP setup on Amazon Lightsail and I want to add a feature for users to upload video files.
Currently I'm uploading the files directly to Amazon S3 using a pre-signed URL. This part is working fine.
But I need to send the files to the API mentioned above. This is where I get stuck!
I think there's two options to explore - (1) find a way to upload directly to the API and skip the S3 upload or (2) continue with uploading to S3 first and then transfer to the API. But I'm not sure if option 1 is even possible or how to do option 2!
With option 1, I'm wondering if there's a way to upload the files from the users directly to the API. If I do this using the regular HTML form upload, then the files are stored temporarily on my server before I can use cURL through PHP to transfer them to the API. This is really time consuming and feels very inefficient. But I don't know how else to send the files to the API without them first being on my server. Maybe there's an option here that I don't know about!
With option 2, I can already upload large files directly to S3 with pre-signed URLs and this process seems to run fine. But I don't know how I would then send the file from S3 to the API. I can use an S3 trigger on new files. But when I looked at Lambda, they have a tiny file size limit. Because my site is hosted on Lightsail, I noticed they have a container option. But I don't know if that can be used for this purpose and if so, how.
Basically, I'm not sure what solution is best, nor how to proceed with that. And maybe there's an option 3 that I'm not aware of!
I would welcome your input.
Many thanks in advance.
I'm using the AWS S3 bucket in a very simple way.
There's a bucket, <somebucket1>
There's a folder, <somebucket1>/sitevideos
And video files in it, <somebucket1>/sitevideos/video.mp4
I use this bucket so playback using HTML5 video (<video></video>) is more optimised and doesn't lag compared to just calling the video from the same server of the website (which is ridiculous).
The video files are encrypted, but they are set to be read-only to Public.
Now, my worries are, because they are public, people can download them from the S3 bucket instead of playing them on the website.
The Question
Is there a way to play a video file in S3 bucket, on an HTML video from a remote website, but will refuse downloads of the file if they are accessed directly via the S3 path?
If there are tutorials for this, I'd appreciate it. If this is already on the S3 documentation, I apologise for the laziness, please show me the link. I also heard that you can set them the permission to private, but they can still play on a remote server (although I haven't made that work).
Cheers & many thanks
A Bucket Policy can be configured to Restrict Access to a Specific HTTP Referrer.
For example, if a web page includes an image on the page, then the HTTP request for that object will include a referer. (I presume this would work for a video, too.)
However, this is not very good security, since the HTTP request can be easily manipulated to include the referer (eg in a web scraper).
A more secure method would be to use a Pre-Signed URL. This is a specially-constructed URL that grants time-limited access to a private Amazon S3 object.
When rendering the web page, your app would determine whether the user is permitted to access the object. If so, it would construct the pre-signed URL using AWS credentials. The URL can then be included in the standard HTML tags (eg <img src='...'>). The user will be able to access the object until the expiry time. If they shared the URL with somebody else (eg in a Tweet), other people would also be able to access the object until the expiry time.
By the way, Amazon CloudFront can also serve video content using various video protocols. It also supports pre-signed URLs (and also signed cookies).
I have setup a cloudfront system for a website.
To serve of the fly picture transformation, i added a custom origin, being the website.
So, in my distribution, i have 2 origins :
- s3 bucket
- mywebsite.com/images
Wen i call cdn.mywebsite.com/500/picture.jpg
It will call my website like : website.com/api.php/file/500/picture.jpg
I get the s3 object, create the thumb, save it on server then upload to s3.
Up to here, it all works.
Now, i would like that the next request to this file does not go to my origin custom website, but to the s3 stored file.
I cannot find a way to define order of importance (weight) for multiple origins.
It seams that once frontcloud has a "route", it keeps the same one.
Any ideas ?
You cannot do multiple origins for a Cloud Front Distributions from the AWS side you have to customize this using either
Amazon CloudFront REST API
Bucket Explorer
Check this guide has both steps http://www.bucketexplorer.com/documentation/amazon-s3--how-to-create-distributions-post-distribution-with-multiple-origin-servers.html
Recently I have moved my existing site in Amazon AWS and I am in trouble to store the uploaded image or any other file type directly from Fck-editor to Aws S3 bucket. I have search my problem in Google, but not found any proper solution in this regard.
Does anyone have any advise or possible solutions?
Thanks for your time.
To do this would require the client (here: the browser) to have your secret credentials to access S3 to store anything. Unless the system is only going to be used by you alone, providing your secret credentials to random website visitors sounds like a bad idea. You're better off uploading to your server first, which can then store the file at S3. This also allows you to do some validation on the uploaded data first.
I think its a unique requirement,
you can just hack the FCk-editor uploading section and implement S3 uploads.
Here this question will help you to get basic idea of S3 uploads.
Hope its helps..
I am creating a video website and I have codeigniter libraries that are successfully uploading the files to amazon S3 and encoding the files with encoding.com to MP4's.
My problem is that if I upload the file to S3 first, and then send to encoding.com it works perfectly but it doesn't send back to amazon S3. Would it be proper to just somehow download the encoding.com URL onto my server and then reupload it to S3 again? The url is like:
http://encoding.com.result.s3.amazonaws.com/684981684684-681684384384_53169775.mp4
I don't see anything in the encoding.com api about reuploading the finished file back to S3 or onto the host server. Is it standard practice to just use the encoding.com generated URL to show the client side files? Like using encoding.com as a CDN?
I'm just confused on the best order to do what I'm trying to accomplish. Anyone have any ideas?
Probably not exactly the answer you're looking for, but this is super straight forward with zencoder. S3 to s3 is fully supported and one of the nice, easy features about the platform.
An s3 URL looks something like:
"url": "s3://mybucket/path/file.mp4"
Works for both inputs and outputs.
Zencoder is straightforward to implement, faster, and has better features so save yourself some time and energy.