We're currently facing a design challenge about dealing with large (>30 MB) file uploads. Our system is currently as follows:
Browser <---AJAX---> PHP <---OAUTH---> REST API
PHP and API are on different servers/domains.
Uploading a file from browser to PHP and then from PHP to API sounds like a waste of bandwidth to me.
So the idea we had in mind is to use the following sequence:
PHP request upload-URL from the API
API creates nonce and sends it back as upload-URL
Browser uploads the file directly to the API using the upload-URL.
So we only transfer the file data once.
Would there be anything wrong with this?
Thanks,
pepijn
If the PHP needs no further processing to do with the file, then it should be fine.
Related
I'm stuck wondering what the best solution is to handling large file uploads and sending them to a third-party API. Any pointers on what would be a good solution would be very welcome. Thank you in advance.
The end goal is to send video files to this API - https://docs.bunny.net/reference/manage-videos#video_uploadvideo. The complication is that the files are often large - up to 5GB in size.
I have an existing website built in PHP7 that runs on a LAMP setup on Amazon Lightsail and I want to add a feature for users to upload video files.
Currently I'm uploading the files directly to Amazon S3 using a pre-signed URL. This part is working fine.
But I need to send the files to the API mentioned above. This is where I get stuck!
I think there's two options to explore - (1) find a way to upload directly to the API and skip the S3 upload or (2) continue with uploading to S3 first and then transfer to the API. But I'm not sure if option 1 is even possible or how to do option 2!
With option 1, I'm wondering if there's a way to upload the files from the users directly to the API. If I do this using the regular HTML form upload, then the files are stored temporarily on my server before I can use cURL through PHP to transfer them to the API. This is really time consuming and feels very inefficient. But I don't know how else to send the files to the API without them first being on my server. Maybe there's an option here that I don't know about!
With option 2, I can already upload large files directly to S3 with pre-signed URLs and this process seems to run fine. But I don't know how I would then send the file from S3 to the API. I can use an S3 trigger on new files. But when I looked at Lambda, they have a tiny file size limit. Because my site is hosted on Lightsail, I noticed they have a container option. But I don't know if that can be used for this purpose and if so, how.
Basically, I'm not sure what solution is best, nor how to proceed with that. And maybe there's an option 3 that I'm not aware of!
I would welcome your input.
Many thanks in advance.
I am trying to develop a RESTFUL API call in PHP , where someone will send me a file through the URL to upload
something like:
script.php?file_name=text.txt
is there away I can take text.txt and upload it in PHP?
To clarify:
lets put it this way , what are the ways that a end user could send a file to a PHP program?
The problem with this is that the REST server is not aware of the end user's machine in any way. So, say for instance that your end user is at yoursite.com/upload where they fill out a form with the upload credentials which posts to api.yoursite.com/uploads/do or whatever. As far as the api is concerned, yoursite.com is making the request, not the end user.
So, no. In my opinion there is no safe way to do this. The best alternative would be to upload the file and then HTTP POST the contents to the rest server. That can be tricky if the file is much larger than a few kilobytes, and you would want to do all sorts of security checking before writing the file to the server. The other option would be to use yoursite.com to upload the file to a temporary location and then send some information to the rest server with details on out to CURL the file contents from the first server. Also, can be insecure.
What problem are you trying to solve? What language framework? Can you give more details please?
i got two questions.
1) i want to upload image into php server and the front end will be android. and i want all processing to be done by php api like checking image and name extension and even error checking .
All the tutorials that i surfed has done in such way that most of the processing is done in android and only for storing uses php api. so any suggestion which supports above requirement will be of great help.
2) $_FILES["fileContents"]["type"] this line in php api is directly used with android uploadimage . i know that it initates file browse option but how to pass parameter value ['fileContents'] from android code.
thanks
We're creating a form that allows users to upload large files. On mobile devices and slow connections, it might take a while to upload, so it seems important for this to be handled by an AJAX call that shows the users a progress bar (or something to let them know it's still working).
Here's the problem: The endpoint for the upload is a 3rd party API which expects our secret API key as one of the parameters. Here's a link directly to the section in their documentation. This API key cannot be exposed to the users on the client side.
My first instinct is to submit the form to an intermediate PHP script on our site, which has the API key, and then uploads the file to the API. But I'm pretty sure this will mean uploading the file twice: once to our server. Then again from our server to the API endpoint. Even if the form is submitted with AJAX, it's not a great result for the user to wait twice as long for it to complete.
So: What's the smoothest way to let users upload files while keeping our API key safe?
Some details that may or may not be important:
Our site is a PHP web app built on the CakePHP framework (v2.x). The files being uploaded are video files of all different formats between 1 and 5 minutes long. The API is a company called Wistia (see link to docs above). The file sizes seem to range from 3-30MB. We have no ability to change the way the 3rd party API works.
Uploading twice shouldn't be an issue - should it?
Its from your server to their API - this is what servers and APIs are meant for - exchanging data.
Javascript is not meant for this.
There is no way to hide it on the client, so your first instinct was correct - you will need to forward the file from the server.
It should be possible to read raw post stream from php://input, you can get the uploaded file from there (if you can parse it :)) and start upload to api server right away.
But even if the communication between mobile device and your script is slow, your script likely will likely upload fast to api server. So is it really needed?
I'm currently working on a Django application using AngularJS for the frontend part.
I want for the moment to upload some images on the server and get as callback a list with the paths of the uploaded files. I want to send afterwards to the API (I am using Tastypie framework) as a POST request the callback and insert it into the database in a specific field. My issue is that I don't know how to approach the images upload (I should use PHP?) part and most important how to receive the callback? I hope that I explain clear enough. :D
Documentation for uploading files in django: https://docs.djangoproject.com/en/dev/topics/http/file-uploads/
HTML only allows to upload one file per upload-controller. To upload multiple files either the user can create a zip file (or similar), upload that and on the server side you extract it into multiple files. Or you have to use a flash component. There are lots of these available, for example http://developer.yahoo.com/yui/uploader/