I'm stuck wondering what the best solution is to handling large file uploads and sending them to a third-party API. Any pointers on what would be a good solution would be very welcome. Thank you in advance.
The end goal is to send video files to this API - https://docs.bunny.net/reference/manage-videos#video_uploadvideo. The complication is that the files are often large - up to 5GB in size.
I have an existing website built in PHP7 that runs on a LAMP setup on Amazon Lightsail and I want to add a feature for users to upload video files.
Currently I'm uploading the files directly to Amazon S3 using a pre-signed URL. This part is working fine.
But I need to send the files to the API mentioned above. This is where I get stuck!
I think there's two options to explore - (1) find a way to upload directly to the API and skip the S3 upload or (2) continue with uploading to S3 first and then transfer to the API. But I'm not sure if option 1 is even possible or how to do option 2!
With option 1, I'm wondering if there's a way to upload the files from the users directly to the API. If I do this using the regular HTML form upload, then the files are stored temporarily on my server before I can use cURL through PHP to transfer them to the API. This is really time consuming and feels very inefficient. But I don't know how else to send the files to the API without them first being on my server. Maybe there's an option here that I don't know about!
With option 2, I can already upload large files directly to S3 with pre-signed URLs and this process seems to run fine. But I don't know how I would then send the file from S3 to the API. I can use an S3 trigger on new files. But when I looked at Lambda, they have a tiny file size limit. Because my site is hosted on Lightsail, I noticed they have a container option. But I don't know if that can be used for this purpose and if so, how.
Basically, I'm not sure what solution is best, nor how to proceed with that. And maybe there's an option 3 that I'm not aware of!
I would welcome your input.
Many thanks in advance.
I'd like to store my asset in the cloud. I thought Zend_Cloud_Storage may be the right library? For now they are saved locally using Zend_Cloud_StorageService_Adapter_Filesystem. Upload works so far.
What is the right way to send files from Zend_Cloud_StorageService_Adapter_Filesystem to the browser? I guess this is only possible by streaming them through php? If yes any code examples out there because I have problems getting it working with all the file header.
Using S3 I probably would make a redirect or directly link the files to the s3 bucket?
I'm pretty new to cloud storage so I really appreciate your help.
Kind regards, Manuel
The Zend Cloud services are fairly new and from my experience the documentation still needs a bit of work. Depending on the cloud service, you have either the choice of fetching your objects and delivering them to the user or in some cases you can give the user a direct URL to the asset (this largely depends on if you need to control access to the object).
To fetch an item programmatically, you should be able to get it to work with right header, for example:
$image = $storage->fetchItem("/my/remote/path/picture.jpg");
header("content-type: image/jpg");
echo file_get_contents($image);
In theory you can get the content type for the header from the metadata, but I'm not sure if it is consistent between adapters.
Linking directly depends on your Cloud. For the filesystem you should know the path to where your files are, so you can link from there if they are accessible. In Amazon, you can set the permissions to public when you upload the file - there is an example in the S3 doco here.
I'd recommend having a look at the Zend Service S3 doco, Zend Cloud is just a wrapper for it and in trying to be generic it looses a lot of functionality. if you work with it for any length of time I suspect you'll eventually end up doing calls straight to the service.
Good luck!
I want to develop a website like file manager. Where user register and will get fix disk space lets say 20MB.
Now user can upload their pdf, doc, txt, jpeg etc files upto their disk limit.
I can develop upto this using PHP.
Now below is my issue:
1) If user's files are corrupted they can rollback their folders before 2-3 days.
Files must be secure and safe from viruses as users are uploading their important documents.
Is there any 3rd party storage server who provides such facility?
2) Also all files should be previewed from browser.
I am using Google doc viewer. Is is good and safe way to preview file in browser?
But google links are accessible from all, I need to add some restrictions as file can be viewed only by their owner.
I know it's a major task, but i just need some sort of logic. Please share your thoughts.
Thanks.
Any cloud storage service can be used for this. You'll get HDD space. There is not storage server who provides revision control system for this. You can use git, svn for this though. But as the files are binary you can not get full facility of these tools.
How file will be previewed depends on you. If you use PHP you make the site and at the backend you use the API to interact with the storage service. Google doc is not an option for this if you use PHP. Also note Google links can be made private.
I suggest you this,
Find a cloud storage service and use the storage in your server. Any will do.
Create UI using PHP and control the access using PHP too.
Manipulate files in your server directly or in 3rd party storage server via API
Use a revision control system to track the changes. And use its API in PHP end.
Some cloud storage service
Amazon S3. It also supports Versioning.
Google Cloud Storage
Microsoft Azure
Try Microsoft SkyDrive or Google Drive or Dropbox
I came across quite a few video distribution services (one is heyspread). The video they have on their home page is like, you upload your video only once and it is distributed to other video sharing sites. To me, it looks like they have coded it in Php.
Can anyone explain me the logic behind it? How is possible? Also, will it take the bandwidth of uploading it only once or uploading it to all the sites that we are using?
Also, if I had to code one like that, are there any links which I can use as a start up to kick off?
Here is my explanation based on what I know and your requirements:
Can anyone explain me the logic behind it?
You basically act as a mediator for all the sites. What the users do is given the site their permission to use their credentials that they set up in this multiple sites and allow you to upload video on their behalf so they don't have to do it themselves thereby saving them time
How is possible?
Many of the video hosting websites operate on HTTP protocol. In order to upload on your behalf the video distribution service do the following for each of the website (I have generalize the steps, there could be more than these steps for some of the sites):
Authenticate using the credentials that you give
Upload the video using one of the following ways:
If there is API available, then this is the preferred way for the service to upload on your behalf as the interface to authenticate/upload are clean and well define
If no API is available for a particular website, then the service has to simulate the HTTP sequence as if the request is made from the browser by the user. This is not the best way but sometimes is the only way. This approach is not as robust as the first one because the contract could change and you don't necessarily get the confirmation message (for success/failure of upload for example) other than via parsing the HTML
Also, will it take the bandwidth of uploading it only once or uploading it to all the sites that we are using?
It will use your bandwidth once to upload the video the first time. In order to upload to other websites on your behalf, the video distribution service will use its datacenter/cloud service bandwidth
Also, if I had to code one like that, are there any links which I can use as a start up to kick off?
I don't know any that does exactly like that, but you should look at each site that you want to mediate and see if they have the API first. You might want to start with those who have before trying to simulate user's click for those who don't have API. For example YouTube even goes further by providing the Client Library for you to use based on your language of choice.
I have a PHP/AJAX/MYSQL chat application. I want to add video chatting to my application. How can I create live video streaming to be used for live video conferences/chatting in a PHP application. What are the key-terms I need to know if I wanted to build such a system? Is it a good idea in the first place to use PHP? Is there something I'm not thinking about? What other languages can I use, perhaps in conjunction with PHP?
I am not saying that you have to abandon PHP, but you need different technologies here.
Let's start off simple (without Akamai :-)) and think about the implications here. Video, chat, etc. - it's all client-side in the beginning. The user has a webcam, you want to grab the signal somehow and send it to the server. There is no PHP so far.
I know that Flash supports this though (check this tutorial on webcams and flash) so you could use Flash to transport the content to the server. I think if you'll stay with Flash, then Flex (flex and webcam tutorial) is probably a good idea to look into.
So those are just the basics, maybe it gives you an idea of where you need to research because obviously this won't give you a full video chat inside your app yet. For starters, you will need some sort of way to record the streams and re-publish them so others see other people from the chat, etc..
I'm also not sure how much traffic and bandwidth this is gonna consume though and generally, you will need way more than a Stackoverflow question to solve this issue. Best would be to do a full spec of your app and then hire some people to help you build it.
HTH!
Please note that the below described service is no longer available as
it was based on FLV media (Flash)
This project which utilizes the Red5, Flex and PHP for Live Video Streaming and Recording has many features
Stream Live video to the viewers
Record the streams from your cam or other video input devices to the server
Preview the recorded streams and files and thumbnail the frame which you would like to display for the video.
Upload the videos from your computer and convert them to FLV which can be streamed using Red5 .
Choose from any resolutions
Can be plugged to any script
Each website user can have a separate Directory for storing their videos and thumbnails use this link
http://code.google.com/p/red5-flex-streamer/
PHP will let you build the pages of your site that make up your video conferencing and chat applications, but it won't deliver or stream video for you - PHP runs on the server only and renders out HTML to a client browser.
For the video, the first thing you'll need is a live streaming account with someone like akamai or the numerous others in the field. Using this account gives you an ingress point for your video - ie: the server that you will stream your live video up to.
Next, you want to get your video out to the browsers - windows media player, flash or silverlight will let you achieve this - embedding the appropriate control for your chosen technology into your page (using PHP or whatever) and given the address of your live video feed.
PHP (or other scripting language) would be used to build the chat part of the application and bring the whole thing together (the chat and the embedded video player).
Hope this helps.
PHP/AJAX/MySQL will not be enough for creating the live video streaming application There is a similar thread here. It primarily suggests using Flex or Silverlight.
For live video conferencing you can't ignore the need of a streaming server.
Yes, flash will let you display video from a webcam within the local flash control, but that won't let you then send that video over the network - for that you need a streaming server to send it to.
If you're going to build something like this it's prudent to think about how you're going to host the video from a very early stage as it will influence how you build the application. Flash/Flex/Silverlight/Windows Media....etc....
There are a lot of "off-the-shelf" 'servers' that will run in your environment.
Most of these utilize the aforementioned Flex or Silverlight to implement the actual video itself but I'm pretty sure all will run under LAMP/PHP.
The challenges will picking the best software from everything that's available and getting your hosting-provider to let you stream video (it goes without saying that streaming is heavy on bandwidth).
You can easily build a website as per the requirements. PHP will be there to handle the website development part. All the hosting and normal website development will work just as it is. However, for the streaming part, you will have to choose a good streaming service. Whether it is Red5 or Adobe, you can choose from plenty of services.
Choose a service that provides a dedicated storage to get something done right. If you do not know how to configure the server properly, you can just choose a streaming service. Good services often give a CDN that helps broadcast the stream efficiently. Simply launch your website in PHP and embed the YouTube player in the said web page to get it working.
Same problem/answer here, quoted below
I'm assuming you mean that you want to run your own private video
calls, not simply link to Skype calls or similar. You really have 2
options here: host it yourself, or use a hosted solution and integrate
it into your product.
Self-Hosted
----------------- This is messy. This can all be accomplished with PHP, but that is probably not the most advisable solution, as it is
not the best tool for the job on all sides. Flash is much more
efficient at a/v capture and transport on the user end. You can try to
do this without flash, but you will have headaches. HTML5 may make
your life easier, but if you're shooting for maximum compatibility,
flash is the simplest way to go for creating the client. Then, as far
as the actual server side that will relay the audio/video, you could
write a chat server in php, but you're better off using an open source
project, like
janenz00's mention
of red5, that's already built and
interfacing with it through your client (if it doesn't already have
one). Or you could homebrew a flash client as mentioned before and
hook it up to a flash streaming server on both sides...either way it
gets complicated fast, and is beyond my expertise to help you with at
all.
Hosted Service
----------------- All in, my recommendation, unless you want to administer a ridiculous setup of many complex servers and failure
points is to use a hosted service like
UserPlane or similar and offload all the
processing and technical work to people who are good at that, and then
worry about interfacing with their api and getting their client well
integrated into your site. You will be a happier developer if you do.