I have logic to upload images to s3 in php. I need to upload the same set of images in a SFTP server. In my view there are 2 options. First is to find a logic to upload image from my local to the server, when I am uploading images to s3 and the other option is to write some script to transfer the images from s3 to sftp server. I need the same set of images to be in the server and s3.
Out of the 2 approaches, which one is optimal? Is there any other way to approach my requirement? Any sample php script available for local to SFTP file transfer, if yes please provide the code.
I cannot say for sure which one is optimal, but I can definitely see a potential issue with option #1. If you perform the second upload (i.e. from your "local" server to the SFTP server) during the first upload, you make PHP wait on that operation before returning the response to the client. This could cause some unnecessary hanging for the user agent connecting to the local server.
I would explore option #2 first. If possible, look into SSHFS. This is a way to mount a remote filesystem over SSH. It uses the SFTP to transfer files. This may be a possible solution where all you have to do is write the file once to the local server's filesystem and then again to the mounted, remote filesystem. SSHFS would take care of the transfer for you.
Related
I want to upload a large file from My computer to S3 Server without editing php.ini. Firstly,I choose file from browse button and submit upload button and then upload to s3 server. But I can't post form file data when I upload a large file. But I don't want to edit php.ini.Is there any way to upload a large local file to s3 server?
I've done this by implementing Fine Uploader's php implementation for S3. As of recently it is through an MIT license. It's an easy way to upload huge files to S3 without changing your php.ini at all.
It's not the worst thing in the world to set up. You'll need to set some environment variables for the public/secret keys, set up CORS settings on the bucket, and write a php page based on one of the examples which will call a php endpoint that'll handle the signing.
One thing that was not made obvious to me was that, when setting the environment variables, they expect you to make two separate AWS users with different privileges for security reasons.
ini_set("upload_max_filesize","300M");
try this
When uploading an image PHP stores the temp image in a local on the server.
Is it possible to change this temp location so its off the local server.
Reason: using loading balancing without sticky sessions and I don't want files to be uploaded to one server and then not avaliable on another server. Note: I don't necessaryly complete the file upload and work on the file in the one go.
Preferred temp location would be AWS S3 - also just interested to know if this is possible.
If its not possible I could make the file upload a complete process that also puts the finished file in the final location.
just interested to know if the PHP temp image/file location can be off the the local server?
thankyou
You can mount S3 bucket with s3fs on your Instances which are under ELB, so that all your uploads are shared between application Servers. About /tmp, don't touch it as destination is S3 and it is shared - you don't have to worry.
If you have a lot of uploads, S3 might be bottleneck. In this case, I suggest to setup NAS. Personally, I use GlusterFS because it scales well and very easy to set up. It has replication issues, but you might not use replicated volumes at all and you are fine.
Another alternatives are Ceph, Sector/Sphere, XtreemFS, Tahoe-LAFS, POHMELFS and many others...
You can directly upload a file from a client to S3 with some newer technologies as detailed in this post:
http://www.ioncannon.net/programming/1539/direct-browser-uploading-amazon-s3-cors-fileapi-xhr2-and-signed-puts/
Otherwise, I personally would suggest using each server's tmp folder for exactly that-- temporary storage. After the file is on your server, you can always upload to S3, which would then be accessible across all of your load balanced servers.
On my FTP server, there is a 4gb XML file and I want it to put data from that file to the database using PHP.
I know how to connect to FTP and basic operations using PHP but my question is there a possibility to do it without having to download the file first?
Unfortunately no, you cannot "stream" file using FTP as you could do say on network drive. It's not possible to open that file without downloading it first locally.
This is given you can only access that file via FTP.
If your FTP server and PHP server are one and the same, you just need to change the path to reference the FTP location rather than whereever you are downloading to.
If they are on the same local network, you may be able to use a network path to reach the file.
Otherwise, you will indeed need to transfer the entire file first by downloading it.
I have a webapp where by people are allowed to upload files, the webapp and upload form run on VPS1(24GB) I have another server called VPS2(1TB). I want user to use the webapp to upload files and for the files to be stored on the VPS2. How ever I'm not sure the best way to do this, would I upload the file to VPS1 and then transfer it to VPS2 via FTP(or other methods)? Or should I upload it directly to VPS2 using a post method on a webserver running on VPS2? This has to be scalable, I will be adding more webservers in the future.
I had thoughts about putting all the storage VPS servers in an array in PHP an array and randomly selecting which one to post files to. But I'm not sure, really lost and would like some advanced help.
1.You can post your files to your vps2 php script and store files there, Thats a good option and for scalability you can choose which server to choose depending on nearest location of server for clients or randomly choose one. this is the best option i see here, And rest work is your database.
2.Also you can backup a certain amount of files to your vps2 server using linux script when the disk is full using their local ip in case you have a local ip to share with other server.
But still first option is better, you can have different subdomains for the different web server like vps1.domain.com/file01 and vps2.domain.com/file02 and similarly other, and obviously script on different servers depends on sessions ,cookies , database.
We are storing a Final Image, Back and Front proof and a thumbnail on one server, but what I need to is when the user goes to took at their images, it needs to transfer the image from the image server on to our web server.
I have tried opening the the server connection and the transferring the image from where it is to where the .php file that has opened the connection is, but i just continually got errors saying that the stream could not be opened because the directory didn't exist.
And i had tried to open up two connections (one for each server) and using get and put to move it from server to server.
These images can't be stored on a intermittent PC.
Any help or advice on how to do this?
You could try transferring the image directly to the user through Apache mod-proxy connection. I'm sure there are ways to work out user permissions between the servers.
What is the purpose of the image server you're using? Are you sure you need to transfer the image across to your other server rather than just having the HTML the web server is producing link to the image server?
Other than that you might want to look into using NFS rather than programatically transferring the files yourself every time they're requested (assuming you're using some form of *nix on both machines and can create NFS shares).