We are storing a Final Image, Back and Front proof and a thumbnail on one server, but what I need to is when the user goes to took at their images, it needs to transfer the image from the image server on to our web server.
I have tried opening the the server connection and the transferring the image from where it is to where the .php file that has opened the connection is, but i just continually got errors saying that the stream could not be opened because the directory didn't exist.
And i had tried to open up two connections (one for each server) and using get and put to move it from server to server.
These images can't be stored on a intermittent PC.
Any help or advice on how to do this?
You could try transferring the image directly to the user through Apache mod-proxy connection. I'm sure there are ways to work out user permissions between the servers.
What is the purpose of the image server you're using? Are you sure you need to transfer the image across to your other server rather than just having the HTML the web server is producing link to the image server?
Other than that you might want to look into using NFS rather than programatically transferring the files yourself every time they're requested (assuming you're using some form of *nix on both machines and can create NFS shares).
Related
I have numerous storage servers, and then more "cache" servers which are used to load balance downloads. At the moment I use RSYNC to copy the most popular files from the storage boxes to the cache boxes, then update the DB with the new server IDs, so my script can route the download requests to a random box which has the file.
I'm now looking at better ways to distribute the content, and wondering if it's possible to route requests to any box at random, and the download script then check if the file exists locally, if it doesn't, it would "get" the file contents from the remote storage box, and output the content in realtime to the browser, whilst keeping the file on the cache box so that the next time the same request is made, it can just serve the local copy, rather than connecting to the storage box again.
Hope that makes sense(!)
I've been playing around with RSYNC, wget and cURL commands, but I'm struggling to find a way to output the data to browser as it comes in.
I've also been reading up on reverse proxies with nginx, which sounds like the right route... but it still sounds like they require the entire file to be downloaded from the origin server to the cache server before it can output anything to the client(?) some of my files are 100GB+ and each server has a 1gbps bandwidth limit, so at best, it would take 100s to download a file of that size to the cache server before the client will see any data at all. There must be a way to "pipe" the data to the client as it streams?
Is what I'm trying to achieve possible?
You can pipe data without downloading the full file using streams. One example for downloading a file as a stream would be the Guzzle sink feature. One example for uploading a file as a stream would be the Symfony StreamedResponse. Using those the following can be done:
Server A has a file the user wants
Server B gets the user request for the file
Server B uses Guzzle to setup a download stream to server A
Server B outputs the StreamedResponse directly to the user
Doing so will serve the download in real-time without having to wait for the entire file to be finished. However I do not know if you can stream to the user and store the file on disk at the same time. There's a stream_copy_to_stream function in PHP which might allow this, but don't know that for sure.
I have two server,
Let's assume first one is content and second is storage.
My viewers see the site from content server and they upload files to that server.
What I want to do is upload them back to storage server and unlink the previous file.
I want to do this in PHP without curl.
Any Idea how to do this ? Remember I have admin access to both servers.
I have a webapp where by people are allowed to upload files, the webapp and upload form run on VPS1(24GB) I have another server called VPS2(1TB). I want user to use the webapp to upload files and for the files to be stored on the VPS2. How ever I'm not sure the best way to do this, would I upload the file to VPS1 and then transfer it to VPS2 via FTP(or other methods)? Or should I upload it directly to VPS2 using a post method on a webserver running on VPS2? This has to be scalable, I will be adding more webservers in the future.
I had thoughts about putting all the storage VPS servers in an array in PHP an array and randomly selecting which one to post files to. But I'm not sure, really lost and would like some advanced help.
1.You can post your files to your vps2 php script and store files there, Thats a good option and for scalability you can choose which server to choose depending on nearest location of server for clients or randomly choose one. this is the best option i see here, And rest work is your database.
2.Also you can backup a certain amount of files to your vps2 server using linux script when the disk is full using their local ip in case you have a local ip to share with other server.
But still first option is better, you can have different subdomains for the different web server like vps1.domain.com/file01 and vps2.domain.com/file02 and similarly other, and obviously script on different servers depends on sessions ,cookies , database.
If I do
$file='http://notmywebsite.com/verybigimage.png';
$newfile='test.png';
copy($file,$newfile);
will my server download http://notmywebsite.com/verybigimage.png?
If yes, how can I make my user download http://notmywebsite.com/verybigimage.png without that my server download it?
It's not possible to have the client (the user) download a file and then copy this file in the server in a seamless way, because these two environments are living in different systems — in order to do this, the client needs to upload the file back to the server after downloading it, which in turn would be a pain for the user and slower than simply letting the server download the file and copying it there, if the ultimate goal is having a copy of the file in the server.
I need to send a file from one PHP page (on which client uploads their files) to another PHP page on another server were files will be finaly stored.
To comunicate now I use JSON-RPC protocol; is it wise to send the file this way?
$string = file_get_contents("uploaded_file_path");
send the string to remote server and then
file_put_contents("file_name", $recieved_string_from_remte);
I understand that this approach takes twice the time than uploading directly to the second server.
Thanks
[edit]
details:
i need to write a service allowing some php (may be joomla) user to use a simple api to upload files and send some other data to my server which analyze them , put in a db and send back a response
[re edit]
i need to create a Simple method allowing the final user to do that, who will use this the interface on server 1 (the uploading) use the php and stop, so remote ssh mount ore strange funny stuff
If I were you, I'd send the file directly to the second server and store its file name and/or some hash of the file name (for easier retrieval) in a database on the first server.
Using this approach, you could query the second server from the first one for the status of the operation. This way, you can leave the file processing to the second machine, and assign user interaction to the first machine.
As i said in my comment, THIS IS NOT RECOMMENDABLE but anyway....
You can use sockets reading byte by byte:
http://php.net/manual/en/book.sockets.php
or you can use ftp:
http://php.net/manual/en/book.ftp.php
Anyway, the problem in your approuch is doing the process async or sync with the user navigation? I really suggest you passed it by sql or ftp and give the user a response based on another event (like a file watching, then email, etc) or using sql (binary, blob, etc)
Use SSHFS on machine 1 to map a file path to machine 2 (using SSH) and save the uploaded file to machine 2. After the file is uploaded, trigger machine 2 to do the processing and report back as normal.
This would allow you to upload to machine 1, but actually stream it to machine 2's HD so it can be processed faster on that machine.
This will be faster than any SQL or manual file copy solution, because the file transfer happens while the user is uploading the file.
If you don't need the files immediately after receiving them (for processing etc), then you can save them all in one folder on Server 1 and set up a cron to scp the contents of the folder to Server 2. All this assuming you are using linux servers, this is one of the most secure and efficient ways to do it.
For more info please take a look at http://en.wikipedia.org/wiki/Secure_copy or google scp.