I am working on a system that will store uploaded files. The metadata will go into a locally-accessible database, but the files themselves are going to be stored on a remote box via NFS so that PHP can interact with the server as if it was a directory.
I identified an issue that may occur if somebody attempts to upload a file when the NFS server is down or otherwise unavailable, which could cause the script to error out or hang. Obviously we want to avoid this scenario and handle it in a graceful manner, but we aren't sure how we can do this.
We are thinking of a) checking the server at page-display time and ghosting out the file upload portion of the form should the server be down, or b) checking the link before executing move_uploaded_file to store the uploaded document.
Is it possible to do this from within PHP, and if so, how?
Checkout http://www.php.net/manual/en/function.stream-set-timeout.php
You could write a simple check that tries to write to the NFS share with a 2 second timeout. If it succeeds, proceed with the move_uploaded_file. If it fails, give the user a graceful error.
I don't know what your setup looks like... If you are mounting it, could you use is_writable?
if (!is_writable('/path/to/nfs/share/mount')) {
die('NFS share is not writable!');
}
I'd try to write a small file for real at nfs-mountpoint, if success you're online and can write the posted file.
If not, cache it at webserver-disk for later (automatic) save.
Check if you can opendir() the directory?
<?php
$dir = "/etc/php5/";
// Open a known directory, and proceed to read its contents
if (is_dir($dir)) {
if ($dh = opendir($dir)) {
// do your stuff
closedir($dh);
}
}
?>
Related
So the situation is that lets say "client" changed their ftp server and I have a script that uploads stuff on their server via ftp_put(). I think they have different permissions now on their ftp server when I upload the files. Uploading behaves really weirdly with their new server, it duplicates some files about 5 times, others it doesn't duplicate at all. Keep in mind the same script worked correctly on their old server.
I noticed from the screenshot they sent me, that there's only read and write permissions on their server now. It used to have delete permission also. When I tried to upload files to our own server, and took away deleting permissions, everything stopped working.
So my thought is that, this is the issue maybe since logs showed, I might be totally wrong here, ftp_put() tries to delete files before it uploads them to a server to avoid duplicates. Is that correct and is there workaround to that?
The function itself what sends the files is really easy:
function send_to_ftp($ftp_server, $ftp_username, $ftp_password, $fileName,
$subFolder = "n/") {
$ftp_conn = ftp_connect($ftp_server);
if (!$ftp_conn) return false;
$login = ftp_login($ftp_conn, $ftp_username, $ftp_password);
$reciver_path = $subFolder . end(explode('/', $fileName));
$local_folder = $fileName;
return (ftp_put($ftp_conn, $reciver_path, $local_folder, FTP_ASCII));
//ftp_close($ftp_conn);
}
It is missing in the documentation on php.net but is mentioned in a comment.
ftp_put tries to overwrite the existing file, and this is why you have issues with the permissions. The doubleing might be, without reading the source code of the php ftp_put function, that the overwrite really is a delete and upload, or that you are running the function multiple times.
I recommend checking if the file already exists before trying to upload it (regardless, checking files are less costly in network traffic then uploading a full sized file).
Use ftp_nlist to get the content in the target path and see if it exists.
http://php.net/manual/en/function.ftp-nlist.php
If you need to upload already existing file you need to look over your permissions with the owner of the server.
My mailserver writes to a file every minute, this is fine and I'm happy for it to do that.
However on my WebServer, I want to check if that file is currently being written to and if it isn't, show the user a download link.
Is there any way I can do this..
For example: run a loop that will keep looking until the file is no longer being written to then, show a download link to the file?
I've read about flock() but I don't think this will help as another process / os is actually creating the file!
Your writting script/app/process should write lock file (empty file like filename.lock before it starts writting to main file, and then it shall remove when done. It's regular locking approach but the your script will just need to check if filename.lock is present or not. If it is, then file is being written to.
You can only acquire a read or write lock if no-one else is currently writing. You shouldn't have to do this.
Also, when the user downloads the file it could be the file has changed in the mean time. Are you sure you've got the right mental image of what you want?
We have a FreeBSD server with samba that employees copy image files on to, which then get uploaded to our web servers (this way they don't have to mess with ftp). Sometimes, if the upload script is running at the same time as files are being copied, it can upload an incomplete file.
We fixed this by getting the list of files along with the file sizes, then waiting 5 seconds and rechecking the file sizes. If the sizes match then its save to upload, if they don't match it checks again in another 5 seconds.
This seems like an odd way to check if the files are being written to. is there a better, more simple way of doing this?
Use a flock function http://php.net/flock - when writing a file obtain an exclusive lock flock($handle, LOCK_EX), after it is written release the lock flock($handle, LOCK_UN).
The upload script could try to obtain the exclusive writing lock too, if it succeeds it is Okay to move the file, otherwise no.
EDIT: Sorry, I forgot about the users copying the files to the server through samba... So there is no space to use flock while copying... But the upload script could still use flock($handle, LOCK_EX) to see, if it is successful or not.
I recommend to shell_exec() smbstatus(1), e.g. smbstatus -LB to check for locked files
Write a script to copy the files to a temp folder on the Samba server and then, when fully copied and flushed, move, (ie, unlink/link, not copy again), them to the upload folder.
I have a PHP script that moves files out of a specific folder on the server(an IBM AS400). The problem I am running into is that sometimes the script runs while the file is still in the process of being moved in to the folder.
Poor logic on my part assumed that if a file was "in use" that PHP wouldn't attempt to move it but it does which results in a corrupted file.
I thought I could do this:
$oldModifyTime = filemtime('thefile.pdf');
sleep(2);
if($oldModifyTime === filemtime('thefile.pdf'){
rename('thefile.pdf','/folder2/thefile.pdf');
}
But the filemtime functions come up with the same value even while the file is being written. I have also tried fileatime with the same results.
If I do Right Click->Properties in Windows the Modified Date and Access Date are constantly changing as the file is being written.
Any ideas how to determine if a file is finished transferring before doing anything to it?
From the PHP manual entry for filemtime():
Note: The results of this function are cached. See clearstatcache() for more details.
I would also suggest that 2 seconds is a bit short to detect whether the file transfer is complete due to network congestion, buffering, etc.
Transfer it as a temporary name or to a different folder, then rename/copy it to the correct folder after the transfer is complete.
Here's a few questions I cannot find in search:
When adding CDN services to your website, do you still maintain/create local dynamic files on your origin server and point the CDN to that location, set a http rule, and have them upload it automatically if they aren't hosting it yet?
Let's say I have an avatar upload form on my origin server and after a cropping function, do I set the save image to the local directory or to the CDN?
The other question I have is if you save files locally first and wait for the CDN to pull them, how do you code for the page to know the difference? Do you use some thing like
// $filename = 'images/image.jpg';
function static_file($filename) {
$cdnfilepath = 'http://cdndomain.com/';
if (fopen($cdnfilepath.$filename, "r")) {
return $cdnfilepath.$filename;
} else {
return $filename;
}
}
Or, do you just PUT every dynamically created file that you would like the CDN to host directly to the CDN?
If anyone knows a good tutorial on this that would helpful. Sorry if any of this has been covered but I having been searching with no clear answers...
Sometimes there's no straight-forward way of uploading directly to your CDN.
For example with AWS you have to PUT the file, which means it still has to be uploaded to your server temporarily. What I do is upload the files to a temp directory then have a cron script run that PUT's the files onto AWS, so as not to cause the upload process to take any longer for the end-user.