I'm writing a photo gallery script in PHP and have a single directory where the user will store their pictures. I'm attempting to set up page caching and have the cache refresh only if the contents of the directory has changed. I thought I could do this by caching the last modified time of the directory using the filemtime() function and compare it to the current modified time of the directory. However, as I've come to realize, the directory modified time does not change as files are added or removed from that directory (at least on Windows, not sure about Linux machines yet).
So my questions is, what is the simplest way to check if the contents of a directory have been modified?
As already mentioned by others, a better way to solve this would be to trigger a function when particular events happen, that changes the folder.
However, if your server is a unix, you can use inotifywait to watch the directory, and then invoke a PHP script.
Here's a simple example:
#!/bin/sh
inotifywait --recursive --monitor --quiet --event modify,create,delete,move --format '%f' /path/to/directory/to/watch |
while read FILE ; do
php /path/to/trigger.php $FILE
done
See also: http://linux.die.net/man/1/inotifywait
What about touching the directory after a user has submitted his image?
Changelog says: Requires php 5.3 for windows to work, but I think it should work on all other environments
with inotifywait inside php
$watchedDir = 'watch';
$in = popen("inotifywait --monitor --quiet --format '%e %f' --event create,moved_to '$watchedDir'", 'r');
if ($in === false)
throw new Exception ('fail start notify');
while (($line = fgets($in)) !== false)
{
list($event, $file) = explode(' ', rtrim($line, PHP_EOL), 2);
echo "$event $file\n";
}
Uh. I'd simply store the md5 of a directory listing. If the contents change, the md5(directory-listing) will change. You might get the very occasional md5 clash, but I think that chance is tiny enough..
Alternatively, you could store a little file in that directory that contains the "last modified" date. But I'd go with md5.
PS. on second thought, seeing as how you're looking at performance (caching) requesting and hashing the directory listing might not be entirely optimal..
IMO edubem's answer is the way to go, however you can do something like this:
if (sha1(serialize(Map('/path/to/directory/', true))) != /* previous stored hash */)
{
// directory contents has changed
}
Or a more weak / faster version:
if (Size('/path/to/directory/', true) != /* previous stored size */)
{
// directory contents has changed
}
Here are the functions used:
function Map($path, $recursive = false)
{
$result = array();
if (is_dir($path) === true)
{
$path = Path($path);
$files = array_diff(scandir($path), array('.', '..'));
foreach ($files as $file)
{
if (is_dir($path . $file) === true)
{
$result[$file] = ($recursive === true) ? Map($path . $file, $recursive) : $this->Size($path . $file, true);
}
else if (is_file($path . $file) === true)
{
$result[$file] = Size($path . $file);
}
}
}
else if (is_file($path) === true)
{
$result[basename($path)] = Size($path);
}
return $result;
}
function Size($path, $recursive = true)
{
$result = 0;
if (is_dir($path) === true)
{
$path = Path($path);
$files = array_diff(scandir($path), array('.', '..'));
foreach ($files as $file)
{
if (is_dir($path . $file) === true)
{
$result += ($recursive === true) ? Size($path . $file, $recursive) : 0;
}
else if (is_file() === true)
{
$result += sprintf('%u', filesize($path . $file));
}
}
}
else if (is_file($path) === true)
{
$result += sprintf('%u', filesize($path));
}
return $result;
}
function Path($path)
{
if (file_exists($path) === true)
{
$path = rtrim(str_replace('\\', '/', realpath($path)), '/');
if (is_dir($path) === true)
{
$path .= '/';
}
return $path;
}
return false;
}
Here's what you may try. Store all pictures in a single directory (or in /username subdirectories inside it to speed things up and to lessen the stress on the FS) and set up Apache (or whaterver you're using) to serve them as static content with "expires-on" set to 100 years in the future. File names should contain some unique prefix or suffix (timestamp, SHA1 hash of file content, etc), so whenever uses changes the file its name gets changed and Apache will serve a new version, which will get cached along the way.
You're thinking the wrong way.
You should execute your directory indexer script as soon as someone's uploaded a new file and it's moved to the target location.
Try deleting the cached version when a user uploads a file to his directory.
When someone tries to view the gallery, look if there's a cached version first. If there's a cached version, load it, otherwise, generate the page, cache it, done.
I was looking for something similar and I just found this:
http://www.franzone.com/2008/06/05/php-script-to-monitor-ftp-directory-changes/
For me looks like a great solution since I'll have a lot of control (I'll be doing an AJAX call to see if anything changed).
Hope that this helps.
Here is a code sample, that would return 0 if the directory was changed.
I use it in backups.
The changed status is determined by presence of files and their filesizes.
You could easily change this, to compare file contents by replacing
$longString .= filesize($file);
with
$longString .= crc32(file_get_contents($file));
but it will affect execution speed.
#!/usr/bin/php
<?php
$dirName = $argv[1];
$basePath = '/var/www/vhosts/majestichorseporn.com/web/';
$dataFile = './backup_dir_if_changed.dat';
# startup checks
if (!is_writable($dataFile))
die($dataFile . ' is not writable!');
if (!is_dir($basePath . $dirName))
die($basePath . $dirName . ' is not a directory');
$dataFileContent = file_get_contents($dataFile);
$data = #unserialize($dataFileContent);
if ($data === false)
$data = array();
# find all files ang concatenate their sizes to calculate crc32
$files = glob($basePath . $dirName . '/*', GLOB_BRACE);
$longString = '';
foreach ($files as $file) {
$longString .= filesize($file);
}
$longStringHash = crc32($longString);
# do changed check
if (isset ($data[$dirName]) && $data[$dirName] == $longStringHash)
die('Directory did not change.');
# save hash do DB
$data[$dirName] = $longStringHash;
file_put_contents($dataFile, serialize($data));
die('0');
Related
I have a number of different hosting accounts set up for clients and need to calculate the amount of storage space being used on each account, which would update regularly.
I have a database set up to record each clients storage usage.
I attempted this first using a PHP file on each account, run by a Cron Job. If run manually by myself, it would output the correct filesize and update the correct size to the database, although when run from the Cron Job, it would output 0.
I then attempted to run this file from a Cron Job from the main account but figured this wouldn't actually work as my hosting would block files from another server and I would end up with the same result as before.
I am now playing around with FTP access to each account from a Cron Job from the main account which looks something like below, the only problem is I don't know how to calculate directory size rather than single file sizes using FTP access, and don't know how to reiterate this way? Hoping somebody might be able to help here before I end up going around in circles?
I will also add the previous first attempt too.
$ftp_conn = ftp_connect($ftp_host, 21, 420) or die("Could not connect to server");
$ftp_login = ftp_login($ftp_conn, $ftp_username, 'mypassword');
$total_size = 0;
$contents = ftp_nlist($ftp_conn, ".");
// output $contents
foreach($contents as $folder){
while($search == true){
if($folder == '..' || $folder == '.'){
} else {
$file = $folder;
$res = ftp_size($ftp_conn, $file);
if ($res != -1) {
$total_size = $total_size + $res;
} else {
$total_size = $total_size;
}
}
}
}
ftp_close($ftp_conn);
This doesn't work as it doesn't calculate folder sizes and I don't know how to open the reiterate using this method?
This second script did work but would only work if opened manually, and return 0 if run by the cron job.
class Directory_Calculator {
function calculate_whole_directory($directory)
{
if ($handle = opendir($directory))
{
$size = 0;
$folders = 0;
$files = 0;
while (false !== ($file = readdir($handle)))
{
if ($file != "." && $file != "..")
{
if(is_dir($directory.$file))
{
$array = $this->calculate_whole_directory($directory.$file.'/');
$size += $array['size'];
$files += $array['files'];
$folders += $array['folders'];
}
else
{
$size += filesize($directory.$file);
$files++;
}
}
}
closedir($handle);
}
$folders++;
return array('size' => $size, 'files' => $files, 'folders' => $folders);
}
}
/* Path to Directory - IMPORTANT: with '/' at the end */
$directory = '../public_html/';
// return an array with: size, total files & folders
$array = $directory_size->size($directory);
$size_of_site = $array['size'];
echo $size_of_site;
Please bare in mind that I am currently testing and none of the MySQLi or PHP scripts are secure yet.
If your server supports MLSD command and you have PHP 7.2 or newer, you can use ftp_mlsd function:
function calculate_whole_directory($ftp_conn, $directory)
{
$files = ftp_mlsd($ftp_conn, $directory) or die("Cannot list $directory");
$result = 0;
foreach ($files as $file)
{
if (($file["type"] == "cdir") || ($file["type"] == "pdir"))
{
$size = 0;
}
else if ($file["type"] == "dir")
{
$size = calculate_whole_directory($ftp_conn, $directory."/".$file["name"]);
}
else
{
$size = intval($file["size"]);
}
$result += $size;
}
return $result;
}
If you do not have PHP 7.2, you can try to implement the MLSD command on your own. For a start, see user comment of the ftp_rawlist command:
https://www.php.net/manual/en/function.ftp-rawlist.php#101071
If you cannot use MLSD, you will particularly have problems telling if an entry is a file or folder. While you can use the ftp_size trick, as you do, calling ftp_size for each entry can take ages.
But if you need to work against one specific FTP server only, you can use ftp_rawlist to retrieve a file listing in a platform-specific format and parse that.
The following code assumes a common *nix format.
function calculate_whole_directory($ftp_conn, $directory)
{
$lines = ftp_rawlist($ftp_conn, $directory) or die("Cannot list $directory");
$result = 0;
foreach ($lines as $line)
{
$tokens = preg_split("/\s+/", $line, 9);
$name = $tokens[8];
if ($tokens[0][0] === 'd')
{
$size = calculate_whole_directory($ftp_conn, "$directory/$name");
}
else
{
$size = intval($tokens[4]);
}
$result += $size;
}
return $result;
}
Based on PHP FTP recursive directory listing.
Regarding cron: I'd guess that the cron does not start your script with a correct working directory, so you calculate a size of a non-existing directory.
Use an absolute path here:
$directory = '../public_html/';
Though you better add some error checking so that you can see yourself what goes wrong.
I have a couple of PHP scripts for deleting error_log, .DS_Store etc files from all folders on my entire server. I simply have these scripts uploaded to my root (public_html) and visit them periodically when I want to do a little cleanup. When I visit the URL of where the scripts are loaded it automatically gets to work. That's all perfect and how I'd like to continue using it.
However, I'd love to consolidate this automation into just one script where I can list an array of the undesirable files like so:
$unwanted_filenames = array(
'.DS_Store',
'.localized',
'Thumbs.db',
'error_log'
);
And simply run through all folders and delete all the files of which I've listed in the array.
The scripts I use now are overkill, listing out every individual file and how much it's freed up etc. I'm a minimalist and would love the simplest script with the least amount of code to just get the job done.
So when I visit the page it automatically get's to work, a white screen of nothing is fine and then maybe a simple "Done. Freed up 3MB." message. That's it.
OK - here's the shortest but of PHP I can think of that'll do it:
$unwanted_filenames = array(
'.DS_Store',
'.localized',
'Thumbs.db',
'error_log'
);
$it = new RecursiveDirectoryIterator("/"); // Set starting directory here
foreach(new RecursiveIteratorIterator($it) as $file) {
if (in_array(basename($file), $unwanted_filenames)) {
#unlink($file); // THe # hides errors, remove if you want to see them
}
}
Hopefully self-explanatory - and yes, it does subdirectories (that's the "recursive" bit).
And you said minamalistic, so I didn't include the freed space, but just add a $FreedSpace += filesize($file) before the unlink if you want to add that in.
I'm using this, you can do it like this:
<?php
$dir = "/var/www/vhosts/"; //Write your dirname here
$rii = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir));
$total_thumbs = 0;
$total_ds = 0;
foreach ($rii as $file) {
if ($file->isDir()){
continue;
}
$parcala = explode('.', $file->getFilename());
$uzanti = end($parcala);
if ($file->getFilename() == 'Thumbs.db') {
unlink($file->getPathname());
$total_thumbs++;
}
if ($uzanti == '.DS_Store' || $file->getFilename() == 'DS_Store' || $file->getFilename() == '.DS_Store') {
unlink($file->getPathname());
$total_ds++;
}
}
echo $total_thumbs . ' Thumbs.db file and ' . $total_ds . ' DS_Store file deleted!';
And if you want automation you can use Cronjob
I'm not a native PHP developer but I make do with what I can find and hack together out there, so please excuse me if this doesn't make too much sense:
I have a simple (so it seems) script that takes two arguments in the URL. One is a simple string (title of the ZIP to be created) and the other is a serialized array of audio tracks that point to the files on the server that need zipping. These then pass just fine and are unserialized etc. Here's the script:
<?php
$audioTitleVar = unserialize(rawurldecode($_GET['audioTitle']));
$audioArrayVar = unserialize(rawurldecode($_GET['audioArray']));
function create_zip( $files = array(), $destination = '', $overwrite = true ) {
if(file_exists($destination) && !$overwrite) { return false; }
$valid_files = array();
if(is_array($files)) {
foreach($files as $file) {
if( file_exists($_SERVER['DOCUMENT_ROOT'] . str_replace("http://mydomain.com","", $file)) ) {
$valid_files[] = $_SERVER['DOCUMENT_ROOT'] . str_replace("http://mydomain.com","", $file);
}
}
}
if(count($valid_files)) {
$zip = new ZipArchive();
if($zip->open($destination,$overwrite ? ZIPARCHIVE::OVERWRITE : ZIPARCHIVE::CREATE) !== true) {
return false;
}
foreach( $valid_files as $file ) {
$just_name = preg_replace("/(.*)\/?([^\/]+)/","$2",$file);
$zip->addFile($file,$just_name);
//$zip->addFile($file,$file);
}
//echo '<p>The zip archive contains ' . $zip->numFiles . ' files with a status of ' . $zip->status . '</p>';
$zip->close();
return file_exists($destination);
} else {
return false;
}
}
$fileName = $_SERVER['DOCUMENT_ROOT'] . '/wp-content/themes/jones/zip/' . $audioTitleVar . '.zip';
create_zip( $audioArrayVar, $fileName, true );
//echo '<p>File Path: ' . $fileName . '</p>';
var_dump(file_exists($fileName));
?>
I guess my real issue here, is that, although the script DOES NOT error...no ZIP is created. I have, over time placed outputs in the function at certain parts to see if it was even getting there, and it is - so I'm stumped on this.
What I really need is for one of you guys to scan the script over to see if there's anything glaringly obvious that just won't work!
Could it be a permissions thing? Should PHP be running in CGI mode or Apache? Or does this not make a difference?
The bones of this script were taken from: http://davidwalsh.name/create-zip-php but it's never worked even with that version alone.
PS - I'd just like to add, that if I uncomment the line that tells me how many files are in the ZIP etc, it seems to return the correct info...but the final check on whether the file exists returns false.
Ok, I have finally got it working.
Seems there was something in the addFile() that corrupted the code.
I changed this:
foreach( $valid_files as $file ) {
$just_name = preg_replace("/(.*)\/?([^\/]+)/","$2",$file);
$zip->addFile($file,$just_name);
//$zip->addFile($file,$file);
}
To this:
foreach( $valid_files as $file ) {
// Use basename to get JUST the file name from the path.
$zip->addFile($file, basename($file) );
}
The basename function gets only the file NAME to then place in the ZIP. Before it was the whole path on the server, and this was causing Windows to choke on it.
Hope this helps someone someday.
$value can = a folder structure to the language file. Example: languages/english.php
$value can also = the files name. Example: english.php
So I need to get the current folder that $value is in and delete the folder ONLY if there are no other files/folders within that directory (after deleting the actual file as I am doing already, ofcourse).
foreach($module['languages'] as $lang => $langFile)
{
foreach ($langFile as $type => $value)
{
#unlink($module_path . '/' . $value);
// Now I need to delete the folder ONLY if there are no other directories inside the folder where it is currently at.
// And ONLY if there are NO OTHER files within that folder also.
}
}
How can I do this?? And wondering if this can be done without using a while loop, since a while loop within a foreach loop could take some time, and need this to be as quick as possible.
And just FYI, the $module_path should never be deleted. So if $value = english.php, it should never delete the $module_path. Ofcourse, there will always be another file in there, so checking for this is not necessary, but won't hurt either way.
Thanks guys :)
EDIT
Ok, now I'm using this code here and it is NOT working, it is not removing the folders or the files, and I don't get any errors either... so not sure what the problem is here:
foreach($module['languages'] as $lang => $langFile)
{
foreach ($langFile as $type => $value)
{
if (#unlink($module_path . '/' . $value))
#rmdir(dirname($module_path . '/' . $value));
}
}
NEVERMIND, this works a CHARM!!! Cheers Everyone!!
The easyest way is try to use rmdir. This don't delete folder if it is not empty
rmdir($module_path);
also you can check is folder empty by
if(count(glob($module_path.'*'))<3)//delete
2 for . and ..
UPD: as I reviewed maybe you should replace $module_path by dirname($module_path.'.'.$value);
Since the directory you care about might be part of the $value, you need to use dirname to figure out what the parent directory is, you can't just assume that it's $module_path.
$file_path = $module_path . '/' . $value;
if (#unlink($file_path)) {
#rmdir(dirname($file_path));
}
if (is_file($value)) {
unlink($value);
} else if (is_dir($value)) {
if (count(scandir($value)) == 2) }
unlink($value)
}
}
http://php.net/manual/en/function.is-dir.php
http://www.php.net/manual/en/function.scandir.php
The code below will take a path, check if it is a file (i.e. not a directory). If it is a file, it will extract the directory name, then delete the file, then iterate over the dir and count the files in it, if the files are zero it'll delete the dir.
Code is as an example and should work, however privileges and environment setup may result in it not working.
<?php
if(!is_dir ( string $filename )){ //if it is a file
$fileDir = dirname ( $filename );
if ($handle = opendir($fileDir)) {
echo "Directory handle: $handle\n";
echo "Files:\n";
$numFiles=0;
//delete the file
unlink($myFile);
//Loop the dir and count the file in it
while (false !== ($file = readdir($handle))) {
$numFiles = $numFiles + 1;
}
if($numFiles == 0) {
//delete the dir
rmdir($fileDir);
}
closedir($handle);
}
}
?>
i wonder if theres a tutorial out there or you could give me a quick and simple approach how i could do the following task.
i have a folder on my server. i want to build a kind of cms where i can easily delete files from a folder. i know how to upload them, i already found a tutorial.
i think of simply running through all files, creating a list of them with a checkbox in front, selecting the checkbox and pressing a DELETE button.
is this a rather difficult task to get done? do you maybe kno any tutorial or something.
thank you very much!
Start with
List all files
<?php
// file_array() by Jamon Holmgren. Exclude files by putting them in the $exclude
// string separated by pipes. Returns an array with filenames as strings.
function file_array($path, $exclude = ".|..", $recursive = false) {
$path = rtrim($path, "/") . "/";
$folder_handle = opendir($path);
$exclude_array = explode("|", $exclude);
$result = array();
while(false !== ($filename = readdir($folder_handle))) {
if(!in_array(strtolower($filename), $exclude_array)) {
if(is_dir($path . $filename . "/")) {
if($recursive) $result[] = file_array($path, $exclude, true);
} else {
$result[] = $filename;
}
}
}
return $result;
}
?>
then create hyperlinks to the above files to a delete function like;
unlink('filename.jpg');