I need to get a total count of JPG files within a specified directory, including ALL it's subdirectories. No sub-sub directories.
Structure looks like this :
dir1/
2 files
subdir 1/
8 files
total dir1 = 10 files
dir2/
5 files
subdir 1/
2 files
subdir 2/
8 files
total dir2 = 15 files
I have this function, which doesn't work fine as it only counts files in the last subdirectory, and total is 2x more than the actual amount of files. (will output 80 if I have 40 files in the last subdir)
public function count_files($path) {
global $file_count;
$file_count = 0;
$dir = opendir($path);
if (!$dir) return -1;
while ($file = readdir($dir)) :
if ($file == '.' || $file == '..') continue;
if (is_dir($path . $file)) :
$file_count += $this->count_files($path . "/" . $file);
else :
$file_count++;
endif;
endwhile;
closedir($dir);
return $file_count;
}
You could do it like this using the RecursiveDirectoryIterator
<?php
function scan_dir($path){
$ite=new RecursiveDirectoryIterator($path);
$bytestotal=0;
$nbfiles=0;
foreach (new RecursiveIteratorIterator($ite) as $filename=>$cur) {
$filesize=$cur->getSize();
$bytestotal+=$filesize;
$nbfiles++;
$files[] = $filename;
}
$bytestotal=number_format($bytestotal);
return array('total_files'=>$nbfiles,'total_size'=>$bytestotal,'files'=>$files);
}
$files = scan_dir('./');
echo "Total: {$files['total_files']} files, {$files['total_size']} bytes\n";
//Total: 1195 files, 357,374,878 bytes
?>
For the fun of it I've whipped this together:
class FileFinder
{
private $onFound;
private function __construct($path, $onFound, $maxDepth)
{
// onFound gets called at every file found
$this->onFound = $onFound;
// start iterating immediately
$this->iterate($path, $maxDepth);
}
private function iterate($path, $maxDepth)
{
$d = opendir($path);
while ($e = readdir($d)) {
// skip the special folders
if ($e == '.' || $e == '..') { continue; }
$absPath = "$path/$e";
if (is_dir($absPath)) {
// check $maxDepth first before entering next recursion
if ($maxDepth != 0) {
// reduce maximum depth for next iteration
$this->iterate($absPath, $maxDepth - 1);
}
} else {
// regular file found, call the found handler
call_user_func_array($this->onFound, array($absPath));
}
}
closedir($d);
}
// helper function to instantiate one finder object
// return value is not very important though, because all methods are private
public static function find($path, $onFound, $maxDepth = 0)
{
return new self($path, $onFound, $maxDepth);
}
}
// start finding files (maximum depth is one folder down)
$count = $bytes = 0;
FileFinder::find('.', function($file) use (&$count, &$bytes) {
// the closure updates count and bytes so far
++$count;
$bytes += filesize($file);
}, 1);
echo "Nr files: $count; bytes used: $bytes\n";
You pass the base path, found handler and maximum directory depth (-1 to disable). The found handler is a function you define outside, it gets passed the path name relative from the path given in the find() function.
Hope it makes sense and helps you :)
error_reporting(E_ALL);
function printTabs($level)
{
echo "<br/><br/>";
$l = 0;
for (; $l < $level; $l++)
echo ".";
}
function printFileCount($dirName, $init)
{
$fileCount = 0;
$st = strrpos($dirName, "/");
printTabs($init);
echo substr($dirName, $st);
$dHandle = opendir($dirName);
while (false !== ($subEntity = readdir($dHandle)))
{
if ($subEntity == "." || $subEntity == "..")
continue;
if (is_file($dirName . '/' . $subEntity))
{
$fileCount++;
}
else //if(is_dir($dirName.'/'.$subEntity))
{
printFileCount($dirName . '/' . $subEntity, $init + 1);
}
}
printTabs($init);
echo($fileCount . " files");
return;
}
printFileCount("/var/www", 0);
Just checked, it's working. But the alignment of results is bad,logic works
The answer by Developer is actually brilliant!
Use it like this to make it work:
System("find . -type f -print | wc -l");
if anyone is looking to count total number of files and directories.
Show/count total dir and sub dir count
find . -type d -print | wc -l
Show/count total number of files in main and sub dir
find . -type f -print | wc -l
Show/count only files from current dir (no sub dir)
find . -maxdepth 1 -type f -print | wc -l
Show/count total directories and files in current dir (no sub dir)
ls -1 | wc -l
A for each loops could do the trick more quickly ;-)
As I remember, opendir is derivated from the SplFileObject class which is a RecursiveIterator , Traversable , Iterator , SeekableIterator class, so, you don't need a while loops if you use the SPL standard PHP Library to retrive the whole images count even on subdirectory.
But, it's been a while that I didn't used PHP so I might made a mistake.
Related
The following code is meant to iterate through a directory of images and rename them.
I had it working well, but what I would like to add is a 'static' token at the beginning of the file that once renamed, it will then in future ignore.
For example, let's say that we have a directory of 100 files.
The first 20 of which go by the name "image-JTzkT1RYWnCqd3m1VXYcmfZ2nhMOCCunucvRhuaR5.jpg"
The last 80 go by the name "FejVQ881qPO5t92KmItkNYpny.jpg" where this could be absolutely anything.
I would like to ignore the files that have already been renamed (denoted by the 'image-" at the beginning of the file name)
How can I do this?
<?php
function crypto_rand_secure($min, $max) {
$range = $max - $min;
if ($range < 0) return $min; // not so random...
$log = log($range, 2);
$bytes = (int) ($log / 8) + 1; // length in bytes
$bits = (int) $log + 1; // length in bits
$filter = (int) (1 << $bits) - 1; // set all lower bits to 1
do {
$rnd = hexdec(bin2hex(openssl_random_pseudo_bytes($bytes)));
$rnd = $rnd & $filter; // discard irrelevant bits
} while ($rnd >= $range);
return $min + $rnd;
}
function getToken($length){
$token = "";
$codeAlphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
$codeAlphabet.= "abcdefghijklmnopqrstuvwxyz";
$codeAlphabet.= "0123456789";
for($i=0;$i<$length;$i++){
$token .= $codeAlphabet[crypto_rand_secure(0,strlen($codeAlphabet))];
}
return $token;
}
$dir = "/path/to/images";
if ( $handle = opendir ( $dir)) {
echo "Directory Handle = $handles<br />" ;
echo "Files: <br />" ;
while ( false !== ($file = readdir($handle))) {
if ( $file != "." && $file != ".." ) {
$isdir = is_dir ( $file ) ;
if ( $isdir == "1" ) {} // if its a directory do nothing
else {
$file_array[] = "$file" ; // get all the file names and put them in an array
//echo "$file<br />" ;
} // closes else
} // closes directory check
} // closes while
} // closes opendir
//Lets go through the array
$arr_count = count( $file_array ) ; // find out how many files we have found so we can initiliase the counter
for ( $counter=1; $counter<$arr_count; $counter++ ) {
echo "Array = $file_array[$counter] - " ; // tell me how many files there are
//$new = str_replace ( "C", "CIMG", $file_array[$counter] ) ; // now create the new file name
$new =getToken(50);
//if (substr($file_array[$counter]), 0, 3) == "gallery_image")
//{
//}
//else
//{
$ren = rename ( "$dir/$file_array[$counter]" , "image-$dir/$new.jpg" ) ; // now do the actual file rename
echo "$new<br />" ; // print out the new file name
//}
}
closedir ( $handle ) ;
?>
It seems like you're doing a lot of unnecessary work for something that should be quite trivial.
As far as I understand you want to get a list of the files from a directory and rename all of the ones not already pre-pended with image-.
You can do that with the following piece of code:
$dir = "/path/to/dir/"; //Trailing slash is necessary or you would have to change $dir.$newname to $dir.'/'.$newname further down the code
if(is_dir($dir)){
$dirHandle = opendir($dir);
while($file = readdir($dirHandle)){
if(is_file($dir.$file) === TRUE){
if(strpos($file, 'image-') === 0)
continue; //Skip if the image- is apready prepended
while(TRUE){
$cryptname = md5(mt_rand(1,10000).$file.mt_rand(1,10000)); //Create random string
$newname = 'image-'.$cryptname.'.jpg'; //Compile new file name
if(is_file($dir.$newname) === FALSE) //Check file name doesn't exist
break; //Exit loop if it doesn't
}
rename($dir.$file, $dir.$newname); //Rename file with new name
}
}
}
If you want to use your own function then you can the change the line:
$cryptname = md5(mt_rand(1,10000).$file.mt_rand(1,10000));
To use your function instead of md5. Like:
$cryptname = getToken(50);
I suggest that you should also check that the file name doesn't already exist otherwise - unlikely as it may be - you run the risk of overwriting files.
/images/ vs ./images/
Firstly, when dealing with the web you have to understand that there are effectively two root directories.
The web root and the document root. Take the example url http://www.mysite.com/images/myimage.jpg as far as the web root is concerned the path name is /images/myimage.jpg however, from php you have yo use the document root which would actually be something like: /home/mysite/pubic_html/images/myimage.jpg
So when you type / in php it thinks that you mean the directory which home is located in.
./images/ works because the ./ means this directory i.e. the one that the script/php is in.
In your case you probably have a file structure like:
>/
>home
>public_html
>images
image-243.jpg
image-243.jpg
index.php
So because index.php is in the same folder as images: ./images/ is equal to /home/public_html/images/. / on the other hand means the parent directory of home which has no images folder let alone the one you're targeting.
If you're more used to windows think of it like this: in php / means the document root directory (on windows that would be something like C:\).
You have the 'image-' in front of the directory portion of your file name:
$ren = rename ( "$dir/$file_array[$counter]" , "image-$dir/$new.jpg" )
should be
$ren = rename ( "$dir/$file_array[$counter]" , "$dir/image-$new.jpg" )
So I'm trying to make a simple script, it will have a list of predefined files, search for anything that's not on the list and delete it.
I have this for now
<?php
$directory = "/home/user/public_html";
$files = glob($directory . "*.*");
foreach($files as $file)
{
$sql = mysql_query("SELECT id FROM files WHERE FileName='$file'");
if(mysql_num_rows($sql) == 0)
unlink($directory . $file);
}
?>
However, I'd like to avoid the query so I can run the script more often (there's about 60-70 files, and I want to run this every 20 seconds or so?) so how would I embedd a file list into the php file and check against that instead of database?
Thanks!
You are missing a trailing / twice.. In glob() you are giving /home/user/public_html*.* as the argument, I think you mean /home/user/public_html/*.*.
This is why I bet nothing matches the files in your table..
This won't give an error either because the syntax is fine.
Then where you unlink() you do this again.. your argument home/user/public_htmltestfile.html should be home/user/public_html/testfile.html.
I like this syntax style: "{$directory}/{$file}" because it's short and more readable. If the / is missing, you see it immediately. You can also change it to $directory . "/" . $file, it you prefer it. The same goes for one line conditional statements.. So here it comes..
<?php
$directory = "/home/user/public_html";
$files = glob("{$directory}/*.*");
foreach($files as $file)
{
$sql = mysql_query("SELECT id FROM files WHERE FileName=\"{$file}\";");
if(mysql_num_rows($sql) == 0)
{
unlink("{$directory}/{$file}");
}
}
?>
EDIT: You requested recursion. Here it goes..
You need to make a function that you can run once with a path as it's argument. Then you can run that function from inside that function on subdirectories. Like this:
<?php
/*
ListDir list files under directories recursively
Arguments:
$dir = directory to be scanned
$recursive = in how many levels of recursion do you want to search? (0 for none), default: -1 (for "unlimited")
*/
function ListDir($dir, $recursive=-1)
{
// if recursive == -1 do "unlimited" but that's no good on a live server! so let's say 999 is enough..
$recursive = ($recursive == -1 ? 999 : $recursive);
// array to hold return value
$retval = array();
// remove trailing / if it is there and then add it, to make sure there is always just 1 /
$dir = rtrim($dir,"/") . "/*";
// read the directory contents and process each node
foreach(glob($dir) as $node)
{
// skip hidden files
if(substr($node,-1) == ".") continue;
// if $node is a dir and recursive is greater than 0 (meaning not at the last level or disabled)
if(is_dir($node) && $recursive > 0)
{
// substract 1 of recursive for ever recursion.
$recursive--;
// run this same function again on itself, merging the return values with the return array
$retval = array_merge($retval, ListDir($node, $recursive));
}
// if $node is a file, we add it to the array that will be returned from this function
elseif(is_file($node))
{
$retval[] = $node;
// NOTE: if you want you can do some action here in your case you can unlink($node) if it matches your requirements..
}
}
return $retval;
}
// Output the result
echo "<pre>";
print_r(ListDir("/path/to/dir/",1));
echo "</pre>";
?>
If the list is not dynamic, store it in an array:
$myFiles = array (
'some.ext',
'next.ext',
'more.ext'
);
$directory = "/home/user/public_html/";
$files = glob($directory . "*.*");
foreach($files as $file)
{
if (!in_array($file, $myFiles)) {
unlink($directory . $file);
}
}
I have a directory that contains subdirectories of Year (e.g. 2009), and within these sub directories contain sub directories of months (e.g. 02)
I need to calculate the total files in each month folder, along with the total file size for each folder, and out put the two totals (Total Size & Total Files) along with the year and month folder names.
Example:
05/2009 - 512MB (12 Files)
06/2009 - 45KB (21 Files)
07/2009 - 6MB (8 Files)
01/2011 - 54MB (2 Files)
04/2011 - 652MB (55 Files)
...
I have created the following function that will calculate the total size and file of a directory. How do I adapt it to cater for muti sub folders lists as outlined above?
function foldersize($path) {
$total_size = 0;
$total_files = 0;
$files = scandir($path);
foreach($files as $file) {
if ($t<>"." && $t <> "..") {
if (is_dir($path.'/'.$file)) {
$size = foldersize($path.'/'.$file);
$total_files++;
$total_size += $size;
}
else {
$size = filesize($path.'/'.$file);
$total_files++;
$total_size += $size;
}
}
}
return $total_size.' | '.$total_files;
}
You should use the RecursiveDirectoryIterator to iterate over subdirectories recursively.
$_recursiveIterator = new RecursiveDirectoryIterator(
$path, FilesystemIterator::SKIP_DOTS);
$iterator = new RecursiveIteratorIterator(
$_recursiveIterator, RecursiveIteratorIterator::SELF_FIRST);
foreach($iterator as $subdirPath => $subdirInfo) {
if ($subdirInfo->isDir()) {
echo $subdirPath . ' ' . foldersize($subdirPath);
}
}
What happens here is a bit difficult to understand and the SPL Iterators are not documented well, so I'll try to explain:
Iterators are objects that implement a special interface so they can
be used in foreach loops
RecursiveDirectoryIterator is a recursive iterator for the file system. Recursive iterators are a special type of iterator that yield an iterator themselves in each iteration. They can be used for nested data structures.
SKIP_DOTS tells it to ignore the . and .. entries
The RecursiveIteratorIterator is used to "flatten" a recursive iterator. It gets passed a recursive iterator and yields all subsequent entries in a single loop.
SELF_FIRST means that parents are yielded before their children (i.e. 2009 before 2009/01)
in the loop, the key $subdirPath is the path of each item and the value $subdirInfo a SplFileInfo object (in fact, it is the RecursiveDirectoryIterator itself which extends from SplFileInfo in the end - but that doesn't matter at this point)
isDir() checks if the current item is a directory
The next step would be to refactor your foldersize() function to use iterators too. It might be sensible to to the iteration over all files and size calculation at once. Edit: Look at the answer from Mike Brant for such a solution, the explanation above applies to that too.
You should look at the recursiveDirectoryIterator and recursiveIteratorIterator classes provided in SPL. This should meet your needs quite nicely.
http://php.net/manual/en/class.recursivedirectoryiterator.php
http://php.net/manual/en/class.recursiveiteratoriterator.php
Sample usage:
$path = realpath('/path/to/directory');
$total_files = 0;
$total_filesize = 0;
$dir_files = 0;
$dir_filesize = 0;
$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($path, FilesystemIterator::SKIP_DOTS), RecursiveIteratorIterator::SELF_FIRST);
$first_dir = true;
foreach($iterator as $path => $object){
if ($path->isDir()) {
// echo out information from last directory
if (false === $first_dir) {
echo "Number of Files in Directory: " . $dir_files . "\nTotal File Size of Directory: " . $dir_filesize . " bytes\n\n";
// reset directory counters
$dir_files = 0;
$dir_filesize = 0;
}
// start new directory output
echo $path->getPathname() . "\n";
} else if ($path->isFile()) {
$total_filesize += $path->getSize();
$dir_filesize += $path->getSize();
$total_files++;
$dir_files++;
}
}
echo "\nTotal Files: " . $total_files . "\nTotal File Size: " . $total_filesize . " bytes";
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
get last modified file in a dir?
I have a folder which contents more than 30000 subfolders in it. How can I get a list of subfolders with last modification date >= one hour ago? Is it possible to do that without getting a list of all files in an array and sorting it? I cannot use a readdir function because it returns files in the order in which they are stored by the filesystem and exhaustive search of the list of files will take a very long time.
Use GNU Find - it is simpler and faster!
find [path] -type d -mmin +60
The linux "find" command is pretty powerful.
$cmd = "find ".$path." type -d -mmin +60";
$out=`$cmd`;
$files=explode("\n",$out);
Give this a try:
<?php
$path = 'path/to/dir';
if (is_dir($path)) {
$contents = scandir($path);
foreach ($contents as $file) {
$full_path = $path . DIRECTORY_SEPARATOR . $file;
if ($file != '.' && $file != '..') {
if (is_dir($full_path)) {
$dirs[filemtime($full_path)] = $file;
}
}
}
// Sort in reverse order to put newest modification at the top
krsort($dirs);
$iteration = 0;
foreach ($dirs as $mtime => $name) {
if ($iteration != 5) {
echo $name . '<br />';
}
$iteration++;
}
}
?>
I have nearly 1 million photos auto-incremented starting at "1".
So, I have the following:
1.jpg
2.jpg
3.jpg
4.jpg
5.jpg
....
1000000.jpg
Currently, I have all of these files in a single Ext4 linux directory.
As you can imagine, the filesystem is crazy slow.
Using PHP, how can I create an algorithm that divides up my images into a directory structure so that each directly has significantly less objects per directory.
For example:
1000/1.jpg
1000/2.jpg
1000/3.jpg
...
1000/999.jpg
1000/1000.jpg
2000/1001.jpg
2000/1002.jpg
2000/1003.jpg
2000/1999.jpg
How would I divide/modulus/implode/shift the image name (id) into a file structure like such above?
UPDATE:
Basically, I want to create a PHP function that does the following.
Only accept positive integers, not including 0.
For values 1-999, return 0
For values 1000-1999, return 1000
For values 10,000-10,999, return 10000
For values 25,000-25,999, return 25000
When I've done things like this in the past, I create subdirectories from the rightmost digits, so that as they increment, they are added to all the directories more evenly:
4/3/1234.jpg
5/3/1235.jpg
6/3/1236.jpg
7/3/1237.jpg
8/3/1238.jpg
Re your comment:
how would I do that with PHP?
Here's an example function in PHP:
function NumToPath($n)
{
$n = (int) $n;
if ($n <= 0) {
return false;
}
$n = str_pad($n, 7, "0", STR_PAD_LEFT);
preg_match("/.*(\d)(\d)$/", $n, $matches);
$path = $matches[2] . "/" . $matches[1] . "/" . $n . ".jpg";
return $path;
}
You should keep the current (incrementing) number in a meta-file, so that you can retrieve it very quickly. Everything else is pretty easy:
$directory = ceil($num / 1000) * 1000;
$filename = $num . '.jpg';
$path = $directory . '/' . $filename;
i guess you think far to complex.
untested very basic way of doing this:
for($i=0;$i<1000000;$i++){
if(file_exists($i.'.jpg'){
$multi = floor($i/1000);
$dir = ($multi <= 1) $dir = 1000 : $dir=$multi*1000;
if(!is_dir($dir)){
mk_dir($dir);
}
move($i.'.jpg',$dir.'/'.$i.'.jpg);
}
}
Untested but should work. Certainly back up your stuff first or try it on just a small sample directory.
<?php
// specify your image directory, no trailing slash
$imagedir = 'your/path/to/images';
// loop through each jpg file in the image directory
foreach (glob($imagedir.'/*.jpg') as $file)
{
// determine new folder name
$newdir = floor(basename($file) / 1000) * 1000;
// ^ php will interpret the filename's number for the calculation
// eg: ('999.jpg' / 1000) returns .999
// ensure the 0 folder name is not null
if (!$newdir) {
$newdir = '0';
}
// add the new folder to the image directory
$newdir = $imagedir.'/'.$newdir;
if (!is_dir($newdir)) {
mkdir($newdir);
}
// move the image
rename($file, $newdir.'/'.basename($file);
}
?>