I am looking to select files over a certain age from a local directory tree. The following code works but is very inefficient, and because the directory is so large, it causes timeouts.
$dir = new RecursiveDirectoryIterator('store/', FilesystemIterator::SKIP_DOTS | FilesystemIterator::UNIX_PATHS);
$files = new RecursiveCallbackFilterIterator($dir, function ($current, $key, $iterator) {
// Allow recursion
if ($iterator->hasChildren()) {
return TRUE;
}
// Check for target folder name
if ($current->isFile() && strpos($current->getPath(), '/target-folder/')) {
return TRUE;
}
return FALSE;
});
$expiry = new DateTime();
$expiry = date_modify($expiry,"-28 days");
foreach (new RecursiveIteratorIterator($files) as $file) {
$cTime = new DateTime();
$cTime->setTimestamp($file->getCTime());
if ($cTime < $expiry) {
echo $file->getPath() . "/" . $file->getFileName() . " file Created " . $cTime->format('Y-m-d j:i:s') . "<br/>\n";
}
}
At the moment the code is scanning every file and folder and just returning the matches, but I know that the target-folder is always 3 levels deep (store/[sub-folder]/[sub-folder]/target-folder/[recursive search from here]) so I am searching the top 2 levels for nothing. However there is not always a target folder and I also need to search recursively within the target folder as the files I need will be in it's children. Is there any way to ignore the top 2 levels within the 'store' tree?
Related
I am creating a online application in PHP and as part of this I need to have a function where I give it a directory and it will count all the files in that folder, all the folders in that folder, and all the folders in the sub-directories etc until all files and sub-files in that folder are counted.
I have tried multiple methods and from doing my own research most people do it the way I have tried to attempt it but I am getting no success with my method.
Here is the code i currently use: (The $uniqueKey parameter is not the issue already debugged that out.)
//Get file count of user
function getFolderFileCount($uniqueKey){
$totalCount = 0;
$directory = "data/storage/uploads/" . $uniqueKey;
$dirContents = scandir($directory);
foreach($dirContents as $row){
if(is_dir($row)){
$totalCount += getFolderFileCount($directory . "/" . $row);
$totalCount++;
}else{
$totalCount++;
}
}
return $totalCount - 2;
}
The tree of my directory is as follows:
-www
--access
--code.php ( <- this is where the code is being ran from)
---data
----storage
-----uploads
------user500493 ( <- this is the $uniqueKey variable and therefore the folder than i need the file count of)
The errors i am getting in my error log is:
https://pastebin.com/g3jGD7u8
You are sending the $directory variable to the function again instead of the $uniqueKey '/' $row.
On the second loop it tries to access 'data/storage/uploads/data/storage/uploads/{$uniqueKey}/{$row}'.
//Get file count of user
function getFolderFileCount($uniqueKey){
$totalCount = 0;
$directory = "data/storage/uploads/" . $uniqueKey;
$dirContents = scandir($directory);
foreach($dirContents as $row){
if($row == '.' || $row == '..')
continue;
if(is_dir($directory. $row)){
$totalCount += getFolderFileCount($uniqueKey . "/" . $row);
$totalCount++;
}else{
$totalCount++;
}
}
return $totalCount;
}
Edit: The code considers subfolders inside the user500493 folder as well.
I did some tests on PHP 7.1.9-1 and it's fully working.
The is_dir() parameters must be the path to the folder, not only $rowbecause it is only the file or directory name.
I am trying to figure out a way of searching through all of the *.php files inside the parent directory, parent directory example:
/content/themes/default/
I am not wanting to search through all of the files in the sub-directories. I am wanting to search for a string embedded in the PHP comment syntax, such as:
/* Name: default */
If the variable is found, then get the file name and/or path. I have tried googling this, and thinking of custom ways to do it, this is what I have attempted so far:
public function build_active_theme() {
$dir = CONTENT_DIR . 'themes/' . $this->get_active_theme() . '/';
$theme_files = array();
foreach(glob($dir . '*.php') as $file) {
$theme_files[] = $file;
}
$count = null;
foreach($theme_files as $file) {
$file_contents = file_get_contents($file);
$count++;
if(strpos($file_contents, 'Main')) {
$array_pos = $count;
$main_file = $theme_files[$array_pos];
echo $main_file;
}
}
}
So as you can see I added all the found files into an array, then got the content of each file, and search through it looking for the variable 'Main', if the variable was found, get the current auto-incremented number, and get the path from the array, however it was always telling me the wrong file, which had nothing close to 'Main'.
I believe CMS's such as Wordpress use a similar feature for plugin development, where it searches through all the files for the correct plugin details (which is what I want to make, but for themes).
Thanks,
Kieron
Like David said in his comment arrays are zero indexed in php. $count is being incremented ($count++) before being used as the index for $theme_files. Move $count++ to the end of the loop, And it will be incremented after the index look up.
public function build_active_theme() {
$dir = CONTENT_DIR . 'themes/' . $this->get_active_theme() . '/';
$theme_files = array();
foreach(glob($dir . '*.php') as $file) {
$theme_files[] = $file;
}
$count = null;
foreach($theme_files as $file) {
$file_contents = file_get_contents($file);
if(strpos($file_contents, 'Main')) {
$array_pos = $count;
$main_file = $theme_files[$array_pos];
echo $main_file;
}
$count++;
}
}
So I'm trying to make a simple script, it will have a list of predefined files, search for anything that's not on the list and delete it.
I have this for now
<?php
$directory = "/home/user/public_html";
$files = glob($directory . "*.*");
foreach($files as $file)
{
$sql = mysql_query("SELECT id FROM files WHERE FileName='$file'");
if(mysql_num_rows($sql) == 0)
unlink($directory . $file);
}
?>
However, I'd like to avoid the query so I can run the script more often (there's about 60-70 files, and I want to run this every 20 seconds or so?) so how would I embedd a file list into the php file and check against that instead of database?
Thanks!
You are missing a trailing / twice.. In glob() you are giving /home/user/public_html*.* as the argument, I think you mean /home/user/public_html/*.*.
This is why I bet nothing matches the files in your table..
This won't give an error either because the syntax is fine.
Then where you unlink() you do this again.. your argument home/user/public_htmltestfile.html should be home/user/public_html/testfile.html.
I like this syntax style: "{$directory}/{$file}" because it's short and more readable. If the / is missing, you see it immediately. You can also change it to $directory . "/" . $file, it you prefer it. The same goes for one line conditional statements.. So here it comes..
<?php
$directory = "/home/user/public_html";
$files = glob("{$directory}/*.*");
foreach($files as $file)
{
$sql = mysql_query("SELECT id FROM files WHERE FileName=\"{$file}\";");
if(mysql_num_rows($sql) == 0)
{
unlink("{$directory}/{$file}");
}
}
?>
EDIT: You requested recursion. Here it goes..
You need to make a function that you can run once with a path as it's argument. Then you can run that function from inside that function on subdirectories. Like this:
<?php
/*
ListDir list files under directories recursively
Arguments:
$dir = directory to be scanned
$recursive = in how many levels of recursion do you want to search? (0 for none), default: -1 (for "unlimited")
*/
function ListDir($dir, $recursive=-1)
{
// if recursive == -1 do "unlimited" but that's no good on a live server! so let's say 999 is enough..
$recursive = ($recursive == -1 ? 999 : $recursive);
// array to hold return value
$retval = array();
// remove trailing / if it is there and then add it, to make sure there is always just 1 /
$dir = rtrim($dir,"/") . "/*";
// read the directory contents and process each node
foreach(glob($dir) as $node)
{
// skip hidden files
if(substr($node,-1) == ".") continue;
// if $node is a dir and recursive is greater than 0 (meaning not at the last level or disabled)
if(is_dir($node) && $recursive > 0)
{
// substract 1 of recursive for ever recursion.
$recursive--;
// run this same function again on itself, merging the return values with the return array
$retval = array_merge($retval, ListDir($node, $recursive));
}
// if $node is a file, we add it to the array that will be returned from this function
elseif(is_file($node))
{
$retval[] = $node;
// NOTE: if you want you can do some action here in your case you can unlink($node) if it matches your requirements..
}
}
return $retval;
}
// Output the result
echo "<pre>";
print_r(ListDir("/path/to/dir/",1));
echo "</pre>";
?>
If the list is not dynamic, store it in an array:
$myFiles = array (
'some.ext',
'next.ext',
'more.ext'
);
$directory = "/home/user/public_html/";
$files = glob($directory . "*.*");
foreach($files as $file)
{
if (!in_array($file, $myFiles)) {
unlink($directory . $file);
}
}
I have a directory that contains subdirectories of Year (e.g. 2009), and within these sub directories contain sub directories of months (e.g. 02)
I need to calculate the total files in each month folder, along with the total file size for each folder, and out put the two totals (Total Size & Total Files) along with the year and month folder names.
Example:
05/2009 - 512MB (12 Files)
06/2009 - 45KB (21 Files)
07/2009 - 6MB (8 Files)
01/2011 - 54MB (2 Files)
04/2011 - 652MB (55 Files)
...
I have created the following function that will calculate the total size and file of a directory. How do I adapt it to cater for muti sub folders lists as outlined above?
function foldersize($path) {
$total_size = 0;
$total_files = 0;
$files = scandir($path);
foreach($files as $file) {
if ($t<>"." && $t <> "..") {
if (is_dir($path.'/'.$file)) {
$size = foldersize($path.'/'.$file);
$total_files++;
$total_size += $size;
}
else {
$size = filesize($path.'/'.$file);
$total_files++;
$total_size += $size;
}
}
}
return $total_size.' | '.$total_files;
}
You should use the RecursiveDirectoryIterator to iterate over subdirectories recursively.
$_recursiveIterator = new RecursiveDirectoryIterator(
$path, FilesystemIterator::SKIP_DOTS);
$iterator = new RecursiveIteratorIterator(
$_recursiveIterator, RecursiveIteratorIterator::SELF_FIRST);
foreach($iterator as $subdirPath => $subdirInfo) {
if ($subdirInfo->isDir()) {
echo $subdirPath . ' ' . foldersize($subdirPath);
}
}
What happens here is a bit difficult to understand and the SPL Iterators are not documented well, so I'll try to explain:
Iterators are objects that implement a special interface so they can
be used in foreach loops
RecursiveDirectoryIterator is a recursive iterator for the file system. Recursive iterators are a special type of iterator that yield an iterator themselves in each iteration. They can be used for nested data structures.
SKIP_DOTS tells it to ignore the . and .. entries
The RecursiveIteratorIterator is used to "flatten" a recursive iterator. It gets passed a recursive iterator and yields all subsequent entries in a single loop.
SELF_FIRST means that parents are yielded before their children (i.e. 2009 before 2009/01)
in the loop, the key $subdirPath is the path of each item and the value $subdirInfo a SplFileInfo object (in fact, it is the RecursiveDirectoryIterator itself which extends from SplFileInfo in the end - but that doesn't matter at this point)
isDir() checks if the current item is a directory
The next step would be to refactor your foldersize() function to use iterators too. It might be sensible to to the iteration over all files and size calculation at once. Edit: Look at the answer from Mike Brant for such a solution, the explanation above applies to that too.
You should look at the recursiveDirectoryIterator and recursiveIteratorIterator classes provided in SPL. This should meet your needs quite nicely.
http://php.net/manual/en/class.recursivedirectoryiterator.php
http://php.net/manual/en/class.recursiveiteratoriterator.php
Sample usage:
$path = realpath('/path/to/directory');
$total_files = 0;
$total_filesize = 0;
$dir_files = 0;
$dir_filesize = 0;
$iterator = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($path, FilesystemIterator::SKIP_DOTS), RecursiveIteratorIterator::SELF_FIRST);
$first_dir = true;
foreach($iterator as $path => $object){
if ($path->isDir()) {
// echo out information from last directory
if (false === $first_dir) {
echo "Number of Files in Directory: " . $dir_files . "\nTotal File Size of Directory: " . $dir_filesize . " bytes\n\n";
// reset directory counters
$dir_files = 0;
$dir_filesize = 0;
}
// start new directory output
echo $path->getPathname() . "\n";
} else if ($path->isFile()) {
$total_filesize += $path->getSize();
$dir_filesize += $path->getSize();
$total_files++;
$dir_files++;
}
}
echo "\nTotal Files: " . $total_files . "\nTotal File Size: " . $total_filesize . " bytes";
I am using this function in codeigniter to try to check and make directories and sub directories only if they do not exist. Only the k_uploads is made but an error occurs in making the sub directory in the main directory 'k_upoloads'. The structure should be as
k_uploads (main directory)
-2012 (subdirectory in main directory - made per year in k_uploads)
-Jan(subdirectory in 2012 - made every month in 2012)
-subdirx (subdirectory in Jan - holds the excel files for that month)
xxyyy.xlsx (month files in subdirx)
Each year and month directories and sub directories should be created. I cant figure where the problem is, it works with plain php but not in codeigniter.
public function makeDir(){
$labref = $this->uri->segment(3);
$dirName='k_uploads';
echo $store_dir= date('Y').'/'.date('M').'/'.$subdirx;
if(!is_dir($dirName))
$k= mkdir($dirName, 0777);
if($k){
echo $dirName. 'dir has been created';
}else{
echo 'An error occured';
}
if(is_dir($dirName))
$w= mkdir($store_dir,0777);
if($w){
echo $sore_dir. 'subdirs have been created';
}else{
echo 'An error occured';
}
}
mkdir has a recursive flag which can be set. This will create the full path. See PHP: mkdir
so you should use mkdir($store_dir,0777, true)
The function could look something like the following:
public function makeDir(){
$subdirx = $this->uri->segment(3);
$store_dir= APPPATH . 'k_uploads/' . date('Y').'/'.date('M').'/'.$subdirx;
if(!is_dir($store_dir)) {
return mkdir($dirName, 0777, true);
} else {
return true;
}
}
In order to do this, you will need to create each sub-directory in a sequence, as PHP will not do this for you. You might want to check if the directory already exists using is_dir() while doing so.
Here's an example:
function createDir($dirToMake) {
$root = "/home/sites/test/www/";
$dArray = explode("/",$dirToMake);
if (file_exists($root) && is_dir($root)) {
// just a quick check
if (substr($root,0,-1) !== "/") $root .= "/";
foreach ($dArray as $v) {
if (strlen($v) == 0) continue;
$root = $root.$v."/";
if (file_exists($root) && is_dir($root)) continue;
mkdir($root);
}
}
else throw new Exception("Root directory does not exist");
}
This function also allows for the usual mistakes (// being one of them), and will loop through, creating the sub-directory architecture needed if it doesn't exist already.