I am connecting to another server via php's ftp connect.
However I need to be able to extract all html files from it's web root which is causing me a bit of a headache...
I found this post Recursive File Search (PHP) which talks about using RecursiveDirectoryIterator function however this is for a directory on the same server as the php script its self.
I've had a go with writing my own function but not sure I've got it right... assuming that the original path sent to the method is the doc root of the server:
public function ftp_dir_loop($path){
$ftpContents = ftp_nlist($this->ftp_connection, $path);
//loop through the ftpContents
for($i=0 ; $i < count($ftpContents) ; ++$i)
{
$path_parts = pathinfo($ftpContents[$i]);
if( in_array($path_parts['extension'], $this->accepted_file_types ){
//call the cms finder on this file
$this->html_file_paths[] = $path.'/'.$ftpContents[$i];
} elseif(empty( $path_parts['extension'] )) {
//run the directory method
$this->ftp_dir_loop( $path.'/'.$ftpContents[$i] );
}
}
}
}
Has anyone seen a premade class to do something like this?
You can try
public function ftp_dir_loop($path) {
$ftpContents = ftp_nlist($this->ftp_connection, $path);
foreach ( $ftpContents as $file ) {
if (strpos($file, '.') === false) {
$this->ftp_dir_loop($this->ftp_connection, $file);
}
if (in_array(pathinfo($file, PATHINFO_EXTENSION), $this->accepted_file_types)) {
$this->html_file_paths[$path][] = substr($file, strlen($path) + 1);
}
}
}
Related
I have ftp server with a lot of subfolders and files in it. I need to retrieve the folder structure from ftp, which shows names of all folders and subfolders from a specified starting path. I'm not interested in files included in each folder, only the directory tree. I'm using PHP and my server does not support mlsd.
Thanks for help.
I implemented my own recursive function, which for some reason is not working.
function ftp_list_files_recursive($ftp_stream, $path) {
$lines = ftp_rawlist($ftp_stream, $path);
$result = [];
if (is_array($lines) || is_object($lines)) {
foreach ($lines as $line) {
$exp0 = explode('<', $line);
if (sizeof($exp0) > 1):
$exp1 = explode('>', $exp0[1]);
if ($exp1[0] == 'DIR') {
$file_path=$path . "/" . ltrim($exp1[1]);
$result = array_merge($result, ftp_list_files_recursive($ftp_stream, $file_path));
} else {
$result[] = $file_path;
}
endif;
}
}
return $result;
}
The ftp_rawlist returns directory info as: 01-18-20 01:00PM <DIR> DirName so first I explode on < and check whether it was successful. If yes, then it means a string had DIR in it and it can be further exploded on >. It could have been done with regular expression, but that works for me now. If I print $file_path variable I see that it changes, so I assume the recursion works. However, the $result array is always empty. Any thoughts on that?
Start here: PHP FTP recursive directory listing.
You just need to adjust the code to:
the DOS-style listing you have from your FTP server (IIS probably) and
to collect only the folders.
function ftp_list_dirs_recursive($ftp_stream, $directory)
{
$result = [];
$lines = ftp_rawlist($ftp_stream, $directory);
if ($lines === false)
{
die("Cannot list $directory");
}
foreach ($lines as $line)
{
// rather lame parsing as a quick example:
if (strpos($line, "<DIR>") !== false)
{
$dir_path = $directory . "/" . ltrim(substr($line, strpos($line, ">") + 1));
$subdirs = ftp_list_dirs_recursive($ftp_stream, $dir_path);
$result = array_merge($result, [$dir_path], $subdirs);
}
}
return $result;
}
How to list all files recursively in remote server via SFTP in PHP?
opendir or scandir only list in current folder.
phpseclib library has recursive listing built-in, just use Net_SFTP.nlist() with $recursive = true:
set_include_path(get_include_path() . PATH_SEPARATOR . 'phpseclib');
require_once("Net/SFTP.php");
$sftp = new Net_SFTP($hostname);
if (!$sftp->login($username, $password))
{
die("Cannot login to the server");
}
if (!($files = $sftp->nlist($path, true)))
{
die("Cannot read directory contents");
}
foreach ($files as $file)
{
echo "$file\n";
}
If you need to list even the folder names (so that you capture even empty folder), you have to re-implement, what nlist does:
function nlist_with_folders($sftp, $dir)
{
$files = $sftp->rawlist($dir);
if ($files === false)
{
$result = false;
}
else
{
$result = array();
foreach ($files as $name => $attrs)
{
if (($name != ".") && ($name != ".."))
{
$path = "$dir/$name";
$result[] = $path;
if ($attrs["type"] == NET_SFTP_TYPE_DIRECTORY)
{
$sub_files = nlist_with_folders($sftp, $path);
$result = array_merge($result, $sub_files);
}
}
}
}
return $result;
}
The following was/is true about phpseclib 1.0 branch, whose latest release 1.0.19 from 2020 is still usable, but it does not seem to be updated anymore. The phpseclib 2.0 and 3.0 use Composer, so their setup is somewhat more complicated.
phpseclib does not require any installation and does not have any mandatory dependencies. It's a "pure PHP" code. You just download an archive with the PHP code and extract it to your webserver (or extract it locally and upload the extracted code, if you do not have a shell access to the webserver). In my example, I have extracted it to phpseclib subfolder.
I have a number of different hosting accounts set up for clients and need to calculate the amount of storage space being used on each account, which would update regularly.
I have a database set up to record each clients storage usage.
I attempted this first using a PHP file on each account, run by a Cron Job. If run manually by myself, it would output the correct filesize and update the correct size to the database, although when run from the Cron Job, it would output 0.
I then attempted to run this file from a Cron Job from the main account but figured this wouldn't actually work as my hosting would block files from another server and I would end up with the same result as before.
I am now playing around with FTP access to each account from a Cron Job from the main account which looks something like below, the only problem is I don't know how to calculate directory size rather than single file sizes using FTP access, and don't know how to reiterate this way? Hoping somebody might be able to help here before I end up going around in circles?
I will also add the previous first attempt too.
$ftp_conn = ftp_connect($ftp_host, 21, 420) or die("Could not connect to server");
$ftp_login = ftp_login($ftp_conn, $ftp_username, 'mypassword');
$total_size = 0;
$contents = ftp_nlist($ftp_conn, ".");
// output $contents
foreach($contents as $folder){
while($search == true){
if($folder == '..' || $folder == '.'){
} else {
$file = $folder;
$res = ftp_size($ftp_conn, $file);
if ($res != -1) {
$total_size = $total_size + $res;
} else {
$total_size = $total_size;
}
}
}
}
ftp_close($ftp_conn);
This doesn't work as it doesn't calculate folder sizes and I don't know how to open the reiterate using this method?
This second script did work but would only work if opened manually, and return 0 if run by the cron job.
class Directory_Calculator {
function calculate_whole_directory($directory)
{
if ($handle = opendir($directory))
{
$size = 0;
$folders = 0;
$files = 0;
while (false !== ($file = readdir($handle)))
{
if ($file != "." && $file != "..")
{
if(is_dir($directory.$file))
{
$array = $this->calculate_whole_directory($directory.$file.'/');
$size += $array['size'];
$files += $array['files'];
$folders += $array['folders'];
}
else
{
$size += filesize($directory.$file);
$files++;
}
}
}
closedir($handle);
}
$folders++;
return array('size' => $size, 'files' => $files, 'folders' => $folders);
}
}
/* Path to Directory - IMPORTANT: with '/' at the end */
$directory = '../public_html/';
// return an array with: size, total files & folders
$array = $directory_size->size($directory);
$size_of_site = $array['size'];
echo $size_of_site;
Please bare in mind that I am currently testing and none of the MySQLi or PHP scripts are secure yet.
If your server supports MLSD command and you have PHP 7.2 or newer, you can use ftp_mlsd function:
function calculate_whole_directory($ftp_conn, $directory)
{
$files = ftp_mlsd($ftp_conn, $directory) or die("Cannot list $directory");
$result = 0;
foreach ($files as $file)
{
if (($file["type"] == "cdir") || ($file["type"] == "pdir"))
{
$size = 0;
}
else if ($file["type"] == "dir")
{
$size = calculate_whole_directory($ftp_conn, $directory."/".$file["name"]);
}
else
{
$size = intval($file["size"]);
}
$result += $size;
}
return $result;
}
If you do not have PHP 7.2, you can try to implement the MLSD command on your own. For a start, see user comment of the ftp_rawlist command:
https://www.php.net/manual/en/function.ftp-rawlist.php#101071
If you cannot use MLSD, you will particularly have problems telling if an entry is a file or folder. While you can use the ftp_size trick, as you do, calling ftp_size for each entry can take ages.
But if you need to work against one specific FTP server only, you can use ftp_rawlist to retrieve a file listing in a platform-specific format and parse that.
The following code assumes a common *nix format.
function calculate_whole_directory($ftp_conn, $directory)
{
$lines = ftp_rawlist($ftp_conn, $directory) or die("Cannot list $directory");
$result = 0;
foreach ($lines as $line)
{
$tokens = preg_split("/\s+/", $line, 9);
$name = $tokens[8];
if ($tokens[0][0] === 'd')
{
$size = calculate_whole_directory($ftp_conn, "$directory/$name");
}
else
{
$size = intval($tokens[4]);
}
$result += $size;
}
return $result;
}
Based on PHP FTP recursive directory listing.
Regarding cron: I'd guess that the cron does not start your script with a correct working directory, so you calculate a size of a non-existing directory.
Use an absolute path here:
$directory = '../public_html/';
Though you better add some error checking so that you can see yourself what goes wrong.
I'm in trouble, I am failing to understand why this error is happening.
So when I only run this code,
function getClientProject($cliente)
{
$file = fopen("Projetos.csv","r");
$ArrayCount = 0;
$bool = false;
while(! feof($file))
{
$data = fgetcsv($file);
if($data[0] != '')
{
if(substr_compare($data[0],$cliente, 0, 3) == 0)
{
if($ArrayCount > 0)
{
$total = count($OpenProject);
for($i=0;$i<$total;$i++)
{
if($OpenProject[$i] == $data[0])
$bool = true;
}
if($bool == false)
{
$OpenProject[$ArrayCount] = $data[0];
$ArrayCount++;
}
}else
{
$OpenProject[$ArrayCount] = $data[0];
$ArrayCount++;
}
}
}
}
fclose($file);
return $OpenProject;
}
It works and returns the Array. But when I call the function this way,
include_once 'CRM files/TSread.php';
$arrayC = getClientProject('SLA');
var_dump($arrayC);
No longer works and me these errors,
What am I doing wrong?
path, the file i'm using is "Projeto.php":
and my CRM files folder:
You are opening the file using a relative path. You assume that Projetos.csv is always in the same directory as the TSread.php file. Although, when including it, you seem to be in a higher directory (outside of the CRM files directory), therefor PHP can no longer find your CSV file, since it's trying to open it relative to the upper directory now.
You could pass the full path to your getClientProject method to avoid this. So, you would get something like:
$arrayC = getClientProject('SLA', __DIR__ . '/CRM files/Projectos.csv');
Obviously, you will need to change your function a little to work with this new constructor, so it should like something like this:
function getClientProject($cliente, $csv) {
$file = fopen($csv, "r");
// Followed by the rest of your function
I have a zip file containing one folder, that contains more folders and files, like this:
myfile.zip
-firstlevel
--folder1
--folder2
--folder3
--file1
--file2
Now, I want to extract this file using PHPs ZipArchive, but without the "firstlevel" folder. At the moment, the results look like this:
destination/firstlevel/folder1
destination/firstlevel/folder2
...
The result I'd like to have would look like this:
destination/folder1
destination/folder2
...
I've tried extractTo, which produces the first mentioned result, and copy(), as suggested here, but this doesn't seem to work at all.
My current code is here:
if($zip->open('myfile.zip') === true) {
$firstlevel = $zip->getNameIndex(0);
for($i = 0; $i < $zip->numFiles; $i++) {
$entry = $zip->getNameIndex($i);
$pos = strpos($entry, $firstlevel);
if ($pos !== false) {
$file = substr($entry, strlen($firstlevel));
if(strlen($file) > 0){
$files[] = $file;
}
}
}
//attempt 1 (extractTo):
//$zip->extractTo('./test', $files);
//attempt 2 (copy):
foreach($files as $filename){
copy('zip://'.$firstlevel.'/'.$filename, 'test/'.$filename);
}
}
How can I achieve the result I'm aiming for?
Take a look at my Quick Unzipper script. I wrote this for personal use a while back when uploading large zip files to a server. It was a backup, and 1,000s of files take forever with FTP so using a zip file was faster. I use Git and everything, but there wasn't another option for me. I place this php file in the directory I want the files to go, and put the zip file in the same directory. For my script, they all have to operate in the same directory. It was an easy way to secure it for my needs, as everything I needed was in the same dir.
Quick Unzipper: https://github.com/incomepitbull/QuickUnzipper/blob/master/unzip.php
I linked the file because I am not showcasing the repo, just the code that makes the unzip tick. With modern versions of PHP, there should't be anything that isn't included on your setup. So you shouldn't need to do any server config changes to use this.
Here is the PHP Doc for the ZipArchive class it uses: http://php.net/manual/en/class.ziparchive.php
There isn't any included way to do what you want, which is a shame. So I would unzip the file to a temp directory, then use another function to copy the contents to where you want. So when using ZipArchive, you will need to return the first item to get the folder name if it is unknown. If the folder is known, ie: the same pesky folder name every time, then you could hard code the name.
I have made it return the first item from the index. So if you ALWAYS have a zip with 1 folder inside it, and everything in that folder, this would work. However, if you have a zip file without everything consolidated inside 1 folder, it would fail. The code I have added will take care of your question. You will need to add further logic to handle alternate cases.
Also, You will still be left with the old directory from when we extract it to the temp directory for "processing". So I included code to delete it too.
NOTE: The code uses a lot of if's to show the processing steps, and print a message for testing purposes. You would need to modify it to your needs.
<?php
public function copyDirectoryContents($source, $destination, $create=false)
{
if ( ! is_dir($source) ) {
return false;
}
if ( ! is_dir($destination) && $create === true ) {
#mkdir($destination);
}
if ( is_dir($destination) ) {
$files = array_diff(scandir($source), array('.','..'));
foreach ($files as $file)
{
if ( is_dir($file) ) {
copyDirectoryContents("$source/$file", "$destination/$file");
} else {
#copy("$source/$file", "$destination/$file");
}
}
return true;
}
return false;
}
public function removeDirectory($directory, $options=array())
{
if(!isset($options['traverseSymlinks']))
$options['traverseSymlinks']=false;
$files = array_diff(scandir($directory), array('.','..'));
foreach ($files as $file)
{
if (is_dir("$directory/$file"))
{
if(!$options['traverseSymlinks'] && is_link(rtrim($file,DIRECTORY_SEPARATOR))) {
unlink("$directory/$file");
} else {
removeDirectory("$directory/$file",$options);
}
} else {
unlink("$directory/$file");
}
}
return rmdir($directory);
}
$file = dirname(__FILE__) . '/file.zip'; // full path to zip file needing extracted
$temp = dirname(__FILE__) . '/zip-temp'; // full path to temp dir to process extractions
$path = dirname(__FILE__) . '/extracted'; // full path to final destination to put the files (not the folder)
$firstDir = null; // holds the name of the first directory
$zip = new ZipArchive;
$res = $zip->open($file);
if ($res === TRUE) {
$firstDir = $zip->getNameIndex(0);
$zip->extractTo($temp);
$zip->close();
$status = "<strong>Success:</strong> '$file' extracted to '$temp'.";
} else {
$status = "<strong>Error:</strong> Could not extract '$file'.";
}
echo $status . '<br />';
if ( empty($firstDir) ) {
echo 'Error: first directory was empty!';
} else {
$firstDir = realpath($temp . '/' . $firstDir);
echo "First Directory: $firstDir <br />";
if ( is_dir($firstDir) ) {
if ( copyDirectoryContents($firstDir, $path) ) {
echo 'Directory contents copied!<br />';
if ( removeDirectory($directory) ) {
echo 'Temp directory deleted!<br />';
echo 'Done!<br />';
} else {
echo 'Error deleting temp directory!<br />';
}
} else {
echo 'Error copying directory contents!<br />';
}
} else {
echo 'Error: Could not find first directory';
}
}