I have an array or over 1,000 files that I need to download to my local PC from my server.
I need to keep it to replicate the same file/folder structure for each file.
Here is an example list of files:
/lib/Zend/EventManager/Filter/FilterIterator.php
/lib/Zend/EventManager/config.php
/lib/Zend/Text/Figlet/themes.php
/lib/Zend/Gdata/Analytics/DataEntry.php
/lib/Zend/Gdata/Analytics/AccountQuery.php
/lib/Zend/Gdata/Calendar/files.php
/lib/Zend/Gdata/Query.php
/lib/Zend/Gdata/Gbase/Feed.php
/lib/Zend/Gdata/Photos.php
/lib/Zend/Gdata/Photos/AlbumFeed.php
/lib/Zend/Gdata/Media/Extension/press.php
/lib/Zend/Gdata/Media/file.php
/lib/Zend/Gdata/Extension/RecurrenceException.php
/lib/Zend/Gdata/Extension/Comments.php
/lib/Zend/Gdata/Extension/Recurrence.php
/lib/Zend/Gdata/Extension/Rating.php
To create the folders, navigate to them with FTP, and then download them would take me all day long! How can I do this with PHP?
These files cannot be accessed in the browser with a URL so I have to use the file path.
UPDATE
Here is what I have tried so far using PHP ZipArchive...
files.txt
Test file to test a sample of the files I will need. Final result will be over 1,000 files
lib/Zend/EventManager/Filter/FilterIterator.php
lib/Zend/EventManager/config.php
lib/Zend/Text/Figlet/themes.php
lib/Zend/Gdata/Analytics/DataEntry.php
lib/Zend/Gdata/Analytics/AccountQuery.php
lib/Zend/Gdata/Calendar/files.php
lib/Zend/Gdata/Query.php
lib/Zend/Gdata/Gbase/Feed.php
lib/Zend/Gdata/Photos.php
lib/Zend/Gdata/Photos/AlbumFeed.php
lib/Zend/Gdata/Media/Extension/press.php
lib/Zend/Gdata/Media/file.php
lib/Zend/Gdata/Extension/RecurrenceException.php
lib/Zend/Gdata/Extension/Comments.php
lib/Zend/Gdata/Extension/Recurrence.php
lib/Zend/Gdata/Extension/Rating.php
lib/Zend/Gdata/Books/VolumeQuery.php
lib/Zend/Gdata/Books/VolumeFeed.php
lib/Zend/Gdata/Exif/themes.php
lib/Zend/Gdata/MimeBodyString.php
lib/Zend/Gdata/HttpAdapterStreamingProxy.php
lib/Zend/Gdata/Spreadsheets/Extension/test.php
lib/Zend/Gdata/Spreadsheets/ListEntry.php
lib/Zend/Gdata/Gapps/Query.php
lib/Zend/Gdata/Gapps/GroupQuery.php
lib/Zend/Gdata/Gapps/EmailListRecipientQuery.php
lib/Zend/Gdata/Gapps/Error.php
lib/Zend/Gdata/Gapps/OwnerFeed.php
lib/Zend/Gdata/Gapps/alias.php
lib/Zend/Gdata/Gapps/MemberQuery.php
lib/Zend/Gdata/Gapps/EmailListQuery.php
lib/Zend/Gdata/Gapps/NicknameFeed.php
lib/Zend/Gdata/Exif.php
lib/Zend/Gdata/App/LoggingHttpClientAdapterSocket.php
lib/Zend/Gdata/App/Extension.php
lib/Zend/Gdata/App/MediaEntry.php
lib/Zend/Gdata/App/FeedEntryParent.php
lib/Zend/Gdata/App/AuthException.php
download.php
$zip = new ZipArchive();
$filename = "./test112.zip";
if ($zip->open($filename, ZipArchive::CREATE)!==TRUE) {
exit("cannot open <$filename>\n");
}else{
echo 'zip good';
}
//$zip->addFromString("testfilephp.txt" . time(), "#1 This is a test string added as testfilephp.txt.\n");
//$zip->addFile("lib/Zend/files2.txt" ,"lib/Zend/EventManager/test.php" );
// list of files to download
$lines = file('files.txt');
// Loop through our array of files from the files.txt file
foreach ($lines as $line_num =>$file) {
//echo "Line #<b>{$line_num}</b> : " . htmlspecialchars($file) . "<br />\n";
// Add files to Zip file incliuding folder structure
$zip->addFile($file,$file);
echo $file;
}
// show number of files in new zip file and close zip archive
echo "numfiles: " . $zip->numFiles . "\n";
echo "status:" . $zip->status . "\n";
$zip->close();
Result
This creates my zip file however instead of adding all files, it only adds the last file in my files array to the zip archive! In this example that is lib/Zend/Gdata/App/AuthException.php
As you have SSH access, you could simply run this on the server:
# Change '*.php' to whatever you want to retrieve:
find . -name '*.php' -print | zip archive.zip -#
Then you can get the file archive.zip via scp or ftp.
Related
What I'm trying to do:
Use PHP ftp_nlist to retrieve the contents of a directory on the FTP server
The problem:
For directories that contain a lot of files (the one I encountered the problem on has nearly 40 thousand files, and no subfolders), the ftp_nlist function is returning false. For directories that are not as large, the ftp_nlist function returns an array of filenames as expected.
What I've tried:
Enabling passive mode (it already was enabled, but I see it as a common suggestion)
Adding ftp_set_option($conn_id, FTP_USEPASVADDRESS, false); after my ftp_login
using ftp_chdir, although my folder names never have spaces anyways
echoing error_get_last() after ftp_nlist returns false. The error show seems unrelated, but is shown below.
My code:
In case it is useful, here is the function I have created. What it is supposed to do is...
take in $fm (filemaker, unrelated to this problem)
take in $FTPConnectionID (the ftp connection I established in the prior to the function call)
take in $FolderPath (the path of the folder on the FTP server for which I want to list files/subfolders recursively - ex: "SomeFolder/Testing")
take in $TextFile (I am writing the paths of every file found on the FTP server to a text file, which was created prior to calling the function)
function createAuditFile($fm, $FTPConnectionID, $FolderPath, $TextFile) {
echo "createAuditFile called for " . $FolderPath . "\n";
//Get the contents of the given path. Will include files and folders.
$FolderContents = ftp_nlist($FTPConnectionID, $FolderPath);
if($FolderContents == false) {
echo "Couldn't get " . $FolderPath . "\n";
echo "ERROR: " . print_r(error_get_last()) . "\n";
} else {
print_r($FolderContents);
}
//Loop through the array, call this function recursively if a folder is found.
if(is_array($FolderContents)) {
foreach($FolderContents as $Content) {
//Create a varaible for the folder path
$ContentPath = $FolderPath . "/" . $Content;
//Call the function recursively if a folder is found
if(pathinfo($Content, PATHINFO_EXTENSION) == "") {
createAuditFile($fm, $FTPConnectionID, $ContentPath, $TextFile);
echo "Recursive call for " . $ContentPath . "\n";
//If a file is found, add the file ftp path to our array
} else {
echo "Writing to file: " . $ContentPath . "\n";
fwrite($TextFile, $ContentPath . "\n");
}
}
}
}
I can provide other code if needed, but I think my question is less of a coding issue, and more of an understanding ftp_nlist issue. I've been stuck on this for hours, so any help is appreciated. And like I said, this function works just fine for most folder paths passed to it, the problem is when there are tens of thousands of files within the folder. Thank you!
I have been trying to reach some files thru glob with the extension .vic thru php file_get_contents.
The problem I ran into is that it does not work.
My code:
<?php
$fileList = glob('*.vic');
echo $_SERVER['DOCUMENT_ROOT'] .'/server/main/';
foreach($fileList as $filename) {
echo file_get_contents($_SERVER['DOCUMENT_ROOT'].'/server/main/' . $filename) . "\r\n";
}
When I run it echo's the folder I need but the file content doesn't show up.
There is a .vic file in the folder with file contents.
Did I do something wrong or is this not posible
PHP Output:
C:/xampp/htdocs/server/main/
Try:
$fileList = glob($_SERVER['DOCUMENT_ROOT'] .'/server/main/*.vic');
The following PHP file creates a ZIP File and works as it should.
<?php
$zip = new ZipArchive();
$ZIP_name = "./path/Prefix_" .$date . ".zip";
if ($zip->open($ZIP_name, ZipArchive::CREATE)!==TRUE) {
exit("There is a ZIP Error");
}
if ($zip->open($ZIP_name, ZipArchive::CREATE)==TRUE) {
echo "ZIP File can be created" . "<br>";
}
foreach($list as $element) {
$path_and_filename = "../path_to_somewhere/product_"
. $element
. ".csv";
$zip->addFile($path_and_filename, basename($path_and_filename));
}
echo "numfiles: " . $zip->numFiles . "\n"; // number of element files
echo "status:" . $zip->status . "\n"; // Status "0" = okay
$zip->close();
?>
There is only a small blemish:
The above foreach-loop retrieves elements from an array where all elements are sorted in alphabetical order. After the ZIP-File creation, the files within the ZIP are in different order, maybe caused by different file size.
Is there a way to sort the csv files within the ZIP with PHP later on? I'm new to ZIP creation with PHP an I have not found something helpful in the documentation.
You can't do that, better just sort the file list in your program, not in the file system (-;
What I am trying to do
Connect to FTP server
Create new directories
Upload file into one of those new directories
What's going wrong?
Ok, so there is nothing wrong with the ftp connection, I am logging in with the credentials, switching to the relevant starting directory, which in this case is a directory on the server called "test".
At the start, this "test" directory is empty, as I just created it.
What I am then doing is taking a file from my local machine and attempting to upload that the remote path test/Folder1/Sub1/Sub2/Myfile.txt on the server.
To do this, I first split the path by "/" and check to see if I can switch to each of those directories, and if I can't, then I create them:
(You'll have to excuse the debugging dumps in the code)
// Split the path by directories and try and create all of them if they don't exist
$workingDir = $this->dir;
$newPath = str_replace($this->dir, "", $path);
$split = array_filter( explode("/", $newPath) );
if ($split)
{
foreach($split as $dir)
{
// Can we change to it?
if (!#ftp_chdir($this->conn, $workingDir . '/' . $dir)){
// Try and make the directory
if (!#ftp_mkdir($this->conn, $workingDir . '/' . $dir)){
var_dump("failed to make: " .$workingDir . '/' . $dir );
return false;
}
// Set the permissions to the default
$this->chmod = 0777;
ftp_chmod($this->conn, $this->chmod, $workingDir . '/' . $dir);
var_dump("Successfully made directory: " . $workingDir . '/' . $dir . " with permissions: " . $this->chmod);
}
$workingDir .= '/' . $dir;
}
}
This works fine, and I get the output of:
'Successfully made directory: /home/conn/test/Folder1 with permissions: 511' (length=74)
'Successfully made directory: /home/conn/test/Folder1/Sub1 with permissions: 511' (length=79)
'Successfully made directory: /home/conn/test/Folder1/Sub1/Sub2 with permissions: 511' (length=84)
If i exit at this point I can confirm the directories now exist on the remote server, with permissions of 0777
Next bit is to do the file upload into the new directory:
var_dump("Use this new name: {$newName}");
$filename = (($newName) ? $newName : $file->getFileName());
var_dump("working in remote directory: {$this->dir}");
var_dump("upload to remote: {$filename}, from: {$file->getFullPath()}");
// Change to that directory?
$split = explode("/", $filename);
$realFileName = array_pop($split);
$filedir = implode("/", $split);
var_dump("calling ftp_chdir() to : {$filedir}");
if (!#ftp_chdir($this->conn, $filedir)){
var_dump("couldn't change to {$filedir}");
return false;
};
var_dump( ftp_pwd($this->conn) );
var_dump("actual file name: " . $realFileName);
return ftp_put($this->conn, $realFileName, $file->getFullPath(), FTP_BINARY);
And again, this works fine. I get all the expected dumps:
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:121:string 'Use this new name: Folder1/Sub1/Sub2/Myfile.txt' (length=47)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:124:string 'working in remote directory: /home/conn/test' (length=44)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:125:string 'upload to remote: Folder1/Sub1/Sub2/Myfile.txt, from: C:\Ampps\www\duckfusioncore\app\lotto\tmp\tmp-nsEMH1RsDX' (length=110)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:131:string 'calling ftp_chdir() to : Folder1/Sub1/Sub2' (length=42)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:139:string '/home/conn/test/Folder1/Sub1/Sub2' (length=33)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:140:string 'actual file name: Myfile.txt' (length=28)
C:\Ampps\www\duckfusioncore\app\lotto\controllers\IndexController.php:28:boolean true
All the directories exist. The file is uploaded. All is well.
Except, if I now change the path I want to upload to and change toa new sub directory, which doesn't yet exist, e.g: Folder1/Sub1/NewSub/Myfile.txt
This is where it all goes wrong.
So the script runs through the create directory part again, and it successfully creates the new directory "NewSub", with the exact same permissions as before, and I can confirm that the directory exists on the remote server.
Except now when we get to the upload part, even though the directory exists, and this worked previously as we saw, it is unable to change to that new directory:
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:121:string 'Use this new name: Folder1/Sub1/NewSub/Myfile.txt' (length=49)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:124:string 'working in remote directory: /home/conn/test' (length=44)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:125:string 'upload to remote: Folder1/Sub1/NewSub/Myfile.txt, from: C:\Ampps\www\duckfusioncore\app\lotto\tmp\tmp-XJYdx8eduL' (length=112)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:131:string 'calling ftp_chdir() to : Folder1/Sub1/NewSub' (length=44)
C:\Ampps\www\duckfusioncore\sys\lib\helpers\datastore\stores\FTPStore.php:135:string 'couldn't change to Folder1/Sub1/NewSub' (length=38)
C:\Ampps\www\duckfusioncore\app\lotto\controllers\IndexController.php:28:boolean false
The directory 100% exists. And it has the same 0777 permissions as the other ones.
I'm just very confused as to why it is unable to switch to the directory, when it defintely exists and it has the same permissions as the other directories, which it had no problem switching to before...
Has anyone got any clue what might be the problem? Is there any extra debugging I can do on the ftp_chdir() function call to see if it gives me a reason?
Thanks.
I would like to check that a file uploaded to my OpenShift app has a text extension (.txt or .tab). Following some advice given here I wrote the following code, with echoes added to help debug:
$AllowedExts = array('txt','tab');
echo "AllowedExts: " . $AllowedExts[0] . " and " . $AllowedExts[1] . "<br>";
$ThisPath = $_FILES['uploadedfile']['tmp_name'];
echo "ThisPath: " . $ThisPath . "<br>";
$ThisExt = pathinfo($ThisPath, PATHINFO_EXTENSION);
echo "ThisExt: " . $ThisExt . "<br>";
if(!in_array($ThisExt,$AllowedExts) ) {
$error = 'Uploaded file must end in .txt or .tab';
}
echo "error echo: " . $error . "<br>";
On uploading any file, the echoed response was:
AllowedExts: txt and tab
ThisPath: /var/lib/openshift/************/php/tmp/phpSmk2Ew
ThisExt:
error echo: Uploaded file must end in .txt or .tab
Does this mean that OpenShift is renaming the file upon upload? How do I get the original filename and then check its suffix? More generally, is there a better way to check the file type?
$_FILES['uploadedfile']['tmp_name'] contains the name of a temporary file on the server (which can be moved with move_uploaded_file()). If you want to check the original name of the uploaded file on the client machine use $_FILES['uploadedfile']['name'].
That's not an Open Shift issue, it's the standard way of PHP.
For further details see http://php.net/manual/en/features.file-upload.post-method.php
For other ways to detect the file type see http://php.net/manual/en/ref.fileinfo.php