Obtaining files via ssh2_scp_recv in php - php

For a project i need to retreive all the files on a sftp server with a total size that amounts to about ~100 mb among all the files. I developed a script thats uses the ssh2 library and fread to get all the files, but it takes arround 10 min to get all the content. I tried ssh2_scp_recv, hoping for better performance, as shown in code bellow, but the script runs rather fast, not obtaining any file, as if the mentioned function never ran.
function get_sftp_content(){
$url = 'someurl.com';
$port = 22;
$username = "user";
$password = "password";
// Make our connection
$connection = ssh2_connect($url);
// Authenticate
if (!ssh2_auth_password($connection, $username, $password)) throw new Exception('Unable to connect.');
// Create our SFTP resource
if (!$sftp = ssh2_sftp($connection)) throw new Exception('Unable to create SFTP connection.');
$localDir = '/ftp_files';
$remoteDir = '/download';
// download all the files
$files = scandir('ssh2.sftp://' . $sftp . $remoteDir);
//echo '<pre>' . print_r($files, true) . '</pre>' ;
if (!empty($files)) {
foreach ($files as $file) {
ssh2_scp_recv($connection,$remoteDir.'/'.$file, $localDir.'/'.$file);
}
}
}
get_sftp_content();
Any ideas on why this happens, and is there a faster alternative to get all the files, without recurring to phpseclib?

Related

how to implement pagination while fetching data from file server in php

I wrote a php script which fetches the data from file server,it lists all the uploaded file on server.
Now problem is there is large no. of file on my server ,i want limited results per page how can i implement pagination for this.
My php code is :
$ftp_server = "ftp.abc.com";
$ftp_user = "uploads#abc.com";
$ftp_password = "xyz";
$conn = ftp_connect($ftp_server) or die("could not connect to ftp server");
if (ftp_login($conn, $ftp_user, $ftp_password)) {
$contents = ftp_nlist($conn, $path);
foreach($contents as $value) {
$i++;
echo "$i <a href='#form' id='url_title' class='links'>$value</a><br><br> <br>";
}
}

PHP : ssh2 : How to connect to a second ssh server

I have to connect to a server via ssh, but to access it I need to first connect to another ssh server. I use standard password access to them.
So my steps are:
ssh root#serverdomain1.com
then when connected in serverdomain1 I do in terminal:
ssh myuseraccount#serverdomain2.com
in php, I tried to use ssh2_exec('ssh serverdomain2.com'); but no results. Then I tried also ss2_tunnel($connection, ...). but nothing worked.
This doesn't work:
$ssh = ssh2_connect('serverdomain1.com', 22);
if (ssh2_auth_password($ssh, $user,$pass)) {
$stream = ssh_exec($ssh, "ssh serverdomain2.com");
stream_set_blocking($stream, true);
$stream_out = ssh2_fetch_stream($stream, SSH2_STREAM_STDIO);
echo stream_get_contents($stream_out); // <== doesn't work!!!
}
This also doesn't work:
$ssh = ssh2_connect('serverdomain1.com', 22);
if (ssh2_auth_password($ssh, $user,$pass)) {
$tunnel = ssh2_tunnel($ssh, 'serverdomain2.com', 22);
if (!$tunnel) {
echo('no tunnel<br/>');
}
else {
fwrite($tunnel, "echo 1\n");
while (!feof($tunnel)) {
echo fgets($tunnel, 128);
}
}
}
The echo result for tunnel:
"SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.4 Protocol mismatch."
How can I do that with SSH2 from PHP?
I recently published a project that allows PHP to obtain and interact with a real Bash shell, through SSH if needed. Get it here: https://github.com/merlinthemagic/MTS
The project lets you keep bouncing from server to server using ssh.
After downloading you would simply use the following code:
//first you get a shell on the first server:
$shellObj = \MTS\Factories::getDevices()->getRemoteHost('ip_address1')->setConnectionDetail('username1', 'password1')->getShell();
//then build on that first shell, the following way.
\MTS\Factories::getDevices()->getRemoteHost('ip_address2')->setConnectionDetail('username2', 'password2')->getShell($shellObj);
//any command executed on the shell will run only on the second host you connected to.
$return1 = $shellObj->exeCmd("hostname");
echo $return1;//hostname of the second host you connected to
//
Make sure that RSSH, PECL, SSH2 libraries installed on your server
You can check this using phpinfo
Here is my working code to access the server using ssh2. Hope it will help!
<?php
$host = 'SERVER_HOST_ADDR';
$port = SERVER_PORT;
$username = 'SERVER_USERNAME';
$password = 'SERVER_PASSWORD';
$remoteDir = './home/'; //DIR_PATH
$localDir = '/var/www/html/'; //LOCAL_DIR_PATH
// Make our connection
$connection = ssh2_connect($host);
// Authenticate
if (!ssh2_auth_password($connection, $username, $password)) {
throw new Exception('Unable to connect.');
}
// Create our SFTP resource
if (!$sftp = ssh2_sftp($connection)) {
throw new Exception('Unable to create SFTP connection.');
}
/**
* Now that we have our SFTP resource, we can open a directory resource
* to get us a list of files. Here we will use the $sftp resource in
* our address string as I previously mentioned since our ssh2://
* protocol allows it.
*/
$files = array();
$dirHandle = opendir("ssh2.sftp://$sftp/$remoteDir");
// Properly scan through the directory for files, ignoring directory indexes (. & ..)
while (false !== ($file = readdir($dirHandle))) {
if ($file != '.' && $file != '..') {
$files[] = $file;
}
}
echo "<pre>";print_r($files);
?>

I am creating PHP page in which i am downloading .CSV file from SFTP server

I want to download file from SFTP so i created page which looks like it.
<?php
$username = "XYZ";
$password = "ABC";
$url ='FTP.abc.COM';
// Make our connection
$connection = ssh2_connect($url);
// Authenticate
if (!ssh2_auth_password($connection, $username, $password)) throw new Exception('Unable to connect.');
// Create our SFTP resource
if (!$sftp = ssh2_sftp($connection)) throw new Exception('Unable to create SFTP connection.');
$localDir = '/path/to/your/local/dir';
$remoteDir = '/path/to/your/remote/dir';
// download all the files
$files = scandir('ssh2.sftp://' . $sftp . $remoteDir);
if (!empty($files))
{
foreach ($files as $file) {
if ($file != '.' && $file != '..')
{
ssh2_scp_recv($connection, "$remoteDir/$file", "$localDir/$file");
}
}
}
?>
When i will call this page from browser. It will show error look like it.
Fatal error: Call to undefined function ssh2_connect() in page.php on line 6
if this method is correct then what should i edit in this page?
and if there is another method then suggest me that method. Thanks in advance.
You'll probably have better success with phpseclib, a pure PHP SFTP implementation. eg.
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
// outputs the contents of filename.remote to the screen
echo $sftp->get('filename.remote');
// copies filename.remote to filename.local from the SFTP server
$sftp->get('filename.remote', 'filename.local');
?>
It has a number of advantages over libssh2:
http://phpseclib.sourceforge.net/ssh/compare.html
SSH2 functions are not standard functions in PHP. You have to install them first. See http://php.net/manual/en/ref.ssh2.php for more details.

FTP Files are not accessible using fopen or file_exists or file_get_contents

I am trying ftp connections for file read.
All I want to do is
read particular ftp dir
list all its text files
and read them one by one
I am successful in listing files in given directory.
For eg: below code returns $emailfiles array with TEST.txt as one value.
But when I try to open/read the same file content using fopen/file_get_contents , it is giving the error: " Failed to open Stream "
I am able to access a file from another ftp : .38, which is public ftp.
The ftp I want to actually use is .94, for which I have full privileges, as I am able to access files from browser with same link. I'm also able to write/paste files from windows explorer.
But I am unable to read the same file from php code.
Below is my code for ftp connection and testing.
$ftp_server = "10.96.5.94";
$ftp_serverpath = "ftp://".$ftp_server;
$ftp_user_name = "myusername";
$ftp_user_pass = "mypassword";
$email_dir = "ebill";
try {
$con = ftp_connect($ftp_server);
if (false === $con) {
throw new Exception('Unable to connect');
}
$loggedIn = ftp_login($con, $ftp_user_name, $ftp_user_pass);
if (true === $loggedIn) {
echo 'Success!';
} else {
throw new Exception('Unable to log in');
}
ftp_pasv($con, true);
$emailfiles = ftp_nlist($con, $email_dir);
print_r($emailfiles);
//$filename = "ftp://10.16.0.38/ebill/TEST.txt";
$filename = "ftp://10.96.5.94/ebill/TEST.txt";
$pr = file_exists($filename);
// OR
$handle = fopen($filename, "r");
$contents = fread($handle, filesize($filename));
Any help is greatly appreciated, Thank you.
Ok, This solved my error.
$filename = "ftp://username:pa‌​ssword#hostname/path/to/file";
Thanks.

Upload files to SFTP server via PHP (phpseclib)

I have CSVs that I want to upload to the 'incoming' folder in the SFTP server. I am using phpseclib to do this. The connection is already there but it does not output anything.
I'm not sure if what I did was correct since I haven't dealt with SFTP before. Here's what my code looks like:
$file = "leads.csv";
$server = "41.160.150.200";
//$server = "ft.bayport.co.za";
$port = "22";
$username = "";
$password = "";
//username and password removed for security reasons
set_include_path(get_include_path() . PATH_SEPARATOR . 'phpseclib');
include 'Net/SFTP.php';
define('NET_SFTP_LOGGING', NET_SFTP_LOG_COMPLEX); // or NET_SFTP_LOG_SIMPLE
$sftp = new Net_SFTP($server);
// Check SFTP Connection
if (!$sftp->login($username, $password)) {
echo 'Login Failed.';
echo $sftp->getSFTPLog();
}else{
echo 'Connected to SFTP.';
echo $sftp->pwd();
// Upload CSVs to SFTP incoming folder
echo $upload = $sftp->put("incoming/".$file, "./bayport/".$file, NET_SFTP_LOCAL_FILE);
}
I would really appreciate any help. Thanks!
Now I know what's the problem with the script.
The remote directory URL was wrong. "incoming/" should be "/incoming/"

Categories