I'm having trouble with ssh2_sftp_rename, or in general with the SSH2/SFTP wrapper.
We have two SFTP servers. One is a test server, which we've setup by ourself, and one is from a service provider. On the test server, everything works fine while on the provided SFTP we cannot rename a file via PHP. Manual renaming via sftp cli works fine. Our SFTP server uses the protocol version 2.0 and remote software version OpenSSH_6.0p1 Debian-4, while the other one uses protocol version 2.0, remote software version OpenSSH_5.3. So I'd guess they should act the same.
Here's the smallest code snippet for testing, which doesn't work:
<?php
$connection = ssh2_connect('sftp.serviceprovider.com', 22);
ssh2_auth_password($connection, 'username', 'password');
$sftp = ssh2_sftp($connection);
$result = ssh2_sftp_rename($sftp, '/new/foo.xml.gz', '/processed/foo.xml.gz');
var_dump($result);
var_dump(error_get_last());
$result is false and error_get_last() returns NULL. I also tried leaving the beginning slashes or any other combination I can think of, but it just "didn't work": the file is still in the 'new' folder.
So, unfortunately I cannot take a look into the log files of the service provider but I have requested an log excerpt or at least an analysis. But I don't want to wait for their support center to answer my question and try to debug this stuff.
Is there any possibility to debug what ssh2_sftp_rename or renameare doing behind the scenes? Does anyone else experienced something like this and found a solution?
Thanks for any hints,
Matthias
Diagnosing issues with libssh2 is nigh impossible. My recommendation: use phpseclib, a pure PHP SFTP implementation. eg.
<?php
include('Net/SFTP.php');
include('Crypt/RSA.php');
define('NET_SFTP_LOGGING', NET_SFTP_LOG_COMPLEX);
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
$sftp->rename('filename.remote', 'newname.remote');
echo $sftp->getSFTPLog();
?>
In lieu of seeing the logs my guess would be a permissions issue but the logs should tell us for sure.
Related
I have to connect to a FTPES server to retrieve data. Connecting and logging in works just fine, but my call with ftp_rawlist always fails and returns "false".
I am using this code for debugging purposes:
$ftp = ftp_ssl_connect($ftp_host);
if (ftp_login($ftp, $ftp_user, $ftp_pass)) {
$p = ftp_pasv($ftp, true);
var_dump($p);
$r = ftp_rawlist($ftp, '/', true);
var_dump($r);
} else {
echo 'Could not login';
}
$p is always true, $r always false.
When I connect to the server through Filezilla everything works fine and I can list directory content and more.
Update #1: Tried to not only list '/' but various subfolders on the server, they all fail through the script.
Update #2: Also tried to use ftp_raw with the commands to get a list, but the LIST command runs for some time and then does not return any result at all. But HELP lists LIST as a valid command for the server... Strange...
Update #3: I tried phpseclib now, but while I can connect, I can't login with the user/password combination. Support from the maintainer of the FTPES server is not happening ("works fine for $somebody else..."), so I need to figure this out another way... :-)
To come to an end with this: As the deadline for this project came closer a solution had to be found. And although this is no real answer in the sense of a question, I'd like to show what I have done to have this fixed. Maybe someone stumbles upon this through googling.
Next to the things mentioned in the OP, I also tried connecting to FTPS using PHP and certificate as auth which didn't work either. As nothing works as it is supposed to, I wonder if the FTPS server is really configured correctly after all.
The people who run the server told me that everything is fine and their CLI CURL-call works fine for them, so they have no need to further investigate issues.
As a result of this I set up a sandbox account on a server which has shell_exec() enabled. There is now a script running which gets a file listing via CURL and then downloads the files via CURL with the commands provided by the server provider. That server can be accessed through normal SFTP and therefore acts as a "Proxy FTP" which regularly mirrors the remote FTPS server file structure.
Although I find this "solution" quite "hacky" it seems to run robust, stable and fast for the moment. We will therefore be able to have the operation running this way in this year (it only runs around three months before christmas) and will have a look into it in the new year and develop a more stable solution.
Maybe the server guys are also less stressed then and willing to help... ;-)
Add the following call to ftp_set_option() in a line before the call to ftp_pasv
ftp_set_option($ftp, FTP_USEPASVADDRESS, false);
ftp_pasv($ftp, true);
Before I begin, I'm aware of phpseclib and from what I've ready it's the way you should normally go. Unfortunately I'm working on an older script, and I'm trying to avoid the time needed to set up phpseclib, test on local environment, put on production, test on production, refactor my script, etc. I'm hoping someone can help resolve my issue with ssh2.
I've used the following script to ssh a file to a client's server. It has worked fine for years.
<?php
$url = 'their.url.com';
$userName = 'name';
$password = 'pass';
$conn = ssh2_connect($url, 22);
$auth = ssh2_auth_password($conn, $userName, $password);
// Determine whether this is a file path that needs to be opened or not
$localFilePath = 'test123.txt';
$openFile = fopen($localFilePath, "r");
$fileName = 'test.txt';
// Their server path
$remoteFilePath = "/Orders/".$fileName;
if ($auth === true) {
// Transfer file
$sftp = ssh2_sftp($conn);
file_put_contents("ssh2.sftp://".$sftp.$remoteFilePath, $openFile);
}
WHAT HAPPENED:
My client moved his stuff to a different server. It broke the connection, he freaked out, etc., etc. He gave me updated credentials to the new server, and I have confirmed the protocal is SFTP - SSH File Transfer Protocal.
THE ISSUE:
The file transfer is no longer working after updating the ssh combo url/credentials.
WHAT I'VE TRIED:
I tried a test file transfer using Linux (not PHP) and it worked. I also ssh'd in using Filezilla and transferred a file that way, no problem. so I have confirmed that the host/user/pass combo is correct.
I dumped out the result of the file_put_contents, and it is false.
I dumped out $conn, $auth, and $sftp, and what "ssh2.sftp://".$sftp.$remoteFilePath looks like, and the results are (in order)
resource(27) of type (SSH2 Session)
bool(true)
resource(30) of type (SSH2 SFTP)
ssh2.sftp://Resource id #30/Orders/test.txt
Note that test.txt is a very tiny file to rule out file size issues.
I have also uninstalled and reinstalled the ssh2 PECL extension.
I am convinced this is due to an issue on their end, because this script has always worked before, I'm just baffled as to why things are failing now that they're on their new server, and I'm not sure what to do further to diagnose the problem. I can't think of any change that's been made to my server that would affect this.
Any ideas would be greatly appreciated.
Why not just use the ssh2.sftp:// stream directly?
file_put_contents("ssh2.sftp://{$userName}:{$password}#{$url}:22/{$remoteFilePath}", $openFile);
It should work without all the ssh connection, authentication, moving to sftp, etc..
I am currently struggling with using the SSH2 built-in libraries for PHP (running version 5.5). I am trying to upload a file to an SFTP server as the title states however I keep getting a "stream operation failed" error message.
After attempting to debug the code itself the connection works, the sftp resource is assigned an ID correctly, however when fopen is called for writing the file directly to the remote server it fails.
// open Live environment if we are not in dev
$connection = ssh2_connect($this->_settings['source_host'], 22);
$authSuccess = ssh2_auth_password($connection, $this- >_settings['source_user'], $this->_settings['source_password']);
$sftp = ssh2_sftp($connection);
And finally the fopen() call:
if($operation == 'export') {
$handle = fopen("ssh2.sftp://".$sftp."/remotecopy/IN/".$filename, $mode);
}
I added debug messages in my own code to verify if the data from the _settings array is also used correctly and it is, however I can't explain the stream error.
Message: fopen(): Unable to open ssh2.sftp://Resource id #173/PATH GOES HERE/filename.xxx on remote host
Message: fopen(ssh2.sftp://Resource id #173/PATH GOES HERE/filename.xxx): failed to open stream: operation failed
As a note the file does not exist on the remote host but according to my knowledge 'w' mode in PHP fopen() should create the file if it does not exist.
I can't use the other PHP library as our whole project uses the builtin ssh2 libraries and the person in charged told me to not use it as it works fine everywhere else.
i think you'd have an easier time if you used phpseclib, a pure PHP SFTP implementation. eg.
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
// puts a three-byte file named filename.remote on the SFTP server
$sftp->put('filename.remote', 'xxx');
// puts an x-byte file named filename.remote on the SFTP server,
// where x is the size of filename.local
$sftp->put('filename.remote', 'filename.local', NET_SFTP_LOCAL_FILE);
?>
One of the nice things about phpseclib is it's logging so if that doesn't work you can do define('NET_SSH2_LOGGING', NET_SSH2_LOG_COMPLEX); after including Net/SFTP.php and then do echo $sftp->getLog() after the point where it fails. That might provide some insight into what's going on if it still isn't working.
The answer was easy, I had incorrectly formatted path on the remote server. After verifying my settings it works just fine.
Thank you all for the hints and help.
I am trying to do a ssh connection in php. But I get this error Unable to startup sftp subsystem. I don't understand it. And I could not find much in google. Please let me know how to debug this. I am using wamp.
$resConnection = ssh2_connect($strServer, $strServerPort);
if(ssh2_auth_password($resConnection, $strServerUsername, $strServerPassword)){
//Initialize SFTP subsystem
$resSFTP = ssh2_sftp($resConnection); -->error coming here
}else{
echo "Unable to authenticate on server";
}
It's hard to say without more information. Unfortunately, libssh2 is pretty limited in the amount of info it provides. I'd recommend you use phpseclib, a pure PHP SFTP implementation. eg.
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
print_r($sftp->nlist());
?>
Put define('NET_SSH2_LOGGING', 3) at the top to get logs if you still encounter the problem, after switching. Then post the logs so that you can get better assistance.
Thanks!
I had the same issue and I solved it with following steps.
1) First I noticed that there is slowliness on server to which we are trying to make sftp connection.
2) There is a script in our side which try to connect and get files from server so because of slowliness issue our scripts can't complete their job and wait for connection because it was very slow. I checked how many processes created for sftp connection with following command
ps aux | grep sftp
3) Then I killed all these processes with following command
pkill sftp
4) After I kill that processes error was gone.
Summary
Main reason of this issue was that. There was slowliness issue in ftp server and we created too many connection. I think there is some limit on server side for max connections.
I have been told this cannot be done but I want to get some other opinions here. I am a bit of a newbie when it comes to things like this.
My Site: ExampleSiteA.com
File to download: ExampleSiteB.com
Basically, I am downloading a csv file from ExampleSiteB.com to make updates to my site, ExampleSiteA.com. To do this, I am downloading the csv file manually through CoreFTP and then uploading it manually to ExampleSiteA.com. The file changes daily and I would like to skip this step so I can automate the process.
Keep in mind that I need to download the csv file from ExampleSiteB.com through SFTP...
I am not sure if it is possible to directly download/upload a file from one server to another if one is SFTP. The file size is also quite large, it averages about 25,000 KB / 25 MB.
Another option that I haven't explored yet is requiring or including a file from another server... is that an option or a possibility? The file is located in a folder exclusively for my site and a login is required for SFTP download.
Any insight will be appreciated. Thanks in advance!
Go here and download what you need: http://phpseclib.sourceforge.net/
UPDATE
FOR SFTP
Then in your script:
<?php
include('Net/SFTP.php');
$url = 'http://www.downloadsite.com';
$fileToDownload = "yourCSV.csv";
$cmd = "wget -q \"$url\" -O $fileToDownload";
exec($cmd);
$sftp = new Net_SFTP('www.uploadsite.com');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
echo $sftp->pwd() . "\r\n";
$sftp->put('remote.file.csv', 'yourCSV.csv', NET_SFTP_LOCAL_FILE);
print_r($sftp->nlist());
?>
If you need to connect to a second server for download:
$sftp2 = new Net_SFTP('www.serverFromWhichToDownload.com');
if (!$sftp2->login('username', 'password')) {
exit('Login Failed');
}
echo $sftp2->pwd() . "\r\n";
$sftp2->get('localFileName.csv', 'remoteFileName.csv');
print_r($sftp2->nlist());
Read the docs for further help and examples: http://phpseclib.sourceforge.net/documentation/net.html#net_sftp_get
To Log what your connection is doing if it fails, etc. use this:
include('Net/SSH2.php');
define('NET_SSH2_LOGGING', true);
$ssh = new Net_SSH2('www.domain.tld');
$ssh->login('username','password');
echo $ssh->getLog();
FOR FTP upload - SO has gone crazy, does not want to format my code, but here it is anyway:
$file = 'somefile.txt';
$remote_file = 'readme.txt';
$conn_id = ftp_connect($ftp_server);
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
if (ftp_put($conn_id, $remote_file, $file, FTP_ASCII)) {
echo "successfully uploaded $file\n";
} else {
echo "There was a problem while uploading $file\n";
}
ftp_close($conn_id);
Yes, that's possible using ssh2_sftp.
http://php.net/manual/en/function.ssh2-sftp.php
I have had good luck with cURL in the past. If you are on a Linux box, it would be trivial to set up a CRON job to do this update process for you. A good reference for CLI HTTP scripting in cURL can be found here, however you may need the -T flag (for file transport) to accomplish the upload portion. Speaking of uploading, if you can run the script/process/crontab from the server you would like to update, I would recommend downloading from the web server to obviate one trip and a third party. Or, if you need to update on demand, you could write a PHP script that uses the built in PHP cURL functions. If you take the Linux+CLI route, you could also use sftp.
Update: In testing curl with sftp (curl -u uname:pword sftp://domain.tld) I get the following error: curl: (1) Protocol sftp not supported or disabled in libcurl on Kubuntu 12.04. So cURL may not be a good idea. I also tested CLI sftp (sftp uname#domain.tld:/dir/file.ext) but could not find a way (short of using ssh keys) to send authentication. Thus, this would necessarily be a manual process unless you did set up ssh keys between the servers. As it does not sound like you have that kind of access to ExampleSiteB.com, this probably isn't acceptable.
Update 2: Since my initial answer turned out to be of little use, I figured I would expand upon one of the above answers. I was trying to find a solution that did not involve a PECL extension, but I did not have much luck with ftp_ssh_connect(). I recommend trying it, you may have better luck and could forgo the PECL extension route.
Sigh, on further reading, it appears ftp_ssh_connect is, understandably, incompatible with the sftp protocol. However, I found a nice blog post about utilizing ssh2_connect() and ssh2_sftp() (as mentioned in a previous answer) and figured I would post that to give you some additional assistance. It is not as simple as calling the functions for most PHP distributions. Here is the blog post. Some of those steps may not be necessary or you may need to do some additional things listed in another blog post I ran across, here.
On my system, all I had to do was run apt-get install libssh2-1-dev libssh2-php and I was able to find ssh2 in my php -m output.
Having an include, as long as you have read/write permissions on the website you're getting the file from should work, however this is just guess work atm as i don't have any means of checking it. Good luck though!
Yes, you should be able to do this.
Whoever told you that you can't do this might be getting confused with JavaScript and cross-site scripting browser restrictions which prevent JavaScript downloaded from one domain to access content in a different domain.
That being said, if you are using PHP which to me implies that you are talking about PHP running on a web sever, you should be able to use PHP or any other scripting or programming language to download the file from SiteB.com, then update the file, and then finally FTP the file to a different web server (SiteA.com).