Before I begin, I'm aware of phpseclib and from what I've ready it's the way you should normally go. Unfortunately I'm working on an older script, and I'm trying to avoid the time needed to set up phpseclib, test on local environment, put on production, test on production, refactor my script, etc. I'm hoping someone can help resolve my issue with ssh2.
I've used the following script to ssh a file to a client's server. It has worked fine for years.
<?php
$url = 'their.url.com';
$userName = 'name';
$password = 'pass';
$conn = ssh2_connect($url, 22);
$auth = ssh2_auth_password($conn, $userName, $password);
// Determine whether this is a file path that needs to be opened or not
$localFilePath = 'test123.txt';
$openFile = fopen($localFilePath, "r");
$fileName = 'test.txt';
// Their server path
$remoteFilePath = "/Orders/".$fileName;
if ($auth === true) {
// Transfer file
$sftp = ssh2_sftp($conn);
file_put_contents("ssh2.sftp://".$sftp.$remoteFilePath, $openFile);
}
WHAT HAPPENED:
My client moved his stuff to a different server. It broke the connection, he freaked out, etc., etc. He gave me updated credentials to the new server, and I have confirmed the protocal is SFTP - SSH File Transfer Protocal.
THE ISSUE:
The file transfer is no longer working after updating the ssh combo url/credentials.
WHAT I'VE TRIED:
I tried a test file transfer using Linux (not PHP) and it worked. I also ssh'd in using Filezilla and transferred a file that way, no problem. so I have confirmed that the host/user/pass combo is correct.
I dumped out the result of the file_put_contents, and it is false.
I dumped out $conn, $auth, and $sftp, and what "ssh2.sftp://".$sftp.$remoteFilePath looks like, and the results are (in order)
resource(27) of type (SSH2 Session)
bool(true)
resource(30) of type (SSH2 SFTP)
ssh2.sftp://Resource id #30/Orders/test.txt
Note that test.txt is a very tiny file to rule out file size issues.
I have also uninstalled and reinstalled the ssh2 PECL extension.
I am convinced this is due to an issue on their end, because this script has always worked before, I'm just baffled as to why things are failing now that they're on their new server, and I'm not sure what to do further to diagnose the problem. I can't think of any change that's been made to my server that would affect this.
Any ideas would be greatly appreciated.
Why not just use the ssh2.sftp:// stream directly?
file_put_contents("ssh2.sftp://{$userName}:{$password}#{$url}:22/{$remoteFilePath}", $openFile);
It should work without all the ssh connection, authentication, moving to sftp, etc..
Related
I have to connect to a FTPES server to retrieve data. Connecting and logging in works just fine, but my call with ftp_rawlist always fails and returns "false".
I am using this code for debugging purposes:
$ftp = ftp_ssl_connect($ftp_host);
if (ftp_login($ftp, $ftp_user, $ftp_pass)) {
$p = ftp_pasv($ftp, true);
var_dump($p);
$r = ftp_rawlist($ftp, '/', true);
var_dump($r);
} else {
echo 'Could not login';
}
$p is always true, $r always false.
When I connect to the server through Filezilla everything works fine and I can list directory content and more.
Update #1: Tried to not only list '/' but various subfolders on the server, they all fail through the script.
Update #2: Also tried to use ftp_raw with the commands to get a list, but the LIST command runs for some time and then does not return any result at all. But HELP lists LIST as a valid command for the server... Strange...
Update #3: I tried phpseclib now, but while I can connect, I can't login with the user/password combination. Support from the maintainer of the FTPES server is not happening ("works fine for $somebody else..."), so I need to figure this out another way... :-)
To come to an end with this: As the deadline for this project came closer a solution had to be found. And although this is no real answer in the sense of a question, I'd like to show what I have done to have this fixed. Maybe someone stumbles upon this through googling.
Next to the things mentioned in the OP, I also tried connecting to FTPS using PHP and certificate as auth which didn't work either. As nothing works as it is supposed to, I wonder if the FTPS server is really configured correctly after all.
The people who run the server told me that everything is fine and their CLI CURL-call works fine for them, so they have no need to further investigate issues.
As a result of this I set up a sandbox account on a server which has shell_exec() enabled. There is now a script running which gets a file listing via CURL and then downloads the files via CURL with the commands provided by the server provider. That server can be accessed through normal SFTP and therefore acts as a "Proxy FTP" which regularly mirrors the remote FTPS server file structure.
Although I find this "solution" quite "hacky" it seems to run robust, stable and fast for the moment. We will therefore be able to have the operation running this way in this year (it only runs around three months before christmas) and will have a look into it in the new year and develop a more stable solution.
Maybe the server guys are also less stressed then and willing to help... ;-)
Add the following call to ftp_set_option() in a line before the call to ftp_pasv
ftp_set_option($ftp, FTP_USEPASVADDRESS, false);
ftp_pasv($ftp, true);
I'm having trouble with ssh2_sftp_rename, or in general with the SSH2/SFTP wrapper.
We have two SFTP servers. One is a test server, which we've setup by ourself, and one is from a service provider. On the test server, everything works fine while on the provided SFTP we cannot rename a file via PHP. Manual renaming via sftp cli works fine. Our SFTP server uses the protocol version 2.0 and remote software version OpenSSH_6.0p1 Debian-4, while the other one uses protocol version 2.0, remote software version OpenSSH_5.3. So I'd guess they should act the same.
Here's the smallest code snippet for testing, which doesn't work:
<?php
$connection = ssh2_connect('sftp.serviceprovider.com', 22);
ssh2_auth_password($connection, 'username', 'password');
$sftp = ssh2_sftp($connection);
$result = ssh2_sftp_rename($sftp, '/new/foo.xml.gz', '/processed/foo.xml.gz');
var_dump($result);
var_dump(error_get_last());
$result is false and error_get_last() returns NULL. I also tried leaving the beginning slashes or any other combination I can think of, but it just "didn't work": the file is still in the 'new' folder.
So, unfortunately I cannot take a look into the log files of the service provider but I have requested an log excerpt or at least an analysis. But I don't want to wait for their support center to answer my question and try to debug this stuff.
Is there any possibility to debug what ssh2_sftp_rename or renameare doing behind the scenes? Does anyone else experienced something like this and found a solution?
Thanks for any hints,
Matthias
Diagnosing issues with libssh2 is nigh impossible. My recommendation: use phpseclib, a pure PHP SFTP implementation. eg.
<?php
include('Net/SFTP.php');
include('Crypt/RSA.php');
define('NET_SFTP_LOGGING', NET_SFTP_LOG_COMPLEX);
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
$sftp->rename('filename.remote', 'newname.remote');
echo $sftp->getSFTPLog();
?>
In lieu of seeing the logs my guess would be a permissions issue but the logs should tell us for sure.
I'm making a utility that provides a GUI to easy edit certain values in a csv file on a remote server. My boss wants the utility in php running on the private webserver. I'm new to php, but I was able to get the GUI file modifier working locally without issues. The final piece now is rather than the local test file I need to grab a copy of the requested file off of the remote server, edit it, and then replace the old file with the edited one. My issue is uploading and downloading the file.
When I searched for a solution I found the following:
(note in each of these I am just trying to move a test file)
$source = "http://<IP REMOTE SERVER>/index.html";
$dest = $_SERVER['DOCUMENT_ROOT']."index.html";
copy($source, $dest);
This solution ran into a permissions error.
$source ="http://<IP REMOTE SERVER>/index.html";
$destination = $_SERVER['DOCUMENT_ROOT']."newfile.html";
$data = file_get_contents($source);
$handle = fopen($destination, "w");
fwrite($handle, $data);
fclose($handle);
This also had a permissions error
$connection = ssh2_connect('<IP REMOTE SERVER>', 22);
ssh2_auth_password($connection, 'cahenk', '<PASSWORD>');
ssh2_scp_recv($connection, '/tmp/CHenk/CHenk.csv', 'Desktop/CHenk.csv');
This solution has the error Fatal error: Call to undefined function ssh2_connect() which I have learned is because the function is not a part of the default php installation.
In closing, is there any easy way to read/write files to the remote server through php either by changing permissions, having the php extension installed, or a different way entirely that will work. Basically I'm trying to find the solution that requires the least settings changes to the server because I am not the administrator and would have to go through a round about process of getting any changes done. If something does need to be changed instructions on doing so or a link to instructions would be greatly appreciated.
Did you set the enable-url-fopen-wrapper in your php.ini?(only if your php version is older)
Please look # php remote files storing in example 2
I have been told this cannot be done but I want to get some other opinions here. I am a bit of a newbie when it comes to things like this.
My Site: ExampleSiteA.com
File to download: ExampleSiteB.com
Basically, I am downloading a csv file from ExampleSiteB.com to make updates to my site, ExampleSiteA.com. To do this, I am downloading the csv file manually through CoreFTP and then uploading it manually to ExampleSiteA.com. The file changes daily and I would like to skip this step so I can automate the process.
Keep in mind that I need to download the csv file from ExampleSiteB.com through SFTP...
I am not sure if it is possible to directly download/upload a file from one server to another if one is SFTP. The file size is also quite large, it averages about 25,000 KB / 25 MB.
Another option that I haven't explored yet is requiring or including a file from another server... is that an option or a possibility? The file is located in a folder exclusively for my site and a login is required for SFTP download.
Any insight will be appreciated. Thanks in advance!
Go here and download what you need: http://phpseclib.sourceforge.net/
UPDATE
FOR SFTP
Then in your script:
<?php
include('Net/SFTP.php');
$url = 'http://www.downloadsite.com';
$fileToDownload = "yourCSV.csv";
$cmd = "wget -q \"$url\" -O $fileToDownload";
exec($cmd);
$sftp = new Net_SFTP('www.uploadsite.com');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
echo $sftp->pwd() . "\r\n";
$sftp->put('remote.file.csv', 'yourCSV.csv', NET_SFTP_LOCAL_FILE);
print_r($sftp->nlist());
?>
If you need to connect to a second server for download:
$sftp2 = new Net_SFTP('www.serverFromWhichToDownload.com');
if (!$sftp2->login('username', 'password')) {
exit('Login Failed');
}
echo $sftp2->pwd() . "\r\n";
$sftp2->get('localFileName.csv', 'remoteFileName.csv');
print_r($sftp2->nlist());
Read the docs for further help and examples: http://phpseclib.sourceforge.net/documentation/net.html#net_sftp_get
To Log what your connection is doing if it fails, etc. use this:
include('Net/SSH2.php');
define('NET_SSH2_LOGGING', true);
$ssh = new Net_SSH2('www.domain.tld');
$ssh->login('username','password');
echo $ssh->getLog();
FOR FTP upload - SO has gone crazy, does not want to format my code, but here it is anyway:
$file = 'somefile.txt';
$remote_file = 'readme.txt';
$conn_id = ftp_connect($ftp_server);
$login_result = ftp_login($conn_id, $ftp_user_name, $ftp_user_pass);
if (ftp_put($conn_id, $remote_file, $file, FTP_ASCII)) {
echo "successfully uploaded $file\n";
} else {
echo "There was a problem while uploading $file\n";
}
ftp_close($conn_id);
Yes, that's possible using ssh2_sftp.
http://php.net/manual/en/function.ssh2-sftp.php
I have had good luck with cURL in the past. If you are on a Linux box, it would be trivial to set up a CRON job to do this update process for you. A good reference for CLI HTTP scripting in cURL can be found here, however you may need the -T flag (for file transport) to accomplish the upload portion. Speaking of uploading, if you can run the script/process/crontab from the server you would like to update, I would recommend downloading from the web server to obviate one trip and a third party. Or, if you need to update on demand, you could write a PHP script that uses the built in PHP cURL functions. If you take the Linux+CLI route, you could also use sftp.
Update: In testing curl with sftp (curl -u uname:pword sftp://domain.tld) I get the following error: curl: (1) Protocol sftp not supported or disabled in libcurl on Kubuntu 12.04. So cURL may not be a good idea. I also tested CLI sftp (sftp uname#domain.tld:/dir/file.ext) but could not find a way (short of using ssh keys) to send authentication. Thus, this would necessarily be a manual process unless you did set up ssh keys between the servers. As it does not sound like you have that kind of access to ExampleSiteB.com, this probably isn't acceptable.
Update 2: Since my initial answer turned out to be of little use, I figured I would expand upon one of the above answers. I was trying to find a solution that did not involve a PECL extension, but I did not have much luck with ftp_ssh_connect(). I recommend trying it, you may have better luck and could forgo the PECL extension route.
Sigh, on further reading, it appears ftp_ssh_connect is, understandably, incompatible with the sftp protocol. However, I found a nice blog post about utilizing ssh2_connect() and ssh2_sftp() (as mentioned in a previous answer) and figured I would post that to give you some additional assistance. It is not as simple as calling the functions for most PHP distributions. Here is the blog post. Some of those steps may not be necessary or you may need to do some additional things listed in another blog post I ran across, here.
On my system, all I had to do was run apt-get install libssh2-1-dev libssh2-php and I was able to find ssh2 in my php -m output.
Having an include, as long as you have read/write permissions on the website you're getting the file from should work, however this is just guess work atm as i don't have any means of checking it. Good luck though!
Yes, you should be able to do this.
Whoever told you that you can't do this might be getting confused with JavaScript and cross-site scripting browser restrictions which prevent JavaScript downloaded from one domain to access content in a different domain.
That being said, if you are using PHP which to me implies that you are talking about PHP running on a web sever, you should be able to use PHP or any other scripting or programming language to download the file from SiteB.com, then update the file, and then finally FTP the file to a different web server (SiteA.com).
I am writing a simple SFTP client in PHP because we have the need to programatically retrieve files via n remote servers. I am using the PECL SSH2 extension.
I have run up against a road block, though. The documentation on php.net suggests that you can do this:
$stream = fopen("ssh2.sftp://$sftp/path/to/file", 'r');
However, I have an ls method that attempts to something similar
public function ls($dir)
{
$rd = "ssh2.sftp://{$this->sftp}/$dir";
$handle = opendir($rd);
if (!is_resource($handle)) {
throw new SFTPException("Could not open directory.");
}
while (false !== ($file = readdir($handle))) {
if (substr($file, 0, 1) != '.'){
print $file . "\n";
}
}
closedir($handle);
}
I get the following error:
PHP Warning: opendir(): Unable to open ssh2.sftp://Resource id #5/outgoing on remote host
This makes perfect sense because that's what happens when you cast a resource to string. Is the documentation wrong? I tried replacing the resource with host, username, and host and that didn't work either. I know the path is correct because I can run SFTP from the command line and it works fine.
Has anyone else tried to use the SSH2 extenstion with SFTP? Am I missing something obvious here?
UPDATE:
I setup sftp on another machine in-house and it works just fine. So, there must be something about the server I am trying to connect to that isn't working.
When connecting to a SFTP server and you need to connect to the root folder (for instance for reading the content of the folder) you would still get the error when using just "/" as the path.
The solution that I found was to use the path "/./", that's a valid path that references to the root folder. This is useful when the user you are logging with has access only to its own root folder and no full path is available.
So the request to the server when trying to read the contents of the root folder should be something like this:
$rd = "ssh2.sftp://{$this->sftp}/./";
For php versions > 5.6.27 use intval()
$sftpConnection = ssh2_connect($host);
$sftp = ssh2_sftp($sftpConnection);
$fp = fopen("ssh2.sftp://" . intval($sftp) . $remoteFile, "r");
https://bugs.php.net/bug.php?id=73597
I'm having a similar issue. I assume you are doing something similar to this:
$dir = "ssh2.sftp://{$sftp}{$remote_dir}";
$handle = opendir($dir);
When $remote_dir is the full path from root then open_dir works. If $remote_dir is just '/' or '', then I get the 'unable to open' error as you did.
In my case, it seems ssh connects at the root folder instead of the 'home' directory as ftp does. You mentioned that it worked on a different server, so I wonder if it is just a config issue.
the most easiest way to get SFTP working within PHP (even on windows) without installing any extension etc is PHPSECLIB: http://phpseclib.sourceforge.net/ . The SSH stuff is completely implemented in a PHP class.
You use is like this:
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
echo $sftp->pwd();
?>
The documentation on that page contains an error. Take a look at the example here instead: http://php.net/ssh2_sftp - what you actually need to do is to open a special SFTP resource using ssh2_sftp() prior to using it with fopen(). And yes, it looks just like that, e.g. "Resource #24" when converted to string... a bit weird but apparently it works.
Another caveat is that SFTP starts in the root directory rather than the home directory of the remote user, so your remote path in the URI should always be an absolute one.
I just had the same issue, but I could figure out the problem.
On my case, when connecting to the server, I was going to the root of the account, and due to server configs I wasn't able to write there.
I have connected to the account using a fireFTP, and so I could see where the root of the account was...it was the root of the server.
I had to include the whole path until the folder where I am allowed to write, and so I could solve the issue.
So, my advice is to get the path using a graphic interface (I have used fireFTP), and add the whole path to your code.
$pathFromAccountRootFolderToMyDestinationFolder = '/Account/Root/Folder/To/My/Folder';
$stream = fopen("ssh2.sftp://".$sftp."/".$pathFromAccountRootFolderToMyDestinationFolder."/myFile.ext", 'r');
Hope this will help you and other people with the same issue!
Cheers!
I recently tried to get SFTP on PHP working and found that phpseclib was a lot easier to use:
http://phpseclib.sourceforge.net/
If you have the luxury of not being on a shared host and can install whatever extensions you want to maybe the PECL extension would be better but not all of us are so lucky. Plus, phpseclib's API looks a bit more intuitive, being OOP and all.
My issue was, that I was connecting in function and returning string URL with resource inside. Unfortunatelly resource is than created in function context and garbage collector is disconnecting resource on function end. Solution: return resource by reference and unset it manually in more complex context.
Solved my issue by enabling sftp support on the (Powershell) server
This is a bug in the ssh2 package that I found years ago and posted a patch to php.net. It fixes this issue but requires a rebuild of the ssh2 pecl package. You can read more here: https://bugs.php.net/bug.php?id=69981. I included a patch there to the ssh2_fopen_wrappers.c file in the package to fix the issue. Here is a comment I included:
Here is a line of code from ssh2_fopen_wrappers.c that causes this bug: (comment included)
/*
Find resource->path in the path string, then copy the entire string from the original path.
This includes ?query#fragment in the path string
*/
resource->path = estrdup(strstr(path, resource->path));
This line of code -- and therefore this bug -- was introduced as a fix for bug #59794. That line of code is attempting to get a string containing the part, query and fragment from the path variable.
Consider this value for the path variable:
ssh2.sftp://Resource id #5/topdir?a=something#heading1
When resource->path is "/topdir", the result is that "/topdir?a=something#heading1" gets assigned to resource->path just like the comment says.
Now consider the case when resource->path is "/". After the line of code is executed, resource->path becomes "//Resource id#5/topdir#heading1". This is clearly not what you want. Here's a line of code that does:
resource->path = estrdup( strstr( strstr( path, "//" ) + 2, "/" ) );
You may also need to apply the patch for bug # 73597 which removes "Resource id #" from the path string before calling php_url_parse().