I'm trying to containerize my PHP script.
But for some reason it's unable to connect to my FTP server when it's running in the container.
Running the script locally on my machine runs without any issues.
Dockerfile:
FROM php:8.1-cli
EXPOSE 8080
COPY . /var/www/html/
WORKDIR /var/www/html
ENTRYPOINT php -f serve.php
PHP Script:
$connection = ftp_connect(FTP_HOST, 21, 90);
ftp_pasv($connection, true);
if(ftp_login($connection, FTP_USERNAME, FTP_PASSWORD))
{
$stream = fopen('data://text/plain,','r');
ftp_fput($connection, $filename, $stream);
}
ftp_close($connection);
After 90 seconds it gives the following warning:
ftp_fput(): Connection timed out in /var/www/html/ftp.php on line 16
I tried bashing into the container and installing an FTP client.
It gives me a similar timeout error - I can connect to the host, but running any commands results in a stall.
I also have tried running the container on a VPS to eliminate any local firewall/network issues.
But the issue is the same.
Please let me know if any additional information is needed to solve the issue.
Mind the documentation of PHP's ftp_pasv function (emphasis mine):
Please note that ftp_pasv() can only be called after a successful login or otherwise it will fail.
Also note that you do not check the status return value of your ftp_pasv call, so you won't notice if that call actually succeeds (which is most likely won't). Because of that, your script will attempt to establish an active FTP connection. That won't work in a container (unless started with --network=host), because the container runs in a private network which is NATed by your host machine.
Solution: Login first, enabling passive mode second (and also, always check your error return values; many of the older functions from the PHP standard library do not throw exceptions, but rely on error return values):
if (ftp_login($connection, FTP_USERNAME, FTP_PASSWORD))
{
if (ftp_pasv($connection, true) === false) {
throw new \Exception("could not enable passive mode")
}
$stream = fopen('data://text/plain,','r');
ftp_fput($connection, $filename, $stream);
}
Related
I have created an tunnel(SSH) in order to access URL's that are blocked from my local PC. This works really well, I can for example access from the browser this url http://localhost:8888 and I will get the intended content. This URL also works with the REST Client that I'm using for testing. However, if I call it from PHP:
file_get_contents('http://localhost:8888')
I get this error each time: failed to open stream: Connection refused
So my guess is that file_get_contents doesn't work well with SSH tunnels. Is there maybe an alternative function that I could use for this purpose ?
I solved this problem using php-ssh2 package.
Instalation on Ubuntu 16
sudo apt-get install php-ssh2
Usage with user name and password:
$conn = ssh2_connect($ip, 22);
ssh2_auth_password($conn, $user,$pass);
ssh2_scp_recv($conn, '/remote_path', '/local_path');
Usage with id_rsa (after ssh-copy-id)
I recommend use this method because of we probably do not want paste plain password in source code.
$conn = ssh2_connect($ip, 22);
ssh2_auth_pubkey_file($conn, $user,
"/home/{$user}/.ssh/id_rsa.pub",
"/home/{$user}/.ssh/id_rsa", 'secret')
ssh2_scp_recv($conn, '/remote_file', '/local_file');
Additional info
Old name of this package: libssh2-php
Docs:
http://php.net/manual/en/function.ssh2-scp-recv.php
Connected question:
How to SFTP with PHP?
Is there also possibility of use:
file_get_contents("ssh2.sftp://{$user}:{$pass}#{$ip}:22{$remote_path}"))
but I do not know how to use it with id_rsa.pub.
Before I begin, I'm aware of phpseclib and from what I've ready it's the way you should normally go. Unfortunately I'm working on an older script, and I'm trying to avoid the time needed to set up phpseclib, test on local environment, put on production, test on production, refactor my script, etc. I'm hoping someone can help resolve my issue with ssh2.
I've used the following script to ssh a file to a client's server. It has worked fine for years.
<?php
$url = 'their.url.com';
$userName = 'name';
$password = 'pass';
$conn = ssh2_connect($url, 22);
$auth = ssh2_auth_password($conn, $userName, $password);
// Determine whether this is a file path that needs to be opened or not
$localFilePath = 'test123.txt';
$openFile = fopen($localFilePath, "r");
$fileName = 'test.txt';
// Their server path
$remoteFilePath = "/Orders/".$fileName;
if ($auth === true) {
// Transfer file
$sftp = ssh2_sftp($conn);
file_put_contents("ssh2.sftp://".$sftp.$remoteFilePath, $openFile);
}
WHAT HAPPENED:
My client moved his stuff to a different server. It broke the connection, he freaked out, etc., etc. He gave me updated credentials to the new server, and I have confirmed the protocal is SFTP - SSH File Transfer Protocal.
THE ISSUE:
The file transfer is no longer working after updating the ssh combo url/credentials.
WHAT I'VE TRIED:
I tried a test file transfer using Linux (not PHP) and it worked. I also ssh'd in using Filezilla and transferred a file that way, no problem. so I have confirmed that the host/user/pass combo is correct.
I dumped out the result of the file_put_contents, and it is false.
I dumped out $conn, $auth, and $sftp, and what "ssh2.sftp://".$sftp.$remoteFilePath looks like, and the results are (in order)
resource(27) of type (SSH2 Session)
bool(true)
resource(30) of type (SSH2 SFTP)
ssh2.sftp://Resource id #30/Orders/test.txt
Note that test.txt is a very tiny file to rule out file size issues.
I have also uninstalled and reinstalled the ssh2 PECL extension.
I am convinced this is due to an issue on their end, because this script has always worked before, I'm just baffled as to why things are failing now that they're on their new server, and I'm not sure what to do further to diagnose the problem. I can't think of any change that's been made to my server that would affect this.
Any ideas would be greatly appreciated.
Why not just use the ssh2.sftp:// stream directly?
file_put_contents("ssh2.sftp://{$userName}:{$password}#{$url}:22/{$remoteFilePath}", $openFile);
It should work without all the ssh connection, authentication, moving to sftp, etc..
I am trying to do a ssh connection in php. But I get this error Unable to startup sftp subsystem. I don't understand it. And I could not find much in google. Please let me know how to debug this. I am using wamp.
$resConnection = ssh2_connect($strServer, $strServerPort);
if(ssh2_auth_password($resConnection, $strServerUsername, $strServerPassword)){
//Initialize SFTP subsystem
$resSFTP = ssh2_sftp($resConnection); -->error coming here
}else{
echo "Unable to authenticate on server";
}
It's hard to say without more information. Unfortunately, libssh2 is pretty limited in the amount of info it provides. I'd recommend you use phpseclib, a pure PHP SFTP implementation. eg.
<?php
include('Net/SFTP.php');
$sftp = new Net_SFTP('www.domain.tld');
if (!$sftp->login('username', 'password')) {
exit('Login Failed');
}
print_r($sftp->nlist());
?>
Put define('NET_SSH2_LOGGING', 3) at the top to get logs if you still encounter the problem, after switching. Then post the logs so that you can get better assistance.
Thanks!
I had the same issue and I solved it with following steps.
1) First I noticed that there is slowliness on server to which we are trying to make sftp connection.
2) There is a script in our side which try to connect and get files from server so because of slowliness issue our scripts can't complete their job and wait for connection because it was very slow. I checked how many processes created for sftp connection with following command
ps aux | grep sftp
3) Then I killed all these processes with following command
pkill sftp
4) After I kill that processes error was gone.
Summary
Main reason of this issue was that. There was slowliness issue in ftp server and we created too many connection. I think there is some limit on server side for max connections.
I need to find a method of deploying a PHP project stored in a git repo to a staging and production server that do not have git installed. Scripts I've found so far (ie Capistrano) require Git on the target server.
Unfortunately, my host does not allow this, and the only way so far is via standard FTP, with which I keep missing files. This makes for an unprofessional look.
I would like to be able to deploy from my local git repo, which will check the .git folder on the target to see which version is on there, then cause the target server to backup the current version and then overwrite it with only the changed files being pushed.
Preferably something in PHP with a web interface.
Not asking much am I ;)
Anyone out there got/seen anything like this?
There are three git-ftp scripts which allow you to "push" a git repository to a FTP server.
git-ftp (bash)
git-ftp (python)
PHPloy (php)
You might be able to use something like Fuse to "mount" the production server as a local drive, and then as far as your copy of git is concerned it's a local operation. Alternatively, rsync.
There's a tool call Dandelion that also does this. From what I can see, it's quite similar to git-ftp, BUT it also supports sftp and Amazon S3, which is handy if you don't want to change deploy tool just because you change server. It comes as a ruby gem, so really easy to install and get going.
I have done something like that using ssh2 and php.
first you need to clone the repo on the server. Once cloned, you can do git pull, checkout, etc from php using ssh2. the most practical way I found was doing.
git fetch;
git reset --hard commit_hash;
in order to get set the commit to the one expected.
To execute a php - ssh2 command (supposing you have ssh2 installed), you can use this method.
public static function SSHCommmand($command,$user,$ip) {
$port = 22;
if (!function_exists("ssh2_connect"))
die("function ssh2_connect doesn't exist.");
$result['debug'] .= " -Connect- 1";
if (!($con = ssh2_connect($ip, $port, array('hostkey' => 'ssh-rsa')) )) {
die("unable to establish connection.");
} else {
// try to authenticate with username root, password secretpassword
if (!(ssh2_auth_pubkey_file($con, $user, '/home/' . $user . '/.ssh/deploy_rsa.pub', '/home/' . $user . '/.ssh/deploy_rsa'/* , 'secret' */))) {
dir("fail: unable to authenticate.");
} else {
// allright, we're in!
// execute a command
if (!($stream = ssh2_exec($con, $command))) {
die("fail: unable to execute command.");
} else {
// collect returning data from command
stream_set_blocking($stream, true);
$data = '';
while ($buf = fread($stream, 4096)) {
$data .= $buf;
}
fclose($stream);
return $data;
}
}
}
}
I'm using ssh-rsa key, the auth method might change. I'm aslo supposing that the keys are in '/home/' . $user . '/.ssh/deploy_rsa.pub' and '/home/' . $user . '/.ssh/deploy_rsa.
The other thing you might take into account is that to execute remote a remote git command, the command should be like:
_GIT_PATH.' --git-dir='.$path.'/.git --work-tree='.$path.' '.$command;
where $path is the toplevel of the working tree.
By using this and the Amazon Api, I've been able to deploy new code to several servers automatically and simultaneously.
I use Beanstalkapp.com, which is great. You can deploy via FTP or SFTP.
I need to write a script that is run as a cron job every night which transfers some report files via sftp to another server.
The report files are created every night using another cron in the format 'support_[date].csv' & 'download_[date].csv'.
I'm wondering if you had any pointers on how to do the following:
Find the 2 files created on latest [date]
Copy these files to another server using SFTP
I've tried several PHP scripts utilising the ssh2 extension, but to no avail. Is there a way to do it using a shell script? It's not something I am hugely familiar with to be honest (hence going down the PHP route initially)
This was one of my PHP scripts which didn't work:
$src = 'test.csv';
$filename = 'test.csv';
$dest = '/destination_directory_on_server/'.$filename;
$connection = ssh2_connect('example.com', 22);
ssh2_auth_password($connection, 'username', 'password');
// Create SFTP session
$sftp = ssh2_sftp($connection);
$sftpStream = fopen('ssh2.sftp://'.$sftp.$dest, 'w');
try {
if (!$sftpStream) {
throw new Exception("Could not open remote file: $dest<br>");
}
$data_to_send = file_get_contents($src);
if ($data_to_send === false) {
throw new Exception("Could not open local file: $src.<br>");
}
if (fwrite($sftpStream, $data_to_send) === false) {
throw new Exception("Could not send data from file: $src.<br>");
} else {
//Upload was successful, post-upload actions go here...
}
fclose($sftpStream);
} catch (Exception $e) {
//error_log('Exception: ' . $e->getMessage());
echo 'Exception: ' . $e->getMessage();
if($sftpStream) {fclose($sftpStream);}
}
This were the error messages I got:
Warning: fopen() [function.fopen]: URL
file-access is disabled in the server
configuration in
/path_to_script/sftp-test.php on line
17
Warning: fopen(ssh2.sftp://Resource id
3/destination_directory_on_server/test.csv)
[function.fopen]: failed to open
stream: no suitable wrapper could be
found in /path_to_script/sftp-test.php
on line 17 Exception: Could not open
remote file:
/destination_directory_on_server/test.csv
using the terminal to find latest date of your file, you can use ls -1tr . Then use scp (not sftp) to copy/transfer files over
example,
#!/bin/bash
latest_download=$(ls -1tr download*csv | tail -1)
latest_support=$(ls -1tr support*csv | tail -1)
scp $latest_download user#somehost.com:somedir # syntax from memory, check man page for correct syntax
scp $latest_support user#somehost.com:somedir
check the man page of scp for usage
Muchos kudos to ghostdog74! Managed to get this working, but with sftp.
First I managed to set up key authentication, then partly using ghostdog74's script I did this and it worked perfectly!
cd /directorywithfilesin
latest_download=$(ls -1tr download* | tail -1)
latest_support=$(ls -1tr support* | tail -1)
sftp username#example.com <<EOF
cd /dir_to_copy_to
put $latest_download
put $latest_support
EOF
Thanks!
Among other problems with ghostdog74's method is that it's non-portable. My recommendation would be to use phpseclib, a pure PHP SFTP implementation.
This will not work from PHP from your server because your php.ini has disabled remote wrappers
allow_url_fopen boolean
This option enables the URL-aware fopen wrappers that enable accessing URL object like files. Default wrappers are provided for the access of remote files using the ftp or http protocol, some extensions like zlib may register additional wrappers.
Note: This setting can only be set in php.ini due to security reasons.
However, you could simply let your cron job call a shell script that that uses sftp or rsync directly. You don't have to do this with PHP.
I'm voting to move this to ServerFault to get better support for shell scripting.
The answer is right there, in the error message:
Warning: fopen() [function.fopen]: URL file-access is disabled in the server configuration
means that file-access through URL wrappers is disabled in the server configuration.
Check your PHP config, especially allow_url_fopen. PHP documentation says "This setting can only be set in php.ini due to security reasons", so check it there.
See also fopen: "If PHP has decided that filename specifies a registered protocol, and that protocol is registered as a network URL, PHP will check to make sure that allow_url_fopen is enabled. If it is switched off, PHP will emit a warning and the fopen call will fail." As far as I can tell, that's exactly what is happening there.
If you can't or won't enable allow_url_fopen, you still have some options:
call sftp directly
mount a share with sshfs and then use it as a normal folder
Try as follows (Shell)
SFTP=<sftp path>
KEY_FILE=<your key>
USERNAME=<remote username>
SERVER =<remote server>
REMOTE_DIR=<remote location>
APP_HOME =<App location>
FILENAME=<file name>
${SFTP} -o IdentityFile=${KEY_FILE} ${USERNAME}#${SERVER} <<_COMMAND
lcd ${APP_HOME}
cd ${REMOTE_DIR}
put ${FILENAME}
bye
_COMMAND