Unable to run FTP commands from AWS AMI - php

I want to connect to an FTP using PHP to upload the reports generated. As per the remote server, the FTP needs to be in ACTIVE mode.
this is my code:
ini_set('display_errors', '1');
error_reporting(E_ALL);
$conn_id = ftp_connect('myftpserver.com', 21);
if($conn_id)
{
// login with username and password
$login_result = ftp_login($conn_id, 'mysuer', 'password');
$passive = ftp_pasv($conn_id,FALSE);
echo "is active?<br/>";
var_dump($passive);
echo 'Login Result:';
var_dump($login_result);
$files_list = ftp_nlist($conn_id, '/MyFolder/');
echo "<br/>files list ";
var_dump($files_list);
}
else
{
var_dump('Unable to connect to FTP Server');
}
When I am running it from the local machine or a normal shared server, I am able fetch the list, but I am unable to run the script from my AWS AMI instance. For testing purpose, I have even opened all inbound traffic too. Still no luck. Also, the point is that if I try with some other ftp details, I am able to get the response of ftp_nlist. But not for this one. I tried it on 3 AWS instances till yet. Yielded the same result.
All I can say is that this is somewhere the issue at my server security group/firewall. But unable to figure it out. Please help.
The response I get from the server:
is active
bool(true)
Login Result:bool(true)
files list bool(false)

Using FTP Active Mode is problematic with AWS Security Groups.
For active mode to work, you will have to open all inbound ports above 1023. If your client supports restricting the range, do so. You will also need to open both port 20 and port 21 inbound and outbound.
The problem is that the FTP client selects a port that it will listen on. Then the FTP client informs the FTP Server of this port number. The FTP Server then connects to this port. This goes against normal AWS Security Group designs meaning only allow specific ports to be open. You can verify this by opening all ports temporarily, test your FTP client and then closing all the ports.
Active Mode is not secure for the FTP client. Passive Mode is not secure for the FTP server (but the better choice).
NOTE: Rotate your FTP credentials often. Your login and password is sent in the clear and are not encrypted.
FTP is a legacy technology, which is still very popular, that should be stored away in the attic.

Related

Cannot connect to external FTPS server with php

I need to extract data from a .gz file which is located on an ftp server. The FTP server works with IP white listing, so I submitted my local IP and my website IP (i used gethostbynameto get the IP) which were both approved.
Locally, I can run this code to reach the file:
$url="ftp://username:password#host/targetfile.gz";
$xml = simplexml_load_file("compress.zlib://$url") or die ("Cannot load file");
This runs perfectly fine and it allows me to extract data from the XML file.
When I run the script on my server however, i'm not getting a connection. I contacted the admin running the ftp server and they told me they only allow FTPS connections.
So, I proceeded with the following code to try and establish a connection:
$conn_id = ftp_ssl_connect($ftp_server);
// login with username and password
$login_result = ftp_login($conn_id, $ftp_user, $ftp_pass);
if (!$login_result) {
die("Cannot login.");
}
echo ftp_pwd($conn_id); // /
// close the ssl connection
ftp_close($conn_id);
This also doesn't connect. I'm new to FTP connections with PHP and FTP in general and i've got no clue on how to proceed from this point.
OpenSSL is enabled.
Can anyone point me in the right direction?
Edit: These are the error messages:
When I use the top block of code, this occurs:
Warning: simplexml_load_file(): I/O warning : failed to load external
entity
"compress.zlib://username:password#ftp.thehost.com/targetfile.gz"; in
/path/to/my/website/folder/htdocs/mydomain.com/feeds/script.php on
line 13
When I use the bottom block of code, I get this error:
Warning: ftp_login() expects parameter 1 to be resource, boolean given
in /path/to/my/website/folder/htdocs/mydomain.com/feeds/script.php on
line 16 Cannot login.
I do not have access to the log file afaik
I do not think there's much we can help you here.
You do not have a network connectivity between the web server and FTP server.
Note that it does not have to be because the web server has not been white-listed on the FTP server. It's also possible that the web server does not allow outgoing connections.

Cannot connect to server from another server ftp php

I work with 2 servers one is my production server other is my resource server.
I cannot connect to my resource server from my production server over ftp.
I can connect to other servers from my production server.
I can also connect to my resource server from my localhost or filezilla.
I use this code to connect :
$conn_id = ftp_connect("resource server ip", "21", "5");
if ($conn_id) {
echo "connected";
ftp_close($conn_id);
}
print_r(error_get_last());
I don't get any output when I run this script on server(no error).
On localhost it runs no problem.
What can be the problem with this? Is this something that server admin has to resolve? Thanks for help.
You should first check from commandline, whether it's a networking/OS issue or not.
So if you've got shell access to the production server try connecting to the resource server via the commandline ftp client.
If that does not work, you've got a network / firewall / access control problem, not related to php or your software, and you should talk to the sysadmin.
If it does work, then the problem is in your stuff, and you should set the log levels to high, and run this script from commandline, also check the logs of php, php-error, syslog and the resource servers ftp access log and syslog too.
Note: ftp is a not-too-exact beast, the servers and clients have a lot of workarounds built in to treat each other in a way, that works somehow. There could be issues from active (multiple back-and-forth connections) and passive mode (it's like http), also with ls formats and timestamps, timezones and ports.
Also some servers only support ftps (ftp with ssl) - which is not the same as sftp (file transfer over ssh - port 22).
Your production server probably has some firewall rules, and your connection get caught on that, to debug this, please use the commandline ftp client, and/or nmap / netcat.

Non-anonymous sessions must use encryption

I am trying to connect to the FTP using php commands. But when I try to login I get the below error
Warning: ftp_login(): Non-anonymous sessions must use encryption
Code That I have tried:
ftp_connect($ftp_server)
ftp_get($ftp_conn, $local_file, $server_file, FTP_ASCII)
Though it is getting connected through FileZilla . Not sure what the error means. Thanks
Probably you should use FTPS instead of FTP, this is an excerpt from the link:
In order to maintain compatibility with existing non-TLS/SSL-aware FTP clients, implicit FTPS was expected to listen on the IANA Well Known Port 990/TCP for the FTPS control channel, and to 989/TCP for the FTPS data channel. This allowed administrators to retain legacy-compatible services on the original 21/TCP FTP control channel.
The FTP server may sit on the port you connect to, just to tell you that you need to connect to another port via FTP over SSL.
In PHP use FTP over SSL: ftp_ssl_connect().

Why can I obtain file size and the time stamp with FTP but not be able to retrieve the actual file?

I've been working on this for about 20 hours now and I need some major help. I'm able to get the file size and the timestamp on the file but I am unable to actually obtain the data.
The server I'm trying to get the data from requires FTP over explicit TLS
I'm receiving the same error(s) with both FTP_BINARY and FTP_ASCII in the ftp_fget()
The server the file is coming from is UNIX
If I refresh the page every few hours the errors I get from PHP are different with no change in code
Error 1: 'ftp_fget(): Transfer mode set to BINARY if ftp_get is binary
or it's 'ftp_fget(): Transfer mode set to ASCII' if ftp_get is ascii
Error 2: 'ftp_fget(): Entering Passive Mode(12.345.678.90.12.34)'
On the above errors I read that PASV mode being FALSE is what triggers Error 1, so I think the switching between the errors is for pasv mode working or not working. Not positive though.
<?php
$server = "12.345.678.90";
$local_file = 'inv3.txt';
$file = 'inventory-alp.txt';
$con = ftp_ssl_connect($server,21) or die("Could not connect to $server");
ftp_login($con,"xxxxxx","xxxxxx") or die("Could not login");
ftp_pasv($con,true);
$fsize = ftp_size($con, $file); // works
if ($fsize != -1)
{
echo "</br>$file is $fsize bytes.</br></br>";
}
else
{
echo "</br>Error getting file size.</br></br>";
}
$lastchanged = ftp_mdtm($con, $file); //works
if ($lastchanged != -1)
{
echo date("F d Y H:i:s.",$lastchanged)."</br></br>";
}
else
{
echo "Could not get last modified</br></br>";
}
if (ftp_get($con,$local_file,$file,FTP_ASCII)) //fails
{
echo "successfully written to $local_file";
}
else
{
echo "There was a problem while downloading $file to $local_file";
}
$var = error_get_last();
echo '<pre>';
var_dump($var);
echo '</pre>';
ftp_close($con);
?>
EDIT 1: Solution: I ended up not being able to access what I needed to change the firewall settings and such in php. While this is not the true answer, I did make it work and it is relatively easy.I ended up running across WINSCP, having the ability to connect to the server in a filezilla type layout and then save the session url was nice. All i did was access the saved session in the .exe and was able to set up my connection in half an hour.
What Martin indicated is very true, the SIZE and MDTM commands run synchronized over the main FTP Command Connection only. The transferring of data files, and usually the directory listing also (unless MLST/MSLD is used) requires a separate connection, the Data Connection, which is negotiated by the client and server over the Control Connection using a series of commands, most notably PORT and PASV.
Without going into a ton of detail (There's a link to our white paper later), when the Client & Server negotiate the terms of the Data Connection, one of the end points will tell the other endpoint the specific IP address and Port number for the connection. One endpoint will listen and wait for a connection from the other endpoint. This works great unless there is a firewall in front of the endpoint that is waiting for the inbound connection. If the client/server session is running in Active mode, the Server will actively connect back to the client on the IP/Port which was received by the server from the client in the form of the PORT command. In Passive mode, the Server will passively wait for the client to connect on the IP/Port which was sent by the server to the client in the response to the PASV command sent by the client to the server.
Again, firewalls tend to block FTP data connections, unless the firewall does active FTP NAT'ing or unless Port Forwarding has been set up on the Firewall and a set of passive-ports has been opened and routed to the endpoint specifically.
So check the firewall settings on the client if you want to use Active/Port mode; check the firewall settings on the server if you want to use Passive/PASV mode.
Here's a link to our white paper which outlines the basics of FTP/PASV/PORT, hopefully it'll help you with your issue.
http://www.webdrive.com/wp-content/uploads/FTP_Explained1.pdf
Best of Luck!
Michael
With the FTP protocol, it's perfectly possible that you are able to obtain the file size and the modification timestamp (using SIZE and MDTM commands respectively), but not the file itself.
The SIZE and MDTM commands use the FTP control connection only.
While a file transfer (or a directory listing) requires a separate data connection. And it's likely that there's something that prevents the data connection from being opened.
See (my) article on the FTP connection modes for more details and typical issues with data connections.
Typically a culprit would be a firewall on your webserver. If you have an SSH/terminal access to the webserver, are you able to connect from it to the FTP server?
Another possibility is a misconfigured FTP server. Is the IP address in "Error 2" routable from your web server? (=Is it the real IP address you connect to?)
It is unlikely this is related to an ASCII/BINARY mode. The messages you are getting (Transfer mode set to ...) are status messages, not error messages. They are not related to your problem. It's indeed strange that you got no other message/error.
You can try to use the active mode, instead of the passive.
ftp_pasv($con, false);
But usually the active mode is more problematic.

Php ftp_connect fails, FTP client connects

(this question might not belong here, but on ServerFault, I don't know exactly where the problem comes from)
I'm trying to connect to an FTP server like this:
$con = ftp_connect( '86.xxx.xx.xxx', 21 ) or die("FTP connect error");
and it always throws me the error message.
I can connect like this to other FTP servers, but not this one. I can also connect to this one via an FTP client.
What server settings could be wrong that permit a client connect but not PHP?
Thanks for any help!
It seems my host was blocking remote FTP access to other servers. Oddly enough it allowed me to connect to my other servers at this host.

Categories