PHP - Can't list files from an FTP - php

I have an FTP account I'm trying to list files from but I can't get it to work
FileZilla is able to list them, but only when strictly setting it to active mode rather than passive
Example output from FileZilla
Status: Retrieving directory listing of "/"...
Command: PORT 192,168,68,115,213,202
Response: 200 PORT command successful
Command: MLSD
Response: 150 Opening connection
Response: 226 Transfer complete
Status: Directory listing of "/" successful
In my PHP code I've tried the following
$connection = ftp_connect($host, 21);
ftp_login($connection, $username, $password);
var_dump(ftp_size($connection, 'tarbert.jpg'));
var_dump(ftp_mlsd($connection, '.'));
var_dump(ftp_rawlist($connection, '.'));
var_dump(ftp_nlist($connection, '.'));
Which outputs
int(227978)
bool(false)
bool(false)
bool(false)
As you can see by the call to ftp_size, there are files there and it can see them, but the calls to list the files all return false
I've tried the usual answer of setting to passive mode but that just makes it timeout
Does anyone have any suggestions on what else I could try?
Mini update
Strangely, if I move the ftp_size call to be after the others then that also returns false so it seems like the initial call to list files breaks something
I've tried moving each of the 3 list functions to be the first call but this makes no difference to the outcome
Update with FileZilla full log
Status: Connecting to xx.xx.xx.xx...
Status: Connection established, waiting for welcome message...
Response: 220-Welcome To Rumpus!
Response: 220 Service ready for new user
Command: AUTH TLS
Response: 530 USER and PASS required first
Command: AUTH SSL
Response: 530 USER and PASS required first
Status: Insecure server, it does not support FTP over TLS.
Command: USER my_user
Response: 331 Password required
Command: PASS ******
Response: 230 User logged in
Command: SYST
Response: 215 MACOS Server
Command: FEAT
Response: 211-Extensions supported:
Response: SIZE
Response: REST STREAM
Response: APPE
Response: MDTM
Response: MACB
Response: PBSZ
Response: PROT
Response: EPSV
Response: MLSD
Response: UTF8
Response: 211 End
Command: OPTS UTF8 ON
Response: 220 UTF8 OPTS ON
Status: Logged in
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is current directory.
Command: TYPE I
Response: 200 Type set to I (Image)
Command: PORT 192,168,68,115,204,128
Response: 200 PORT command successful
Command: MLSD
Response: 150 Opening connection
Response: 226 Transfer complete
Status: Directory listing of "/" successful
Update with curl log
curl -v --list-only ftp://XX.XX.XX.XX --user "username:password"
* Trying XX.XX.XX.XX:21...
* TCP_NODELAY set
* Connected to XX.XX.XX.XX (XX.XX.XX.XX) port 21 (#0)
< 220-Welcome To Rumpus!
< 220 Service ready for new user
> USER username
< 331 Password required
> PASS password
< 230 User logged in
> PWD
< 257 "/" is current directory.
* Entry path is '/'
* Request has same path as previous transfer
> EPSV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 229 Entering Extended Passive Mode (|||3004|)
* Trying XX.XX.XX.XX:3004...
* TCP_NODELAY set
* Connecting to XX.XX.XX.XX (XX.XX.XX.XX) port 3004
* Connected to XX.XX.XX.XX (XX.XX.XX.XX) port 21 (#0)
> TYPE A
< 200 Type set to A (ASCII)
> NLST
< 150 Opening connection
* Maxdownload = -1
# FILE LIST HERE
* Remembering we are in dir ""
< 226 Transfer complete
* Connection #0 to host XX.XX.XX.XX left intact

Just in case anyone else stumbles on this, it appears the FTP is behind a NAT
See this link for details
http://www.elitehosts.com/blog/php-ftp-passive-ftp-server-behind-nat-nightmare/
I couldn't find a fix as this particular FTP doesn't allow passive connections but if yours does then that link will help

Related

Weird Elasticsearch and Curl problem. Curl and wget can connect to elascticsearch but receives no data

Today i am having a weird problem with elasticsearch and curl. Elasticsearch is running fine but when i try to connect it with curl, it waits forever and no data returns.
I tried to get verbose output from curl, it can connect to elasticsearch, but after connect, it receives no data.
curl -v http://10.0.0.1:9200/_cat/indices
* Trying 10.0.0.1...
* TCP_NODELAY set
* Connected to 10.0.0.1 (10.0.0.1) port 9200 (#0)
> GET /_cat/indices HTTP/1.1
> Host: 10.0.0.1:9200
> User-Agent: curl/7.52.1-DEV
> Accept: */*
>
It waits forever. Also elasticsearch php is not working as same.
When i tried to open same url from same machine with lynx, it can get data successfully. But when i tried to open same url with wget, it waits forever too. Curl can connect other external elasticsearch hosts without problem. Also we can connect to this elasticsearch host from other hosts without problem.
The other weird thing is, when i tried to get /_cat/health with curl or wget, they can get the data.
curl -v http://10.0.0.1:9200/_cat/health
* Trying 10.0.0.1...
* TCP_NODELAY set
* Connected to 10.0.0.1 (10.0.0.1) port 9200 (#0)
> GET /_cat/health HTTP/1.1
> Host: 10.0.0.1:9200
> User-Agent: curl/7.52.1-DEV
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: text/plain; charset=UTF-8
< content-length: 63
<
1545812391 11:19:51 mycluster green 2 2 382 381 0 0 0 0 - 100.0%
* Curl_http_done: called premature == 0
* Connection #0 to host 10.0.0.1 left intact
There is no firewall on both machines. I tried to update curl, nothing changed. I also coudln't find any usefull information on syslog, elasticsearch logs etc.
Any idea why this happens?
This system is working perfectly for 4 months. I am having this issue since 2 days. Then i tried to upgrade system packages, rebooted the servers but nothing changed. I cleared the arp cache also.
Thank you.

requests library: Failed to establish a new connection: [Errno 13] Permission denied

I have a python script which uses requests library to connect to a device with its web-interface. It uses a url with https connection.
sess = requests.Session()
try:
resp = sess.post('https://' + device + '/url_admin/login.cgi', data=login_data, verify=False)
except Exception as e:
template = "Exception: {0}\nArguments: {1!r}"
print(template.format(type(e).__name__, e.args))
When I try to execute it from command line it works perfectly (even with apache user). However, when it is triggered with a php code (exec function) from a web-portal. it throws this error
Exception: ConnectionError
Arguments: (MaxRetryError("HTTPSConnectionPool(host='some.device.com', port=443): Max retries exceeded with url: /url_admin/login.cgi (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f55aa8e6550>: Failed to establish a new connection: [Errno 13] Permission denied',))",),)
I am not sure what is the reason because the apache user is able to execute it and make connection. Why it is not connecting when executed by php code?
setsebool -P httpd_can_network_connect on

PHP file upload to a remote FTP directory using Implicit FTP over TLS encryption: uploaded file is empty

Hi everyone i'm trying to upload file using PHP to a remote FTP directory using Implicit FTP over TLS encryption, this is what i have tried so far:
According to one of this article answers if you need implicit, use curl:
ftp_ssl_connect with implicit ftp over tls
1.The code:
$remote_filename='data.csv';
$path = 'php://memory';
$ftp_server = '0.0.0.0';
$ftp_port = 990;
$ftp_user = 'user';
$ftp_pass = 'pass';
$fp = fopen($path, 'rw+');
$ftp_url= 'ftps://'.$ftp_server.'/'.$remote_filename;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $ftp_url);
curl_setopt($ch, CURLOPT_PORT, $ftp_port);
curl_setopt($ch, CURLOPT_USERPWD,$ftp_user .':'.$ftp_pass );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_FTP_SSL, CURLFTPSSL_TRY);
curl_setopt($ch, CURLOPT_FTPSSLAUTH, CURLFTPAUTH_TLS);
curl_setopt($ch, CURLOPT_UPLOAD, 1);
curl_setopt($ch, CURLOPT_INFILE, $fp);
$response = curl_exec($ch);
$error_no = curl_errno($ch); // error number
$error = curl_error($ch); // error buffer
//var_dump(curl_error($ch));
curl_close($ch);
However this is not working for me. File on the remote ftp directory is created but is empty (0 in size).
Response says: false
Also I'm getting error no:35 which is according to manual:
CURLE_SSL_CONNECT_ERROR (35)
A problem occurred somewhere in the SSL/TLS handshake. You really want the error buffer and read the message there as it pinpoints the problem slightly more. Could be certificates (file formats, paths, permissions), passwords, and others.
Have looked into error buffer as it shows more details:
Error buffer is saying "Encountered end of file"
My question:
Does anyone know how to solve this problem?
UPDATE 1
Have added curl logging into script above:
curl_setopt($ch, CURLOPT_VERBOSE, TRUE); // enable curl log
curl_setopt($ch, CURLOPT_STDERR, $fp); // curl log
, and here is the log content:
* About to connect() to #.#.#.# port 990 (#0)
* Trying #.#.#.#...
* Connected to #.#.#.# (#.#.#.#) port 990 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
* subject: E=admin#website.com,OU=1.4,O=Company Ltd,ST=United Kingdom,C=44,CN=#.#.#.#
* start date: Jan 05 13:31:18 2016 GMT
* expire date: Jan 04 13:31:18 2017 GMT
* common name: #.#.#.#
* issuer: E=admin#website.com,OU=1.4,O=Company Ltd,ST=United Kingdom,C=44,CN=#.#.#.#
< 220-WELCOME TO THE Company Ltd FTP SERVER
< 220-
< 220-If you have any issues, please email IT#website.com
< 220-
< 220-Thank you
< 220 ------------------------
> USER #######
< 331 Password required for #######
> PASS #######
< 230 Logged on
> PBSZ 0
< 200 PBSZ=0
> PROT P
< 200 Protection level set to P
> PWD
< 257 "/" is current directory.
* Entry path is '/'
> EPSV
* Connect data stream passively
* ftp_perform ends with SECONDARY: 0
< 229 Entering Extended Passive Mode (|||55857|)
* Trying #.#.#.#...
* Connecting to #.#.#.# (#.#.#.#) port 55857
* Connected to #.#.#.# (#.#.#.#) port 990 (#0)
> TYPE I
< 200 Type set to I
> STOR data_1.csv
< 150 Opening data channel for file upload to server of "/data_1.csv"
* Doing the SSL/TLS handshake on the data stream
* skipping SSL peer certificate verification
* NSS error -5938 (PR_END_OF_FILE_ERROR)
* Encountered end of file
* Failure sending ABOR command: SSL connect error
* Closing connection 0
UPDATE 2
When using Ftp Client FileZilla upload works fine, however i have uploaded file from my computer not from the server we are running the PHP script. Not sure if that is of any help as we don't have ftp client on the server.
FileZilla log when uploading the file:
Status: Starting upload of C:\data.csv
Command: PASV
Response: 227 Entering Passive Mode (0,0,0,0,216,218)
Command: STOR data.csv
Response: 150 Opening data channel for file upload to server of "/data.csv"
Response: 226 Successfully transferred "/data.csv"
Status: File transfer successful, transferred 251,542 bytes in 1 second
Status: Retrieving directory listing of "/"...
Command: PASV
Response: 227 Entering Passive Mode (0,0,0,0,215,110)
Command: MLSD
Response: 150 Opening data channel for directory listing of "/"
Response: 226 Successfully transferred "/"
Status: Directory listing of "/" successful
Server error log is not logging any error.
UPDATE 3
FileZilla loggin level set to 2(info) and the content is below:
Status: Connecting to #.#.#.#:990...
Status: Connection established, initializing TLS...
Trace: TLS Handshake successful
Trace: Protocol: TLS1.2, Key exchange: ECDHE-RSA, Cipher: #, MAC: #
Status: Verifying certificate...
Status: TLS connection established, waiting for welcome message...
Response: 220-WELCOME TO THE COMPANY FTP SERVER
Response: 220-
Response: 220-If you have any issues, please email IT#company.com
Response: 220-
Response: 220-Thank you
Response: 220 ------------------------
Command: USER ####
Response: 331 Password required for ####
Command: PASS *************
Response: 230 Logged on
Command: PBSZ 0
Response: 200 PBSZ=0
Command: PROT P
Response: 200 Protection level set to P
Status: Connected
Trace: Measured latency of 16 ms
Status: Starting upload of C:\data.csv
Command: CWD /
Response: 250 CWD successful. "/" is current directory.
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (#,#,#,#,216,202)
Trace: Binding data connection source IP to control connection source IP 192.168.1.50
Command: STOR data.csv
Trace: Trying to resume existing TLS session.
Trace: TLS Handshake successful
Trace: TLS Session resumed
Trace: Protocol: TLS1.2, Key exchange: ECDHE-RSA, Cipher: #, MAC: #
Response: 150 Opening data channel for file upload to server of "/data.csv"
Response: 226 Successfully transferred "/data.csv"
Status: File transfer successful, transferred 251,542 bytes in 1 second
Status: Retrieving directory listing of "/"...
Command: PASV
Response: 227 Entering Passive Mode (#,#,#,#,217,234)
Trace: Binding data connection source IP to control connection source IP 192.168.1.50
Command: MLSD
Trace: Trying to resume existing TLS session.
Response: 150 Opening data channel for directory listing of "/"
Trace: TLS Handshake successful
Trace: TLS Session resumed
Trace: Protocol: TLS1.2, Key exchange: ECDHE-RSA, Cipher: #, MAC: #
Response: 226 Successfully transferred "/"
Status: Directory listing of "/" successful
Status: Sending keep-alive command
Command: TYPE I
Response: 200 Type set to I
Trace: Skipping reply after cancelled operation or keepalive command.
Status: Sending keep-alive command
Command: TYPE I
Response: 200 Type set to I
Trace: Skipping reply after cancelled operation or keepalive command.
$ftpServer = "xxxx.xxxx.xxxx";
$portNo = "990";
$username = "xxxxxxx";
$password = "xxxxxxx";
$local_file = "Local Directoy Path/filename;
$ftp_path = "FTP Directoy Path/filename";
$fp = fopen($local_file, 'rw+');
$ftp_url= 'ftps://'.$ftpServer.'/'.$ftp_path;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $ftp_url);
curl_setopt($ch, CURLOPT_PORT, $portNo);
curl_setopt($ch, CURLOPT_USERPWD,$username .':'.$password );
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_FTP_SSL, CURLFTPSSL_TRY);
curl_setopt($ch, CURLOPT_FTPSSLAUTH, CURLFTPAUTH_TLS);
curl_setopt($ch, CURLOPT_UPLOAD, 1);
curl_setopt($ch, CURLOPT_INFILE, $fp);
$response = curl_exec($ch);
$error_no = curl_errno($ch); // error number
$error = curl_error($ch); // error buffer
//var_dump(curl_error($ch));
curl_close($ch);

Upload failed via ftp.exe

I am using xampp on Windows and want to upload an file via ftp.exe.
I created a .vbs script creating an ftp.dat file and run ftp.exe.
Set FS = CreateObject("Scripting.FileSystemObject")
Set FileIn = fs.OpenTextFile("C:\xampp\htdocs\ftp.dat", 2, true)
filein.WriteLine "open"
filein.WriteLine "IP"
filein.WriteLine "USER"
filein.WriteLine "PASSWORD"
filein.WriteLine "put C:\path\to\file\file.txt"
filein.WriteLine "quit"
filein.close
Set WSHShell = WScript.CreateObject("WScript.Shell")
WshShell.run "cmd /c ftp.exe -s:C:\xampp\htdocs\ftp.dat > log"
I printed the output to a log file giving me this error:
ftp> To Connected to XXX.XXX.XXX.XXX.
open
XXX.XXX.XXX.XXX
220 hostname FTP server ready.
User (XXX.XXX.XXX.XXX:(none)):
331 Password required for USER.
230 User USER logged in.
ftp> put C:\path\to\file\file.txt
200 PORT command successful.
553 file.txt: Permission denied.
ftp> quit
221 Goodbye. You uploaded 0 bytes and downloaded 0 bytes.
When I run this .vbs file in the command line, everything works fine.
But if I run this using exec/system I get the error from above.
PHP File:
<? php
exec('C:\\xampp\htdocs\\upload.vbs');
?>

Permissions issue when using PHP system and cURL?

I'm trying to create a php script that will put a file on a remote server.
The following line will work from the CLI:
curl -u username:pass -T myfile.jpg sftp://my.domain.com/dir/
However, if I take this into PHP and try to run it through 'system' like:
<? system('curl -u username:pass -T myfile.jpg sftp://my.domain.com/dir/'); ?>
I get an error:
curl: (7) Failed to connect to xxx.xxx.xx.xx: Permission denied
Also, to complicate things, this is going from a RHEL to a Windows Server. Anyone have a suggestion or a fix?
Below is the result from the PHP cURL library.
* About to connect() to my.domain.com port 22 (#0)
* Trying xxx.xxx.xx.xx... * connected
* Connected to my.domain.com (xxx.xxx.xx.xx) port 22 (#0)
* SSH authentication methods available: publickey,password
* Initialized password authentication
* Authentication complete
* Upload failed: Permission denied (3/-31)
* Connection #0 to host my.domain.com left intact
* Closing connection #0
PHP has it's own cURL library that you should probably use. php cURL
Your problem is that system() doesn't allow curl to ask you for a password.
Consider other authentication methods.

Categories