Transfer files using SFTP and either PHP or shell/terminal script - php

Question

I need to write a script that is run as a cron job every night which transfers some report files via sftp to another server.
The report files are created every night using another cron in the format 'support_[date].csv' & 'download_[date].csv'.
I'm wondering if you had any pointers on how to do the following:
Find the 2 files created on latest [date]
Copy these files to another server using SFTP
I've tried several PHP scripts utilising the ssh2 extension, but to no avail. Is there a way to do it using a shell script? It's not something I am hugely familiar with to be honest (hence going down the PHP route initially)
This was one of my PHP scripts which didn't work:
$src = 'test.csv';
$filename = 'test.csv';
$dest = '/destination_directory_on_server/'.$filename;
$connection = ssh2_connect('example.com', 22);
ssh2_auth_password($connection, 'username', 'password');
// Create SFTP session
$sftp = ssh2_sftp($connection);
$sftpStream = fopen('ssh2.sftp://'.$sftp.$dest, 'w');
try {
if (!$sftpStream) {
throw new Exception("Could not open remote file: $dest<br>");
}
$data_to_send = file_get_contents($src);
if ($data_to_send === false) {
throw new Exception("Could not open local file: $src.<br>");
}
if (fwrite($sftpStream, $data_to_send) === false) {
throw new Exception("Could not send data from file: $src.<br>");
} else {
//Upload was successful, post-upload actions go here...
}
fclose($sftpStream);
} catch (Exception $e) {
//error_log('Exception: ' . $e->getMessage());
echo 'Exception: ' . $e->getMessage();
if($sftpStream) {fclose($sftpStream);}
}
This were the error messages I got:
Warning: fopen() [function.fopen]: URL
file-access is disabled in the server
configuration in
/path_to_script/sftp-test.php on line
17
Warning: fopen(ssh2.sftp://Resource id
3/destination_directory_on_server/test.csv)
[function.fopen]: failed to open
stream: no suitable wrapper could be
found in /path_to_script/sftp-test.php
on line 17 Exception: Could not open
remote file:
/destination_directory_on_server/test.csv

Answer 1

using the terminal to find latest date of your file, you can use ls -1tr . Then use scp (not sftp) to copy/transfer files over
example,
#!/bin/bash
latest_download=$(ls -1tr download*csv | tail -1)
latest_support=$(ls -1tr support*csv | tail -1)
scp $latest_download user#somehost.com:somedir # syntax from memory, check man page for correct syntax
scp $latest_support user#somehost.com:somedir
check the man page of scp for usage

Answer 2

Muchos kudos to ghostdog74! Managed to get this working, but with sftp.
First I managed to set up key authentication, then partly using ghostdog74's script I did this and it worked perfectly!
cd /directorywithfilesin
latest_download=$(ls -1tr download* | tail -1)
latest_support=$(ls -1tr support* | tail -1)
sftp username#example.com <<EOF
cd /dir_to_copy_to
put $latest_download
put $latest_support
EOF
Thanks!

Answer 3

Among other problems with ghostdog74's method is that it's non-portable. My recommendation would be to use phpseclib, a pure PHP SFTP implementation.

Answer 4

This will not work from PHP from your server because your php.ini has disabled remote wrappers
allow_url_fopen boolean
This option enables the URL-aware fopen wrappers that enable accessing URL object like files. Default wrappers are provided for the access of remote files using the ftp or http protocol, some extensions like zlib may register additional wrappers.
Note: This setting can only be set in php.ini due to security reasons.
However, you could simply let your cron job call a shell script that that uses sftp or rsync directly. You don't have to do this with PHP.
I'm voting to move this to ServerFault to get better support for shell scripting.

Answer 5

The answer is right there, in the error message:
Warning: fopen() [function.fopen]: URL file-access is disabled in the server configuration
means that file-access through URL wrappers is disabled in the server configuration.
Check your PHP config, especially allow_url_fopen. PHP documentation says "This setting can only be set in php.ini due to security reasons", so check it there.
See also fopen: "If PHP has decided that filename specifies a registered protocol, and that protocol is registered as a network URL, PHP will check to make sure that allow_url_fopen is enabled. If it is switched off, PHP will emit a warning and the fopen call will fail." As far as I can tell, that's exactly what is happening there.
If you can't or won't enable allow_url_fopen, you still have some options:
call sftp directly
mount a share with sshfs and then use it as a normal folder

Answer 6

Try as follows (Shell)
SFTP=<sftp path>
KEY_FILE=<your key>
USERNAME=<remote username>
SERVER =<remote server>
REMOTE_DIR=<remote location>
APP_HOME =<App location>
FILENAME=<file name>
${SFTP} -o IdentityFile=${KEY_FILE} ${USERNAME}#${SERVER} <<_COMMAND
lcd ${APP_HOME}
cd ${REMOTE_DIR}
put ${FILENAME}
bye
_COMMAND