PHP LFTP data mirror output - php

Question

I'm using a Linux local computer and need to backup/mirror some very large file structures regularly. I only have access to SFTP.
I was after a simple one click solution. I originally tried to write the little script in BASH but I've never used it before and am not up to scratch with the syntax so I resorted to PHP. (I do understand PHP is not designed for this kind of work, but I'm on a tight time scale and don't have the time to get into BASH atm)
<?php
//init
parse_str(implode('&', array_slice($argv, 1)), $_GET);
$error = array();
$lPrefix = '/home/hozza/Sites/';
$archiveLocation = '/home/hozza/Backups/';
$lDir = isset($_GET['l']) ? $_GET['l'] : $error[] = 'Local Directory Required';
$rDir = isset($_GET['r']) ? $_GET['r'] : $error[] = 'Remote Directory Required';
$bookmark = isset($_GET['b']) ? $_GET['b'] : $error[] = 'lftp Bookmark Required';
//Check for args
if(count($error) == 0) {
$archiveName = end(explode('/', $lDir)) . '_' . date('Y-m-d_H-i');
//Validate local dir
if(is_dir($lPrefix . $lDir)) {
//preserve Sublime Text 2 config SFTP files
$ST2_SFTP_conf = false;
if(file_exists($lPrefix . $lDir . '/sftp-config.json')) {
$ST2_SFTP_conf = file_get_contents($lPrefix . $lDir . '/sftp-config.json');
unlink($lPrefix . $lDir . '/sftp-config.json');
}
//Start mirror
$lftOutput = explode("\n", shell_exec('lftp -e "mirror -e -p --parallel=10 --log=' . $archiveLocation . 'logs/' . $archiveName . '.txt ' . $rDir . '/ ' . $lPrefix . $lDir . '/; exit top" ' . $bookmark));
//Tar regardless of lftp error or success
$tarOutput = shell_exec('cd ' . $lPrefix . ' && tar -czf ' . $archiveLocation . $archiveName . '.tar.gz ' . $lDir);
//Output completion or errors
shell_exec('notify-send -i gnome-network-properties -t 0 "Mirror & Archive Complete" "' . $archiveName . '\n\n' . implode('\n', $lftOutput) . $tarOutput . '"');
//put back ST2 SFTP conf
if($ST2_SFTP_conf != false) file_put_contents($lPrefix . $lDir . '/sftp-config.json', $ST2_SFTP_conf);
exit;
}
else shell_exec('notify-send -i error -t 0 "Mirror & Archive Error" "' . date('Y-m-d') . ' ' . date('H-i') . '\n' . $lDir . ' \n Does not exist! D:"');
}
else shell_exec('notify-send -i error -t 0 "Mirror & Archive Error" "' . date('Y-m-d') . ' ' . date('H-i') . '\n' . implode('\n', $error) . '"');
?>
It can be run for many sites via a short-cut like so...
terminator -T "Mirror & Archive" -e "php ~/Programs/mirror.php l=local-dir_path r=./ b=lftp-bookmark-name"
If no password is in the LFTP bookmark (there shouldn’t be as it's stored in plain text) the terminal prompts for a password, after the script has run, a nice notification is given with some info about files/folders/speed etc.
However, when the script is running in a terminal, only the "input password" bit is output to the terminal, I would like all the output displayed in the terminal (normally that would display what file/folder is currently working with etc.)
Anyone know how to do that?

Answer 1

IIRC the reason that you see the password prompt output to the terminal is that it is using stderr. You could try redirecting stdout to stderr for your commands which should show you the 'real-time' progress. Tack this on to the end of the shell_exec() command: 1>&2
ie:
shell_exec('lftp -e "mirror -e -p --parallel=10 --log=' . $archiveLocation . 'logs/' . $archiveName . '.txt ' . $rDir . '/ ' . $lPrefix . $lDir . '/; exit top" ' . $bookmark . ' 1>&2')
However, this will preclude you from having anything returned by shell_exec for logging purposes. What I would suggest is something like:
$log_stem = '/tmp/' . time() . '_'; // ie: /tmp/1357581737_
$lfOutfile = $log_stem . 'lftp.log';
$tarOutfile = $log_stem . 'tar.log';
shell_exec('lftp -blah | tee ' . $lfOutfile ' 1>&2' );
shell_exec('tar -blah | tee ' . $tarOutfile ' 1>&2' );
$lfOut = file_get_contents($lfOutfile);
$tarOut = file_get_contetns(tarOutfile);
// remove tmp files
unlink($lfOutfile);
unlink($tarOutfile);
Which will capture a copy of the output to a file before redirecting the output to stderr so you can watch it live.
However, if you want to run this via cron I would recommend against writing anything to stderr that is not an error, otherwise cron will send a warning email every time it is run.

Answer 2

I think the last answer was close:
Either this:
shell_exec('lftp -blah |& tee ' . $lfOutfile );
shell_exec('tar -blah |& tee ' . $tarOutfile );
Or if that still doesn't work try this:
shell_exec('lftp -blah 2>&1 | tee ' . $lfOutfile );
shell_exec('tar -blah 2>&1 | tee ' . $tarOutfile );