DFT Wiki

Both SCP and RSYNC, use SSH Protocol as a way to transfer files and directories.

While SCP just uploads or downloads files and folders, Rsync can be used to synchronize the content of both folders copying only what is new.

The basic Syntax of both looks like this:

[scp/rsync] [OPTION] … [SRC] … [USER@]HOST:DEST
[scp/rsync] [OPTION] … [USER@]HOST:SRC [DEST]

SCP

scp /home/user/file.zip [email protected]:/home/user/file.zip
scp file.zip [email protected]:/home/user
scp file.zip 200.100.50.3:/home/user
scp file.zip [email protected]:
scp file.zip 200.100.50.3:

scp [email protected]:/home/user/file.zip /home/user/file.zip
scp [email protected]:/home/user/file.zip /home/user
scp [email protected]:/home/user/file.zip .
scp 200.100.50.3:file.zip /home/user
scp 200.100.50.3:file.zip .

The examples above are multiple ways of doing basically upload (1 to 5) and download (6 to 10).

The first of each block in bold show the complete syntax except if you need to apply any additional option.

By omitting the name of the file at the destiny location it will copy with the same name.

By omitting the full path on the origin means the current directory.

By omitting the full path at the destiny means to save at the home directory of the user.

By omitting the username means you will use the same user remotely as you are logged in locally.

Use the option -r to copy recursively entire directories, -v  (verbose) to print debugging messages, and -P (same as –progress) to see real-time progress.

RSYNC

The big feature of RSYNC is the ability to check the dates to define which is the most recent version of the file and compare the content of both directories to understand what is missing on both sides.

rsync -r /directory [email protected]:/directory
rsync -r [email protected]:/directory /directory

The usage option -r (recursively) will copy all the content but option -a (archive) would only copy after comparing both directories (local and remote), and copy what is missing or update files with a new version.

Option -b (backup) would not overwrite the old with the new version but would make a backup copy of the old version first (file.zip~), and ‐‐delete would delete files at the destination that were deleted from the source.

Using option -z (compress) the traffic will be compressed, saving time in slow connections but increasing CPU usage on both ends. The option ‐‐dry-run is going to show everything like if it was going to do but in fact will not perform anything because it is only informative.

The attribute –exclude ‘dir1’ will prevent the synchronization to compare and copy the content of a file or directory.

You can also try the option ‐‐backup-dir=$(date +%F) and see what it does.

---

BONUS

There is a tool called LSYNCD that runs as a service and periodically compare and sync files (using RSYNC) according to the provided configuration.

sudo apt install lsyncd -y
sudo mkdir /etc/lsyncd
sudo nano /etc/lsyncd/lsyncd.conf.lua

Add the following content customizing the parameters in bold:

settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status",
statusInterval = 20,
nodaemon   = false
}

sync {
default.rsync,
source = "/path/",
target = "/path"
}

Note: the interval (20) is in seconds.

systemctl start lsyncd
systemctl enable lsyncd
systemctl status lsyncd

For troubleshooting:

tail -f /var/log/lsyncd/lsyncd.log
tail -f /var/log/lsyncd/lsyncd.status

---

Backing up additional or newer files from source to destination.

rsync -rlptgo /source /destination