[ALUG] The Manifold Uses Of rsync

[Titus T.]

Hello;

I thought this is worth sharing;


rsync(1) is a remarkably versatile program. At its simplest, you can use it
as a bulk-copying tool:

rsync --archive --delete --verbose <src-dir>/ <dst-dir>/

except it works equally well between machines:

rsync --archive --delete --verbose <host>:<src-dir>/ <dst-dir>/

or

rsync --archive --delete --verbose <src-dir>/ <host>:<dst-dir>/

Inter-machine transfers are done securely via SSH.

But it is more robust than either “cp -a” (for local copies) or “scp -rp”
(for remote copies), because if the transfer should be aborted part-way for
any reason (e.g. a network link going down for an inter-machine transfer),
you can resume the transfer from where it left off just by re-executing the
*exact same* rsync command. rsync automatically figures out that the
destination directory has stuff in it that doesn’t need to be copied again,
and goes from there.

This is because rsync is actually a tool for answering the question “given
an existing directory <src> and a directory <dst>, what is the minimum that
has to be done to <dst> to make it an exact copy of <src>?”. And it manages
to do this with a minimum of network traffic. Thus, for example, it can
pick up small changes to a large file *without* having to transfer the
entire file across the network to compare it with the other version. The
algorithm for doing this was invented by Andrew Tridgell (a name which may
be familiar as one of the principal SAMBA developers) and published in 1999
as part of his PhD thesis.

Thus, rsync is useful for doing full backups, and also incremental backups.
Given a directory containing a previous backup, rsync can bring it
up-to-date with the latest state of the source directory. Alternatively, if
you want to maintain snapshots of the state of that directory at previous
points in time, you can use the --copy-dest or --link-dest options, for
example

rsync --archive --delete --verbose --link-dest=<previous-backup-dir>/
<src-dir>/ <new-backup-dir>/

will make <new-backup-dir> a mirror of <src-dir>, but it will also look at
<previous-backup-dir>, and any files in there that are copies matching the
current contents of <src-dir> will be hard-linked into <new-backup-dir>,
saving the disk space of an extra copy. The relevant marketing buzzword,
should you want one, is “file-level deduping”.

If you have heard of Apple’s “Time Machine” backup utility, rsync offers
the same functionality, but in a more robust form, without requiring any
dodgy filesystem hacks.

Notice in all the examples above I use the --verbose option, so rsync tells
me about each file it touches. You can also add the --dry-run option, so
rsync will tell you what it would do, without actually doing it. Very handy
for avoiding mistakes...

The --verbose option may slow the transfer down a bit. For really large
transfers, you may want to run it without --verbose (after having tested it
will do the right thing, at least part of the way, with both --verbose and
--dry-run, perhaps!). Then when it finishes, rerun it with --verbose, just
to confirm that it reports nothing left to do.

You can also use rsync to minimize downtime when e.g. moving an
actively-used filesystem to a new disk. Run the initial rsync transfer to
the new volume while the current volume is still live; files will change
while the transfer is progress, but that’s a minor issue, you should still
get the bulk of the data. Then stop general access to the old volume while
rerunning the rsync to pick up all the changes since the previous run; this
should complete a lot quicker than the first transfer. Then you can
re-enable general access on the new volume.

To minimize impact on other users of the filesystem during the initial
transfer, use the ionice(1) command to drop rsync’s I/O priority down to
“idle”. This way it only gets to access the disk when nobody else needs to,
and its impact on the system becomes minimal.

regards,
...............................................................
Titus Turinawe-Computer Tech
*Email: ttituskayz16 at gmail.com <ttituskayz16 at gmail.com>,*
*          titus at bioscomputers.co.tz <ttitus.turinawe at habari.co.tz>*
*Phone: +255 756258434, (0)686122151*
"The more you ignore me, the closer I get" -- Morrissey
<http://true-to-you.net/>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.habari.co.tz/pipermail/linux/attachments/20141021/ce5594cf/attachment.html>