rsync ‐ #files #backups - five4nets/Linux-Knowledgebase GitHub Wiki

Tutorial: Using the rsync Command in Linux

rsync (remote sync) is a powerful and versatile command-line tool in Linux for synchronizing files and directories between two locations, either locally or remotely. It is efficient, supports incremental backups, and preserves file permissions, timestamps, and more. This tutorial explains rsync, its common options, and provides practical examples.

Table of Contents

What is rsync?

rsync is a fast and efficient utility that synchronizes files and directories from one location to another while minimizing data transfer by copying only the differences. It can operate locally (between directories on the same machine) or remotely (over a network using SSH or rsync daemon).

Key features:

  • Efficient delta-transfer algorithm (only changed parts of files are transferred).
  • Supports copying file permissions, ownership, timestamps, and symbolic links.
  • Can compress data during transfer.
  • Widely used for backups, mirroring, and file transfers.

Basic Syntax

rsync [options] source destination
  • source: The file or directory to copy from.
  • destination: The location to copy to (local path or remote host).
  • options: Flags to customize rsync behavior (e.g., -a, -v, -z).

Common rsync Options

Here are some frequently used options:

  • -a (archive): Enables archive mode, preserving symbolic links, permissions, timestamps, ownership, and recursive copying.
  • -v (verbose): Displays detailed output of the sync process.
  • -r (recursive): Copies directories recursively.
  • -z (compress): Compresses file data during transfer to save bandwidth.
  • --progress: Shows progress during file transfer.
  • -u (update): Skips files that are newer on the destination.
  • -t (times): Preserves modification times.
  • --delete: Deletes files in the destination that no longer exist in the source.
  • --exclude: Excludes specific files or directories from the sync.
  • -e: Specifies the remote shell (e.g., -e "ssh" for SSH).

Examples of rsync Usage

Example 1: Local File Synchronization

Copy a single file from one directory to another locally.

rsync -v /home/user/document.txt /backup/
  • Explanation: Copies document.txt from /home/user/ to /backup/. The -v flag shows the transfer details.
  • Output: Displays the file being copied and a summary.

Example 2: Synchronizing Directories

Synchronize an entire directory, including subdirectories, while preserving permissions and timestamps.

rsync -av /home/user/projects/ /backup/projects/
  • Explanation: The -a flag ensures archive mode (preserving permissions, timestamps, etc.), and -v provides verbose output. The trailing slash (/) on projects/ ensures the directory contents are copied, not the directory itself.
  • Note: Without the trailing slash, the projects directory itself would be copied into /backup/projects/projects.

Example 3: Remote File Transfer

Copy a directory to a remote server over SSH.

rsync -avz -e "ssh" /home/user/projects/ user@remote-server:/var/www/projects/
  • Explanation: Transfers the projects directory to a remote server at user@remote-server. The -z flag compresses data, and -e "ssh" specifies SSH as the transport protocol.
  • Prerequisite: SSH access must be configured (e.g., SSH keys or password).

Example 4: Incremental Backup with Compression

Create an incremental backup of a directory with compression.

rsync -avz --progress --delete /home/user/data/ /mnt/backup/data/
  • Explanation: Backs up the data directory to an external drive at /mnt/backup/data/. The --progress flag shows transfer progress, --delete removes files in the destination not present in the source, and -z compresses data.
  • Use Case: Ideal for regular backups to external storage.

Example 5: Excluding Files or Directories

Sync a directory but exclude specific files or directories.

rsync -av --exclude 'logs/' --exclude '*.log' /home/user/app/ /backup/app/
  • Explanation: Copies the app directory but excludes the logs subdirectory and any .log files. The --exclude flag can take patterns or specific paths.
  • Use Case: Useful when you want to skip temporary or large files.

Example 6: Dry Run

Test an rsync command without making changes.

rsync -av --dry-run /home/user/projects/ /backup/projects/
  • Explanation: The --dry-run flag simulates the sync process, showing what would happen without actually copying files. Useful for testing complex commands.
  • Output: Lists files that would be transferred.

Best Practices

  1. Test with --dry-run: Always use --dry-run to preview changes, especially with --delete.
  2. Use Archive Mode (-a): For most use cases, -a ensures files are copied with their metadata intact.
  3. Secure Remote Transfers: Use -e "ssh" for secure transfers over untrusted networks.
  4. Exclude Unnecessary Files: Use --exclude to skip temporary or irrelevant files.
  5. Monitor Progress: Use --progress for large transfers to track status.
  6. Verify SSH Access: For remote sync, ensure SSH keys are set up to avoid password prompts.
  7. Backup Before Using --delete: The --delete option can remove files, so ensure you have a backup.

References