backup and restore - vincios/rpi-setup GitHub Wiki

Overview

In case of factory reset, don't forget to backup this services. And then to restore them!

Home

Don't forget to save user home directories

Backrest configuration

Backup made each day at 01:00
Retention policy:
- Daily: 3 - Keep a backup for the last 3 days that have a backup
- Weekly: 2 - Keep a backup for the last 2 weeks (the last one of each week)
  
  NB: if one of them will coincide with a daily backup, only the daily will be kept
- Monthly: 4 - Keep a backup for the last 4 months (the last one of each month)
  
  NB: if one of them will coincide with a daily or a weekly backup, only one will be kept
- Yearly: 1 - Keep a backup for the last year (the last one of the year)

{
  "id": "Home",
  "repo": "Synlogy-NetBackup",
  "paths": [
    "/home/raspi/.backup",
    "/home/raspi/.bash_aliases",
    "/home/raspi/.config",
    "/home/raspi/.ssh",
    "/home/raspi/Apps/instaloader",
    "/home/raspi/Bookshelf",
    "/home/raspi/Desktop",
    "/home/raspi/Documenti",
    "/home/raspi/Immagini",
    "/home/raspi/Modelli",
    "/home/raspi/Musica",
    "/home/raspi/Pubblici",
    "/home/raspi/Scaricati",
    "/home/raspi/Video"
  ],
  "excludes": [],
  "iexcludes": [],
  "retention": {
    "policyTimeBucketed": {
      "yearly": 0,
      "monthly": 4,
      "weekly": 2,
      "daily": 3,
      "hourly": 0
    }
  },
  "schedule": {
    "cron": "0 3 * * *"
  }
}

Home Assistant

On Home Assistant, create an automation to backup your configuration. For example, this automation will make a backup each Monday at 3AM

alias: ⚙️ [System] Backup each Monday
description: Make a configuration backup each Monday at 3AM
trigger:
  - platform: time
    at: "03:00:00"
condition:
  - condition: time
    weekday:
      - mon
action:
  - service: backup.create
    data: {}
mode: single

The compressed backups will be stored in the directory

/home/homeassistant/.homeassistant/backups

So, you only have to backup this folder.

To restore, Home Assistant core doesn't have a restore utility, so you have to manually extract the backup tar and replace the file inside the .homeassistant folder.

Backrest configuration

Backup made each Monday at 03:30
Retention policy:
- Monthly: 4 - Since the backup folder always contains all the oldest backups (except if manually removed), we just keep a snapshot for the last 4 months (the last one of each month)

{
  "id": "Home-Assistant",
  "repo": "Synlogy-NetBackup",
  "paths": [
    "/home/homeassistant/.homeassistant/backups"
  ],
  "excludes": [],
  "iexcludes": [],
  "retention": {
    "policyTimeBucketed": {
      "yearly": 0,
      "monthly": 4,
      "weekly": 0,
      "daily": 0,
      "hourly": 0
    }
  },
  "schedule": {
    "cron": "20 3 * * 1"
  }
}

Docker

Backup the docker compose configuration under the dockers folder. Exclude the immich-app and vaultwarden folders, since we will backup separately.

Backrest configuration

Backup made each Wednesday at 02:00
Retention policy:
- Daily: 3 - Keep a backup of the last 3 days with a snapshot. Since the backups are one per week, it keep the backups of the last 3 weeks.
- Monthly: 4 - Keep a backup for the last 4 months (the last one of each month)

{
  "id": "Dockers",
  "repo": "Synlogy-NetBackup",
  "paths": [
    "/home/raspi/dockers"
  ],
  "excludes": [
    "immich-app",
    "vaultwarden"
  ],
  "iexcludes": [],
  "retention": {
    "policyTimeBucketed": {
      "yearly": 0,
      "monthly": 4,
      "weekly": 0,
      "daily": 5,
      "hourly": 0
    }
  },
  "schedule": {
    "cron": "40 3 * * 3"
  }
}

Immich

To backup and restore the Immich database, follow the official wiki.

Don't backup the data folder, because it cannot be read.

Backrest configuration

Backup made every two days at 02:40
Retention policy:
- Daily: 2 - Keep a backup of the last 2 days with a snapshot.
- Weekly: 4 - Keep a backup for the last 4 weeks (the last one of each week)
- Monthly: 3 - Keep a backup for the last 3 months (the last one of each month)

Hooks: the hook executes the postgres dump

Condition: CONDITION_SNAPSHOT_START
Error Behavior: ON_ERROR_FATAL

#!/bin/bash
PLAN_DIR="{{ index .Plan.Paths 0 }}"

if [ ! -d $PLAN_DIR/data-backups ]; then
  mkdir $PLAN_DIR/data-backups 
fi

docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres > $PLAN_DIR/data-backups/immich-database.sql

{
  "id": "Immich",
  "repo": "Synlogy-NetBackup",
  "paths": [
    "/home/raspi/dockers/immich-app"
  ],
  "excludes": [
    "data"
  ],
  "iexcludes": [],
  "retention": {
    "policyTimeBucketed": {
      "yearly": 0,
      "monthly": 3,
      "weekly": 4,
      "daily": 2,
      "hourly": 0
    }
  },
  "hooks": [
    {
      "conditions": [
        "CONDITION_SNAPSHOT_START"
      ],
      "onError": "ON_ERROR_CANCEL",
      "actionCommand": {
        "command": "#!/bin/bash\nPLAN_DIR=\"{{ index .Plan.Paths 0 }}\"\n\nif [ ! -d $PLAN_DIR/data-backups ]; then\n  mkdir $PLAN_DIR/data-backups \nfi\n\ndocker exec -t immich_postgres pg_dumpall --clean --if-exists --username=postgres > $PLAN_DIR/data-backups/immich-database.sql"
      }
    }
  ],
  "schedule": {
    "cron": "40 2 */2 * *"
  }
}

Traefik

Save the traefik static/dynamic configuration.

Don't backup the data folder, because it cannot be read.

Backrest configuration

Backup made two times per month, on 1st and 16th of the month, at 02:30
Retention policy:
- Count: 5 - keep the last 5 snapshot

{
  "id": "Traefik",
  "repo": "Synlogy-NetBackup",
  "paths": [
    "/etc/traefik"
  ],
  "excludes": [
    "acme"
  ],
  "iexcludes": [],
  "schedule": {
    "cron": "30 3 1,16 * *"
  },
  "retention": {
    "policyKeepLastN": 5
  }
}

Vaultwarden

The official documentation explain what and how to backup relevant data. The default data folder is ./vw-data.

Backup

Don't backup the vw-data folder directly, the backup could be invalid if the sqlite database is in use!

This script automatize the backup process according the official documentation.

Important

This script needs sqlite3, so make sure you have it somewhere. You can set the path of the executable through the SQLITE3 environment variable.

You can just download the script, place it in the same folder of where the wv-data folder is located and run. It will create a vw-data-backup folder next to the vw-data folder containing all the file and folder needed to backup.

Optionally, you can customize the behavior setting some environment variables:

Variable	Description	Default Value	Note
`VAULTWARDEN_ROOT`	Path of the root Vaultwarden folder, where the `VAULTWARDEN_DATA_FOLDER` resides	`pwd` output	Set this variable if the `backup.sh` script is not placed in the same directory of the data folder
`VAULTWARDEN_DATA_FOLDER`	Name of the Vaultwarden data folder, as set in the Vaultwarden `DATA_LOCATION` variable	wv-data
`SQLITE3`	Path of the `sqlite3` executable	/usr/bin/sqlite3	No need to change if sqlite was installed through `apt`

Important

Don't forget to backup the vw-data-backup folder, possibly in multiple locations.

Backrest configuration

This configuration backups entire Vaultwarden folder, including the `docker-compose.yml` and the `.env` file.

⚠️ This configuration uses the backup.sh script, it must be located into the Vaultwarden root folder.

Backup made every day, at 02:30
Retention policy:
- Daily: 7 - keep (the last one) backup of the last 7 days that have a backup
- Weekly: 4 - keep (the last one) backup of the last 4 weeks that have a backup
- Monthly: 3 - keep (the last one) backup of the last 3 months that have a backup
Hooks: the hook calls the backup.sh script placed into the defined Plan path
- Condition: CONDITION_SNAPSHOT_START
- Error Behavior: ON_ERROR_FATAL
```
#!/bin/bash
PLAN_DIR="{{ index .Plan.Paths 0 }}"
VAULTWARDEN_ROOT=${PLAN_DIR}

source ${VAULTWARDEN_ROOT}/backup.sh
```

Full configuration

{
  "id": "Vaultwarden-Synlogy",
  "repo": "Synlogy-NetBackup",
  "paths": [
    "/home/raspi/dockers/vaultwarden"
  ],
  "excludes": [
    "secrets",
    "vw-data"
  ],
  "iexcludes": [],
  "retention": {
    "policyTimeBucketed": {
      "yearly": 0,
      "monthly": 3,
      "weekly": 4,
      "daily": 7,
      "hourly": 0
    }
  },
  "hooks": [
    {
      "conditions": [
        "CONDITION_SNAPSHOT_START"
      ],
      "onError": "ON_ERROR_FATAL",
      "actionCommand": {
        "command": "#!/bin/bash\nPLAN_DIR=\"{{ index .Plan.Paths 0 }}\"\nVAULTWARDEN_ROOT=${PLAN_DIR}\nsource ${VAULTWARDEN_ROOT}/backup.sh"
      }
    }
  ],
  "schedule": {
    "cron": "30 2 * * *"
  }
}

Extra Due the cruciality of these data, other two Plans backups the data off-site (Dropbox and GDrive), each two days.

The configuration is the same, but the fist backups in the odd days, the latter backups in the even days.

Backup made every 2 days, at 06:00
Retention policy:
- Count: 3 - to keep the data usage low, only last 3 backups are kept
Hooks: same as above

{
  "id": "Vaultwarden-GDrive-Vinciosdev",
  "repo": "GDrive-Vinciosdev",
  "paths": [
    "/home/raspi/dockers/vaultwarden"
  ],
  "excludes": [
    "secrets",
    "vw-data"
  ],
  "iexcludes": [],
  "hooks": [
    {
      "conditions": [
        "CONDITION_SNAPSHOT_START"
      ],
      "onError": "ON_ERROR_FATAL",
      "actionCommand": {
        "command": "#!/bin/bash\nPLAN_DIR=\"{{ index .Plan.Paths 0 }}\"\nVAULTWARDEN_ROOT=${PLAN_DIR}\nsource ${VAULTWARDEN_ROOT}/backup.sh"
      }
    }
  ],
  "schedule": {
    "cron": "0 6 */2 * *"
  },
  "retention": {
    "policyKeepLastN": 3
  }
}

{
  "id": "Vaultwarden-Dropbox",
  "repo": "Dropbox",
  "paths": [
    "/home/raspi/dockers/vaultwarden"
  ],
  "excludes": [
    "secrets",
    "vw-data"
  ],
  "iexcludes": [],
  "hooks": [
    {
      "conditions": [
        "CONDITION_SNAPSHOT_START"
      ],
      "onError": "ON_ERROR_FATAL",
      "actionCommand": {
        "command": "#!/bin/bash\nPLAN_DIR=\"{{ index .Plan.Paths 0 }}\"\nVAULTWARDEN_ROOT=${PLAN_DIR}\nsource ${VAULTWARDEN_ROOT}/backup.sh"
      }
    }
  ],
  "schedule": {
    "cron": "0 6 2-30/2 * *"
  },
  "retention": {
    "policyKeepLastN": 3
  }
}

Restore

After you have restored the vw-data-backup folder into the Vaultwarden root folder, follow these steps to restore the backup.

Stop Vaultwarden
Rename the vw-data-backup folder
```
$ mv vw-data-backup/ vw-data
```

If necessary, recreate your Docker secrets

$ mkdir secrets
$ nano secrets/your_secret.txt

Change the vw-data owner to root
```
$ sudo chown -R root:root vw-data/
```
Start Vaultwarden
If the Admin page is enabled, force the clients to refresh their data
- Users > Force clients to resync
If you have fail2ban setup, restart it to reload the logs file
```
$ sudo systemctl restart fail2ban.service
```

Backup plan schedule recap

Plan	Days	Time	Cron Expression
Vaultwarden-Synlogy	every day	02:30	30 2 * * *
Immich	every 2 days	02:40	40 2 * * */2
Home	every day	03:00	0 3 * * *
Home-Assistant	every Monday	03:20	20 3 * * 1
Traefik	1st and 16th day of the month	03:30	30 3 1,16 * *
Dockers	each Wednesday	03:40	40 3 * * 3
Vaultwarden-Dropbox	every odd days	06:00	0 6 2-30/2 * *
Vaultwarden-GDrive-Vinciosdev	every even days	06:00	0 6 /2 *

Restic Hooks

This hooks sends some Telegram notification (via Shoutrrr).

Repository Synlogy-NetBackup

CONDITION_SNAPSHOT_START

🗄️ <b>Ho avviato il backup di <code>{{ .Plan.Id }}</code></b>.

Ti avviso in caso di problemi.

CONDITION_SNAPSHOT_END

{{ if .Error -}}
🟥 <b>Si è verificato un problema mentre creavo lo snapshot per <code>{{ .Plan.Id }}</code></b>.

Cerco di capirci qualcosa e ti aggiorno...
{{ else -}}
✅ <b>Lo snapshot per <code>{{ .Plan.Id }}</code> è stato creato con successo!</b>
{{ if .SnapshotStats -}}
Ci sono voluti appena {{ .SnapshotStats.TotalDuration }} secondi 😎

<b>Dati aggiunti</b>: {{ .FormatSizeBytes .SnapshotStats.DataAdded }}
<b>File processati in totale</b>: {{ .SnapshotStats.TotalFilesProcessed }}
<b>Bytes processati in totale</b>: {{ .FormatSizeBytes .SnapshotStats.TotalBytesProcessed }}
{{ end }}
{{ end }}

CONDITION_SNAPSHOT_ERROR

🟥 <b>Si è verificato un problema nella creazione dello snapshot per <code>{{ .Plan.Id }}</code></b>.

Dunque, durante la creazione dello snapshot si è verificato questo errore:

<blockquote>{{ .Error }}</blockquote>

Dovresti dargli un'occhiata.

CONDITION_SNAPSHOT_WARNING

🟨 <b>Si è verificato un problema nella creazione dello snapshot per <code>{{ .Plan.Id }}</code></b>.

Dunque, durante la creazione dello snapshot si è verificato questo errore:

<blockquote>{{ .Error }}</blockquote>

Non preoccuparti però, lo snapshot è stato creato lo stesso.

CONDITION_CHECK_START

🩺 <b>Verifico lo stato del repository <code>{{ .Repo.Id }}</code></b>.
Ti avviso in caso di problemi.

CONDITION_CHECK_ERROR

🟥 🩺 <b>C'è un problema nel repository {{ .Repo.Id }}</b>

{{ if .Error -}}
<blockquote>{{ .Error }}</blockquote>
{{end}}

Dovresti dargli un'occhiata.

[DEPRECATED] Backup utility

You can use this utility to schedule the backup of your Raspberry folders.

RSYNC backup script

Just clone the repository and follow the Readme to create the backup tasks.

tasks.txt

This tasks.txt file is a good starting point to backup the services listed in this wiki.

⚠️ Make sure to adjust it with your paths!

# Backup Home Assistant
-> /home/homeassistant/.homeassistant/
@ /media/qnas/Vincenzo/Raspberry/Backups/homeassistant/
+backups/***
-*
<-

# Backup all the docker configurations
-> /home/raspi/dockers/
@ /media/qnas/Vincenzo/Raspberry/Backups/dockers/
<-

# Backup the rest of the raspi folder
-> /home/raspi/
@ /media/qnas/Vincenzo/Raspberry/Backups/raspi/
+Apps/
+Apps/instaloader/***
+.backup/***
+.bash_aliases
+Bookshelf/***
+Desktop/***
+Documenti/***
+Immagini/***
+Modelli/***
+Musica/***
+Pubblici/***
+Scaricati/***
+Video/***
-*
<-

backup and restore - vincios/rpi-setup GitHub Wiki

Overview

Home

Home Assistant

Docker

Immich

Traefik

Vaultwarden

Backup

Restore

Backup plan schedule recap

Restic Hooks

Repository Synlogy-NetBackup

[DEPRECATED] Backup utility

tasks.txt

⚠️ **GitHub.com Fallback** ⚠️

⚠️ GitHub.com Fallback ⚠️