How to setup automatic snapshots on a Linux ZFS pool

Lets say you have a Proxmox hypervisor with a VM running with an id of 100. The VM's OS resides in largepool/base-100-disk-0. To persist data in an isolated fashion, you've provided it with a different volume on a different pool: smallpool/data.

Listing current ZFS pools

# zfs list
NAME                        USED  AVAIL     REFER  MOUNTPOINT
largepool                   735G  2.29T      205K  /largepool
largepool/base-100-disk-0  7.69G  2.29T     7.69G  /largepool/base-100-disk-0
smallpool                  52.8M  2.11T      192K  /smallpool
smallpool/data             49.5M  2.11T      280K  /smallpool/data

Listing ZFS snapshots, an empty list for now...

# zfs list -t snapshot
NAME  USED AVAIL REFER MOUNTPOINT

Needs

largepool/base-100-disk-0 and smallpool/data need to be snapshotted according to different schedules. This table shows how many snapshots to keep for a given frequency.

Pool	Monthly	Daily	Hourly
largepool/base-100-disk-0	0	3	24
smallpool/data	3	31	24

Smallpool contains more sensible data that need to be kept in an offsite backup. It could also be part of an audit, so we're going to keep snapshots for 3 months. The latest 31 days will contain one snapshot each. The last 24 hours will also contain hourly backups. Any snapshot older than this schedule will be purged to save disk space.

Configuration file

By convention, zman looks for a /etc/zman/zman.yaml file to figure out this schedule. It could be configured otherwise with the --config command line argument, but we'll keep things simple and opt for the former option.

---
zfs: /sbin/zfs
pools:
- name: largepool/base-100-disk-0
  frequencies:
    - type: daily
      quantity: 3
    - type: hourly
      quantity: 24

- name: smallpool/data
  frequencies:
    - type: monthly
      quantity: 3
    - type: daily
      quantity: 31
    - type: hourly
      quantity: 24

Show zman status

Now, zman knows which pools it has to monitor. Let's issue a zman status command.

# zman status

NO ACTIVE SNAPSHOTS

NO EXPIRED SNAPSHOTS

OVERDUE POOLS

POOL                       FREQUENCY   
largepool/base-100-disk-0  daily     
largepool/base-100-disk-0  hourly
smallpool/data             monthly       
smallpool/data             daily       
smallpool/data             hourly 

As might be expected, there are no related snapshot yet. Moreover, the 5 frequencies configured in zman.yaml are reflected here in a different format as overdue pools. Time to create them all in one shot with the renew command.

Renew overdue pools

The renew command outputs all zfs commands it executes. Here, 5 snapshots have been created.

# zman renew
/sbin/zfs snapshot largepool/base-100-disk-0@zman-daily-2019-11-11-14:00
/sbin/zfs snapshot largepool/base-100-disk-0@zman-hourly-2019-11-11-14:00
/sbin/zfs snapshot smallpool/data@zman-monthly-2019-11-11-14:00
/sbin/zfs snapshot smallpool/data@zman-daily-2019-11-11-14:00
/sbin/zfs snapshot smallpool/data@zman-hourly-2019-11-11-14:00

# zman status
ACTIVE SNAPSHOTS

POOL                       FREQUENCY   CREATION DATE      EXPIRATION DATE    EXPIRE IN   NAME
largepool/base-100-disk-0  daily       2019-11-11 14:00   2019-11-12 14:00   3 days      largepool/base-100-disk-0@zman-daily-2019-11-11-14:00
largepool/base-100-disk-0  hourly      2019-11-11 14:00   2019-11-11 15:00   a day       largepool/base-100-disk-0@zman-daily-2019-11-11-14:00
smallpool/data             monthly     2019-11-11 14:00   2020-02-11 14:00   3 months    smallpool/data@zman-monthly-2019-11-11-14:00
smallpool/data             daily       2019-11-11 14:00   2019-12-12 14:00   a month     smallpool/data@zman-daily-2019-11-11-14:00
smallpool/data             hourly      2019-11-11 14:00   2020-11-12 14:00   a day       smallpool/data@zman-hourly-2019-11-11-14:00


NO EXPIRED SNAPSHOT

NO OVERDUE POOL

A quick status shows that all snapshots are up-to-date. As time goes, some will need to be renewed. For sure, both hourly snapshots will one hour later. Another call to zman renew would create two new snapshots, keeping all the previous ones. A day later, hourly snapshots and will expire. The command zman purge removes those snapshots from ZFS, liberating space on the filesystem.

Automating the whole process

Now it seems running zman renew and zman purge once per hour could do the trick of automating this whole process. On unix systems, cron is pretty well suited for that task. Let's create two cron jobs.

# crontab -e

An editor will pop up, send the following content:

0 * * * * root /usr/local/sbin/zman renew
0 * * * * root /usr/local/sbin/zman purge

Cron requires full path of executables, as it has by default no notion of path.

This will run both commands on the hour, every hour, essentially maintaining snapshots according to the schedule set in /etc/zman/zman.yaml.