Backup - epam/hubctl GitHub Wiki

Hub CLI organize stack backup as a new verb - backup, and restore from backup is performed as integral part of new stack deployment. Ie. there is no separate restore verb on deployed stack but there is backup verb. Alternatively, invoke could be used to re-apply snapshot in-place.

Actual backup and restore implementation is delegated to components. Component backup verb emits outputs that are arbitrary name = value strings. Outputs are passed to the component as parameters during deploy, then component decides - due to presence of a non-empty parameter(s) that a restore must be performed.

The output / parameter could be an RDS snapshot name, S3 object URL, small base64 encoded file, etc.

Backups are managed by command backup. There are 2 sub-commands:

  • create
  • unbundle - transform backup bundle(s) into parameters file (params.yaml)

Create

Result of the backup is a backup bundle.

hub backup create hub.yaml.elaborate [-s hub.yaml.state] \
    [-o bundle.yaml,s3://...] \
    [--json] \
    [-c component] \
    [--allow-partial] \
    [-y / --dry]

Each stack component that supports backup verb is invoked. Outputs are captured:

Outputs:

component.postgresql.rds.snapshot = friday
kind = postgresql-rds
timestamp = 2018-03-15T14:00:39.556948+02:00

kind and timestamp are optional. kind defaults to component name (in the stack). Thus, for example, in case there are two PostgreSQL component in the stack named pg1 and pg2 it is crucial to output kind so that later the backup could be matched to the true kind of the component (ie. component type - postgresql-rds) instead of just using the name.

The outputs are saved into bundle file (or printed to stdout):

version: 1
kind: backup
timestamp: 2018-03-15T14:00:39.556948+02:00
status: success
components:
  postgresql:
    timestamp: 2018-03-15T14:00:39.556948+02:00
    status: success
    kind: postgresql-rds
    outputs:
    - name: component.postgresql.rds.snapshot
      value: friday

No template processing is performed during backup. Code that relies on templates cannot assume they are processed because backup may happen from the clean source tree. Use render to create templates - parameters will be substituted from state file (if any) and stack level parameters.

status on component level is one of success or error. Additionally, partial on bundle level is allowed if --allow-partial is supplied. partial backup has at least one component status set to error, and at least one set to success.

Restore

Restore process is part of deploy verb. The component must react to non-empty parameter and perform restore as part of the deployment process.

For example, declare snapshot parameter with empty: allow in hub-component.yaml:

version: 1
kind: component
meta:
  name: postgresql-rds

parameters:
- name: component.postgresql.rds.snapshot
  empty: allow

Use the parameter in Terraform script template:

variable "snapshot_identifier" {
  default = "${component.postgresql.rds.snapshot}"
}

resource "aws_db_instance" "postgresql" {
  snapshot_identifier = "${var.snapshot_identifier}"
}

If there is no snapshot defined, then variable content is empty string - that is fine from Terraform perspective. Also, env: is fully supported and may be used in shell scripts and/or Makefile.

Bundle to parameters

Backup bundle should be converted into parameters file before use with deploy:

hub backup unbundle bundle.yaml,s3://... [bundle2.yaml ...] \
    [-o restore-params.yaml,s3://...] \
    [-r / --rename pg1:postgresql,pg2:postgresql-rds]
    [-e / --erase etcd,vault]
    [-i / --include-only postgresql,postgresql-rds]

The unbundle sub-command will unroll backup bundles into params.yaml (or to stdout):

parameters:
- name: component.postgresql
  component: postgresql
  parameters:
  - name: rds.snapshot
    value: friday

The bundles will be merged - a single backup from multiple backups for the same component name is selected based on the following priorities:

  1. Backups from a later file - as ordered on command line, take over previous files.
  2. success status on component level from an earlier file take over subsequent backup with error status and a warning is emitted.
  3. If a backup with an earlier timestamp on component level overwrites later backup (due to file ordering) then a warning is emited. We may want to fine-tune this with a command-line switch.

Component name from the bundle is placed into component: params.yaml qualifier. It is possible to rename components in the output with --rename.

To select specific components from the backup bundle (by name) use --erase or --include-only.

Templates

Templating engine has unbase64 parameter formatting option. Assuming component.vault.app-roles parameter's content is base64-encoded YAML file, it can be recreated by creating a template file with following content:

${component.vault.app-roles/unbase64}