Deploying - dsblank/gramps-connect GitHub Wiki

๐ŸŒ Also available in: Deutsch ยท Espaรฑol ยท Franรงais ยท ็ฎ€ไฝ“ไธญๆ–‡

Deploying

deploy/ is the real multi-user shape of Gramps Connect: a containerized app/ frontend + gramps-web-api backend, backed by real Postgres (via the SharedPostgreSQL addon), fronted by Caddy for TLS, meant to actually be hosted somewhere โ€” real secrets, a real domain/certificate, and multiple users each with their own login. It's also the only way to see live collaboration in action; the standalone desktop build is single-user by design, so there's no one else's edits to watch appear.

The backend is the official, unmodified dmstraub/gramps-webapi image โ€” the same one gramps-project/gramps-web-api's own CI publishes on every release, and the one gramps-project/gramps-web itself builds on โ€” not a from-source build this repo maintains. That keeps this deployment a plain, standard gramps-web-api instance any compatible client can talk to, not just Gramps Connect's own frontend. SharedPostgreSQL/PostgreSQL/ FilterRules/JSON addons, multi-tree support, and compiled translations all come from that image already; the only thing deploy/Dockerfile adds is app/'s frontend, layered on top as static files. The trade-off: that upstream image is built on gramps-web-base (~4.3GB โ€” torch, sentence-transformers, opencv, 45 tesseract language packs) with AI extras installed unconditionally, since there's no official slim variant to pull instead.

Frontend and backend share one container/origin by default (gramps_webapi serves the built SPA via STATIC_PATH and handles /api/* in the same process), so Gramps Connect itself needs no CORS configuration โ€” that's only relevant for a different, separately-hosted frontend calling in (set GRAMPSWEB_CORS_ORIGINS in deploy/.env for that case).

Services: caddy (TLS termination, the only published entrypoint), app (gunicorn, serves the frontend + /api/*), worker (Celery, runs import/media/search-reindex jobs), postgres (tree data via SharedPostgreSQL), redis (Celery broker).

Running it locally

cp deploy/.env.example deploy/.env    # then edit it -- see the file's comments
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d --build

Then visit https://localhost (Caddy issues itself a local, self-signed certificate automatically โ€” accept the browser warning).

Running it on a real host

Build once on GitHub's runners instead of on the host (gh workflow run build-docker.yml, or trigger it from the Actions tab), which pushes ghcr.io/<owner>/gramps-connect:latest, then on the host:

cp deploy/.env.example deploy/.env    # real secrets/domain this time
docker compose -f deploy/docker-compose.yml --env-file deploy/.env pull
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d

Point a domain at the host and edit deploy/Caddyfile (see TLS below) to get a real certificate instead of the self-signed one.

Example: hosting on a DigitalOcean droplet

A walkthrough from an empty account to a live site on a small cloud server (DigitalOcean calls these "droplets"). Any VPS provider works the same way once you have a Linux server with Docker on it.

1. Create the droplet

In the DigitalOcean console, Create โ†’ Droplets:

  • Image: Marketplace โ†’ Docker on Ubuntu (Docker preinstalled).
  • Size: Basic, 1 vCPU / 2 GB RAM (s-1vcpu-2gb) is the smallest that works well. The backend image is large (~4 GB, see above), so don't go smaller.
  • Region: whichever is closest to your users.
  • Authentication: an SSH key (safer than a password).

Then Networking โ†’ Firewalls โ†’ Create Firewall: allow inbound 22, 80, and 443 only, and apply it to the droplet.

2. Add swap

2 GB of RAM is tight during imports and startup, so add a 2 GB swap file (SSH in as root):

fallocate -l 2G /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstab

3. Point a domain at it

Optional but recommended, since it's what gets you a real HTTPS certificate. A free DuckDNS name works, as does an A record on a domain you own. Either way, point it at the droplet's public IP.

4. Install and start Gramps Connect

git clone https://github.com/dsblank/gramps-connect /opt/gramps-connect
cd /opt/gramps-connect/deploy
cp .env.example .env
# edit .env: GRAMPSWEB_ADMIN_USER / GRAMPSWEB_ADMIN_PASSWORD,
# PUBLIC_URL=https://your.domain, and the email settings (see below)
docker compose pull
docker compose up -d

This pulls the prebuilt, public ghcr.io/dsblank/gramps-connect:latest image, so nothing is built on the droplet itself.

5. Turn on HTTPS

Edit deploy/Caddyfile as described under TLS: replace the :443 { tls internal ... } block with your.domain { reverse_proxy app:5000 } and delete the :80 block, then docker compose up -d again. Caddy requests and renews a Let's Encrypt certificate on its own.

6. Create the first tree

Follow First-time setup and Importing data below, using https://your.domain in place of https://localhost.

Email (password resets, invitations)

  • New DigitalOcean accounts block outbound SMTP. Use a relay on port 587 (e.g. Brevo, SendGrid, Mailgun) and, if mail still doesn't go out, open a DigitalOcean support ticket (category Networking) to have the block lifted.
  • GRAMPSWEB_EMAIL_USE_STARTTLS must be lowercase true. True is silently read as a string and the connection fails.
  • Brevo's SMTP login is a generated address (like [email protected]), not the email address you signed up with.

Updating later

See Updating a hosted server below.

Updating a hosted server

These steps apply to any host running the deploy/ stack, a droplet or otherwise. Commands assume the repo is cloned at /opt/gramps-connect and that you're in its deploy/ directory.

1. Back up

cd /opt/gramps-connect/deploy
# Tree data (Postgres)
docker compose exec -T postgres sh -c 'pg_dumpall -U "$POSTGRES_USER"' > backup-$(date +%F).sql
# User accounts (sqlite) and media files
docker compose cp app:/app/users  ./backup-users-$(date +%F)
docker compose cp app:/app/media  ./backup-media-$(date +%F)

Updates can run database migrations on startup that don't run backwards, so a backup is the only way to fully undo one.

2. Make sure the new image is built

The server pulls ghcr.io/dsblank/gramps-connect:latest, which only changes when the build-docker.yml workflow runs. Right after a release is tagged, check that the Docker build has finished (the repo's Actions tab) before updating, or you'll pull the previous version.

3. Update

cd /opt/gramps-connect && git pull       # updated deploy/ config
cd deploy
docker compose pull app                  # the new image
docker compose up -d                     # recreate what changed

Only app and worker are recreated. postgres, redis, and caddy keep running, and all data stays in Docker volumes. Never add -v to docker compose down unless you mean to delete everything (see Notes).

If git pull refuses because of local changes: a real-domain host has an edited deploy/Caddyfile (see TLS) that is never committed, so an update that changes the repo's copy conflicts with it. Keep your edits with:

git stash && git pull && git stash pop    # then fix any conflict in deploy/Caddyfile

deploy/.env is gitignored, so it's never affected.

4. Check for new settings

diff .env.example .env

Copy any newly added settings from .env.example into .env, then run docker compose up -d again if you changed anything.

5. Verify

docker compose ps                  # app and worker recently created, "Up"
docker compose logs --tail 50 app  # gunicorn "Booting worker" lines, no tracebacks

Then load the site and log in. One Gtk-CRITICAL ... gtk_icon_theme_get_for_screen line per gunicorn worker is normal and harmless: the server has GTK installed but no display.

Rolling back

Every image is also tagged with the commit it was built from, ghcr.io/dsblank/gramps-connect:<commit-sha> (see the repo's Packages page for the list). To go back, change the image: line under x-app-image at the top of deploy/docker-compose.yml to that tag, then:

docker compose up -d

If the newer version migrated the database, also restore the backup from step 1.

Running your own fork

A fork publishes its own image. Run build-docker.yml in your fork (pushes ghcr.io/<your-user>/gramps-connect) and set the image: line above to it, or build on the server with docker compose up -d --build. Building on the server needs more disk and memory than the 2 GB droplet above, since the base image alone is around 4 GB.

Docker commands

All commands below assume you're in the repo root. docker compose is the plugin form; if your Docker install only has the standalone binary, use docker-compose instead (same flags either way).

# Status of all five services
docker compose -f deploy/docker-compose.yml ps

# Logs (add -f to follow, --tail=100 to limit)
docker compose -f deploy/docker-compose.yml logs app
docker compose -f deploy/docker-compose.yml logs worker

# Restart one service (e.g. after an env change in deploy/.env)
docker compose -f deploy/docker-compose.yml up -d
# ^ recreates any service whose config (image, env, volumes) changed;
#   add --build first if you edited deploy/Dockerfile or app/backend source.

# Stop everything, keep data (volumes survive)
docker compose -f deploy/docker-compose.yml down

# Stop and delete all data (careful -- drops Postgres, media, users, etc.)
docker compose -f deploy/docker-compose.yml down -v

# Shell into a running container
docker compose -f deploy/docker-compose.yml exec app sh

Login credentials: this deploy vs. the standalone build

This docker deploy has no default password โ€” you set GRAMPSWEB_ADMIN_USER/GRAMPSWEB_ADMIN_PASSWORD yourself in deploy/.env before first boot, and that's what the entrypoint seeds. Nothing generates or prints a password for you.

That's different from gramps-connect-desktop (the single-user PyInstaller demo build, unrelated to this docker deploy): it always seeds a fixed admin/admin account, which is fine there since it's a throwaway local demo, not something exposed on a real server.

First-time setup: the seeded admin

On first boot (first time the app-users/app-db volumes are empty), the entrypoint:

  • generates and persists a Flask secret key if GRAMPSWEB_SECRET_KEY was left blank
  • runs user-database migrations
  • seeds a site admin user (treeless, role 5) from GRAMPSWEB_ADMIN_USER / GRAMPSWEB_ADMIN_PASSWORD (both required in deploy/.env โ€” no admin/admin fallback)

A treeless site admin can create/list/delete trees via the API, but โ€” since app/'s frontend has no tree-selection UI and always expects the logged-in user's JWT to already carry a tree โ€” cannot browse a tree's data until assigned to one. There's no CLI command for tree creation in multi-tree mode, so create the first tree via the API (one-time):

TOKEN=$(curl -sk -X POST https://localhost/api/token/ \
  -H 'Content-Type: application/json' \
  -d '{"username":"<admin>","password":"<admin password>"}' \
  | python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])")

TREE_ID=$(curl -sk -X POST https://localhost/api/trees/ \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d '{"name":"My Family Tree"}' \
  | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")

echo "$TREE_ID"

Then either assign the site admin itself to that tree, or (recommended โ€” keeps site administration and tree ownership separate) create a dedicated tree-scoped user:

# Option A: assign the existing site admin to the tree.
curl -sk -X PUT "https://localhost/api/users/<admin>/" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "{\"tree\":\"$TREE_ID\"}"

# Option B: create a separate regular user tied to the tree instead.
# email and full_name are required by the schema even if unused.
curl -sk -X POST "https://localhost/api/users/<username>/" \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -d "{\"email\":\"<email>\",\"full_name\":\"<full name>\",\"password\":\"<password>\",\"role\":4,\"tree\":\"$TREE_ID\"}"

role is an integer (gramps_webapi.auth.const):

Role Value Notes
ADMIN 5 Site admin; only role allowed to be treeless
OWNER 4 Full control of their tree
EDITOR 3 Can edit tree data
CONTRIBUTOR 2 Can add data
MEMBER 1 Read access
GUEST 0 Minimal read access

Then log in at https://localhost/ as whichever user you just created โ€” existing tokens issued before a tree assignment won't carry it, so log out/in again if you reused an already-open session.

Importing data (e.g. example.gramps, with media)

TOKEN=$(curl -sk -X POST https://localhost/api/token/ \
  -H 'Content-Type: application/json' \
  -d '{"username":"<owner>","password":"<password>"}' \
  | python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])")

# 1. Import the .gramps XML file. IMPORTANT: this endpoint reads the raw
#    request body and writes it straight to disk -- it does NOT parse
#    multipart/form-data, so use --data-binary, not curl's -F/--form (a
#    multipart upload silently corrupts the file with boundary/header
#    bytes, and Gramps' importer then fails with a generic, unhelpful
#    "Import failed" and no further detail).
curl -sk -X POST https://localhost/api/importers/gramps/file \
  -H "Authorization: Bearer $TOKEN" --data-binary @example.gramps

# Response is a task handle (import runs on the `worker` service via
# Celery) -- poll it until "state":"SUCCESS" (or "FAILURE"):
curl -sk https://localhost/api/tasks/<task_id> -H "Authorization: Bearer $TOKEN"

# 2. Media: zip up the referenced media files (any subset/superset is fine
#    -- files are matched by checksum, unreferenced ones are ignored) and
#    upload the archive, again as a raw body:
curl -sk -X POST https://localhost/api/media/archive/upload/zip \
  -H "Authorization: Bearer $TOKEN" --data-binary @media.zip

# 3. Verify: object counts for the logged-in user's tree.
curl -sk https://localhost/api/metadata/ -H "Authorization: Bearer $TOKEN" \
  | python3 -m json.tool

Known gotcha: with GRAMPSWEB_MEDIA_PREFIX_TREE=True (set by default in this compose file), media is stored under MEDIA_BASE_DIR/<tree_id>/, but nothing creates that per-tree subdirectory automatically โ€” neither POST /api/trees/ nor the media-archive-upload task. Uploading media for a tree fails with Directory /app/media/<tree_id> does not exist until it's created once:

docker compose -f deploy/docker-compose.yml exec app mkdir -p /app/media/<tree_id>

Large media uploads timing out: both the single-file (POST /api/media/) and ZIP-archive (POST /api/media/archive/upload/zip) upload endpoints read the request body synchronously inside app's gunicorn worker before handing off to Celery, so a large file or a slow connection can outlast gunicorn's per-request timeout โ€” the browser then sees a bare Internal Server Error with no JSON body (from gunicorn/Caddy, not from Gramps). deploy/docker-compose.yml sets GUNICORN_TIMEOUT to 600s by default; raise it further via GUNICORN_TIMEOUT in deploy/.env (see that file's comments) if uploads still fail.

TLS

caddy is the only service with published ports (80/443) and terminates TLS in front of app. With no domain configured, it generates and persists its own local CA and issues a self-signed certificate from it automatically (deploy/Caddyfile's tls internal) โ€” browsers will show a trust warning the first time; accept it to proceed (or add the generated CA to your system/browser trust store if you want the warning gone without a real domain). Port 80 redirects to 443.

Once a real domain points at this server, edit deploy/Caddyfile: replace the :443 { tls internal ... } block with example.com { reverse_proxy app:5000 } and delete the :80 block โ€” Caddy handles ACME issuance, renewal, and the port-80 redirect automatically for a real domain, no tls directive needed. Also update PUBLIC_URL in deploy/.env to the real https:// domain, then docker compose -f deploy/docker-compose.yml up -d to pick up both changes.

Running gramps-web alongside Gramps Connect

Not wired up by default (no grampsweb service in docker-compose.yml), but worth knowing: since the backend is a plain, unmodified gramps-web-api instance, gramps-project/gramps-web (the other official Gramps frontend) can run against this same backend too โ€” same data, same trees/users, just a different UI on a different port.

gramps-web publishes ghcr.io/gramps-project/grampsjs:latest for exactly this: nginx serving its static build only, no backend baked in. Its nginx config reverse-proxies /api to an API_HOST env var at container startup, so from the browser's perspective it's same-origin โ€” no GRAMPSWEB_CORS_ORIGINS needed for this path specifically (that setting is for a gramps-web instance hosted somewhere else entirely, calling in cross-origin instead of through this proxy).

To add it, a docker-compose.yml service along these lines:

grampsweb:
  image: ghcr.io/gramps-project/grampsjs:latest
  environment:
    API_HOST: http://app:5000
    # Docker's embedded DNS resolver -- default.conf.template's `resolver`
    # directive requires this to be set explicitly.
    NAME_SERVER: 127.0.0.11
  depends_on:
    - app
  restart: unless-stopped

published on its own port (either directly, e.g. ports: ["8081:80"], or fronted by Caddy with a second :8443 { reverse_proxy grampsweb:80 }-style block in deploy/Caddyfile for TLS to match the app service).

Notes

  • Data persists in named Docker volumes (app-db, app-media, app-indexdir, app-users, app-secret, app-cache, app-tmp, postgres-data, caddy-data, caddy-config). docker compose down (without -v) keeps them; docker compose down -v deletes everything (including the generated CA โ€” that's a new browser trust warning next boot, not just data loss).
  • redis (Celery broker/result backend) is required for background tasks (search indexing, large import/export jobs) โ€” not optional. So is the worker service: import, media-archive-upload, and search reindexing all dispatch through Celery once GRAMPSWEB_CELERY_CONFIG__* is set, and sit forever as an unfulfilled task without something consuming the queue. app-cache (/app/cache) must be a volume shared between app and worker for the same reason: the app container's request handler writes the uploaded file there, then the worker container's Celery task reads it back.
  • The worker service runs with --pool=solo (no forking), out of caution around Celery's default prefork pool forking a process that has already touched real PyGObject/GTK state (Gramps' gi.repository.GLib import). A single worker instance doesn't need the concurrency --pool=solo gives up, but this is worth revisiting if worker throughput ever becomes a bottleneck.
  • app/'s browser-side local cache (sql.js, OPFS) is keyed by a fixed filename per view, not by backend URL or tree ID โ€” see Architecture for the gotcha this causes and how to clear it.
  • .github/workflows/build-docker.yml (manual trigger โ€” gh workflow run build-docker.yml, or the Actions tab) builds deploy/Dockerfile and pushes ghcr.io/<owner>/gramps-connect:latest. docker compose -f deploy/docker-compose.yml --env-file deploy/.env pull grabs that instead of building locally; up -d --build remains there for local iteration on the Dockerfile itself.
โš ๏ธ **GitHub.com Fallback** โš ๏ธ