Deploying - dsblank/gramps-connect GitHub Wiki
๐ Also available in: Deutsch ยท Espaรฑol ยท Franรงais ยท ็ฎไฝไธญๆ
deploy/ is the real multi-user shape of Gramps Connect: a containerized
app/ frontend + gramps-web-api backend, backed by real
Postgres (via the SharedPostgreSQL addon), fronted by Caddy for TLS,
meant to actually be hosted somewhere โ real secrets, a real
domain/certificate, and multiple users each with their own login. It's
also the only way to see live collaboration in action; the
standalone desktop build is single-user by design, so
there's no one else's edits to watch appear.
The backend is the official, unmodified dmstraub/gramps-webapi image โ
the same one gramps-project/gramps-web-api's own CI publishes on every
release, and the one gramps-project/gramps-web itself builds on โ not a
from-source build this repo maintains. That keeps this deployment a plain,
standard gramps-web-api instance any compatible client can talk to, not
just Gramps Connect's own frontend. SharedPostgreSQL/PostgreSQL/
FilterRules/JSON addons, multi-tree support, and compiled translations all
come from that image already; the only thing deploy/Dockerfile adds is
app/'s frontend, layered on top as static files. The trade-off: that
upstream image is built on gramps-web-base (~4.3GB โ torch,
sentence-transformers, opencv, 45 tesseract language packs) with AI extras
installed unconditionally, since there's no official slim variant to pull
instead.
Frontend and backend share one container/origin by default (gramps_webapi
serves the built SPA via STATIC_PATH and handles /api/* in the same
process), so Gramps Connect itself needs no CORS configuration โ that's
only relevant for a different, separately-hosted frontend calling in (set
GRAMPSWEB_CORS_ORIGINS in deploy/.env for that case).
Services: caddy (TLS termination, the only published entrypoint),
app (gunicorn, serves the frontend + /api/*), worker (Celery, runs
import/media/search-reindex jobs), postgres (tree data via
SharedPostgreSQL), redis (Celery broker).
cp deploy/.env.example deploy/.env # then edit it -- see the file's comments
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d --buildThen visit https://localhost (Caddy issues itself a local, self-signed
certificate automatically โ accept the browser warning).
Build once on GitHub's runners instead of on the host
(gh workflow run build-docker.yml, or trigger it from the Actions tab),
which pushes ghcr.io/<owner>/gramps-connect:latest, then on the host:
cp deploy/.env.example deploy/.env # real secrets/domain this time
docker compose -f deploy/docker-compose.yml --env-file deploy/.env pull
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -dPoint a domain at the host and edit deploy/Caddyfile (see TLS
below) to get a real certificate instead of the self-signed one.
A walkthrough from an empty account to a live site on a small cloud server (DigitalOcean calls these "droplets"). Any VPS provider works the same way once you have a Linux server with Docker on it.
In the DigitalOcean console, Create โ Droplets:
- Image: Marketplace โ Docker on Ubuntu (Docker preinstalled).
-
Size: Basic, 1 vCPU / 2 GB RAM (
s-1vcpu-2gb) is the smallest that works well. The backend image is large (~4 GB, see above), so don't go smaller. - Region: whichever is closest to your users.
- Authentication: an SSH key (safer than a password).
Then Networking โ Firewalls โ Create Firewall: allow inbound 22, 80, and 443 only, and apply it to the droplet.
2 GB of RAM is tight during imports and startup, so add a 2 GB swap file
(SSH in as root):
fallocate -l 2G /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile
echo '/swapfile none swap sw 0 0' >> /etc/fstabOptional but recommended, since it's what gets you a real HTTPS
certificate. A free DuckDNS name works, as
does an A record on a domain you own. Either way, point it at the
droplet's public IP.
git clone https://github.com/dsblank/gramps-connect /opt/gramps-connect
cd /opt/gramps-connect/deploy
cp .env.example .env
# edit .env: GRAMPSWEB_ADMIN_USER / GRAMPSWEB_ADMIN_PASSWORD,
# PUBLIC_URL=https://your.domain, and the email settings (see below)
docker compose pull
docker compose up -dThis pulls the prebuilt, public ghcr.io/dsblank/gramps-connect:latest
image, so nothing is built on the droplet itself.
Edit deploy/Caddyfile as described under TLS: replace the
:443 { tls internal ... } block with your.domain { reverse_proxy app:5000 }
and delete the :80 block, then docker compose up -d again. Caddy
requests and renews a Let's Encrypt certificate on its own.
Follow First-time setup and
Importing data below,
using https://your.domain in place of https://localhost.
- New DigitalOcean accounts block outbound SMTP. Use a relay on port 587 (e.g. Brevo, SendGrid, Mailgun) and, if mail still doesn't go out, open a DigitalOcean support ticket (category Networking) to have the block lifted.
-
GRAMPSWEB_EMAIL_USE_STARTTLSmust be lowercasetrue.Trueis silently read as a string and the connection fails. - Brevo's SMTP login is a generated address (like
[email protected]), not the email address you signed up with.
See Updating a hosted server below.
These steps apply to any host running the deploy/ stack, a droplet or
otherwise. Commands assume the repo is cloned at /opt/gramps-connect
and that you're in its deploy/ directory.
cd /opt/gramps-connect/deploy
# Tree data (Postgres)
docker compose exec -T postgres sh -c 'pg_dumpall -U "$POSTGRES_USER"' > backup-$(date +%F).sql
# User accounts (sqlite) and media files
docker compose cp app:/app/users ./backup-users-$(date +%F)
docker compose cp app:/app/media ./backup-media-$(date +%F)Updates can run database migrations on startup that don't run backwards, so a backup is the only way to fully undo one.
The server pulls ghcr.io/dsblank/gramps-connect:latest, which only
changes when the build-docker.yml workflow runs. Right after a release
is tagged, check that the Docker build has finished (the repo's
Actions tab) before updating, or you'll pull the previous version.
cd /opt/gramps-connect && git pull # updated deploy/ config
cd deploy
docker compose pull app # the new image
docker compose up -d # recreate what changedOnly app and worker are recreated. postgres, redis, and caddy
keep running, and all data stays in Docker volumes. Never add -v to
docker compose down unless you mean to delete everything (see
Notes).
If git pull refuses because of local changes: a real-domain host
has an edited deploy/Caddyfile (see TLS) that is never
committed, so an update that changes the repo's copy conflicts with it.
Keep your edits with:
git stash && git pull && git stash pop # then fix any conflict in deploy/Caddyfiledeploy/.env is gitignored, so it's never affected.
diff .env.example .envCopy any newly added settings from .env.example into .env, then run
docker compose up -d again if you changed anything.
docker compose ps # app and worker recently created, "Up"
docker compose logs --tail 50 app # gunicorn "Booting worker" lines, no tracebacksThen load the site and log in. One Gtk-CRITICAL ... gtk_icon_theme_get_for_screen line per gunicorn worker is normal and
harmless: the server has GTK installed but no display.
Every image is also tagged with the commit it was built from,
ghcr.io/dsblank/gramps-connect:<commit-sha> (see the repo's
Packages page for the list). To go back, change the image: line
under x-app-image at the top of deploy/docker-compose.yml to that
tag, then:
docker compose up -dIf the newer version migrated the database, also restore the backup from step 1.
A fork publishes its own image. Run build-docker.yml in your fork
(pushes ghcr.io/<your-user>/gramps-connect) and set the image: line
above to it, or build on the server with docker compose up -d --build.
Building on the server needs more disk and memory than the 2 GB droplet
above, since the base image alone is around 4 GB.
All commands below assume you're in the repo root. docker compose is the
plugin form; if your Docker install only has the standalone binary, use
docker-compose instead (same flags either way).
# Status of all five services
docker compose -f deploy/docker-compose.yml ps
# Logs (add -f to follow, --tail=100 to limit)
docker compose -f deploy/docker-compose.yml logs app
docker compose -f deploy/docker-compose.yml logs worker
# Restart one service (e.g. after an env change in deploy/.env)
docker compose -f deploy/docker-compose.yml up -d
# ^ recreates any service whose config (image, env, volumes) changed;
# add --build first if you edited deploy/Dockerfile or app/backend source.
# Stop everything, keep data (volumes survive)
docker compose -f deploy/docker-compose.yml down
# Stop and delete all data (careful -- drops Postgres, media, users, etc.)
docker compose -f deploy/docker-compose.yml down -v
# Shell into a running container
docker compose -f deploy/docker-compose.yml exec app shThis docker deploy has no default password โ you set
GRAMPSWEB_ADMIN_USER/GRAMPSWEB_ADMIN_PASSWORD yourself in deploy/.env
before first boot, and that's what the entrypoint seeds. Nothing generates
or prints a password for you.
That's different from gramps-connect-desktop (the
single-user PyInstaller demo build, unrelated to this docker deploy): it
always seeds a fixed admin/admin account, which is fine there since
it's a throwaway local demo, not something exposed on a real server.
On first boot (first time the app-users/app-db volumes are empty), the
entrypoint:
- generates and persists a Flask secret key if
GRAMPSWEB_SECRET_KEYwas left blank - runs user-database migrations
- seeds a site admin user (treeless, role 5) from
GRAMPSWEB_ADMIN_USER/GRAMPSWEB_ADMIN_PASSWORD(both required indeploy/.envโ noadmin/adminfallback)
A treeless site admin can create/list/delete trees via the API, but โ
since app/'s frontend has no tree-selection UI and always expects the
logged-in user's JWT to already carry a tree โ cannot browse a tree's
data until assigned to one. There's no CLI command for tree creation in
multi-tree mode, so create the first tree via the API (one-time):
TOKEN=$(curl -sk -X POST https://localhost/api/token/ \
-H 'Content-Type: application/json' \
-d '{"username":"<admin>","password":"<admin password>"}' \
| python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])")
TREE_ID=$(curl -sk -X POST https://localhost/api/trees/ \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"name":"My Family Tree"}' \
| python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")
echo "$TREE_ID"Then either assign the site admin itself to that tree, or (recommended โ keeps site administration and tree ownership separate) create a dedicated tree-scoped user:
# Option A: assign the existing site admin to the tree.
curl -sk -X PUT "https://localhost/api/users/<admin>/" \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d "{\"tree\":\"$TREE_ID\"}"
# Option B: create a separate regular user tied to the tree instead.
# email and full_name are required by the schema even if unused.
curl -sk -X POST "https://localhost/api/users/<username>/" \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d "{\"email\":\"<email>\",\"full_name\":\"<full name>\",\"password\":\"<password>\",\"role\":4,\"tree\":\"$TREE_ID\"}"role is an integer (gramps_webapi.auth.const):
| Role | Value | Notes |
|---|---|---|
| ADMIN | 5 | Site admin; only role allowed to be treeless |
| OWNER | 4 | Full control of their tree |
| EDITOR | 3 | Can edit tree data |
| CONTRIBUTOR | 2 | Can add data |
| MEMBER | 1 | Read access |
| GUEST | 0 | Minimal read access |
Then log in at https://localhost/ as whichever user you just created โ
existing tokens issued before a tree assignment won't carry it, so log
out/in again if you reused an already-open session.
TOKEN=$(curl -sk -X POST https://localhost/api/token/ \
-H 'Content-Type: application/json' \
-d '{"username":"<owner>","password":"<password>"}' \
| python3 -c "import sys,json;print(json.load(sys.stdin)['access_token'])")
# 1. Import the .gramps XML file. IMPORTANT: this endpoint reads the raw
# request body and writes it straight to disk -- it does NOT parse
# multipart/form-data, so use --data-binary, not curl's -F/--form (a
# multipart upload silently corrupts the file with boundary/header
# bytes, and Gramps' importer then fails with a generic, unhelpful
# "Import failed" and no further detail).
curl -sk -X POST https://localhost/api/importers/gramps/file \
-H "Authorization: Bearer $TOKEN" --data-binary @example.gramps
# Response is a task handle (import runs on the `worker` service via
# Celery) -- poll it until "state":"SUCCESS" (or "FAILURE"):
curl -sk https://localhost/api/tasks/<task_id> -H "Authorization: Bearer $TOKEN"
# 2. Media: zip up the referenced media files (any subset/superset is fine
# -- files are matched by checksum, unreferenced ones are ignored) and
# upload the archive, again as a raw body:
curl -sk -X POST https://localhost/api/media/archive/upload/zip \
-H "Authorization: Bearer $TOKEN" --data-binary @media.zip
# 3. Verify: object counts for the logged-in user's tree.
curl -sk https://localhost/api/metadata/ -H "Authorization: Bearer $TOKEN" \
| python3 -m json.toolKnown gotcha: with GRAMPSWEB_MEDIA_PREFIX_TREE=True (set by default in
this compose file), media is stored under MEDIA_BASE_DIR/<tree_id>/, but
nothing creates that per-tree subdirectory automatically โ neither
POST /api/trees/ nor the media-archive-upload task. Uploading media for a
tree fails with Directory /app/media/<tree_id> does not exist until it's
created once:
docker compose -f deploy/docker-compose.yml exec app mkdir -p /app/media/<tree_id>Large media uploads timing out: both the single-file (POST /api/media/) and ZIP-archive (POST /api/media/archive/upload/zip)
upload endpoints read the request body synchronously inside app's
gunicorn worker before handing off to Celery, so a large file or a slow
connection can outlast gunicorn's per-request timeout โ the browser then
sees a bare Internal Server Error with no JSON body (from gunicorn/Caddy,
not from Gramps). deploy/docker-compose.yml sets GUNICORN_TIMEOUT to
600s by default; raise it further via GUNICORN_TIMEOUT in deploy/.env
(see that file's comments) if uploads still fail.
caddy is the only service with published ports (80/443) and terminates
TLS in front of app. With no domain configured, it generates and
persists its own local CA and issues a self-signed certificate from it
automatically (deploy/Caddyfile's tls internal) โ browsers will show a
trust warning the first time; accept it to proceed (or add the generated
CA to your system/browser trust store if you want the warning gone without
a real domain). Port 80 redirects to 443.
Once a real domain points at this server, edit deploy/Caddyfile:
replace the :443 { tls internal ... } block with
example.com { reverse_proxy app:5000 } and delete the :80 block โ
Caddy handles ACME issuance, renewal, and the port-80 redirect
automatically for a real domain, no tls directive needed. Also update
PUBLIC_URL in deploy/.env to the real https:// domain, then
docker compose -f deploy/docker-compose.yml up -d to pick up both
changes.
Not wired up by default (no grampsweb service in docker-compose.yml),
but worth knowing: since the backend is a plain, unmodified
gramps-web-api instance, gramps-project/gramps-web (the other official
Gramps frontend) can run against this same backend too โ same data, same
trees/users, just a different UI on a different port.
gramps-web publishes ghcr.io/gramps-project/grampsjs:latest for
exactly this: nginx serving its static build only, no backend baked in.
Its nginx config reverse-proxies /api to an API_HOST env var at
container startup, so from the browser's perspective it's same-origin โ no
GRAMPSWEB_CORS_ORIGINS needed for this path specifically (that setting
is for a gramps-web instance hosted somewhere else entirely, calling in
cross-origin instead of through this proxy).
To add it, a docker-compose.yml service along these lines:
grampsweb:
image: ghcr.io/gramps-project/grampsjs:latest
environment:
API_HOST: http://app:5000
# Docker's embedded DNS resolver -- default.conf.template's `resolver`
# directive requires this to be set explicitly.
NAME_SERVER: 127.0.0.11
depends_on:
- app
restart: unless-stoppedpublished on its own port (either directly, e.g. ports: ["8081:80"], or
fronted by Caddy with a second :8443 { reverse_proxy grampsweb:80 }-style
block in deploy/Caddyfile for TLS to match the app service).
- Data persists in named Docker volumes (
app-db,app-media,app-indexdir,app-users,app-secret,app-cache,app-tmp,postgres-data,caddy-data,caddy-config).docker compose down(without-v) keeps them;docker compose down -vdeletes everything (including the generated CA โ that's a new browser trust warning next boot, not just data loss). -
redis(Celery broker/result backend) is required for background tasks (search indexing, large import/export jobs) โ not optional. So is theworkerservice: import, media-archive-upload, and search reindexing all dispatch through Celery onceGRAMPSWEB_CELERY_CONFIG__*is set, and sit forever as an unfulfilled task without something consuming the queue.app-cache(/app/cache) must be a volume shared betweenappandworkerfor the same reason: the app container's request handler writes the uploaded file there, then the worker container's Celery task reads it back. - The
workerservice runs with--pool=solo(no forking), out of caution around Celery's default prefork pool forking a process that has already touched real PyGObject/GTK state (Gramps'gi.repository.GLibimport). A single worker instance doesn't need the concurrency--pool=sologives up, but this is worth revisiting if worker throughput ever becomes a bottleneck. -
app/'s browser-side local cache (sql.js, OPFS) is keyed by a fixed filename per view, not by backend URL or tree ID โ see Architecture for the gotcha this causes and how to clear it. -
.github/workflows/build-docker.yml(manual trigger โgh workflow run build-docker.yml, or the Actions tab) buildsdeploy/Dockerfileand pushesghcr.io/<owner>/gramps-connect:latest.docker compose -f deploy/docker-compose.yml --env-file deploy/.env pullgrabs that instead of building locally;up -d --buildremains there for local iteration on the Dockerfile itself.