Production Deployment - ptweezy/cronstable GitHub Wiki
This page covers running cronstable in hardened Linux container and Kubernetes
environments: the published image, the security context it is built to satisfy,
the Kubernetes Deployment manifest, building a custom image from the published
image, and the few cases that require a writable path. The published Docker
image is Linux-only. See installation for the package/binary
install methods and the HTTP control API for the optional web
interface. For native Windows deployment (the MSI, winget, the
one-directory zip, or the standalone .exe), see
running on Windows.
At runtime the daemon only reads its configuration and secrets and writes its output to stdout/stderr. It needs no writable working directory, no temp files, and no log files. It runs in the foreground, makes no exotic syscalls, and requires no special privileges. The published image runs under a restricted Kubernetes/container security context with no writable paths or elevated privileges required.
The single exception is the optional per-job user/group switching feature,
which calls os.initgroups/os.setgid/os.setuid (cronstable/job.py) and so
requires the daemon to run as root. It is unavailable in the non-root
published image. If you do not use that feature, drop root entirely. See
commands and environment for the feature itself.
Per-run resource limits (resourceLimits) also need more
than the published image grants: root and a cgroup that cronstable can
write. See resource limits in a container.
Prebuilt, multi-architecture (linux/amd64, linux/arm64, linux/386,
linux/arm/v7, linux/ppc64le, linux/s390x, and linux/riscv64) images are
published to the GitHub Container Registry on every release.
For compatible x86-64-v3 hosts, the explicit -amd64v3 tags are recommended
(for example, latest-amd64v3 and latest-alpine-amd64v3). Existing tags
retain their current platform coverage and CPU requirements. The deployment
examples below use those existing tags; select the v3 tag for a compatible
host, including the CPU features exposed to a VM.
See CPU requirements and variant selection.
| Property | Value |
|---|---|
| Registry/image | ghcr.io/ptweezy/cronstable |
| Tags | the release version (for example, 1.2.60), plus latest
|
| Base |
python:3.14-slim (multi-stage; runtime stage copies a self-contained venv) |
| User |
65534:65534 (nobody), set with USER in the Dockerfile
|
| Entrypoint | cronstable |
| Default command | -c /etc/cronstable.d |
| Config path |
/etc/cronstable.d (a file or a directory of *.yaml/*.yml files) |
| Env |
PYTHONUNBUFFERED=1, PYTHONDONTWRITEBYTECODE=1, PATH=/opt/venv/bin:$PATH
|
PYTHONUNBUFFERED flushes stdout/stderr immediately (where cronstable logs) and
PYTHONDONTWRITEBYTECODE prevents .pyc writes. Both matter under a read-only
root filesystem.
A second
python:3.14-slim-based Dockerfile underexample/docker/exists for demonstration. Itpip installs cronstable and is not non-root. For production, use the published GHCR image (or base your own image on it, described later), not the example.
Mount a crontab read-only and run:
docker run --rm \
-v "$PWD/cronstable.yaml:/etc/cronstable.d/cronstable.yaml:ro" \
ghcr.io/ptweezy/cronstable:latestFor production, pin a specific version instead of latest
(for example, ghcr.io/ptweezy/cronstable:1.2.60).
If you would rather bake the configuration into an image, base it on the published image. Your image inherits the non-root user, entrypoint, and config path:
FROM ghcr.io/ptweezy/cronstable:latest
# The base image already runs as the non-root user 65534.
COPY cronstable.yaml /etc/cronstable.d/cronstable.yamlThe published image satisfies a fully restricted pod security context:
-
runAsNonRoot/ non-root UID: the daemon needs no privileges, so it runs as an unprivileged UID (65534). Only per-jobuser/groupswitching and per-run resource limits need root, and neither is available here. -
RuntimeDefaultseccomp: cronstable makes no exotic syscalls, so the default seccomp profile (or an equivalently strict custom one) works. -
readOnlyRootFilesystem: neither the image nor apip/pipxinstall requires runtime writes. Mount the crontab read-only. See writable-path exceptions for the two features that need a small writable volume: a Unix-socket web listener (with the image) and the standalone binary's self-extraction temp directory (only if you deploy the binary). -
fsGroup-mounted config/secrets: mount config and secret volumes with anfsGroup(for example,65534) so the non-root process can read them. -
drop: [ALL]capabilities andallowPrivilegeEscalation: false: cronstable needs no Linux capabilities and never escalates privileges.
A Deployment mounting the crontab from a ConfigMap, read-only, under a fully
restricted security context:
apiVersion: apps/v1
kind: Deployment
metadata:
name: cronstable
spec:
replicas: 1
selector:
matchLabels:
app: cronstable
template:
metadata:
labels:
app: cronstable
spec:
securityContext: # pod-level
runAsNonRoot: true
runAsUser: 65534
runAsGroup: 65534
fsGroup: 65534 # lets the non-root process read mounted volumes
seccompProfile:
type: RuntimeDefault
containers:
- name: cronstable
image: ghcr.io/ptweezy/cronstable:latest
args: ["-c", "/etc/cronstable.d"]
securityContext: # container-level
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
resources:
limits:
cpu: 200m
memory: 128Mi
requests:
cpu: 10m
memory: 64Mi
volumeMounts:
- name: crontab
mountPath: /etc/cronstable.d
readOnly: true
volumes:
- name: crontab
configMap:
name: cronstablereplicas: 1 is the safe default: cronstable holds the schedule in-process, so
two replicas with no coordination each run every job independently.
To coordinate scheduled jobs across replicas, enable
leader election with a cluster section.
Execution guarantees depend on the backend and each job's clusterPolicy.
Choose a backend based on the coordination services you already run:
-
backend: kubernetes(recommended on Kubernetes). Acoordination.k8s.io/v1Leaseprovides fenced leadership with a single lease holder while the apiserver is reachable. No peer mTLS, no peer list, no odd-replica rule: a plainDeploymentwith any replica count works. You only grant theLeaseRBAC (get/create/update). Likewisebackend: etcdif you run etcd. See operating the lease backends andexample/kubernetes/. -
backend: gossip(the default, no coordination store). A quorum-gated, mutual-TLS election that keeps no shared state, so:-
Use an odd replica count. 3 replicas tolerate one failure, 5 tolerate
two. Even counts buy no extra fault tolerance, and
replicas: 2is worse than 1: a majority of 2 is 2, so both must be up to run anything. The daemon refuses to start withelectLeaderand a 2-node cluster, and warns on even sizes. Spread replicas across nodes/zones withtopologySpreadConstraints. Correlated failures defeat quorum regardless of count. -
A minority partition skips
Leaderjobs once it detects lost quorum. Each node's view is only as fresh as its last poll, so a leadership change can briefly cause skipped or duplicate runs. Use a lease backend for fenced leadership. No backend guarantees that every scheduled job runs exactly once; see the per-job policies. - Provision the per-pod certificates from your own PKI (for example,
cert-manager) and give each pod a stable
nodeName. A StatefulSet's ordinal hostnames make both the cert SANs and the peer list straightforward.
-
Use an odd replica count. 3 replicas tolerate one failure, 5 tolerate
two. Even counts buy no extra fault tolerance, and
See clustering and leader election for the full
trust model, quorum math, per-job clusterPolicy, the lease backends, and a
runnable local example.
The gossip backend adds a second listener, the cluster.listen mTLS port
(0.0.0.0:8443 in the examples), that peers poll over
mutual TLS. It is
separate from the optional web API port, so plan the pod network for it:
-
Open the port for pod-to-pod traffic. Every node both serves
/peeroncluster.listenand connects out to each peer'scluster.listen, so the port must be reachable between the replicas. Expose it on the container (containerPort: 8443). -
Add a
NetworkPolicyin a default-deny cluster. If you run default-deny ingress, allow thecluster.listenport from the pods carrying the sameapplabel (the peers), so the mTLS gossip is not silently dropped. A blocked port shows up asunreachablepeers and a lost quorum, not a clear error. -
Use a headless
Servicefor peer DNS. Pair a gossip StatefulSet with a headlessService(clusterIP: None) so each pod is addressable at a stable per-ordinal DNS name (cronstable-0.<svc>.<ns>.svc,cronstable-1.<svc>…). You list those names undercluster.peers, and the certificate SANs must match them. A lease backend needs none of this: it has no peer listener.
On the web API, cronstable serves a GET /version that returns
200 after the daemon is up, which makes a cheap liveness probe. Enable an
http:// web listener (for example, a web.listen entry of
http://0.0.0.0:8080) and point the probe at it:
ports:
- name: web
containerPort: 8080
livenessProbe:
httpGet:
path: /version
port: web
initialDelaySeconds: 5
periodSeconds: 10If you protect the web API with a web.authToken, only the dashboard page (/)
stays unauthenticated, so /version returns 401 without the bearer
token. Either add the Authorization header to the probe
(httpGet.httpHeaders) or point it at / (when the dashboard is enabled; with
web.ui: false there is no / route).
For clustered deployments, do not turn quorum loss into a pod restart: a
livenessProbe that fails on lost quorum would restart exactly the nodes you
need to reform it. Keep the probe scoped to daemon liveness (/version) and
alert on cluster health separately. quorate on GET /cluster is the field to
alert on: a node that cannot see a majority, or that on a lease backend cannot
reach the store, stands its Leader jobs down. For the full signal set, see the
monitoring and alerting
table.
Two different
tls:blocks. This section is aboutcluster.tls, the peer certificates the gossip mesh presents to other cronstable nodes on thecluster.listenport. They are unrelated toweb.tls, the listener certificates the HTTP control API serves to operators, browsers, and clients on theweb.listenaddresses, and tostate.jobApi.tls, which serves the job-facing state endpoint. Keep their PKI separate: the cluster CA is a membership allowlist (explained later), so a certificate issued for a dashboard user must not chain to it. Provision each from its own issuer. For the listener side, see listener TLS.
This applies to the gossip backend only (the lease backends use no per-node
mTLS certs). Day-to-day operation needs little intervention: cronstable reloads
its mTLS contexts in place. On each config-reload pass it compares the
on-disk ca/cert/key against what it loaded, and if any changed it
rebuilds the TLS contexts within ~1 minute, with no restart. It dry-runs
the new material first, so a half-written file is retried, not fatal. It
therefore picks up in-place renewals from cert-manager, Vault, and
mounted-secret refreshes (same paths, new bytes) automatically. The Clustering
page covers the full mechanism under
certificate rotation; the
operational cases:
-
The cluster
cais the membership allowlist. The daemon trusts any cert the configuredcasigns to assert a node identity and gossip state, so thecamust be a dedicated, single-purpose CA issued only to cronstable nodes, not a shared service-mesh or organization-wide CA. Any cert the CA admits could otherwise fabricate the/peerpayload (fake agreement, trip the conflict gate, or suppress an@rebootjob). Provision leaf certs from that dedicated PKI (a private cert-manager issuer, an internal CA). The daemon only consumes them. -
Rotating a single node's cert. If a leaf your PKI renews still chains to the same
ca, rotating it needs no coordination: write the newcert/keyin place, the node reloads within ~1 minute, and its peers keep trusting it. Issue the replacement with a comfortable overlap (well before the old cert expires) so a slow refresh never leaves the node serving an expired cert. -
Rolling the cluster CA. Changing the CA itself needs care, because a node only trusts peers whose certs chain to the CA bundle it currently holds. Roll it so trust always overlaps:
- Distribute a bundle file containing both the old and new CA as the
tls.caon every node first (an additional trust anchor). Each node reloads within ~1 minute and now trusts either CA. - Confirm every node still shows its peers
agreedonGET /cluster(nountrusted). - Re-issue each node's leaf cert from the new CA, one node at a time,
watching
GET /clusterafter each until the rotated node and its peers return toagreed. - After every node presents a new-CA cert, narrow the bundle back to the new CA alone and distribute it everywhere.
Never cut the CA over in a single step: if some nodes trust only the new CA while others still present old-CA certs, they reject each other as
untrustedand the cluster loses quorum until trust overlaps again. - Distribute a bundle file containing both the old and new CA as the
-
Recovering from an
untrustedcascade. If peers start showinguntrustedonGET /cluster(or thepeer … is untrustedWARNINGin the logs) after a rotation, certs and CA trust have diverged. The usual causes are a CA roll that skipped the overlap step, or a node whose refresh lagged. Recovery needs no restart: restore the trust overlap (push a CA bundle covering whichever CA the still-untrustedpeers were issued from), or finish rolling the lagging nodes onto the new CA. Each node reloads within ~1 minute and peers return toagreedautomatically.Leaderjobs stand down (fail closed) while quorum is lost, so the cascade skips firings rather than double-running them: no split-brain during recovery, only missed firings until trust reconverges.
Before scaling past replicas: 1, work through this checklist for your backend.
Each row links to the authoritative section on the Clustering page.
| Concern |
gossip (default) |
Lease (kubernetes / etcd) |
|---|---|---|
| Trust / auth (gossip mTLS / lease store) | per-node certs from a dedicated cluster CA; open the cluster.listen mTLS port |
apiserver Lease RBAC (get/create/update) or reachable etcd endpoints; no mTLS |
| Node identity (gossip / lease) | stable, unique nodeName (distinct cert SANs / stable hostnames) |
stable name too, but the lease is the fence, so a duplicate name cannot double-lead |
| Listener / port (attestation) |
cluster.listen reachable pod-to-pod; headless Service for peer DNS |
none (no peer listener) |
| Replica count (sizing) |
odd (3/5/7); cronstable rejects an electLeader 2-node cluster and warns on even sizes |
any count; the store fences, not a quorum |
| Rotation (cert rotation) | in-place cert/CA reload (earlier); roll the CA with an overlap | n/a (no per-node mTLS certs) |
| Preflight |
cronstable -c <path> --validate-config (catches lease ordering, CA/cert paths, an electLeader 2-node cluster) |
cronstable -c <path> --validate-config (same) |
cronstable -c <path> --validate-config parses the config and exits nonzero on
any error, so it belongs in CI before a cluster deploy. It enforces the lease
timing ordering (leaseDurationSeconds > renewDeadlineSeconds,
retryPeriodSeconds < renewDeadlineSeconds, and other lease timing rules), the
RFC1123 leaseName/leaseNamespace shapes, the etcd endpoint/TLS rules, and
the electLeader 2-node rejection. See
command-line reference.
In the published image, resourceLimits caps do not apply. The image runs
as uid 65534, and container runtimes mount /sys/fs/cgroup read-only by
default, so cronstable cannot give a run a cgroup of its own. A capped job
runs uncapped, the daemon logs one warning naming the capped jobs, and each
run's record says its caps were not enforced. A job with
resourceLimits.required: true fails each run instead. The container's own
limits, such as a pod's resources.limits or docker run --memory and
--cpus, still bound cronstable and all of its jobs together.
For per-run caps, the container needs:
- cronstable running as root inside it, from an image of your own;
- a writable cgroupfs, in the container's own cgroup namespace or in a
container cgroup that the runtime delegated. On a cgroup v2 host, Docker
and Podman give each container its own cgroup namespace by default, and
--privilegedor, on Docker Engine 28 and later,--security-opt writable-cgroups=truemakes the cgroupfs writable.
Kubernetes has no pod setting for this. In a privileged pod, which shares
the host's cgroup namespace, whether caps apply depends on whether the
runtime delegates the container's cgroup; check limits on
GET /node. An unprivileged pod gets a writable
cgroupfs only from a node-level runtime setting, such as containerd's
cgroup_writable option.
When the caps can apply, cronstable moves every process in the container's
cgroup, its own included, into a daemon/ child cgroup, and gives each
capped run a cgroup under runs/. A docker exec shell joins the daemon/
cgroup, so a second cronstable started that way shares the first daemon's
daemon/ and runs/, and its cpu controller too: either daemon's config
apply can turn cpu control on or off for both, and neither turns it off
while a run of either one holds a cpu cap. For independent cpu control, run
a second capped daemon in a cgroup of its own (see
the cgroup layout). A job that runs as
root, as jobs do in such a container unless they set user, can write its
own cgroup's files, so the caps there do not sandbox it (see
security).
A read-only root filesystem is sufficient for the published image in the normal
case. Two features need a small writable mount, described in the following
sections. The durable state store (state.path) and the filesystem election
backend also need a writable volume, as does an out-of-cluster kubeconfig that
embeds client-certificate-data/client-key-data (set TMPDIR to a writable
mount).
If you enable the optional HTTP control API on a Unix socket
(a web.listen entry such as unix:///path/cronstable.sock), cronstable binds a UnixSite and
creates the socket file at that path, a write. Point the socket at a small
writable volume (a Kubernetes emptyDir) rather than the root filesystem:
volumeMounts:
- name: crontab
mountPath: /etc/cronstable.d
readOnly: true
- name: run
mountPath: /run/cronstable
volumes:
- name: crontab
configMap:
name: cronstable
- name: run
emptyDir: {}with unix:///run/cronstable/cronstable.sock as a web.listen entry in your
configuration. (TCP listeners such as http://0.0.0.0:8080 need no writable
path.) The optional web.socketMode config key, if set, chmods the socket
after bind.
Windows does not support
unix://web listeners (the Proactor event loop lackscreate_unix_server). The daemon skips such a listen URL with the warningIgnoring web listen url <url>: unix-socket listeners are not supported on this platform. Use anhttp://listener instead.web.socketModeonly applies to unix sockets, so it is irrelevant on Windows. See running on Windows.
This applies only if you deploy the standalone binary (from the GitHub
releases) instead of the published image. The binary self-extracts on each
start, so it needs a temp directory that is both writable and executable, and a
read-only root filesystem makes /tmp fail that test. For the error messages
and the Docker --tmpfs exec recipe, Kubernetes emptyDir mount, and
TMPDIR override, see the
standalone binary temp-directory requirement
on the Installation page.
-
Logging: cronstable logs to stdout/stderr. Collect logs at the platform
level (
kubectl logs, the container runtime's log driver). Adjust verbosity with-l/--log-level(defaultINFO) or alogging:config section. -
Shutdown: on POSIX, cronstable installs handlers for
SIGINTandSIGTERMand shuts down gracefully, so the default pod termination path needs no extra configuration. On Windows (where the Proactor loop has noadd_signal_handler) it instead handlesSIGINT/SIGBREAKwithsignal.signalplus a heartbeat timer, and console close or OS shutdown with a native console-control handler. Logoff is excluded deliberately: a service receives it for every user's sign-out. Either way, the daemon finishes the currently-running jobs first. Supervised deployments can also stop it over HTTP with the authenticatedPOST /shutdownroute. See running on Windows and the HTTP control API. -
Validate before deploy:
cronstable -c <path> --validate-configparses the config and exits, useful as a CI/pre-deploy gate. See command-line reference. -
Config not found: the default config path is platform-specific:
/etc/cronstable.don POSIX. On Windows it is%ProgramData%\cronstablewhen that directory holds configuration, else%APPDATA%\cronstable, falling back to the user profile~ifAPPDATAis unset. With no arguments and no default configuration, cronstable prints setup guidance and exits0. Pass-cexplicitly in a supervised command so a missing configuration exits1. In the container, mount the config volume at/etc/cronstable.d.