Add unsealer, internal CA/TLS, auto-unseal, and automated cert+backup

Build out the home-lab OpenBAO deployment beyond the basic node: - docker-compose: add openbao-unsealer sidecar; main node now transit auto-unseals against it (seal config kept in gitignored config/seal.hcl) - policies/admin.hcl: non-root admin policy; per-engine rules for ssh/pki/pki_int/totp/transit - Internal two-tier CA (pki/ root + pki_int/ intermediate) issues the openbao.famfi.home leaf Traefik serves; root CA published under ca/ - scripts/ + systemd/: daily cert renewal and Raft snapshot backups (both instances), with scoped tokens stored outside the repo - README: full runbook (auto-unseal, PKI, renewal, backups, DR/restore) Secrets (init/unsealer keys, tokens, seal stanza) stay gitignored. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 14:28:49 +02:00
parent 95e1b676e2
commit 14f46e1082
12 changed files with 389 additions and 23 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -1,7 +1,9 @@
 # NEVER commit unseal keys, root tokens, or storage data
 unseal-keys.json
 init-output.json
+unsealer-init.json
 admin-credentials.txt
+config/seal.hcl
 *.token
 config/tls/
 data/
--- a/README.md
+++ b/README.md
@@ -7,9 +7,13 @@ storage. Suitable for a self-hosted home lab.

 | File | Purpose |
 |------|---------|
-| `docker-compose.yml` | Container definition (port 8200, Raft data volume, IPC_LOCK) |
-| `config/openbao.hcl`  | Server config: Raft storage, TCP listener, UI |
-| `.gitignore`          | Keeps unseal keys / tokens / data out of git |
+| `docker-compose.yml`    | Main node + `openbao-unsealer` sidecar, Raft volumes |
+| `config/openbao.hcl`    | Main server config: Raft storage, TCP listener, UI |
+| `config/seal.hcl`       | Transit auto-unseal stanza (git-ignored — holds a token) |
+| `config-unsealer/`      | Config for the transit unsealer instance |
+| `policies/admin.hcl`    | Non-root admin policy |
+| `scripts/`, `systemd/`  | Automated cert renewal (timer + script) |
+| `ca/`                   | Internal root CA cert (public — for trusting on devices) |

 ## 1. Start the server

@@ -33,13 +37,38 @@ docker compose exec openbao bao operator init \
 in a password manager and delete the file afterward. It is git-ignored, but
 treat it like the master key to everything — because it is.

-## 3. Unseal
+## 3. Unseal — automatic (transit auto-unseal)

-OpenBAO starts sealed after every restart. Provide 3 of the 5 keys:
+The main node **auto-unseals** via the `openbao-unsealer` sidecar (transit
+seal, see "Auto-unseal" below). You normally never unseal it by hand. The
+original 5 Shamir keys are now **recovery keys** (for root-token regen /
+recovery operations), not unseal keys.
+
+> First-time bring-up only: a brand-new install is Shamir-sealed until you run
+> `bao operator init` + `bao operator unseal` (3 keys) once, before migrating to
+> transit auto-unseal.
+
+## Auto-unseal (transit via the unsealer sidecar)
+
+`openbao-unsealer` is a tiny second OpenBAO instance that holds one transit key
+(`autounseal`). The main node's `config/seal.hcl` points at it and unwraps its
+root key on every start — so main-node restarts/upgrades need no manual unseal.

-```bash
-docker compose exec openbao bao operator unseal   # run 3x, paste a key each time
 ```
+main openbao ──seal:transit──▶ openbao-unsealer (transit/autounseal)
+```
+
+- **The unsealer itself is Shamir-sealed (1 key).** Its key + root token are in
+  `unsealer-init.json` (git-ignored). On a **full host reboot** the unsealer
+  comes up sealed, so unseal it once and the main node follows automatically:
+
+  ```bash
+  docker compose exec openbao-unsealer bao operator unseal <unsealer-key>
+  ```
+
+- Caveat: the unsealer currently runs on the same host (SPOF). For real
+  separation, relocate it to another host and point `seal.hcl` at it. To make
+  host reboots fully hands-off, give the unsealer its own boot-unseal.

 ## 4. Log in & use

@@ -60,7 +89,19 @@ The `admin` policy ([policies/admin.hcl](policies/admin.hcl)) grants full
 day-to-day administration but **not** root-only operations (sys/raw, root-token
 generation, rekey). Keep the root token offline.

-A KV v2 secrets engine is mounted at `secret/`:
+### Enabled secrets engines
+
+| Engine  | Path      | Notes |
+|---------|-----------|-------|
+| KV v2   | `secret/` | static key/value secrets |
+| SSH     | `ssh/`    | SSH CA — sign short-lived host/client certs (needs CA + roles configured) |
+| PKI     | `pki/`    | internal CA, max lease 10y (needs root/intermediate CA generated) |
+| TOTP    | `totp/`   | 2FA code generation/validation |
+| Transit | `transit/`| encryption-as-a-service (needs a key created) |
+
+These are mounted but **not yet configured** (no CAs/keys/roles). Each engine
+mounted at a new path has a matching rule in [policies/admin.hcl](policies/admin.hcl);
+add one per future engine.

 ```bash
 docker compose exec openbao bao kv put secret/myapp/db password=s3cr3t
@@ -79,22 +120,98 @@ by Traefik (default self-signed cert).
 - **DNS action required:** add an A record `openbao.famfi.home → 192.168.0.142`
  (the `websecure` entrypoint IP) on your LAN DNS, or a hosts entry on clients.
 - The built-in **web UI** is served at https://openbao.famfi.home once DNS is set.
- Because the cert is self-signed, CLI clients pointed at the HTTPS name need the
-  CA trusted or `BAO_SKIP_VERIFY=true` (local CLI can just use the loopback
-  http://127.0.0.1:8200 instead).
+- TLS uses a **trusted cert from OpenBAO's own internal CA** (see PKI section
+  below) — once you install the root CA on a device, no browser warnings and
+  no `BAO_SKIP_VERIFY` needed.

-## Backups (Raft snapshots)
+## Internal CA (PKI) & the openbao.famfi.home cert
+
+OpenBAO runs a two-tier internal CA and issues the cert Traefik serves:
+
+| Mount | Role | TTL |
+|-------|------|-----|
+| `pki/`     | Root CA (`famfi.home Internal Root CA`)        | 10y |
+| `pki_int/` | Intermediate CA (`famfi.home Intermediate CA`) | 5y  |
+| `pki_int/roles/famfi-home` | issuing role for `*.famfi.home` | 90d max |
+
+The leaf for `openbao.famfi.home` lives in Traefik at
+`/srv/TRAEFIK/etc/traefik/tls/openbao/` and is loaded via
+`/srv/TRAEFIK/etc/traefik/traefik.d/tls-openbao.yml`.
+
+**Trust the CA on your devices** (one time) using [ca/famfi-home-root-ca.pem](ca/famfi-home-root-ca.pem):

 ```bash
-docker compose exec openbao bao operator raft snapshot save /openbao/file/snap.bak
-docker compose cp openbao:/openbao/file/snap.bak ./snap-$(date +%F).bak
+# Linux (Debian/Ubuntu family):
+sudo cp ca/famfi-home-root-ca.pem /usr/local/share/ca-certificates/famfi-home-root-ca.crt
+sudo update-ca-certificates
+# macOS: add to Keychain and mark trusted.  Windows: import to "Trusted Root CAs".
+# Browsers (Firefox) use their own store — import there too.
+```
+
+**Issue a cert for another `.home` service:**
+
+```bash
+docker compose exec openbao bao write pki_int/issue/famfi-home \
+  common_name="gitea.famfi.home" ttl=2160h
+```
+
+**Renewal is automated.** `scripts/renew-openbao-cert.sh` re-issues the leaf and
+reinstalls it for Traefik (rewriting `traefik.d/tls-openbao.yml` to force a
+reload — changing the cert file alone does *not* trigger one). A systemd timer
+(`systemd/openbao-cert-renew.timer`, installed to `/etc/systemd/system`) runs it
+daily; the script no-ops until the cert is within 21 days of expiry. The scoped
+renewal token lives at `/etc/openbao-cert-renew.token` (root-only).
+
+```bash
+sudo systemctl list-timers openbao-cert-renew.timer   # next run
+sudo /home/lutz/Projects/OpenBAO/scripts/renew-openbao-cert.sh --force  # renew now
+```
+
+## Backups (Raft snapshots) — automated
+
+`scripts/backup-raft-snapshots.sh` snapshots **both** instances and prunes to
+the newest `KEEP` (default 14). A systemd timer (`openbao-backup.timer`) runs it
+daily at ~02:30. Snapshots land in `/var/backups/openbao/{main,unsealer}/`
+(root-only, 0600). Scoped backup tokens: `/etc/openbao-backup.token`,
+`/etc/openbao-unsealer-backup.token`.
+
+```bash
+sudo /home/lutz/Projects/OpenBAO/scripts/backup-raft-snapshots.sh   # run now
+sudo systemctl list-timers openbao-backup.timer                     # next run
+```
+
+> ⚠️ **Backups are local to this Pi** — if the disk dies you lose data *and*
+> backups. Add an offsite copy (e.g. rsync the snapshot dirs to a Synology) for
+> real DR. This is the most valuable next hardening step.
+
+### Disaster-recovery set (keep these together, offsite)
+
+To rebuild from nothing you need **all** of:
+1. A `main` snapshot **and** an `unsealer` snapshot (same run).
+2. `unsealer-init.json` — the unsealer's unseal key (without it the unsealer
+   can't be unsealed, so the main node can't be transit-unsealed).
+3. `init-output.json` — the main node's recovery keys + root token.
+
+### Restore outline
+
+```bash
+# 1. Restore the unsealer, unseal it (so transit auto-unseal works again):
+docker compose cp <unsealer.snap> openbao-unsealer:/tmp/u.snap
+docker compose exec openbao-unsealer bao operator raft snapshot restore /tmp/u.snap
+docker compose exec openbao-unsealer bao operator unseal <unsealer-key>
+# 2. Restore the main node (it auto-unseals via the unsealer):
+docker compose cp <main.snap> openbao:/tmp/m.snap
+docker compose exec openbao bao operator raft snapshot restore /tmp/m.snap
 ```

 ## Hardening checklist (before storing real secrets)

- [ ] Put TLS in front (reverse proxy) or enable native TLS in `openbao.hcl`
- [ ] Create a non-root admin policy + token; stop using the root token day-to-day
- [ ] Revoke or store the root token offline
- [ ] Enable auto-unseal (e.g. transit/KMS) if you don't want manual unseal on reboot
- [ ] Schedule the snapshot backup above
+- [x] Put TLS in front (Traefik, LAN-only, trusted internal-CA cert)
+- [x] Create a non-root admin policy + user; stop using the root token day-to-day
+- [x] Enable auto-unseal (transit via the `openbao-unsealer` sidecar)
+- [x] Automate cert renewal (systemd timer)
+- [ ] Move the root token + recovery keys offline (out of `init-output.json`)
+- [x] Schedule the snapshot backup (systemd timer, both instances)
+- [ ] Copy snapshots offsite (e.g. rsync to a Synology) — backups are local-only
 - [ ] Disable or encrypt swap on the host (OpenBAO 2.x dropped mlock support)
+- [ ] (Optional) Relocate the unsealer to a second host; consider 3-node HA
--- a/ca/famfi-home-root-ca.pem
+++ b/ca/famfi-home-root-ca.pem
@@ -0,0 +1,31 @@
+-----BEGIN CERTIFICATE-----
+MIIFPTCCAyWgAwIBAgIUDlqH3oDZSYz6DCiHV8a23qxaeXAwDQYJKoZIhvcNAQEL
+BQAwJjEkMCIGA1UEAxMbZmFtZmkuaG9tZSBJbnRlcm5hbCBSb290IENBMB4XDTI2
+MDYyODExNTgzOVoXDTM2MDYyNTExNTkwN1owJjEkMCIGA1UEAxMbZmFtZmkuaG9t
+ZSBJbnRlcm5hbCBSb290IENBMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
+AgEA56+K6oFpJHmJ8b5RVJ3WeaMA12zKWXJCsHZN2xY654m2dioNj81g/ZluT5am
+pNN6vGB15UXjYw4qyFFqlP7bPrZ34ud8G3uH24ryGOhWm5xOjPHSItLrfhHwUmZs
+0eA2Aami0o5R5NMxGspe13xtuKAcq6DuFPtl0w1b1VgzbqPv2ouCX3HEF8q1c7Gh
+PtpxJSpcb8PxoX7sgLBu45+cNGibkiBzplWtliGhe5s6MrkyhEkoLxSvH3GjD9ao
+cZH6VKtwVHJIqAvylk84VXGO2Ycqsmuc33nsoWcUUGsa6NL8+d0J8s/2yTM1p4Kw
+JqtBFybHbaVn1K9wkcTef6KTLAI8Nd88j/m3B3mWUcQr7H05MbyJS5HoOOvZdRiF
+OvnGqu5yjZlZ4Ne74YTUxAnlz3sEJcgRtokWxfSAVLs+u3Uwva7D7XqeuJ1GJXKG
+IOeaXXdULGVUZyYblBFigDWzTF22rMpyJSjNxK0uVIHqcakKKfpcq+7ehS7A+bNI
+4gbDwRca0fbX+HMurNya1r5agQqsFZnpchVINuBGYITI8TQyyq6/kd8GqYP+KUYs
+sAj7TlVfnhAfLg2ADHRBCoc5u3Fj0Pc6bvoI6ANgVBtTFP2ysP5wBjFXqXk58h7s
+MikwtQyzdRvDkiJLQSG0UoGofybJvhKHEQ5oQwP2MioOd/MCAwEAAaNjMGEwDgYD
+VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFKQhX4TGAuhu
+QC3DedmQwhXkfpoZMB8GA1UdIwQYMBaAFKQhX4TGAuhuQC3DedmQwhXkfpoZMA0G
+CSqGSIb3DQEBCwUAA4ICAQBaJG9aYgIxflz0KmEdJmKtjaCvjX7E7onZ9qM7jFqr
+Rjtl0zJZG1oQ8sDmM4T5fNbORpqLtcYD/Sn2grzZja+yxJFx5/a/jxfkBWwvDiGT
+hS2UuKmVZ1k1thA2PlzzjOyBh2KFP7BpjYdUIiU09K/hZrvrSS7/Vn3g6jbpiTMT
+WtxBbRgt4AdjG/rYLQPNr/yygerllcPAzMr8i7ROuvj8rddi6yCyef2MT1ayF7/w
+J2ufdbLIosYb9qanp50PpS/rWMC4vAVVFlaYEBBACQIo7N0v9learskdNpPw4vBY
+30gXn0kN15QeuYYPkqwBNVJVuXKsz/+T/WAI9Vxj8EdsxrslbflVy+7VzNn1os+R
+GTGbwdXlnK0ie3QHXISB6uKWQz6H91F9HVnzB6uypkMH9iFOdXC7tgqOjdl/ov2T
+MHYxHqXJ6NNvvWHjqIQCCft6i8AglIsP4xIj/CdpOZabkxnqabD82TXW6oQ6qPpV
+muUv8z3yqQRKHIpxgnXUOlN954L0iBIg1DkilWpAPGeA3DHZ/CkOAIxJmdkLmOu3
+4KFzvkh4D8YBzXZqsI6gBhJhriEHiNNyu6Idsah65k2+aPY5nqfYVao5t5+nXJtL
+88xi+NZUqD/79YvLoT6kBHFQbn+Gii6uOM2A/5IcdFd+xcTDh7vk86QKWxBMUQbX
+3g==
+-----END CERTIFICATE-----
--- a/config-unsealer/openbao.hcl
+++ b/config-unsealer/openbao.hcl
@@ -0,0 +1,19 @@
+ui = false
+
+# Tiny dedicated instance whose ONLY job is to hold a transit key that
+# auto-unseals the main OpenBAO node. It is itself Shamir-sealed (unsealed
+# manually or by a boot script). Keep its footprint minimal.
+storage "raft" {
+  path    = "/openbao/file"
+  node_id = "unsealer-node-1"
+}
+
+listener "tcp" {
+  address     = "0.0.0.0:8200"
+  tls_disable = 1
+}
+
+api_addr     = "http://openbao-unsealer:8200"
+cluster_addr = "https://openbao-unsealer:8201"
+
+# OpenBAO 2.x: no mlock; disable/encrypt host swap instead.
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -26,6 +26,30 @@ services:
      timeout: 5s
      retries: 5
      start_period: 10s
+    depends_on:
+      - openbao-unsealer
+
+  # Minimal sidecar instance that provides transit auto-unseal for the main
+  # node above. Shamir-sealed itself; only reachable on the internal network.
+  openbao-unsealer:
+    image: openbao/openbao:2.5.5
+    container_name: openbao-unsealer
+    restart: unless-stopped
+    command: server
+    # No host port published — only the main node needs it, over the default net.
+    environment:
+      BAO_ADDR: "http://127.0.0.1:8200"
+    networks:
+      - default
+    volumes:
+      - ./config-unsealer:/openbao/config:ro
+      - openbao-unsealer-data:/openbao/file
+    healthcheck:
+      test: ["CMD", "bao", "status", "-address=http://127.0.0.1:8200"]
+      interval: 30s
+      timeout: 5s
+      retries: 5
+      start_period: 10s

 networks:
  # Project-internal network (default).
@@ -37,3 +61,4 @@ networks:

 volumes:
  openbao-data:
+  openbao-unsealer-data:
--- a/policies/admin.hcl
+++ b/policies/admin.hcl
@@ -16,12 +16,15 @@ path "sys/policies/acl"   { capabilities = ["list"] }
 path "sys/mounts/*"    { capabilities = ["create", "read", "update", "delete", "sudo"] }
 path "sys/mounts"      { capabilities = ["read"] }

-# Work with secrets data in mounted engines.
+# Work with secrets data in mounted engines. One rule per engine mount path.
+# NOTE: each secrets engine mounted at a NEW path needs its own rule here.
+# The UI runs a preflight capability check on the mount path, so missing = 403.
 path "secret/*"        { capabilities = ["create", "read", "update", "patch", "delete", "list"] }
 path "ssh/*"           { capabilities = ["create", "read", "update", "patch", "delete", "list", "sudo"] }
-# NOTE: each secrets engine mounted at a NEW path needs its own rule here
-# (e.g. add `path "pki/*"` when you enable a PKI engine). The UI runs a
-# preflight capability check on the mount path, so a missing rule = 403.
+path "pki/*"           { capabilities = ["create", "read", "update", "patch", "delete", "list", "sudo"] }
+path "pki_int/*"       { capabilities = ["create", "read", "update", "patch", "delete", "list", "sudo"] }
+path "totp/*"          { capabilities = ["create", "read", "update", "patch", "delete", "list"] }
+path "transit/*"       { capabilities = ["create", "read", "update", "patch", "delete", "list"] }

 # Lease management
 path "sys/leases/*"    { capabilities = ["create", "read", "update", "delete", "list", "sudo"] }
--- a/scripts/backup-raft-snapshots.sh
+++ b/scripts/backup-raft-snapshots.sh
@@ -0,0 +1,55 @@
+#!/usr/bin/env bash
+# Take Raft snapshots of BOTH OpenBAO instances (main + unsealer) and prune old
+# ones. Run as root (reads the root-only backup tokens, writes /var/backups).
+#
+# DR note: a restored MAIN snapshot can only be unsealed with the unsealer's
+# transit key — so the unsealer snapshot (+ its unseal key in unsealer-init.json
+# + the recovery keys in init-output.json) are part of the same recovery set.
+#
+# Installed as a systemd timer (openbao-backup.timer). Manual run:
+#   sudo /home/lutz/Projects/OpenBAO/scripts/backup-raft-snapshots.sh
+set -euo pipefail
+
+PROJECT_DIR="/home/lutz/Projects/OpenBAO"
+BACKUP_ROOT="${BACKUP_ROOT:-/var/backups/openbao}"
+KEEP="${KEEP:-14}"                       # how many snapshots to retain per instance
+STAMP="$(date '+%Y%m%d-%H%M%S')"
+
+log() { printf '%s [backup] %s\n' "$(date '+%F %T')" "$*"; }
+die() { log "ERROR: $*"; exit 1; }
+
+cd "$PROJECT_DIR" || die "cannot cd to $PROJECT_DIR"
+
+# snapshot <service> <token-file> <label>
+snapshot() {
+  local svc="$1" tokfile="$2" label="$3"
+  [ -r "$tokfile" ] || die "token file $tokfile not readable (run as root?)"
+  local tok dir out incontainer="/tmp/${label}-${STAMP}.snap"
+  tok="$(cat "$tokfile")"
+  dir="${BACKUP_ROOT}/${label}"
+  out="${dir}/openbao-${label}-${STAMP}.snap"
+  install -d -o root -g root -m 0700 "$dir"
+
+  docker compose exec -T -e BAO_TOKEN="$tok" "$svc" \
+    bao operator raft snapshot save "$incontainer" >/dev/null \
+    || die "snapshot save failed for $svc"
+  docker compose cp "${svc}:${incontainer}" "$out" >/dev/null \
+    || die "copy-out failed for $svc"
+  docker compose exec -T "$svc" rm -f "$incontainer" >/dev/null 2>&1 || true
+  chmod 0600 "$out"
+
+  # Validate: non-trivial size + gzip integrity (a raft snapshot is a gzip archive)
+  local size; size=$(stat -c %s "$out")
+  [ "$size" -ge 1024 ] || die "snapshot $out suspiciously small (${size}B)"
+  gzip -t "$out" 2>/dev/null && log "OK ${label}: ${out} (${size}B, gzip-valid)" \
+    || die "snapshot $out failed gzip integrity check"
+
+  # Retention: keep newest $KEEP, delete the rest
+  ls -1t "${dir}"/openbao-${label}-*.snap 2>/dev/null | tail -n +$((KEEP + 1)) | while read -r old; do
+    rm -f -- "$old"; log "pruned old snapshot $(basename "$old")"
+  done
+}
+
+snapshot "openbao"          "/etc/openbao-backup.token"           "main"
+snapshot "openbao-unsealer" "/etc/openbao-unsealer-backup.token"  "unsealer"
+log "done; retained up to ${KEEP} snapshots per instance under ${BACKUP_ROOT}"
--- a/scripts/renew-openbao-cert.sh
+++ b/scripts/renew-openbao-cert.sh
@@ -0,0 +1,75 @@
+#!/usr/bin/env bash
+# Renew the openbao.famfi.home leaf cert from OpenBAO's pki_int engine and
+# install it where Traefik serves it. Idempotent: only renews when the current
+# cert expires within $RENEW_WINDOW_DAYS. Run as root (writes Traefik's tls dir).
+#
+# Installed as a systemd timer (openbao-cert-renew.timer). Manual run:
+#   sudo /home/lutz/Projects/OpenBAO/scripts/renew-openbao-cert.sh [--force]
+set -euo pipefail
+
+ADDR="${BAO_ADDR:-http://127.0.0.1:8200}"
+TOKEN_FILE="/etc/openbao-cert-renew.token"
+ROLE="pki_int/issue/famfi-home"
+CN="openbao.famfi.home"
+TTL="2160h"                                   # 90 days
+DEST="/srv/TRAEFIK/etc/traefik/tls/openbao"
+DYN="/srv/TRAEFIK/etc/traefik/traefik.d/tls-openbao.yml"
+RENEW_WINDOW_DAYS="${RENEW_WINDOW_DAYS:-21}"  # renew when <= this many days left
+FORCE="${1:-}"
+
+log() { printf '%s [renew-cert] %s\n' "$(date '+%F %T')" "$*"; }
+die() { log "ERROR: $*"; exit 1; }
+
+[ -r "$TOKEN_FILE" ] || die "token file $TOKEN_FILE not readable (run as root?)"
+TOKEN="$(cat "$TOKEN_FILE")"
+
+# Skip if the current cert is still good (unless --force)
+if [ "$FORCE" != "--force" ] && [ -f "$DEST/fullchain.pem" ]; then
+  end="$(openssl x509 -in "$DEST/fullchain.pem" -noout -enddate | cut -d= -f2)"
+  days_left=$(( ( $(date -d "$end" +%s) - $(date +%s) ) / 86400 ))
+  if [ "$days_left" -gt "$RENEW_WINDOW_DAYS" ]; then
+    log "cert valid ${days_left}d (> ${RENEW_WINDOW_DAYS}d) — nothing to do"
+    exit 0
+  fi
+  log "cert has ${days_left}d left (<= ${RENEW_WINDOW_DAYS}d) — renewing"
+fi
+
+# Issue a fresh cert
+resp="$(curl -sS --fail-with-body --max-time 15 \
+  -H "X-Vault-Token: ${TOKEN}" \
+  --data "{\"common_name\":\"${CN}\",\"ttl\":\"${TTL}\"}" \
+  "${ADDR}/v1/${ROLE}")" || die "issue request failed"
+
+tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
+export RESP="$resp"
+python3 - "$tmp" <<PY
+import json, sys, os
+d = json.loads(os.environ["RESP"])["data"]
+t = sys.argv[1]
+open(f"{t}/fullchain.pem", "w").write(d["certificate"] + "\n" + "\n".join(d.get("ca_chain", [])) + "\n")
+open(f"{t}/privkey.pem", "w").write(d["private_key"] + "\n")
+PY
+
+# Validate cert/key match before installing
+cmod="$(openssl x509 -in "$tmp/fullchain.pem" -noout -modulus | openssl md5)"
+kmod="$(openssl rsa  -in "$tmp/privkey.pem"  -noout -modulus 2>/dev/null | openssl md5)"
+[ "$cmod" = "$kmod" ] || die "cert/key modulus mismatch — refusing to install"
+
+install -d -o root -g root -m 0755 "$DEST"
+install -o root -g root -m 0644 "$tmp/fullchain.pem" "$DEST/fullchain.pem"
+install -o root -g root -m 0600 "$tmp/privkey.pem"  "$DEST/privkey.pem"
+
+# Rewrite the watched dynamic config (with a fresh timestamp) so Traefik's file
+# provider reparses and reloads the cert from disk — changing the cert file
+# alone does NOT trigger a reload.
+cat > "$DYN" <<YML
+# AUTO-MANAGED by renew-openbao-cert.sh — last renewed $(date -Iseconds).
+# Loads OpenBAO's leaf cert for openbao.famfi.home; the openbao router uses
+# \`tls: {}\` and serves this by SNI.
+tls:
+  certificates:
+    - certFile: /etc/traefik/tls/openbao/fullchain.pem
+      keyFile: /etc/traefik/tls/openbao/privkey.pem
+YML
+chmod 0644 "$DYN"
+log "installed new cert ($(openssl x509 -in "$DEST/fullchain.pem" -noout -enddate | cut -d= -f2)); Traefik reload triggered"
--- a/systemd/openbao-backup.service
+++ b/systemd/openbao-backup.service
@@ -0,0 +1,9 @@
+[Unit]
+Description=Raft snapshot backup of both OpenBAO instances (main + unsealer)
+After=network-online.target docker.service
+Wants=network-online.target
+Requires=docker.service
+
+[Service]
+Type=oneshot
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/backup-raft-snapshots.sh
--- a/systemd/openbao-backup.timer
+++ b/systemd/openbao-backup.timer
@@ -0,0 +1,10 @@
+[Unit]
+Description=Daily Raft snapshot backup of OpenBAO
+
+[Timer]
+OnCalendar=*-*-* 02:30:00
+Persistent=true
+RandomizedDelaySec=15m
+
+[Install]
+WantedBy=timers.target
--- a/systemd/openbao-cert-renew.service
+++ b/systemd/openbao-cert-renew.service
@@ -0,0 +1,10 @@
+[Unit]
+Description=Renew openbao.famfi.home cert from OpenBAO PKI and reload Traefik
+After=network-online.target docker.service
+Wants=network-online.target
+Requires=docker.service
+
+[Service]
+Type=oneshot
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/renew-openbao-cert.sh
+# Script no-ops unless the cert is within its renewal window.
--- a/systemd/openbao-cert-renew.timer
+++ b/systemd/openbao-cert-renew.timer
@@ -0,0 +1,10 @@
+[Unit]
+Description=Daily check/renew of the openbao.famfi.home leaf cert
+
+[Timer]
+OnCalendar=daily
+Persistent=true
+RandomizedDelaySec=1h
+
+[Install]
+WantedBy=timers.target