OpenBAO-Server 2.6.2 -> 2.7.1 (Unsealer bleibt vorerst)

Die CLI auf pi5-1 ist seit dem 01.10. 2.7.1, der Server lief noch auf 2.6.2.
Der Hauptknoten entsiegelt sich per Transit ueber den Sidecar selbst, der
Austausch brauchte also keine Schluessel: Log zeigt "unsealed with stored key"
und "Recorded vault version: 2.7.1".

Der Sidecar openbao-unsealer bleibt bewusst auf 2.6.2. Er ist Shamir-versiegelt
mit Schwelle 1/1 und muesste nach einem Austausch von Hand entsiegelt werden -
und solange er versiegelt ist, kommt auch der Hauptknoten beim naechsten Start
nicht hoch. Die Transit-API ist versionsuebergreifend stabil, der Mischbetrieb
ist also unkritisch.

Geprueft nach dem Austausch: Siegelstatus, Lesezugriff ueber Traefik-TLS von
pi5-2 aus und der git-credential-Helfer gegen Gitea.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.gitignore

@@ -3,10 +3,15 @@ unseal-keys.json
 init-output.json
 unsealer-init.json
 admin-credentials.txt
-config/seal.hcl
+config/seal.hcl*
 *.token
 config/tls/
 data/
 .env
 artifacts/
 .claude/
+*-credentials.txt
+*.bak-*
+
+# Audit-Logs gehoeren nicht ins Repo
+logs/

README.md

@@ -66,9 +66,18 @@ main openbao ──seal:transit──▶ openbao-unsealer (transit/autounseal)
   docker compose exec openbao-unsealer bao operator unseal <unsealer-key>
   ```
 
-- Caveat: the unsealer currently runs on the same host (SPOF). For real
-  separation, relocate it to another host and point `seal.hcl` at it. To make
-  host reboots fully hands-off, give the unsealer its own boot-unseal.
+- **Since 2026-09-20 this is automatic on boot.** `systemd/openbao-auto-unseal.service`
+  (installed to /etc/systemd/system/, enabled) runs `scripts/unseal-unsealer.sh` (installed to
+  /usr/local/sbin/openbao-unseal-unsealer.sh) after docker.service: it waits for the sidecar,
+  unseals it with the key from **/etc/openbao-unsealer.key** (root, 0400), restarts the main
+  node and writes the metric `openbao_unsealer_unsealed` into node_exporter's textfile dir.
+  The script is idempotent, so it can be run at any time.
+  Two pitfalls it works around: `bao operator unseal -` (key on stdin) is rejected, so the key
+  goes in as an env var; and `bao status` returns exit code 2 when sealed, which still means
+  "the container answers".
+- Caveat: the unsealer runs on the same host (SPOF) and now its key lives here too, so anyone
+  with root on this host can unseal. That key was already in `unsealer-init.json`. For real
+  separation, relocate the unsealer to another host and point `seal.hcl` at it.
 
 ## 4. Log in & use
 

config/openbao.hcl

@@ -31,3 +31,39 @@ cluster_addr = "https://openbao:8201"
 # NOTE: OpenBAO 2.x removed mlock support. Instead, disable or encrypt swap
 # on the host so secrets can't leak to disk. See:
 # https://openbao.org/docs/install/#post-installation-hardening
+
+# ---------------------------------------------------------------------------
+# Audit-Log, eingerichtet 2026-10-03.
+#
+# Warum deklarativ und nicht per `bao audit enable`: OpenBAO 2.x lehnt das ueber
+# die API ab ("cannot enable audit device via API; use declarative, config-based
+# audit device management instead") - anders als Vault. Der Block wird bei
+# Neustart UND bei SIGHUP uebernommen, ein Reload genuegt also.
+#
+# Warum ueberhaupt: ohne Audit-Device gibt es keinerlei Aufzeichnung, wer welches
+# Geheimnis gelesen hat. Bei der Fehlersuche am 2026-10-03 (Identitaet "claude"
+# ohne Rechte) haette ein Blick in dieses Log die Ursache sofort gezeigt, weil
+# die beim Login vergebenen Policies darin stehen.
+#
+# log_raw bleibt auf dem Standard false: sensible Werte werden gehasht (HMAC),
+# nicht im Klartext geschrieben. Das Log zeigt also WER WAS WANN angefasst hat,
+# nicht die Geheimnisse selbst.
+#
+# ACHTUNG, Betriebsrisiko: Audit-Devices sind BLOCKIEREND. Kann das Device nicht
+# schreiben (Platte voll, Rechte weg), weist OpenBAO Anfragen ab. Deshalb ist
+# logrotate Pflicht, siehe logrotate/openbao-audit im Repo.
+#
+# ACHTUNG, Haltbarkeit: /openbao/logs ist derzeit ein ANONYMES Docker-Volume aus
+# dem Image, nicht in der compose deklariert. Es ueberlebt Neustarts und normale
+# Recreates, aber nicht `docker compose down -v`. Die compose-Datei enthaelt
+# deshalb jetzt einen Bind-Mount auf ./logs; er greift beim naechsten
+# `docker compose up -d openbao`. Der Pfad IM Container bleibt gleich, dieser
+# Block muss dafuer also nicht angepasst werden.
+audit "file" "audit-file" {
+  description = "Audit-Log als JSON in /openbao/logs/audit.log"
+  options {
+    file_path = "/openbao/logs/audit.log"
+    mode      = "0600"
+    format    = "json"
+  }
+}

docker-compose.yml

@@ -1,6 +1,10 @@
 services:
   openbao:
-    image: openbao/openbao:2.5.5
+    # 2026-10-24: 2.6.2 -> 2.7.1. Die CLI auf dem Host ist seit dem 01.10. 2.7.1,
+    # der Server hinkte hinterher. Der Unsealer-Sidecar unten bleibt vorerst auf
+    # 2.6.2: er ist Shamir-versiegelt (1/1) und muesste nach einem Austausch von
+    # Hand entsiegelt werden - die Transit-API ist versionsuebergreifend stabil.
+    image: openbao/openbao:2.7.1
     container_name: openbao
     restart: unless-stopped
     # The image entrypoint already runs `bao server -config=/openbao/config`
@@ -18,6 +22,9 @@ services:
     volumes:
       - ./config:/openbao/config:ro
       - openbao-data:/openbao/file
+      # Audit-Log auf den Host: /openbao/logs war vorher ein anonymes
+      # Volume aus dem Image und haette ein `down -v` nicht ueberlebt.
+      - ./logs:/openbao/logs
     healthcheck:
       # 200 = unsealed+active. Accept sealed (501) and standby (429) too,
       # so the container is "healthy" once the API is responding at all.
@@ -32,7 +39,7 @@ services:
   # Minimal sidecar instance that provides transit auto-unseal for the main
   # node above. Shamir-sealed itself; only reachable on the internal network.
   openbao-unsealer:
-    image: openbao/openbao:2.5.5
+    image: openbao/openbao:2.6.2
     container_name: openbao-unsealer
     restart: unless-stopped
     command: server

keys/openbao-backup-recipient.asc

@@ -0,0 +1,41 @@
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+
+mQGNBGcxBS4BDADO4C2YrddiphCk0S7ZevxawkLonz1qFGIN5eCO5BNv5Slv+IqG
+7rr0oepQ2ZEOrb6eytV/vD4SBC2IlhpGI2HNkrZfrJgPuR3cjW+KhPUEL9s11jTu
+8fPxXO+B6Ps5mdwmR6BcLLKp/mOYUR10+1ExChl5nlYN9ZCIJM/Hn4cXlV2NqInX
+v35+qQvfvYkjUO0Y5JV7asDj3dA3YJJVfjQwggR/MEhapIBX0VHF7nEeANnHV0dT
+rlgj1krJYGrPuHHXvEK76v2jBsm24070j1jwVzqmTUohYVYaOKLpQA/V4OOTuJDQ
++NN04yrq5R8jWGtvmNcr97UZHVt7K/7XLZkfV4wp2BGfzhdgTqj+bZZJWR5EE1+4
+O4UYmd1Xt29cS2+hy48HHXF0rzBwLd9Enb/USLIjO6L5o7N25umCfc6ck7WRq2Cd
+zIM/bdveH5z3eloscxIC/ROLToTpyf+E86iXoN7F2EkAhwZShHUrx8NVHTAGpxWq
+EZsZ1R+M+3BHs30AEQEAAbQoSE9NRV9TRUNVUkUgPGx1dHouZmluc3RlcmxlQHQt
+b25saW5lLmRlPokBzgQTAQoAOBYhBKWn2wAPyhwkPzWhKAfiPcVcD892BQJnMQUu
+AhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAAAoJEAfiPcVcD892VykL/2tBpAfY
+gmsjr+UZPWWnL+S32/5ao4WAkOWJj95eeixDQtuCCCfJxkW4n3O1B97EzHpyTsUa
+U/kMujHXznyLLxwJfPiwVv3IQLr4Kd57Rf0pEjJLeDnnc91gCAtfxgu6aEC31fBJ
+bD+kNJuJlBEySMHIC/4pMptlx4AtauqwQcdJBmZKTJ6BhLeKXS+ZDsszK/l4jbg8
+h2Xo9/VqjZWLnzhzEn7eoexPtRiILYJJmf+50p5l5pJ7+okmwHHNRb8ASw14XcYW
+lnA24oe6EVot/IHLT1nOcIUrE1qJrGaqCopgV9nkF5jFoAz3S06DPTVSWDUgw+lQ
+AmyVMXjHys35CQ8gUUg+liS0xYfPupVy1H+Durch12cLEXg3J09blQyRgm4bkgEq
+bP9bXpb80SEe/lhNukytSVzc0gdPWA56nZjvT1ClToPEWMe1NCIZxKviaqCJNrOF
+R8/bPann5Jq5+Cneq3q+6UZzQ/194Sv3P1UqIVjJwG6sKcr2opOq87JSAbkBjQRn
+MQUuAQwAukFh6xHN+6n0KKN+027H6yfcaBGXPY1d3YGwgY0lH5CuxmJ+EvUg1Og2
+U+gYoMVuP+SnC9iwH/6ST8Zbir4D22sC/6yJuRBCOJfQSVGLBUymBHzj6o7Yjy4T
+4ugjDH/LkPZbw+/89T29IQiN48NaQGG7z1013LjFLqDg7u65LZwDZ98tGOeXlAyw
+vGtjYpc7WeYAzR53tv2XaUelloTPxcx0oGIHUcv8+XwTfV6K8BnNzZKJKg9WgvIC
+bcMDaQYbhn8GRNfUDAcIXnwgHwb58Id5zWZbaWycqmgUgZ/qsbVKanIplxtvkv64
+cF+dgqx0nCFldm0/bGDShHh4bCkJMhpgZuw0KYmeI++MD60Hwv1nkLHWJba5BoLi
+zgwAxmLq88nT35Qt0HIcHwqxvWPpaPd37GdxKNtohBc5LS1IfyS8iEwFbOzgP5J+
+VLmPHHTsiVI3eNHIXbxV9C84wX0uyzHNYAnbaLybr+OMnZZHRWrjTKm6cNY82/H4
+DlelR6UJABEBAAGJAbYEGAEKACAWIQSlp9sAD8ocJD81oSgH4j3FXA/PdgUCZzEF
+LgIbDAAKCRAH4j3FXA/Pdsp7DACHAFABWazH/7pdSmguyJ6I0yXFIXpyXBFNweKL
+8eDJpJZ+0ezjZvJ/tWKJ7pSjzTBYtzIfG9h9+jtHi95VOTMc8mTBed2fYzq8OQUG
+GpMF2YPe9OL/U21fwKYiDEkcy1us0vtH2ZzH7MZbCvJpUUzk4bert9btqTps0Q7g
+TnIOi8QxXTVb0L6rtyrZfkDjQSYBvs2zdMY8QjG1YqbRjAJaJ53sXgPDwHjE2Wvq
+ZMZcF1JHyycO/PnMaJYsVZYQ6OZZsfsrnyEYTS8tlqyvUIQf03qWvYcgYnZq8SYx
+D3F97/bLC891NAdSS96tT4LNq8xQbhB8oqNzUuy8OMeDsKfY755wqd01RFmp2RR2
+zkMuY0IPHxmTrMCTAQ/NDEJJ5xBeZAuokPFSLyQI9Dum8d+pO56oEPk2/dUamhhs
+igt1u1dV/J/4x5WLr1/rRF80e38+GQn67ortvitmUBKnCaxae5SuhuiLJkL3Sgum
+MN8dCQmDviD+FWnBQKDpdJ9/vV8=
+=Onln
+-----END PGP PUBLIC KEY BLOCK-----

policies/approle-rotate.hcl

@@ -0,0 +1,31 @@
+# approle-rotate — lets the rotation timer mint and prune SecretIDs for the
+# demo-app AppRole, and nothing else. Consumed via a periodic token at
+# /etc/openbao-approle-rotate.token by scripts/rotate-approle-secret-id.sh.
+#
+# Deliberately scoped to the single role: no access to the RoleID, no ability
+# to read any secret, and no reach into other AppRoles. A leak of this token
+# lets an attacker mint credentials for demo-app only -- which is why the role
+# itself stays limited to secret/demo-app/*.
+
+# Read the role config -- only for secret_id_num_uses, so the rotator knows
+# whether a verification login would burn a limited-use SecretID. This exposes
+# role settings (TTLs, bound policies) but NOT the RoleID, which lives at the
+# separate .../role-id path and stays denied.
+path "auth/approle/role/demo-app" {
+  capabilities = ["read"]
+}
+
+# Generate a new SecretID, and list existing accessors for pruning.
+path "auth/approle/role/demo-app/secret-id" {
+  capabilities = ["update", "list"]
+}
+
+# Inspect an accessor (creation_time drives which ones are safe to prune).
+path "auth/approle/role/demo-app/secret-id-accessor/lookup" {
+  capabilities = ["update"]
+}
+
+# Destroy a superseded SecretID by accessor.
+path "auth/approle/role/demo-app/secret-id-accessor/destroy" {
+  capabilities = ["update"]
+}

policies/cert-renew.hcl

@@ -1,2 +1,9 @@
 # cert-renew — materialized from live OpenBAO (policy-as-code).
 path "pki_int/issue/famfi-home" { capabilities = ["update"] }
+
+# Ergaenzt 2026-10-06 fuer renew-mikrotik-cert.sh: das Zertifikat von
+# mt-az.famfi.home liegt nicht auf einem Host, sondern muss per SSH in den
+# Router importiert werden. Damit das Skript dessen Passwort nicht auf Platte
+# oder in einer Kommandozeile braucht, holt es sich das Geheimnis zur Laufzeit.
+# Bewusst nur dieser eine Pfad, nicht secret/mikrotik/*.
+path "secret/data/mikrotik/MT-AZ/apics" { capabilities = ["read"] }

policies/claude-rw.hcl

@@ -0,0 +1,77 @@
+# Rechte der Agentenidentitaet "claude" (Entity claude-automations).
+# Umgebaut 2026-10-03 als Ersatz fuer das zu grobe path "claude/*".
+#
+# Warum nicht einfach claude/*: dieses Muster deckt auch claude/ssh/config/ca und
+# claude/ssh/roles/* MIT SCHREIBRECHT ab (mit einem Testtoken belegt). Damit koennte
+# diese Identitaet die SSH-CA ueberschreiben und eigene Rollen mit beliebigen
+# Principals anlegen - auf jedem Host, der die CA kennt, faktisch Root.
+# Aufteilung daher: Rollen definiert der Admin, signieren darf claude.
+#
+# Hinweis zur Syntax: in OpenBAO ist * NUR als letztes Zeichen eine Wildcard;
+# mitten im Pfad ist es ein literales Sternchen. Fuer ein einzelnes Segment: +.
+# "list" ist eine eigene Faehigkeit und steckt nicht in "read".
+
+# ---------- KV v2 ----------
+# Die Daten liegen unter data/, das Auflisten laeuft ueber metadata/ - zwei
+# getrennte Pfadbaeume desselben Mounts.
+path "claude/kv/data/*" {
+  capabilities = ["create", "read", "update", "patch", "delete"]
+}
+
+# Beide Formen absichtlich: claude/kv/metadata/* hat den Praefix
+# "claude/kv/metadata/" und trifft den Wurzel-Listing-Pfad ohne Schraegstrich nicht.
+path "claude/kv/metadata" {
+  capabilities = ["list"]
+}
+path "claude/kv/metadata/" {
+  capabilities = ["list"]
+}
+path "claude/kv/metadata/*" {
+  capabilities = ["list", "read", "delete"]
+}
+
+# Soft-Delete, Wiederherstellen und endgueltiges Loeschen von Versionen.
+path "claude/kv/delete/*" {
+  capabilities = ["update"]
+}
+path "claude/kv/undelete/*" {
+  capabilities = ["update"]
+}
+path "claude/kv/destroy/*" {
+  capabilities = ["update"]
+}
+
+# ---------- SSH ----------
+# Signieren mit vorhandenen Rollen ja, Rollen oder CA aendern nein.
+path "claude/ssh/sign/*" {
+  capabilities = ["update"]
+}
+path "claude/ssh/roles" {
+  capabilities = ["list"]
+}
+path "claude/ssh/roles/*" {
+  capabilities = ["read"]
+}
+# claude/ssh/config/* fehlt hier BEWUSST - siehe Kopf.
+
+# ---------- TOTP ----------
+path "claude/totp/code/*" {
+  capabilities = ["read", "update"]
+}
+path "claude/totp/keys" {
+  capabilities = ["list"]
+}
+path "claude/totp/keys/*" {
+  capabilities = ["read", "create", "update", "delete"]
+}
+
+# ---------- Sichtbarkeit der Mounts ----------
+# Die Mount-Liste der Weboberflaeche kommt aus sys/internal/ui/mounts und ist
+# policy-gefiltert, braucht das hier nicht. `bao secrets list` auf der
+# Kommandozeile liest dagegen sys/mounts und bliebe sonst leer.
+path "sys/mounts" {
+  capabilities = ["read"]
+}
+
+# Selbstverwaltung des eigenen Tokens (lookup-self, renew-self, revoke-self)
+# kommt aus der Policy "default" und steht hier deshalb nicht noch einmal.

policies/smtp-healthcheck-ro.hcl

@@ -0,0 +1,6 @@
+# Read-only access to the SMTP credentials used by the ClaudeAdmin healthcheck
+# email notifier (run-healthcheck.sh). Scoped to exactly one KV v2 path.
+# Consumed via a periodic token at ~/.config/openbao/healthcheck-smtp.token.
+path "secret/data/smtp/healthcheck" {
+  capabilities = ["read"]
+}

scripts/apply-policies.sh

@@ -20,7 +20,7 @@ TOKEN="${BAO_TOKEN:-}"
 # functional comparison: ignore comments and blank lines
 norm(){ grep -vE '^[[:space:]]*#' | grep -vE '^[[:space:]]*$' | sed 's/[[:space:]]*$//'; }
 
-echo "${DRY:+[dry-run] }policies under $DIR -> $ADDR"
+echo "$([ "$DRY" = 1 ] && printf '[dry-run] ')policies under $DIR -> $ADDR"
 for f in "$DIR"/*.hcl; do
   name="$(basename "$f" .hcl)"; filetxt="$(cat "$f")"
   live="$(curl -sS -H "X-Vault-Token: $TOKEN" "$ADDR/v1/sys/policies/acl/$name" \

scripts/offsite-backup.sh

@@ -0,0 +1,211 @@
+#!/usr/bin/env bash
+# Ship an ENCRYPTED OpenBAO disaster-recovery bundle to the NAS (fids2).
+#
+# Local snapshots live on the same Pi as OpenBAO, so a dead SD card takes both.
+# This copies them off the host. Note fids2 is on the same LAN (192.168.0.234),
+# so this is genuinely OFF-HOST but not off-site: it does not protect against
+# fire, theft, or ransomware that reaches the whole LAN.
+#
+# WHY ENCRYPTED: the CIFS share is mounted uid=1000,file_mode=0664, so the
+# root-only 0600 protection on /var/backups/openbao is LOST the moment a file
+# lands there. The bundle is therefore GPG-encrypted to HOME_SECURE before it
+# ever touches the share -- the NAS only ever holds ciphertext, which is also
+# what makes it safe to include the unseal material.
+#
+# NOT encrypted with OpenBAO's transit engine, deliberately: you would need a
+# working OpenBAO to decrypt the backup you are restoring because OpenBAO is
+# broken. GPG keeps the decryption path independent of the thing being backed up.
+#
+# This script never talks to OpenBAO, so it still works while OpenBAO is down.
+# It ships whatever the newest local snapshot is -- and FAILS LOUDLY if that
+# snapshot is stale, which is the exact failure that went unnoticed for 24 days.
+set -uo pipefail
+
+SRC_ROOT="${SRC_ROOT:-/var/backups/openbao}"
+PROJECT_DIR="/home/lutz/Projects/OpenBAO"
+SHARE_ROOT="${SHARE_ROOT:-/home/lutz/nfs_projects}"
+DEST_DIR="${DEST_DIR:-${SHARE_ROOT}/backups/openbao}"
+RECIPIENT_KEY="/etc/openbao-backup-recipient.asc"
+RECIPIENT="07E23DC55C0FCF76"          # HOME_SECURE
+KEEP="${KEEP:-30}"                    # encrypted bundles to retain on the NAS
+MAX_SNAP_AGE_H="${MAX_SNAP_AGE_H:-48}" # fail if newest local snapshot older than this
+STAMP="$(date '+%Y%m%d-%H%M%S')"
+
+log() { printf '%s [offsite] %s\n' "$(date '+%F %T')" "$*"; }
+die() { log "ERROR: $*"; exit 1; }
+
+WORK=""
+cleanup() {
+  [ -n "$WORK" ] && [ -d "$WORK" ] && rm -rf "$WORK"
+}
+trap cleanup EXIT INT TERM
+
+command -v gpg >/dev/null || die "gpg not found"
+[ -r "$RECIPIENT_KEY" ] || die "recipient public key $RECIPIENT_KEY not readable"
+
+# --- 1. locate newest local snapshots, and refuse to ship stale ones --------
+newest() { ls -1t "${SRC_ROOT}/$1"/openbao-"$1"-*.snap 2>/dev/null | head -1; }
+MAIN_SNAP="$(newest main)"
+UNSEALER_SNAP="$(newest unsealer)"
+[ -n "$MAIN_SNAP" ]     || die "no main snapshot found under ${SRC_ROOT}/main"
+[ -n "$UNSEALER_SNAP" ] || die "no unsealer snapshot found under ${SRC_ROOT}/unsealer"
+
+for s in "$MAIN_SNAP" "$UNSEALER_SNAP"; do
+  age_h=$(( ( $(date +%s) - $(stat -c %Y "$s") ) / 3600 ))
+  [ "$age_h" -le "$MAX_SNAP_AGE_H" ] \
+    || die "newest snapshot $(basename "$s") is ${age_h}h old (limit ${MAX_SNAP_AGE_H}h) -- the LOCAL backup is broken; fix that first, shipping a stale DR copy would give false confidence"
+done
+log "local snapshots fresh: $(basename "$MAIN_SNAP"), $(basename "$UNSEALER_SNAP")"
+
+# --- 2. assemble the DR set in a root-only workdir (never on the share) -----
+WORK="$(mktemp -d /root/.openbao-offsite.XXXXXX)" || die "cannot create workdir"
+chmod 0700 "$WORK"
+BUNDLE="openbao-dr-${STAMP}"
+STAGE="${WORK}/${BUNDLE}"
+mkdir -p "$STAGE" || die "cannot create staging dir"
+
+cp -p "$MAIN_SNAP" "$STAGE/" || die "cannot stage main snapshot"
+cp -p "$UNSEALER_SNAP" "$STAGE/" || die "cannot stage unsealer snapshot"
+for f in init-output.json unsealer-init.json; do
+  [ -r "${PROJECT_DIR}/${f}" ] || die "missing ${PROJECT_DIR}/${f} -- the DR set is incomplete without it"
+  cp -p "${PROJECT_DIR}/${f}" "$STAGE/" || die "cannot stage $f"
+done
+
+cat > "$STAGE/RESTORE.md" <<EOF
+# OpenBAO disaster recovery — bundle ${BUNDLE}
+
+Created: $(date -R) on $(hostname -s)
+Decrypt with the HOME_SECURE GPG private key (${RECIPIENT}).
+
+## Contents
+- $(basename "$MAIN_SNAP")      — raft snapshot, MAIN instance
+- $(basename "$UNSEALER_SNAP")  — raft snapshot, UNSEALER instance
+- init-output.json              — main: recovery keys + root token
+- unsealer-init.json            — unsealer: 1-of-1 unseal key + root token
+
+All four are needed together. The main node is transit-auto-unsealed BY the
+unsealer, so a main snapshot alone cannot be opened.
+
+## Restore order (order matters)
+
+This procedure was executed end to end as a drill on 2026-08-22 against an
+isolated copy; the non-obvious steps below are the ones that drill exposed.
+
+### Two gotchas that will bite you first
+- A FRESH node cannot accept a snapshot restore until it has been initialised
+  and unsealed. You must 'operator init' it with THROWAWAY keys first, restore
+  over the top, and then unseal with the ORIGINAL keys from this bundle. The
+  throwaway keys stop working the moment the restore lands.
+- 'docker cp' preserves the snapshot's 0600 root ownership, but since 2.6.0
+  the container runs as uid 100 (openbao) and cannot read it. Fix inside the
+  container: 'docker exec -u 0 <ctr> chmod 0644 /tmp/restore.snap'. The error
+  if you skip this is misleadingly worded:
+  "Error opening policy file: open /tmp/restore.snap: permission denied".
+
+### 1. Unsealer first
+  a. Start the unsealer container on an empty volume.
+  b. bao operator init -key-shares=1 -key-threshold=1   (throwaway keys)
+     then unseal it with that throwaway key.
+  c. docker cp the unsealer .snap in, chmod 0644 it as root (see above), then:
+     bao operator raft snapshot restore -force /tmp/restore.snap
+  d. The node now SEALS itself: the barrier is the original one again.
+     Unseal it with the key from unsealer-init.json (1-of-1 shamir).
+  e. Confirm: bao list transit/keys  ->  must show 'autounseal'.
+     The original root token in unsealer-init.json works again from here.
+
+### 2. Then main
+  a. Mint a transit token on the RESTORED unsealer for main's seal stanza:
+     bao token create -policy=autounseal -period=24h
+     (the original seal token is inside the restored data too, but minting a
+     fresh one avoids depending on a value you would have to go dig out)
+  b. Write config/seal.hcl pointing at the restored unsealer with that token,
+     mode 0644 so container uid 100 can read it (0600 breaks startup).
+  c. Start main. It comes up transit-sealed and uninitialised.
+     bao operator init -recovery-shares=1 -recovery-threshold=1  (throwaway)
+  d. docker cp the main .snap in, chmod 0644, then:
+     bao operator raft snapshot restore -force /tmp/restore.snap
+  e. Main AUTO-UNSEALS via the unsealer's transit key. No manual unseal.
+
+### 3. Verify
+- bao status: Sealed=false, Seal Type=transit, and the Cluster ID should match
+  the production cluster (proving you restored real data, not a fresh init).
+- The original root token in init-output.json authenticates.
+- Strongest check: have the restored instance decrypt a transit ciphertext
+  produced by the old instance, and issue a cert from pki_int -- that proves
+  the CA private keys and transit key material survived, not just the configs.
+
+Note: OpenBAO does NOT support downgrading a raft data dir. Restore onto the
+same version the snapshot came from (or newer), never older. These snapshots
+came from OpenBAO $(docker exec openbao bao status 2>/dev/null | awk '/^Version/{print $2}' || echo '2.6.x').
+
+## Verify integrity
+Each .snap is a gzip tar: 'gzip -t' it, extract, then 'sha256sum -c SHA256SUMS'.
+EOF
+
+# --- 3. tar + encrypt in one pass; plaintext never hits disk unencrypted ----
+GNUPGHOME_TMP="${WORK}/gnupg"
+mkdir -p "$GNUPGHOME_TMP" && chmod 0700 "$GNUPGHOME_TMP"
+GNUPGHOME="$GNUPGHOME_TMP" gpg --batch --quiet --import "$RECIPIENT_KEY" 2>/dev/null \
+  || die "could not import recipient key"
+
+ENC="${WORK}/${BUNDLE}.tar.gz.gpg"
+tar -czf - -C "$WORK" "$BUNDLE" \
+  | GNUPGHOME="$GNUPGHOME_TMP" gpg --batch --quiet --trust-model always \
+      --recipient "$RECIPIENT" --encrypt --output "$ENC" \
+  || die "tar/encrypt pipeline failed"
+[ -s "$ENC" ] || die "encrypted bundle is empty"
+
+# Plaintext staging is no longer needed — remove before touching the network.
+rm -rf "$STAGE"
+
+ENC_SIZE=$(stat -c %s "$ENC")
+ENC_SHA=$(sha256sum "$ENC" | cut -d' ' -f1)
+log "encrypted bundle built (${ENC_SIZE}B, sha256 ${ENC_SHA:0:16}...)"
+
+# Sanity: it must actually be a GPG message, not a tar that slipped through.
+head -c 3 "$ENC" | grep -q $'\x85\|\x84\|\x8c' 2>/dev/null || true
+file_type="$(file -b "$ENC" 2>/dev/null || echo unknown)"
+case "$file_type" in
+  *PGP*|*GPG*|*encrypted*) : ;;
+  *) die "refusing to upload: bundle does not look encrypted ($file_type)" ;;
+esac
+
+# --- 4. ship to the NAS ----------------------------------------------------
+# Touch the automount first so the share is live before we probe it.
+ls "$SHARE_ROOT" >/dev/null 2>&1
+
+# CRITICAL: prove the destination really is the CIFS share before writing.
+# Running as root, a bare `mkdir -p "$DEST_DIR"` SUCCEEDS even when the share
+# is not mounted -- it just creates a local directory under the automount
+# point, and every "offsite" backup silently lands on the same Pi we are
+# trying to survive the loss of. So check the filesystem type of the deepest
+# existing ancestor of DEST_DIR, not merely that some path is a mountpoint.
+ancestor="$DEST_DIR"
+while [ ! -d "$ancestor" ] && [ "$ancestor" != "/" ]; do
+  ancestor="$(dirname "$ancestor")"
+done
+fstype="$(stat -f -c %T "$ancestor" 2>/dev/null || echo unknown)"
+case "$fstype" in
+  cifs|smb2|smb3) : ;;
+  *) die "destination $DEST_DIR resolves to a '$fstype' filesystem, not the CIFS share -- refusing to write a fake 'offsite' copy onto local disk (is the automount for $SHARE_ROOT down?)" ;;
+esac
+
+mkdir -p "$DEST_DIR" || die "cannot create $DEST_DIR on the share"
+
+DEST="${DEST_DIR}/${BUNDLE}.tar.gz.gpg"
+cp "$ENC" "${DEST}.part" || die "copy to NAS failed"
+mv -f "${DEST}.part" "$DEST" || die "atomic rename on NAS failed"
+
+# --- 5. verify the copy that actually landed -------------------------------
+REMOTE_SHA=$(sha256sum "$DEST" | cut -d' ' -f1)
+[ "$REMOTE_SHA" = "$ENC_SHA" ] \
+  || die "checksum mismatch after upload (local ${ENC_SHA:0:16} vs NAS ${REMOTE_SHA:0:16})"
+log "uploaded + verified: $DEST"
+
+# --- 6. retention on the NAS ----------------------------------------------
+ls -1t "${DEST_DIR}"/openbao-dr-*.tar.gz.gpg 2>/dev/null | tail -n +$((KEEP + 1)) | while read -r old; do
+  rm -f -- "$old" && log "pruned old offsite bundle $(basename "$old")"
+done
+
+COUNT=$(ls -1 "${DEST_DIR}"/openbao-dr-*.tar.gz.gpg 2>/dev/null | wc -l)
+log "done; ${COUNT} encrypted bundle(s) offsite, retaining ${KEEP}"

scripts/renew-app-tokens.sh

@@ -0,0 +1,79 @@
+#!/usr/bin/env bash
+# Renew every scoped OpenBAO app token so periodic tokens never lapse. Periodic
+# tokens live only as long as they're renewed within their period; nothing was
+# renewing these, so they expired (2026-08-01). Runs daily via a systemd timer.
+#
+# Covers TWO sets of tokens:
+#   1. $USER_DIR/*.token         — user-owned app tokens, renewed against main.
+#   2. /etc/openbao-*.token      — root-owned infra tokens (backup, cert-renew).
+# The /etc set lapsed unnoticed for 24 days (2026-07-29..08-22) because this
+# script only walked the user dir, which broke nightly backups and would have
+# broken cert renewal — hence it now runs as root to read both.
+#
+# The unsealer's backup token belongs to the SEPARATE openbao-unsealer instance,
+# which publishes no host port, so it is renewed via `docker compose exec`
+# rather than curl. Renewing it against main would 403.
+#
+# The main node's transit SEAL token (config/seal.hcl) also lives on the
+# unsealer and was NOT covered until 2026-09-18 — it lapsed, and after the next
+# reboot main could not auto-unseal (403 on transit/encrypt/autounseal). It is
+# renewed via the same exec path, parsed out of seal.hcl.
+#
+# Best-effort: a dead token logs a failure but never aborts the rest. No secrets
+# are printed.
+set -uo pipefail
+ADDR="${BAO_ADDR:-http://127.0.0.1:8200}"
+PROJECT_DIR="/home/lutz/Projects/OpenBAO"
+USER_DIR="${TOKEN_DIR:-/home/lutz/.config/openbao}"   # explicit: $HOME is /root under the timer
+UNSEALER_TOKEN="/etc/openbao-unsealer-backup.token"
+SEAL_HCL="$PROJECT_DIR/config/seal.hcl"
+LOG="$USER_DIR/token-renew.log"
+ts="$(date '+%F %T %Z')"
+shopt -s nullglob
+
+renewed=0; failed=0
+
+ok()   { echo "$ts $1 renewed ttl=${2}s" >> "$LOG"; renewed=$((renewed+1)); }
+bad()  { echo "$ts $1 RENEW FAILED: $2" >> "$LOG"; failed=$((failed+1)); }
+
+# renew_via_api <token-file>
+renew_via_api() {
+  local tf="$1" name tok resp ttl
+  name="$(basename "$tf")"
+  tok="$(cat "$tf" 2>/dev/null)"
+  [ -n "$tok" ] || { bad "$name" "EMPTY"; return; }
+  resp="$(curl -sS --max-time 10 -H "X-Vault-Token: $tok" -X POST "$ADDR/v1/auth/token/renew-self" 2>/dev/null)"
+  ttl="$(printf '%s' "$resp" | jq -r '.auth.lease_duration // empty' 2>/dev/null)"
+  if [ -n "$ttl" ]; then ok "$name" "$ttl"
+  else bad "$name" "$(printf '%s' "$resp" | jq -c '.errors // .' 2>/dev/null)"; fi
+}
+
+# renew_via_exec <name> <token>  — for the unsealer instance (no published port)
+renew_via_exec() {
+  local name="$1" tok="$2" resp ttl
+  [ -n "$tok" ] || { bad "$name" "EMPTY"; return; }
+  # Bare `bao token renew` (no TOKEN arg) is the renew-self form; there is no
+  # -self flag in OpenBao's CLI.
+  resp="$(cd "$PROJECT_DIR" && docker compose exec -T -e BAO_TOKEN="$tok" openbao-unsealer \
+            bao token renew -format=json 2>&1)"
+  ttl="$(printf '%s' "$resp" | jq -r '.auth.lease_duration // empty' 2>/dev/null)"
+  if [ -n "$ttl" ]; then ok "$name" "$ttl"
+  else bad "$name" "$(printf '%s' "$resp" | jq -c '.errors // .' 2>/dev/null || printf '%s' "$resp" | tr '\n' ' ')"; fi
+}
+
+# 1. user-owned app tokens (*.token only — backups like *.token.bak.* don't match)
+for tf in "$USER_DIR"/*.token; do renew_via_api "$tf"; done
+
+# 2. root-owned infra tokens; the unsealer one needs the exec path
+for tf in /etc/openbao-*.token; do
+  [ -r "$tf" ] || { bad "$(basename "$tf")" "not readable (run as root)"; continue; }
+  if [ "$tf" = "$UNSEALER_TOKEN" ]; then renew_via_exec "$(basename "$tf")" "$(cat "$tf" 2>/dev/null)"; else renew_via_api "$tf"; fi
+done
+
+# 3. main's transit seal token (on the unsealer)
+renew_via_exec "seal.hcl" "$(sed -nE 's/^[[:space:]]*token[[:space:]]*=[[:space:]]*"([^"]+)".*/\1/p' "$SEAL_HCL" 2>/dev/null)"
+
+echo "$ts summary: renewed=$renewed failed=$failed" >> "$LOG"
+# Running as root must not leave the log root-owned for the next user-context read.
+chown lutz:lutz "$LOG" 2>/dev/null || true
+[ "$failed" -eq 0 ]

scripts/renew-gitea-cert.sh

@@ -0,0 +1,75 @@
+#!/usr/bin/env bash
+# Renew the gitea.famfi.home leaf cert (Kopie von renew-openbao-cert.sh, 2026-09-20) from OpenBAO's pki_int engine and
+# install it where Traefik serves it. Idempotent: only renews when the current
+# cert expires within $RENEW_WINDOW_DAYS. Run as root (writes Traefik's tls dir).
+#
+# Installed as a systemd timer (gitea-cert-renew.timer). Manual run:
+#   sudo /home/lutz/Projects/OpenBAO/scripts/renew-gitea-cert.sh [--force]
+set -euo pipefail
+
+ADDR="${BAO_ADDR:-http://127.0.0.1:8200}"
+TOKEN_FILE="/etc/openbao-cert-renew.token"
+ROLE="pki_int/issue/famfi-home"
+CN="gitea.famfi.home"
+TTL="2160h"                                   # 90 days
+DEST="/srv/TRAEFIK/etc/traefik/tls/gitea"
+DYN="/srv/TRAEFIK/etc/traefik/traefik.d/tls-gitea.yml"
+RENEW_WINDOW_DAYS="${RENEW_WINDOW_DAYS:-21}"  # renew when <= this many days left
+FORCE="${1:-}"
+
+log() { printf '%s [renew-cert] %s\n' "$(date '+%F %T')" "$*"; }
+die() { log "ERROR: $*"; exit 1; }
+
+[ -r "$TOKEN_FILE" ] || die "token file $TOKEN_FILE not readable (run as root?)"
+TOKEN="$(cat "$TOKEN_FILE")"
+
+# Skip if the current cert is still good (unless --force)
+if [ "$FORCE" != "--force" ] && [ -f "$DEST/fullchain.pem" ]; then
+  end="$(openssl x509 -in "$DEST/fullchain.pem" -noout -enddate | cut -d= -f2)"
+  days_left=$(( ( $(date -d "$end" +%s) - $(date +%s) ) / 86400 ))
+  if [ "$days_left" -gt "$RENEW_WINDOW_DAYS" ]; then
+    log "cert valid ${days_left}d (> ${RENEW_WINDOW_DAYS}d) — nothing to do"
+    exit 0
+  fi
+  log "cert has ${days_left}d left (<= ${RENEW_WINDOW_DAYS}d) — renewing"
+fi
+
+# Issue a fresh cert
+resp="$(curl -sS --fail-with-body --max-time 15 \
+  -H "X-Vault-Token: ${TOKEN}" \
+  --data "{\"common_name\":\"${CN}\",\"ttl\":\"${TTL}\"}" \
+  "${ADDR}/v1/${ROLE}")" || die "issue request failed"
+
+tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT
+export RESP="$resp"
+python3 - "$tmp" <<PY
+import json, sys, os
+d = json.loads(os.environ["RESP"])["data"]
+t = sys.argv[1]
+open(f"{t}/fullchain.pem", "w").write(d["certificate"] + "\n" + "\n".join(d.get("ca_chain", [])) + "\n")
+open(f"{t}/privkey.pem", "w").write(d["private_key"] + "\n")
+PY
+
+# Validate cert/key match before installing
+cmod="$(openssl x509 -in "$tmp/fullchain.pem" -noout -modulus | openssl md5)"
+kmod="$(openssl rsa  -in "$tmp/privkey.pem"  -noout -modulus 2>/dev/null | openssl md5)"
+[ "$cmod" = "$kmod" ] || die "cert/key modulus mismatch — refusing to install"
+
+install -d -o root -g root -m 0755 "$DEST"
+install -o root -g root -m 0644 "$tmp/fullchain.pem" "$DEST/fullchain.pem"
+install -o root -g root -m 0600 "$tmp/privkey.pem"  "$DEST/privkey.pem"
+
+# Rewrite the watched dynamic config (with a fresh timestamp) so Traefik's file
+# provider reparses and reloads the cert from disk — changing the cert file
+# alone does NOT trigger a reload.
+cat > "$DYN" <<YML
+# AUTO-MANAGED by renew-gitea-cert.sh — last renewed $(date -Iseconds).
+# Loads the internal cert for gitea.famfi.home; the gitea router serves it by SNI
+# (oeffentlich laeuft gitea.famfi.dyndns.org weiter ueber Lets Encrypt).
+tls:
+  certificates:
+    - certFile: /etc/traefik/tls/gitea/fullchain.pem
+      keyFile: /etc/traefik/tls/gitea/privkey.pem
+YML
+chmod 0644 "$DYN"
+log "installed new cert ($(openssl x509 -in "$DEST/fullchain.pem" -noout -enddate | cut -d= -f2)); Traefik reload triggered"

scripts/renew-mikrotik-cert.sh

@@ -0,0 +1,159 @@
+#!/usr/bin/env bash
+# Renew the mt-az.famfi.home API-SSL cert (Kopie von renew-gitea-cert.sh, 2026-10-06)
+# from OpenBAO's pki_int engine and install it on the MikroTik.
+#
+# Warum es dieses Skript gibt: das Zertifikat lief am 2026-09-26 ab und hat damit
+# den CrowdSec-MikroTik-Bouncer getoetet - 967 Neustartversuche mit
+# "x509: certificate has expired", 10 Tage lang keine Sperren in der
+# Router-Firewall. Niemand hat es bemerkt, weil mt-az:8729 in keiner
+# Blackbox-Probe steht und es - anders als fuer openbao und gitea - kein
+# Erneuerungsskript gab.
+#
+# Idempotent: erneuert nur, wenn das Zertifikat am Router innerhalb von
+# $RENEW_WINDOW_DAYS ablaeuft. Als root laufen (liest das Token).
+#   sudo /home/lutz/Projects/OpenBAO/scripts/renew-mikrotik-cert.sh [--force]
+set -euo pipefail
+
+ADDR="${BAO_ADDR:-http://127.0.0.1:8200}"
+TOKEN_FILE="/etc/openbao-cert-renew.token"
+ROLE="pki_int/issue/famfi-home"
+CN="mt-az.famfi.home"
+TTL="2160h"                                   # 90 Tage, wie bei openbao/gitea
+RT_HOST="192.168.0.2"
+RT_USER="apics"                               # group=full, address=192.168.0.0/24
+RENEW_WINDOW_DAYS="${RENEW_WINDOW_DAYS:-21}"
+# Fuer die vollstaendige Kette muss die Zwischen-CA im Zertifikatsspeicher des
+# Routers liegen (2026-10-22 als famfi-int-ca-20261022.crt_0 importiert). RouterOS
+# verknuepft Blatt und CA beim IMPORT des Blattes; fehlt die CA, liefert api-ssl nur
+# das Blatt aus und strenge Clients scheitern mit "unable to get local issuer
+# certificate". Wird die CA je entfernt, faellt das unten in der Kettenpruefung auf.
+ROOT_CA="${ROOT_CA:-/home/lutz/.config/famfi-ca/famfi-home-root-ca.pem}"
+FORCE="${1:-}"
+
+log() { printf '%s [renew-mikrotik-cert] %s\n' "$(date '+%F %T')" "$*"; }
+die() { log "ERROR: $*"; exit 1; }
+
+[ -r "$TOKEN_FILE" ] || die "token file $TOKEN_FILE not readable (run as root?)"
+TOKEN="$(cat "$TOKEN_FILE")"
+
+# Passwort des Router-Benutzers: zur Laufzeit aus OpenBAO, damit es nirgends
+# auf Platte liegt. Die Policy cert-renew braucht dafuer read auf
+# secret/data/mikrotik/MT-AZ/apics. Solange das nicht erteilt ist, kann der
+# Aufrufer MIKROTIK_PASS setzen (nur fuer manuelle Laeufe gedacht).
+SSHPASS="${MIKROTIK_PASS:-}"
+if [ -z "$SSHPASS" ]; then
+  SSHPASS="$(curl -sS --max-time 15 -H "X-Vault-Token: ${TOKEN}" \
+    "${ADDR}/v1/secret/data/mikrotik/MT-AZ/apics" 2>/dev/null \
+    | jq -r '.data.data.password // empty')" || true
+fi
+[ -n "$SSHPASS" ] || die "kein Router-Passwort (Policy cert-renew erweitern oder MIKROTIK_PASS setzen)"
+export SSHPASS
+
+rt() { sshpass -e ssh -o PreferredAuthentications=password -o PubkeyAuthentication=no \
+        -o StrictHostKeyChecking=accept-new -o ConnectTimeout=15 "${RT_USER}@${RT_HOST}" "$@"; }
+
+# Restlaufzeit direkt am Router messen - verlaesslicher als eine lokale Kopie,
+# denn entscheidend ist, was api-ssl tatsaechlich vorzeigt.
+if [ "$FORCE" != "--force" ]; then
+  end="$(echo | timeout 20 openssl s_client -connect "${RT_HOST}:8729" 2>/dev/null \
+         | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2 || true)"
+  if [ -n "$end" ]; then
+    days_left=$(( ( $(date -d "$end" +%s) - $(date +%s) ) / 86400 ))
+    if [ "$days_left" -gt "$RENEW_WINDOW_DAYS" ]; then
+      log "cert valid ${days_left}d (> ${RENEW_WINDOW_DAYS}d) - nothing to do"; exit 0
+    fi
+    log "cert has ${days_left}d left (<= ${RENEW_WINDOW_DAYS}d) - renewing"
+  else
+    log "konnte die Restlaufzeit nicht lesen - erneuere vorsorglich"
+  fi
+fi
+
+resp="$(curl -sS --fail-with-body --max-time 15 -H "X-Vault-Token: ${TOKEN}" \
+  --data "{\"common_name\":\"${CN}\",\"ttl\":\"${TTL}\"}" "${ADDR}/v1/${ROLE}")" \
+  || die "issue request failed"
+
+tmp="$(mktemp -d)"; chmod 700 "$tmp"
+cleanup() { rm -rf "$tmp"; }
+trap cleanup EXIT
+export RESP="$resp"
+python3 - "$tmp" <<'PY'
+import json, os, sys
+d = json.loads(os.environ["RESP"])["data"]
+t = sys.argv[1]
+open(f"{t}/cert.pem", "w").write(d["certificate"] + "\n")
+open(f"{t}/key.pem", "w").write(d["private_key"] + "\n")
+PY
+chmod 600 "$tmp/key.pem"
+
+cmod="$(openssl x509 -in "$tmp/cert.pem" -noout -modulus | openssl md5)"
+kmod="$(openssl rsa  -in "$tmp/key.pem"  -noout -modulus 2>/dev/null | openssl md5)"
+[ "$cmod" = "$kmod" ] || die "cert/key modulus mismatch - refusing to install"
+
+# Datumsstempel im Namen: RouterOS leitet den Zertifikatsnamen vom Dateinamen ab.
+# Mit gleichem Namen gaebe es Kollisionen mit dem noch vorhandenen alten Objekt.
+STAMP="$(date +%Y%m%d%H%M)"
+BASE="mt-az-${STAMP}"
+sshpass -e scp -o PreferredAuthentications=password -o PubkeyAuthentication=no \
+  -o StrictHostKeyChecking=accept-new -o ConnectTimeout=15 \
+  "$tmp/cert.pem" "${RT_USER}@${RT_HOST}:${BASE}.crt" >/dev/null
+sshpass -e scp -o PreferredAuthentications=password -o PubkeyAuthentication=no \
+  -o StrictHostKeyChecking=accept-new -o ConnectTimeout=15 \
+  "$tmp/key.pem" "${RT_USER}@${RT_HOST}:${BASE}.key" >/dev/null
+log "Dateien hochgeladen: ${BASE}.crt/.key"
+
+# Leeres passphrase, damit nichts Geheimes in der Kommandozeile steht - RouterOS
+# protokolliert Kommandos mit.
+rt "/certificate import file-name=${BASE}.crt passphrase=\"\"" >/dev/null
+rt "/certificate import file-name=${BASE}.key passphrase=\"\"" >/dev/null
+
+# Feldweise zerlegen statt mit einem Regex ueber die ganze Zeile: in der
+# terse-Ausgabe steht auch subject-alt-name=, und ein gieriges ".*name=" trifft
+# genau das. Beim ersten Lauf kam dadurch "DNS:mt-az.famfi.home" heraus.
+NEWNAME="$(rt "/certificate print terse where name~\"${BASE}\"" \
+           | tr ' ' '\n' | grep -m1 '^name=' | cut -d= -f2)"
+[ -n "$NEWNAME" ] || die "importiertes Zertifikat nicht gefunden"
+log "importiert als ${NEWNAME}"
+
+OLD="$(rt '/ip service print terse where name="api-ssl"' \
+       | tr ' ' '\n' | grep -m1 '^certificate=' | cut -d= -f2)"
+# Direkt beim Namen ansprechen statt ueber [find ...]: letzteres liefert bei
+# /ip service auch den dynamischen Verbindungseintrag und quittierte das
+# Setzen mit einer irritierenden Meldung ueber telnet.
+rt "/ip service set api-ssl certificate=${NEWNAME}"
+log "api-ssl nutzt jetzt ${NEWNAME} (vorher ${OLD:-unbekannt})"
+
+# Hochgeladene Dateien entfernen - der private Schluessel hat im Flash nichts zu suchen.
+rt "/file remove [find name~\"${BASE}\"]" >/dev/null 2>&1 || true
+# Altes Objekt erst nach dem Umschalten loeschen.
+if [ -n "${OLD:-}" ] && [ "$OLD" != "$NEWNAME" ] && [ "$OLD" != "none" ]; then
+  rt "/certificate remove [find name=\"${OLD}\"]" >/dev/null 2>&1 \
+    && log "altes Zertifikat ${OLD} entfernt" || log "altes Zertifikat ${OLD} blieb stehen"
+fi
+
+sleep 2
+got="$(echo | timeout 20 openssl s_client -connect "${RT_HOST}:8729" 2>/dev/null \
+       | openssl x509 -noout -subject -enddate 2>/dev/null || true)"
+log "api-ssl zeigt jetzt vor: $(echo "$got" | tr '\n' ' ')"
+# Abschliessend pruefen, dass wirklich ein gueltiges Zertifikat ausgeliefert
+# wird - sonst war die Erneuerung wirkungslos und muss auffallen.
+end_new="$(echo "$got" | sed -n 's/^notAfter=//p')"
+[ -n "$end_new" ] || die "api-ssl liefert kein Zertifikat mehr aus"
+left=$(( ( $(date -d "$end_new" +%s) - $(date +%s) ) / 86400 ))
+[ "$left" -gt 0 ] || die "ausgeliefertes Zertifikat ist nicht gueltig (${left}d)"
+log "OK - noch ${left} Tage gueltig"
+
+# Kette pruefen: gegen die ROOT allein verifizieren. Das gelingt nur, wenn der
+# Router die Zwischen-CA mitschickt. Kein die(), denn das Zertifikat selbst ist
+# in Ordnung - es ist ein Mangel, kein Ausfall, und ein Abbruch hier wuerde einen
+# erfolgreichen Austausch als Fehlschlag melden.
+if [ -r "$ROOT_CA" ]; then
+  if echo | timeout 20 openssl s_client -connect "${RT_HOST}:8729" \
+       -CAfile "$ROOT_CA" 2>&1 | grep -q "Verify return code: 0"; then
+    log "Kette vollstaendig (gegen die Root-CA verifiziert)"
+  else
+    log "WARNUNG: api-ssl liefert keine vollstaendige Kette - liegt die Zwischen-CA"
+    log "         noch im Speicher? /certificate print where common-name~\"Intermediate\""
+  fi
+else
+  log "Kettenpruefung uebersprungen: $ROOT_CA nicht lesbar"
+fi

scripts/rotate-approle-secret-id.sh

@@ -0,0 +1,114 @@
+#!/usr/bin/env bash
+# Rotate the demo-app AppRole SecretID before its 30d TTL expires.
+#
+# WHY THIS EXISTS: unlike tokens, a SecretID cannot be renewed -- it must be
+# re-issued by a privileged caller. So renew-app-tokens.sh structurally cannot
+# cover it, and the original SecretID silently expired on ~2026-07-28, leaving
+# scripts/app-get-secret.sh broken until it was noticed on 2026-08-22.
+#
+# ORDER OF OPERATIONS IS THE SAFETY PROPERTY. We mint, then PROVE the new
+# SecretID actually authenticates, and only then overwrite the file and prune
+# old accessors. A failure at any step leaves the previous working credential
+# untouched and exits non-zero, so OnFailure= mails the alert.
+#
+# Old accessors are kept one cycle deep (KEEP=2: current + previous) so a
+# consumer that read the file microseconds before rotation can still log in.
+#
+# Installed as openbao-approle-rotate.timer (weekly -- 4x margin on a 30d TTL,
+# so three consecutive failures can occur before anything actually breaks).
+set -uo pipefail
+
+ADDR="${BAO_ADDR:-http://127.0.0.1:8200}"
+ROLE="${ROLE:-demo-app}"
+TOKEN_FILE="/etc/openbao-approle-rotate.token"
+OUT="${OUT:-/home/lutz/.config/openbao/approle/secret_id}"
+OWNER="lutz:lutz"
+KEEP="${KEEP:-2}"          # accessors to retain: current + previous
+
+log() { printf '%s [approle-rotate] %s\n' "$(date '+%F %T')" "$*"; }
+die() { log "ERROR: $*"; exit 1; }
+
+[ -r "$TOKEN_FILE" ] || die "token file $TOKEN_FILE not readable (run as root?)"
+TOK="$(cat "$TOKEN_FILE")"
+[ -n "$TOK" ] || die "token file $TOKEN_FILE is empty"
+command -v jq >/dev/null || die "jq not found"
+
+api() { # <method> <path> [data]
+  local m="$1" p="$2" d="${3:-}"
+  if [ -n "$d" ]; then
+    curl -sS --max-time 15 -H "X-Vault-Token: $TOK" -X "$m" -d "$d" "$ADDR/v1/$p"
+  else
+    curl -sS --max-time 15 -H "X-Vault-Token: $TOK" -X "$m" "$ADDR/v1/$p"
+  fi
+}
+
+# --- 0. sanity: a test login must not burn a limited-use SecretID -----------
+NUM_USES="$(api GET "auth/approle/role/$ROLE" | jq -r '.data.secret_id_num_uses // empty')"
+[ -n "$NUM_USES" ] || die "cannot read role $ROLE (token scoped wrong, or role gone)"
+VERIFY=1
+if [ "$NUM_USES" != "0" ] && [ "$NUM_USES" -le 1 ] 2>/dev/null; then
+  VERIFY=0
+  log "NOTE secret_id_num_uses=$NUM_USES -- skipping verification login so it is not consumed"
+fi
+
+# --- 1. mint ---------------------------------------------------------------
+RESP="$(api POST "auth/approle/role/$ROLE/secret-id" \
+        "{\"metadata\":\"{\\\"rotated_at\\\":\\\"$(date -u '+%FT%TZ')\\\",\\\"by\\\":\\\"rotate-approle-secret-id.sh\\\"}\"}")"
+NEW_SID="$(printf '%s' "$RESP" | jq -r '.data.secret_id // empty')"
+NEW_ACC="$(printf '%s' "$RESP" | jq -r '.data.secret_id_accessor // empty')"
+[ -n "$NEW_SID" ] && [ -n "$NEW_ACC" ] \
+  || die "mint failed: $(printf '%s' "$RESP" | jq -c '.errors // .')"
+log "minted new SecretID (accessor ${NEW_ACC:0:8}...)"
+
+# --- 2. PROVE it works before touching anything ----------------------------
+if [ "$VERIFY" = "1" ]; then
+  RID="$(cat "$(dirname "$OUT")/role_id" 2>/dev/null)"
+  [ -n "$RID" ] || die "role_id file missing next to $OUT; refusing to rotate blind"
+  LOGIN="$(curl -sS --max-time 15 --data "$(RID="$RID" SID="$NEW_SID" python3 -c \
+            'import json,os;print(json.dumps({"role_id":os.environ["RID"],"secret_id":os.environ["SID"]}))')" \
+            "$ADDR/v1/auth/approle/login" 2>/dev/null)"
+  TESTTOK="$(printf '%s' "$LOGIN" | jq -r '.auth.client_token // empty')"
+  if [ -z "$TESTTOK" ]; then
+    api POST "auth/approle/role/$ROLE/secret-id-accessor/destroy" \
+        "{\"secret_id_accessor\":\"$NEW_ACC\"}" >/dev/null 2>&1
+    die "new SecretID failed verification login; destroyed it, left existing credential in place: $(printf '%s' "$LOGIN" | jq -c '.errors // .')"
+  fi
+  # don't leave a live token lying around just because we tested
+  curl -sS --max-time 10 -H "X-Vault-Token: $TESTTOK" -X POST \
+       "$ADDR/v1/auth/token/revoke-self" >/dev/null 2>&1 || true
+  log "verification login OK (test token revoked)"
+fi
+
+# --- 3. install atomically -------------------------------------------------
+TMP="${OUT}.tmp.$$"
+( umask 077; printf '%s' "$NEW_SID" > "$TMP" ) || die "cannot write $TMP"
+chmod 0600 "$TMP" || die "chmod failed on $TMP"
+chown "$OWNER" "$TMP" 2>/dev/null || log "WARN could not chown $TMP to $OWNER"
+mv -f "$TMP" "$OUT" || die "atomic replace of $OUT failed"
+log "installed new SecretID at $OUT"
+
+# --- 4. prune superseded accessors (keep current + previous) ---------------
+mapfile -t ACCS < <(api LIST "auth/approle/role/$ROLE/secret-id" | jq -r '.data.keys[]? // empty')
+if [ "${#ACCS[@]}" -gt "$KEEP" ]; then
+  # order by creation_time, newest first
+  ORDERED="$(for a in "${ACCS[@]}"; do
+      ct="$(api POST "auth/approle/role/$ROLE/secret-id-accessor/lookup" \
+             "{\"secret_id_accessor\":\"$a\"}" | jq -r '.data.creation_time // empty')"
+      [ -n "$ct" ] && printf '%s\t%s\n' "$ct" "$a"
+    done | sort -r)"
+  n=0
+  while IFS=$'\t' read -r ct a; do
+    [ -n "$a" ] || continue
+    n=$((n+1))
+    [ "$n" -le "$KEEP" ] && continue
+    if api POST "auth/approle/role/$ROLE/secret-id-accessor/destroy" \
+         "{\"secret_id_accessor\":\"$a\"}" >/dev/null 2>&1; then
+      log "pruned superseded accessor ${a:0:8}... (created $ct)"
+    else
+      log "WARN could not prune accessor ${a:0:8}..."
+    fi
+  done <<< "$ORDERED"
+fi
+
+REMAIN="$(api LIST "auth/approle/role/$ROLE/secret-id" | jq -r '.data.keys | length // 0')"
+log "done; $REMAIN SecretID accessor(s) live for role $ROLE"

scripts/send-failure-alert.sh

@@ -0,0 +1,114 @@
+#!/usr/bin/env bash
+# OnFailure handler for the OpenBAO systemd timers. Invoked as:
+#   send-failure-alert.sh <failed-unit-name>
+# by openbao-alert@.service, which the timer units reference via OnFailure=.
+#
+# Backups died silently for 24 days (2026-07-29..08-22) because a failed
+# oneshot unit makes no noise. This makes it make noise.
+#
+# CIRCULAR-DEPENDENCY NOTE: the SMTP password lives in OpenBAO, but the most
+# likely reason one of these units failed is that OpenBAO itself is down or
+# sealed -- in which case fetching the password would fail too, and the alert
+# would be lost exactly when it matters most. So:
+#   1. every alert is ALWAYS appended to $ALERT_LOG first, before any network
+#      call, so a durable record exists even with no OpenBAO and no internet;
+#   2. creds are fetched from OpenBAO when it is reachable, and cached to a
+#      root-only 0600 file that is used as the fallback when it is not.
+# The cache is a deliberate trade-off, consistent with the house rule that
+# secrets live in root-owned 0600 files or come from the store at runtime.
+set -uo pipefail
+
+UNIT="${1:-unknown.unit}"
+HOST="$(hostname -s 2>/dev/null || echo pi)"
+TS="$(date '+%F %T %Z')"
+
+BAO_ADDR="${BAO_ADDR:-http://127.0.0.1:8200}"
+TOKEN_FILE="/etc/openbao-alert-smtp.token"
+CACHE_FILE="/etc/openbao-smtp-cache"
+ALERT_LOG="/var/log/openbao-alerts.log"
+
+SMTP_HOST="securesmtp.t-online.de"; SMTP_PORT="587"
+MAIL_FROM="lutz.finsterle@t-online.de"; MAIL_TO="lutz.finsterle@t-online.de"
+
+# --- 1. gather context ------------------------------------------------------
+STATUS="$(systemctl status --no-pager --full "$UNIT" 2>&1 | head -40)"
+JOURNAL="$(journalctl -u "$UNIT" -n 40 --no-pager 2>&1 | tail -40)"
+RESULT="$(systemctl show -p Result --value "$UNIT" 2>/dev/null)"
+EXECMAIN="$(systemctl show -p ExecMainStatus --value "$UNIT" 2>/dev/null)"
+
+BODY="OpenBAO maintenance unit FAILED on ${HOST}.
+
+Unit:        ${UNIT}
+When:        ${TS}
+Result:      ${RESULT:-unknown} (exit status ${EXECMAIN:-?})
+
+This unit is part of the OpenBAO safety net (raft snapshots, TLS cert renewal,
+scoped-token renewal). A failure here is silent by default -- if you are reading
+this, the alerting is doing its job. Investigate promptly: a lapsed token or a
+missed snapshot degrades quietly and is easy to miss for weeks.
+
+--- systemctl status ---
+${STATUS}
+
+--- last 40 journal lines ---
+${JOURNAL}
+"
+
+# --- 2. durable local record FIRST (never depends on OpenBAO or the network) --
+{
+  echo "===== ${TS} ${UNIT} ====="
+  printf '%s\n\n' "$BODY"
+} >> "$ALERT_LOG" 2>/dev/null
+chmod 0600 "$ALERT_LOG" 2>/dev/null || true
+
+# --- 3. resolve SMTP creds: OpenBAO first, cached copy as fallback -----------
+user=""; pass=""; cred_src=""
+
+if [ -r "$TOKEN_FILE" ]; then
+  tok="$(cat "$TOKEN_FILE" 2>/dev/null)"
+  if [ -n "$tok" ]; then
+    resp="$(curl -sS --max-time 10 -H "X-Vault-Token: $tok" \
+              "$BAO_ADDR/v1/secret/data/smtp/healthcheck" 2>/dev/null)"
+    user="$(printf '%s' "$resp" | jq -r '.data.data.username // empty' 2>/dev/null)"
+    pass="$(printf '%s' "$resp" | jq -r '.data.data.password // empty' 2>/dev/null)"
+    if [ -n "$user" ] && [ -n "$pass" ]; then
+      cred_src="openbao"
+      # refresh the offline fallback copy
+      umask 077
+      printf '%s\n%s\n' "$user" "$pass" > "${CACHE_FILE}.tmp" 2>/dev/null \
+        && chmod 0600 "${CACHE_FILE}.tmp" 2>/dev/null \
+        && mv -f "${CACHE_FILE}.tmp" "$CACHE_FILE" 2>/dev/null
+    fi
+  fi
+fi
+
+if { [ -z "$user" ] || [ -z "$pass" ]; } && [ -r "$CACHE_FILE" ]; then
+  user="$(sed -n 1p "$CACHE_FILE" 2>/dev/null)"
+  pass="$(sed -n 2p "$CACHE_FILE" 2>/dev/null)"
+  cred_src="cache (OpenBAO unreachable -- it may itself be the problem)"
+fi
+
+if [ -z "$user" ] || [ -z "$pass" ]; then
+  echo "${TS} ${UNIT}: ALERT EMAIL NOT SENT -- no SMTP creds from OpenBAO or cache" >> "$ALERT_LOG"
+  logger -t openbao-alert "FAILED unit ${UNIT}; could not send email (no SMTP creds)"
+  exit 0   # never fail the handler: that would just add noise, not signal
+fi
+
+# --- 4. send ----------------------------------------------------------------
+msg="$(printf 'From: %s\r\nTo: %s\r\nSubject: [ALERT] %s: %s failed\r\nDate: %s\r\nContent-Type: text/plain; charset=UTF-8\r\n\r\n%s\r\n' \
+      "$MAIL_FROM" "$MAIL_TO" "$HOST" "$UNIT" "$(date -R)" "$BODY (creds via ${cred_src})")"
+
+if printf '%s' "$msg" | curl -sS --max-time 30 --ssl-reqd \
+     --url "smtp://$SMTP_HOST:$SMTP_PORT" --user "$user:$pass" \
+     --mail-from "$MAIL_FROM" --mail-rcpt "$MAIL_TO" --upload-file - 2>>"$ALERT_LOG"; then
+  echo "${TS} ${UNIT}: alert email sent (creds via ${cred_src})" >> "$ALERT_LOG"
+  logger -t openbao-alert "FAILED unit ${UNIT}; alert email sent"
+else
+  echo "${TS} ${UNIT}: ALERT EMAIL FAILED TO SEND (creds via ${cred_src})" >> "$ALERT_LOG"
+  logger -t openbao-alert "FAILED unit ${UNIT}; alert email could NOT be sent"
+fi
+
+# Keep the log bounded.
+tail -n 2000 "$ALERT_LOG" > "${ALERT_LOG}.tmp" 2>/dev/null && mv -f "${ALERT_LOG}.tmp" "$ALERT_LOG" 2>/dev/null
+chmod 0600 "$ALERT_LOG" 2>/dev/null || true
+exit 0

scripts/unseal-unsealer.sh

@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+# Entsiegelt die openbao-unsealer-Sidecar nach einem Host-Neustart.
+#
+# Warum: der Unsealer ist Shamir-versiegelt (1 Schluessel). Bis er offen ist, kann die
+# Hauptinstanz ihren Transit-Seal nicht aufmachen und startet endlos neu (2026-09-20 beobachtet,
+# erster Reboot seit 106 Tagen). Danach entsiegelt sich die Hauptinstanz von selbst.
+#
+# Der Schluessel steht in /etc/openbao-unsealer.key (root, 0400). Das ist bewusst ein Kompromiss:
+# wer root auf diesem Host hat, kann entsiegeln — er liegt aber ohnehin schon in
+# ~/Projects/OpenBAO/unsealer-init.json. Echte Trennung braeuchte den Unsealer auf einem
+# anderen Host (siehe README, Abschnitt Auto-unseal).
+set -uo pipefail
+
+KEY_FILE="${KEY_FILE:-/etc/openbao-unsealer.key}"
+CONTAINER="${CONTAINER:-openbao-unsealer}"
+MAIN="${MAIN:-openbao}"
+TEXTFILE_DIR="${TEXTFILE_DIR:-/var/lib/prometheus/node-exporter}"
+PROM="${TEXTFILE_DIR}/openbao_unseal.prom"
+TIMEOUT="${TIMEOUT:-300}"     # Sekunden warten, bis Docker und der Container da sind
+
+log() { printf '%s [unseal] %s\n' "$(date '+%F %T')" "$*"; }
+
+metric() {  # $1 = 1 (offen) | 0 (zu)
+  [ -d "$TEXTFILE_DIR" ] || return 0
+  {
+    echo "# HELP openbao_unsealer_unsealed 1 = openbao-unsealer ist entsiegelt"
+    echo "# TYPE openbao_unsealer_unsealed gauge"
+    echo "openbao_unsealer_unsealed $1"
+    echo "# HELP openbao_unsealer_last_attempt_timestamp_seconds Letzter Entsiegelungsversuch"
+    echo "# TYPE openbao_unsealer_last_attempt_timestamp_seconds gauge"
+    echo "openbao_unsealer_last_attempt_timestamp_seconds $(date +%s)"
+  } > "${PROM}.tmp" && mv "${PROM}.tmp" "$PROM" && chmod 644 "$PROM"
+}
+
+[ -r "$KEY_FILE" ] || { log "ABBRUCH: $KEY_FILE fehlt oder ist nicht lesbar"; metric 0; exit 1; }
+
+# 1. Warten, bis der Container laeuft und seine API antwortet.
+# 'bao status' liefert 0 = offen, 2 = versiegelt, 1 = nicht erreichbar. 0 und 2 heissen
+# beide "der Container antwortet" — genau darauf wird gewartet.
+deadline=$(( $(date +%s) + TIMEOUT ))
+while :; do
+  docker exec "$CONTAINER" bao status >/dev/null 2>&1
+  rc=$?
+  case "$rc" in 0|2) break;; esac
+  [ "$(date +%s)" -lt "$deadline" ] || { log "ABBRUCH: $CONTAINER antwortet nicht binnen ${TIMEOUT}s (rc=$rc)"; metric 0; exit 1; }
+  sleep 5
+done
+
+# 2. Schon offen? Dann nichts tun (idempotent — die Unit darf jederzeit laufen).
+if docker exec "$CONTAINER" bao status 2>/dev/null | grep -qE '^Sealed[[:space:]]+false'; then
+  log "$CONTAINER ist bereits entsiegelt"
+  metric 1
+  exit 0
+fi
+
+# 3. Entsiegeln. Der Schluessel geht als Umgebungsvariable in den Container;
+#    'bao operator unseal -' (stdin) funktioniert nicht, es lehnt den Schluessel ab.
+UK="$(tr -d '\r\n' < "$KEY_FILE")"
+docker exec -e UK="$UK" "$CONTAINER" sh -c 'bao operator unseal "$UK"' >/dev/null 2>&1
+unset UK
+
+if docker exec "$CONTAINER" bao status 2>/dev/null | grep -qE '^Sealed[[:space:]]+false'; then
+  log "$CONTAINER entsiegelt"
+  metric 1
+else
+  log "FEHLER: $CONTAINER ist weiterhin versiegelt"
+  metric 0
+  exit 1
+fi
+
+# 4. Die Hauptinstanz anstupsen, damit sie nicht erst den naechsten Restart-Zyklus abwartet.
+if docker inspect "$MAIN" >/dev/null 2>&1; then
+  docker restart "$MAIN" >/dev/null 2>&1 && log "$MAIN neu gestartet"
+  for i in $(seq 1 24); do
+    docker exec "$MAIN" bao status 2>/dev/null | grep -qE '^Sealed[[:space:]]+false' && { log "$MAIN ist entsiegelt"; exit 0; }
+    sleep 5
+  done
+  log "HINWEIS: $MAIN ist nach 120s noch nicht entsiegelt — Logs pruefen"
+fi
+exit 0

systemd/gitea-cert-renew.service

@@ -0,0 +1,9 @@
+[Unit]
+Description=Erneuert gitea.famfi.home aus der OpenBAO-PKI und laedt Traefik neu
+After=network-online.target docker.service
+Wants=network-online.target
+Requires=docker.service
+
+[Service]
+Type=oneshot
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/renew-gitea-cert.sh

systemd/gitea-cert-renew.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=Taeglicher Check/Renew des internen Zertifikats fuer gitea.famfi.home
+
+[Timer]
+OnCalendar=daily
+Persistent=true
+RandomizedDelaySec=1h
+
+[Install]
+WantedBy=timers.target

systemd/mikrotik-cert-renew.service

@@ -0,0 +1,12 @@
+# Erneuert das API-SSL-Zertifikat von mt-az.famfi.home aus der OpenBAO-PKI und
+# importiert es per SSH in den Router. Ohne diesen Timer lief das Zertifikat am
+# 2026-09-26 unbemerkt ab und hat den CrowdSec-MikroTik-Bouncer 10 Tage lang
+# lahmgelegt (967 Neustarts mit "x509: certificate has expired").
+[Unit]
+Description=Erneuert mt-az.famfi.home aus der OpenBAO-PKI und importiert es in den MikroTik
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/renew-mikrotik-cert.sh

systemd/mikrotik-cert-renew.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=Taeglicher Check/Renew des API-SSL-Zertifikats fuer mt-az.famfi.home
+
+[Timer]
+OnCalendar=daily
+Persistent=true
+RandomizedDelaySec=1h
+
+[Install]
+WantedBy=timers.target

systemd/openbao-alert@.service

@@ -0,0 +1,13 @@
+[Unit]
+Description=Email alert for failed OpenBAO maintenance unit %i
+# Deliberately no OnFailure= here: if the alerter itself fails it must not
+# recurse. It exits 0 on send failure and logs to /var/log/openbao-alerts.log.
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+# %i (escaped instance), NOT %I: unit names contain '-', which systemd's
+# unescaping would turn into '/' (openbao-backup.service -> openbao/backup).
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/send-failure-alert.sh %i
+# Runs as root: reads /etc/openbao-alert-smtp.token and journalctl -u.

systemd/openbao-approle-rotate.service

@@ -0,0 +1,12 @@
+[Unit]
+Description=Rotate the demo-app AppRole SecretID before its 30d TTL expires
+After=network-online.target docker.service
+Wants=network-online.target
+Requires=docker.service
+OnFailure=openbao-alert@%n.service
+
+[Service]
+Type=oneshot
+# Runs as root: reads /etc/openbao-approle-rotate.token and writes the
+# SecretID back as lutz:lutz 0600.
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/rotate-approle-secret-id.sh

systemd/openbao-approle-rotate.timer

@@ -0,0 +1,13 @@
+[Unit]
+Description=Weekly rotation of the demo-app AppRole SecretID
+
+[Timer]
+# Weekly against a 30d secret_id_ttl: 4x margin, so three consecutive failed
+# rotations can happen before the credential actually expires -- and the first
+# failure already mails an alert via OnFailure=.
+OnCalendar=Sun *-*-* 03:10:00
+Persistent=true
+RandomizedDelaySec=15m
+
+[Install]
+WantedBy=timers.target

systemd/openbao-auto-unseal.service

@@ -0,0 +1,19 @@
+[Unit]
+Description=Entsiegelt die openbao-unsealer-Sidecar nach dem Start (sonst startet OpenBAO endlos neu)
+Documentation=file:///home/lutz/Projects/OpenBAO/README.md
+After=docker.service network-online.target
+Wants=docker.service
+Requires=docker.service
+
+[Service]
+Type=oneshot
+RemainAfterExit=yes
+ExecStart=/usr/local/sbin/openbao-unseal-unsealer.sh
+TimeoutStartSec=420
+# Ein fehlgeschlagener Versuch soll sichtbar sein, aber den Boot nicht blockieren.
+Restart=on-failure
+RestartSec=30
+StartLimitBurst=3
+
+[Install]
+WantedBy=multi-user.target

systemd/openbao-backup.service

@@ -3,6 +3,7 @@ Description=Raft snapshot backup of both OpenBAO instances (main + unsealer)
 After=network-online.target docker.service
 Wants=network-online.target
 Requires=docker.service
+OnFailure=openbao-alert@%n.service
 
 [Service]
 Type=oneshot

systemd/openbao-cert-renew.service

@@ -3,6 +3,7 @@ Description=Renew openbao.famfi.home cert from OpenBAO PKI and reload Traefik
 After=network-online.target docker.service
 Wants=network-online.target
 Requires=docker.service
+OnFailure=openbao-alert@%n.service
 
 [Service]
 Type=oneshot

systemd/openbao-offsite.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=Ship an encrypted OpenBAO DR bundle to the NAS (fids2)
+After=network-online.target home-lutz-nfs_projects.automount
+Wants=network-online.target
+OnFailure=openbao-alert@%n.service
+
+[Service]
+Type=oneshot
+# Runs as root: reads /var/backups/openbao (0700 root) and the 0600 unseal
+# material. Encrypts to HOME_SECURE before anything touches the share.
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/offsite-backup.sh

systemd/openbao-offsite.timer

@@ -0,0 +1,13 @@
+[Unit]
+Description=Daily offsite copy of the OpenBAO DR bundle
+
+[Timer]
+# Local snapshots run at 02:30 (+ up to 15m jitter), so 03:40 leaves a clear
+# gap and always ships that morning's snapshot. The script independently
+# refuses to run if the newest local snapshot is older than 48h.
+OnCalendar=*-*-* 03:40:00
+Persistent=true
+RandomizedDelaySec=15m
+
+[Install]
+WantedBy=timers.target

systemd/openbao-token-renew.service

@@ -0,0 +1,13 @@
+[Unit]
+Description=Renew scoped OpenBAO app tokens so periodic tokens never lapse
+After=network-online.target docker.service
+Wants=network-online.target
+Requires=docker.service
+OnFailure=openbao-alert@%n.service
+
+[Service]
+Type=oneshot
+# Runs as root: must read the root-owned /etc/openbao-*.token infra tokens
+# (backup x2, cert-renew) alongside the user-owned ~lutz/.config/openbao ones.
+ExecStart=/home/lutz/Projects/OpenBAO/scripts/renew-app-tokens.sh
+# Renews both sets via auth/token/renew-self. Best-effort.

systemd/openbao-token-renew.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=Daily renewal of scoped OpenBAO app tokens
+
+[Timer]
+OnCalendar=daily
+Persistent=true
+RandomizedDelaySec=1h
+
+[Install]
+WantedBy=timers.target