Ran the documented restore end to end against an isolated copy (separate
compose project, volumes and network; live stack untouched throughout). It
worked, but only after two steps the docs never mentioned -- so the docs were
wrong in the way that matters: they would have stalled someone mid-outage.
1. A fresh node cannot accept a snapshot restore at all until it has been
'operator init'-ed and unsealed with THROWAWAY keys. You restore over the
top, and the throwaway keys stop working the instant the restore lands --
from there only the original keys in this bundle open it. The old text said
"restore its snapshot, then unseal it", skipping init entirely.
2. 'docker cp' preserves the snapshot's 0600 root ownership, but since 2.6.0
the container runs as uid 100 and cannot read its own restore file. The
error is misleadingly worded ("Error opening policy file: ... permission
denied") and sends you looking at policies instead of file modes.
Also now documented: main needs a transit token minted on the RESTORED
unsealer for its seal stanza; the unsealer re-seals itself after its restore
and needs the original shamir key; and main then auto-unseals with no manual
step.
Drill results, for the record: Cluster ID of the restored main matched
production, the original root token authenticated, all 13 mounts / 14 policies
/ 4 auth methods came back, and the restored instance decrypted a transit
ciphertext the live instance had produced seconds earlier while issuing a
fresh cert from pki_int -- proving CA private keys and transit key material
survived, not merely the configs. Decrypted plaintext was shredded and the
drill stack removed with its volumes.
Verification guidance now recommends those last two checks specifically,
since data-identical is a weaker claim than operationally functional.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NFtVLA7VVqXL5G2S18c4Jk
Lutz Finsterle
055676c186