Operations · Backups & Restore

What survives

A brain deployment holds exactly three things worth protecting: the knowledge, the agents' state, and the wiring that stands the boxes back up. Everything else is derived. Both boxes push those three to object storage — encrypted before they leave, into buckets neither box can touch for the other.

Brain box master vault · proxy + tunnel config systemd units · sync authorized_keys backup-master.sh · hourly Agents box hermes state zips (one per person) compose .env · docker-compose.yml · crontab backup-agents-offsite.sh · nightly restic encrypts client-side — storage sees only ciphertext BUCKET · BRAIN ONLY <company>-brain-backups 24 hourly · 14 daily · 8 weekly · 12 monthly token installed only on the brain box repo: brain-master BUCKET · AGENTS ONLY <company>-agents-backups 14 daily · 8 weekly · 6 monthly token installed only on the agents box repo: agents no cross-bucket access — a compromised box cannot delete the other box's history
One bucket per box, because object-storage tokens scope to whole buckets and R2 offers no version-undelete. The bucket boundary is the blast-radius boundary. Separate restic passwords mean neither box can even decrypt the other's data.

The three things, and why each matters

Irreplaceable

The knowledge

The master vault — the company's memory, with its full git history. Nothing can rebuild it. Compiled vaults, search indexes, and agent /vault mounts are all derived from it and need no backup. It backs up hourly because losing an hour is annoying; losing a week is a different company.

Per person

The agents' state

Each container's hermes backup zip: sessions, pairing approvals, config, deploy key. Taken inside the running container via SQLite's backup API, so it's consistent while the gateway chats. Restoring one brings a person's agent back with no re-pairing and no re-authorized keys.

Often forgotten

The wiring

Compose .env secrets, the live docker-compose.yml, proxy and tunnel config, systemd units, the sync authorized_keys. The zips restore state into containers — the wiring is what lets you start containers at all. Skip it and a dead box costs a day of re-provisioning.

Failure → recovery

You lostYou do
A note, or the vault restic restore the brain-master repo — the vault returns as a working git repo, history included.
One agent's state Restore that person's newest zip, hermes import <zip> --force into a fresh container, restart. Pairing, sessions, deploy key all return; the vault re-clones itself.
The agents box New box → restore compose dir + crontab → docker compose up -d → import each person's zip.
The brain box New box → restore vault, config, units, and authorized_keys → reinstall brainkit → re-point the tunnel. Every agent reconnects without re-authorizing a single key.

Three rules that keep it honest

The password is the backup

restic encrypts client-side; without the repo password you own ciphertext. Generate it on the box, put it in a password manager the same day, and never let it transit chat or shell history.

Silence must alarm

Every run pings a dead-man's-switch URL — start, success, or fail. A missed backup pages you; it does not wait quietly in a log for the day you need it.

Unrestored = untested

Finish setup with a drill: restore to a temp dir and diff against the live tree. Repeat occasionally. The 3 a.m. restore should be the second time, never the first.