kestrel.markets/docs/adr/0054-the-shared-out-of-tree-secrets-home.md

A typed, token-efficient language + runtime for agentic trading: agents author bounded plans, the runtime fires them at the tick. CLI + typed library + MCP server.

# OSS-ADR-0054 — The shared out-of-tree secrets home (`~/.kestrel/.env`)

Status: accepted (2026-07-18, owner-ratified 2026-07-17). Names and extends
epic `kestrel-jh9w` (CLI-managed credential residency); folds in the loading
half of `kestrel-63r1` and the "no secret file in the tree" invariant of
`kestrel-jh9w.5` / `kestrel-933z`.

## Context

Local development spans **30+ worktrees across two repos** (`~/projects/kestrel`
and `~/projects/kestrel.markets`, plus their worktree roots). Dev/test API keys
have historically lived as per-checkout `.env` files. That has two standing
costs:

1. **Propagation.** A key configured in one worktree is invisible to the other
   29 — every fresh worktree re-exports the same secrets, and an agent working
   across context windows must re-establish them every session.
2. **Leak surface.** The OSS package's `package.json` `files` whitelist ships
   `src/` and `examples/`. A `.env` misplaced anywhere under a whitelisted
   directory **would publish to npm** — and a published tarball is immutable.
   `git`-tree copies are a commit-leak risk in parallel; a gitignored `.env`
   under `src/` is *still* shipped by npm (`git`-ignore is not `npm`-ignore).

The CLI already establishes a single owner-only home — `src/cli/credentials.ts`
creates `~/.kestrel` (`0700`) and writes `credentials.json` / `agent-key.json`
(`0600`), with a `KESTREL_HOME` override that keeps tests hermetic. The operator
secret store (`~/.kestrel/.env`, `jh9w.1`/`63r1`) and its CLI verbs
(`kestrel secrets set|list|unset|path`, `jh9w.2`) already ride that home. This
ADR ratifies the *residency policy* those pieces implement, and closes the two
gaps that remained: a process-environment **loading shim** and a fail-closed
**no-secret-file-in-the-tree** guard.

## Decision

**All local dev/test secrets live in one shared out-of-tree home,
`~/.kestrel/.env`. No repo tree ever carries a secret file — not even a
gitignored one.**

1. **Home.** `~/.kestrel/.env`, file `0600`, dir `0700`, ensured by the existing
   `credentials.ts` machinery (atomic temp-and-rename writes; `KESTREL_HOME`
   override for hermetic tests). Plain dotenv format so tooling in both repos can
   source it.

2. **Precedence — `process.env` wins; `~/.kestrel/.env` is the fallback.** An
   explicitly-exported env var always overrides the at-rest store. This is the
   12-factor rule and it is what keeps CI (GitHub Actions secrets) and production
   (`wrangler secret`) **untouched**: they inject via `process.env`, so the store
   is never consulted there.

3. **The loading shim.** `loadEnvFallback()` (in `credentials.ts`) reads the
   store once and populates any **UNSET** `process.env` key; a set key is left
   exactly as-is. It is fail-quiet and dependency-light (node built-ins only),
   never logs a value, and respects `KESTREL_HOME`. It is wired into the OSS
   entrypoints that read these keys: the CLI router (`src/cli/index.ts` — the
   `kestrel paper` path resolves the IBKR gateway config from `process.env`) and
   the IBKR harness probes (`src/adapters/broker/ibkr/*-smoke.ts`,
   `order-dryrun.ts`, `equity-order.ts`, `tick-probe.ts`). The bench data-pull
   scripts already resolve through the per-key form (`loadSecret`,
   `scripts/bench/lib/resolve-secret.ts`, `kestrel-3oei`).

4. **Contents (dev/paper only).** `ALPACA_PAPER_API_KEY_ID`,
   `ALPACA_PAPER_SECRET_KEY`, `DATABENTO_API_KEY`, the GPU-cloud training key
   (`n_CLOUD_KEY` — the public alias the repo already uses; the literal provider
   name stays obfuscated per the kestrel-djm.10 launch gate),
   `CLOUDFLARE_PIPELINE_TOKEN`, and the IBKR identifiers (`KESTREL_IBKR_*`).
   BYO-model keys (`FIREWORKS_API_KEY`, `AZURE_OPENAI_*`, `LOCAL_LLM_*`) ride the
   same home (credential residency, `jh9w`).

5. **NEVER in `~/.kestrel/.env` (paper-only doctrine).**
   - **Live broker keys** — `wrangler`-only, behind `ALPACA_LIVE_ENABLED` + an
     allowlist. `jh9w.5` refuses storing any `*_LIVE_*`-named key with a pointer
     to `wrangler`.
   - **Platform production secrets** — Cloudflare `wrangler secret put`.
   - **CI credentials** — GitHub Actions secrets.

6. **The no-secret-file invariant, enforced.** `scripts/check-no-secret-files.ts`
   (in `bun run check` and CI) fails closed if a `.env`-style file carrying an
   API-key-shaped assignment appears **anywhere in the tracked tree or the npm
   shipped set** — regardless of whether the value matches a known vendor shape.
   Sanctioned scaffolding (`.env.example` / `.sample` / `.template` / `.dist`,
   and placeholder-only values) does not trip it. This is complementary to, not a
   replacement for, the shape-based `check-public-provenance.ts` credential scan
   (which catches *known-shaped* key material in the shipped set) and the
   `kestrel-933z` prepack tripwire.

## Considered and rejected

- **Per-repo loader shims.** One `.env` loader per repo re-introduces the
  propagation problem this ADR exists to kill, and multiplies the surface where a
  secret file could sit. `jh9w.4`'s `kestrel env -- <cmd>` exec wrapper is the
  cross-**process** complement (e.g. `kestrel env -- wrangler dev` on the
  platform, tracked as a `kestrel.markets` bead) so external tools that only read
  a project `.env` get the same resolution without a second in-tree file.
- **A secret file in the repo, gitignored.** Rejected outright: npm ships
  gitignored files inside a whitelisted `files[]` dir, so "gitignored" is not
  "unpublished." The invariant is *no secret file in the tree at all*.
- **Store-hydration inside the pure config resolver** (`resolveIbkrConfig`).
  Rejected: it would mutate `process.env` as a side effect of a pure resolve and
  muddy the injected-env test seam. Hydration belongs at the entrypoint.

## Consequences

- Configure a key once (`kestrel secrets set KEY`), and every worktree of both
  repos — and every future agent context — reads it. Zero secret files in any
  checkout.
- Determinism is untouched: `loadEnvFallback` only fills config env (never a
  graded runtime input), only when unset, and is a no-op under the hermetic
  `KESTREL_HOME` tests use.
- The guard makes the invariant structural, not conventional.

## Appendix — operator migration checklist (run by a human; never by an agent)

An agent must never read, print, commit, or handle a real secret value. These
steps are for the human operator to consolidate scattered keys into the shared
home and delete the copies.

1. **Create the home (idempotent).**
   ```sh
   mkdir -p ~/.kestrel && chmod 700 ~/.kestrel
   ```

2. **Import each key** with the CLI (value via prompt or `--stdin`, never argv —
   argv leaks to shell history / `ps` / supervisor logs). Use the canonical
   names:
   ```sh
   kestrel secrets set ALPACA_PAPER_API_KEY_ID      # paper only
   kestrel secrets set ALPACA_PAPER_SECRET_KEY      # paper only
   kestrel secrets set DATABENTO_API_KEY
   kestrel secrets set n_CLOUD_KEY                  # GPU-cloud training key
   kestrel secrets set CLOUDFLARE_PIPELINE_TOKEN
   kestrel secrets set KESTREL_IBKR_HOST            # + PORT / CLIENT_ID / ACCOUNT as used
   ```
   Or pipe from an existing file line without echoing it:
   ```sh
   printf %s "$VALUE" | kestrel secrets set DATABENTO_API_KEY --stdin
   ```

3. **Verify** names only (values are never printed):
   ```sh
   kestrel secrets list
   kestrel secrets path      # prints ~/.kestrel/.env
   ```

4. **Harden permissions** (the CLI already writes `0600`/`0700`; confirm):
   ```sh
   chmod 600 ~/.kestrel/.env
   chmod 700 ~/.kestrel
   ```

5. **Delete the scattered copies** once the store resolves them — every
   `0dte/.env`, every per-worktree `.env`, any `~/projects/*/.env` holding dev
   keys. Confirm nothing in a repo tree still holds a secret:
   ```sh
   bun run check          # runs check-no-secret-files.ts (fails closed on a tracked/shipped .env secret)
   ```

   The guard's scope is the tracked tree ∪ the npm shipped set (the two leak
   vectors). A purely-local `.env` that is gitignored, untracked, *and* outside
   `files[]` is not caught by the guard — deleting it is exactly what this step
   is for.

6. **Live and production keys do NOT move here.** Live broker keys stay in
   `wrangler` (behind `ALPACA_LIVE_ENABLED` + allowlist); platform prod secrets
   stay in `wrangler secret`; CI creds stay in GitHub Actions secrets.