kestrel.markets/docs/adr/0013-first-party-agent-harness-byok-ai-sdk.md

A typed, token-efficient language + runtime for agentic trading: agents author bounded plans, the runtime fires them at the tick. CLI + typed library + MCP server.

# The first-party agent harness — a BYOK AI-SDK driver, dual-harness fidelity, the config matrix, and dogfooding the SDK

**Status:** Proposed (kestrel-5zl / kestrel-m9i). Extends ADR-0004 (typed core, text as
projection), ADR-0006 (grade everything, names are data), ADR-0011 (the Blotter projector is
a pure function of the Bus), and the Simulator seam ADR. The byte-stable
inline-comments ADR referenced under Decision (b) is now written and its guarantee delivered:
**ADR-0033** (comments round-trip byte-identically under `print(parse(x))`), so the Decision (b)
dependency below is **satisfied**, not pending.

## Context

The Simulator ADR defines the *seam* — a typed, session-stateful `Agent` (`open / decide / close?`)
consuming a frozen, date-blind ActingFrame and returning an `AgentTurn`, behind one
`runSimulateSession` driver with four adapters. It deliberately leaves the `liveAgent`
adapter's provider boundary — Fork 1's `liveAgent(config, llm: LlmClient)` — **undefined**:
`LlmClient` is a name with no body, and the deterministic core imports no provider SDK today
(`grep` finds no `@ai-sdk`, no `generateText`, no key handling anywhere in `src/`). This ADR
fills exactly that hole: it decides what the first-party agent *is*, how a live model call
sits inside `liveAgent`, and how that call stays fenced off the graded path.

Kestrel already exposes authorship four ways — the api, the cli, the mcp face, and the SDK
(the djm epic). Why build a *first-party* agent alongside them rather than leave every caller
to bring their own? Two reasons, and they are the same reason twice. **Flexibility:** a
built-in, provider-agnostic driver lets an owner point Simulate at any model with a key and
nothing else — no bespoke integration per experiment. **Evaluation:** Simulate is the
platform's primary harness (the Simulator ADR), and the thing it measures is a model-interface
*system*, not a model in the abstract. m9i draws the line the whole research program turns on:
the **controlled** division measures raw model capability (one baseline prompt-profile held
constant across every model); the **optimized** division measures the best deployable system
per model (model-specific variants allowed). Those two divisions must **never** be conflated
(m9i acceptance). A first-party agent is what makes both divisions executable through one
seam, and — because it is built on the public SDK — building it *is* the acid test of that
SDK (Decision f). The agent is the system-under-test for m9i, and the driver of the
Simulator seam.

The tension: the live model call is the one deliberately non-deterministic act in the system
(judgment is the variable), yet the platform's non-negotiables demand a deterministic record,
a byte-stable round-trip, fail-closed authorship, bounded risk, and — for anything OSS —
provably key-free artifacts. Every decision below is placed to keep the wire call *above* the
determinism line of ADR-0011 and the Simulator ADR while giving the benchmark a real, swept, multi-provider
model to test.

## Decision

**(a) The agent authors Kestrel; a parse escape ⇒ STAND_DOWN.** The agent's act at a Wake is
not a tool-call schema or a JSON blob — it is a **Kestrel document** (View / Wake / Plan),
which is decision, record, and executable in one object (ADR-0001, ADR-0004). `liveAgent`
renders the ActingFrame to a prompt, calls the model, and feeds the raw reply straight to the
deterministic `parse()`. Valid surface statements build the `AgentTurn`
(`supersede / scheduleWake / placeOrder / cancelOrder`); a **parse escape becomes a
`standDown` turn** — the fail-closed default (de-arm clean, inventory rides its TP/EXIT, log
the reason, the Session still settles). Output is **never silently repaired**. Following m9i,
the harness keeps three outcomes distinct and never collapses them: *invalid output*,
*provider failure*, and an *explicit STAND_DOWN* the agent chose. `print(parse(text))`
round-trips, so an authored document survives capture and replay byte-for-byte and the golden
fixtures hold.

**(b) Reasoning is captured evidence, not a second authored artifact.** Three separable
channels, each with its own owner, and none of them a mandate:

- **The raw model reasoning trace is captured as free evidence.** Reasoning models already
  emit it; capturing it costs no extra generation. The harness records it as wire evidence
  (Decision c) — it never asks the agent to *re-summarize* its own thinking, which would cost
  tokens to manufacture a worse copy of something already on the wire.
- **Inline comments carry the human-readable "why" *with* the action**, byte-stably, as part
  of the authored Kestrel. This was a separate grammar workstream; its ADR is now written —
  **ADR-0033 (byte-stable inline comments)** — and its guarantee **delivered**: a `#` comment
  round-trips verbatim under `print(parse(x))` and folds into `sha256(print(module))`, so the
  reasoning-in-the-document is preserved and tamper-evident once armed. This ADR's dependency is
  therefore satisfied, not pending.
- **A JOURNAL note is OPTIONAL and MINIMAL:** a falsifiable pre-hoc commitment (thesis +
  invalidation), written *before* arming and graded by a57.2 (claim vs realized outcome).
  Whether a JOURNAL commitment is *required* is itself a **swept benchmark variable** in the
  config matrix, not a fixed rule — some cells demand it, some don't, and Grade tells us
  whether demanding it helps. `AgentTurn.journal → a57.2 pre-hoc author`, `Agent.close? →
  a57.2 debrief`; the harness consumes that stream and never re-mints the JOURNAL schema.

**(c) A dual harness: the Vercel AI SDK (BYOK) is the default driver; a direct-provider path
supplies the fidelity rungs.** The default `LlmClient` behind `liveAgent` is the **Vercel AI
SDK** — TypeScript, runs on Bun, uniform multi-provider model-swap — constructed with a
**pinned direct-provider model object** (never a gateway or fallback router) selected by
`AgentConfig`. This is the flexible path: change the model identity, keep everything else.
Alongside it, a thin **direct-provider path** supplies the wire-fidelity rungs the SDK
abstracts away — prompt-cache economics (m9i.5), latency-causal evidence (m9i.6), and
provider-native usage/cache counts as the shadow oracle. Retries, silent fallbacks,
middleware repair, and unsupported-setting coercion are **disabled or fail the cell
explicitly** (m9i.1) — the harness must never quietly turn one config into another. Both
drivers meet the *same* `Agent` seam; **m9i.7 tournaments HTTP × SDK × CLI × MCP × model ×
harness** over identical Frames, so the AI-SDK path is one face under measurement, not a
privileged one. The invariant across both: **the deterministic core imports no provider SDK**
— BYOK and the wire live only at the `LlmClient` seam inside `liveAgent` (or m9i.1's external
runner), and only parsed Kestrel (or STAND_DOWN) plus redacted wire-evidence/usage cross back
into the graded path.

**(d) The AgentConfig axis IS the point of Simulate, and prompt-profile is its most loaded
dimension.** The comparison grid sweeps **model × tokenizer × Rendering/format × temperature
× thinking-level × prompt-profile**. The first five are mechanical. The sixth is where the
two m9i divisions live:

- **prompt-profile = baseline ⊕ model-variant**, resolved to a **hashed, versioned artifact**
  (`AuthorPolicy.prompt_sha256` over the exact system bytes). The **active variant-set is the
  division selector**: baseline-only, held constant across all models ⇒ the **controlled**
  division (raw capability); baseline plus model-specific variants ⇒ the **optimized**
  division (best deployable system per model). A run is stamped with which division it belongs
  to, and **the two are never averaged into one number** (m9i acceptance) — a controlled
  result and an optimized result are different questions and rank on different boards.
- **Confidentiality rides the profile split.** A generic baseline may be public / OSS; a
  strategy-bearing variant is private — the same public/private separation the provenance
  scrub already enforces (Decision e). The variant-set selector is therefore also the
  publish-safety selector.
- Every run's full config is **stamped on its Blotter** as the a57.14 experimental envelope
  (`AgentConfig` *is* that envelope's input half — a single definition, never a second one)
  and **frozen into a System Profile** (m9i.2 / m9i.8) before any private forward evaluation.
  `ConfigId = sha256(canonical config)` using the Blotter's existing sorted-key /
  drop-undefined canonicalizer, so any edit mints a new grid column — a config change can
  never silently contaminate an old one. Config identity is content; the credential is not
  part of it (Decision e).

**(e) BYOK: keys via env / keyring only, scrubbed at capture, artifacts provably key-free.**
The API key is supplied through environment or keyring configuration and injected into the
`LlmClient` at construction. It **never enters `AgentConfig`** (which is content-hashed into
`ConfigId`), **never touches the Bus, Blotter, System Profile, or any committed fixture**, and
a57.14 stamps *model identity*, not credentials. The recording `fetch` that captures wire
evidence **redacts auth headers and key material at write time** — redaction happens before
bytes ever hit disk, not after. Committed OSS artifacts are provably key-free, gated by
`scripts/check-public-provenance.ts` (which already fails the build on private-provenance
leakage; the BYOK rule extends its remit to credentials). No credential in a committed
artifact is an m9i acceptance criterion, and this is how it is met structurally rather than by
convention.

**(f) The built-in agent dogfoods the SDK: it is the first client of djm.5, not a privileged
internal path.** `liveAgent` drives the runtime through the **public TypeScript SDK
(djm.5)** — deliver the date-blind OPEN Frame, accept a Kestrel document or STAND_DOWN,
advance only to eligible Wakes, accept revisions, finalize through the same Bus / Blotter /
Grade — exactly the surface an external caller gets. Building the first-party agent this way
**validates the external surface**: if the SDK is missing something the agent needs, that is a
djm bug found by dogfooding, not a private back-channel the agent quietly uses. The cli face
(djm.6, agent-mode as an SDK projection) and the mcp face (djm.7) are sibling projections of
the same SDK, and m9i.7 tournaments all four. This is the reconciliation of the two drivers
the Simulator ADR flagged: the first-party agent does not fork a second incremental-Session controller;
it is a *client* of djm.4/djm.5.

**(g) Phasing: an eval-internal driver now, graduating to a shipped reference agent later.**
Phase 1 builds the agent as the **benchmark/Simulator driver** — the `liveAgent` adapter and
its BYOK `LlmClient`, exercised through `runSimulateSession`. It is the system-under-test for
m9i and needs no shipped-product polish. It **graduates** to a shipped *reference* agent once
the surface and the config discipline are proven — same seam, same SDK, additional operational
hardening. The phasing changes only maturity and packaging, not the seam or the determinism
contract, so a phase-1 recorded run re-grades identically under the phase-2 agent.

## Consequences

- **A new external dependency enters the tree — but only at the seam.** The Vercel AI SDK and
  the pinned provider packages are added, imported **exclusively** inside `liveAgent` /
  `src/session/agent.ts` and the m9i external runner. The deterministic core, the projector
  (ADR-0011), and the record/replay path stay provider-free; `recordedAgent` and
  `fixedPlanAgent` carry no provider import. A BYOK live run touches the wire **once**; every
  re-projection, re-grade, and Backtest-through-the-seam is provider-free and byte-identical.
- **BYOK becomes an operational surface.** Keys are an env/keyring concern with a redact-at-
  capture rule and a build-time provenance gate; running the benchmark now requires supplying
  credentials the platform deliberately refuses to persist. Owner approval gates paid model
  calls, System-Profile freeze, and any publication wording (m9i).
- **Building the agent validates djm.** Because the first-party agent is the SDK's first
  client, the SDK's completeness is tested by real use before any third party depends on it —
  and the cli/mcp faces inherit that validation as SDK projections.
- **The benchmark can now sweep the full system.** With the agent in place, Simulate sweeps
  **prompt-profile × model × tokenizer × Rendering × temperature × thinking-level** over a
  fixed tape and fixed fill model, producing a grid of graded Blotters and a config
  leaderboard (the Simulator ADR). Controlled and optimized divisions run through the same seam and
  never share a board. The raw reasoning trace and provider-native usage land as free evidence,
  feeding a57.2's claim-vs-outcome grade and the leaderboard's `ev_per_ktoken` attention axis.
- **Reasoning is evidence, not overhead.** By capturing the emitted trace rather than
  commissioning a summary, and by making the JOURNAL commitment optional and itself a swept
  variable, the harness measures whether structured reasoning helps instead of assuming it and
  paying for it everywhere.