kestrel.markets
Version:
A typed, token-efficient language + runtime for agentic trading: agents author bounded plans, the runtime fires them at the tick. CLI + typed library + MCP server.
472 lines (412 loc) • 36.4 kB
Markdown
# The AgentTurn is an envelope-free, pure Kestrel document
**Status:** **Accepted (owner, 2026-07-15).** *(Originally Proposed 2026-07-13 — "design for owner
review"; ratified in the 2026-07-15 canon-reconciliation grill, see the [Ratification note](#ratification--owner-2026-07-15)
directly below.)* The pure-Kestrel turn is now the **canonical** author-turn wire; the two
grammar-design questions below (the stand-down keyword, the request-vs-registration marker) are **not**
retracted — they are converted into the blocking beads **kestrel-qim.1** and **kestrel-qim.2**, which
gate code exactly as the old "no code lands until confirmed" clause did. Extends ADR-0001 (one language, four statement kinds),
ADR-0004 (the typed object model IS the language; `print(parse(x))` round-trips), ADR-0011 (the
Blotter projector is a pure function of the Bus), ADR-0012 (the Simulator seam — the determinism line
at the returned `AgentTurn`). **Completes ADR-0013 (a)** ("the agent authors Kestrel, not a JSON
tool-call"). **Reconciles ADR-0029 §1** (which chose the JSON `requestView` envelope and *rejected*
authoring a bare `VIEW` document) and **ADR-0030** (the grammar is measured, not designed — this is a
proposed grammar extension, so it is an A/B arm, not a fiat change).
## Ratification note — owner, 2026-07-15
Ratified in the cross-repo canon-reconciliation grill (tracker: **kestrel-2sy**; contradiction
enumeration: `docs/adr/NOTE-agent-architecture-reconciliation-0029-0031-0032.md`). This note is an
**append-only amendment** — nothing in the body below is rewritten. Three owner decisions:
1. **The pure-Kestrel turn is CANONICAL.** The whole agent reply IS Kestrel; the driver routes on
statement-kind plus the `SHOW VIEW` and `STAND-DOWN` directives (§1–§3). The JSON `{requestView:{…}}`
envelope from ADR-0029 §1 (Option C) is **demoted to forgiving-ingest compatibility only**: the
driver *may* accept a `{requestView}` object and normalize it to the same typed request, but the
**logged / canonical / corpus** form is always the Kestrel text projection (`SHOW VIEW`). One
grammar in, one grammar logged — never a bifurcated corpus. **Rationale (recorded):** prior testing
showed JSON envelopes add heavy **token + cognitive overhead** vs pure text — the *same physics as
the percept thesis*, now applied to the **input/authoring** side; **one grammar = one corpus = one
CFG constraint** for the model program; and the **free-is-licensed training corpus starts
accumulating at platform L2**, so the wire format must not bifurcate. This **amends ADR-0029's
Option C** (which had *rejected* a bare Kestrel verb and chosen the JSON key); ADR-0029's bounded
loop, `viewRequestCap`, `authoringTokenBudget`, fail-closed `standDown`, and emergence log **all
survive verbatim** (see ADR-0029's dated amendment).
2. **The landed envelope projection (a57.14) is KEPT as the A/B control arm** (§8), not retired. It is
measured via the platform's experimentation overlay on **parse-failure rate,
stand-down honesty, and grade**, and retired **on evidence** — never by fiat. Season SEAM
dependencies on a57.14 are unaffected meanwhile. (This is exactly §8's "keep the envelope profile
registered as the control regardless"; the decision fixes it as policy.)
3. **This ADR is ratified Proposed → Accepted (2026-07-15).** Its two open grammar questions (§2a/§2b,
Open q1) are converted into **blocking beads inside epic kestrel-qim**: **kestrel-qim.1** (confirm
the `STAND-DOWN` keyword + rest-of-line reason placement) and **kestrel-qim.2** (confirm the
`SHOW VIEW` request-vs-registration marker). This resolves the **authority inversion** in which
Accepted ADR-0032 §7/§8.4 depended on a Proposed ADR-0031.
### Naming — three spellings are three LAYERS (do NOT mass-rename)
Contradiction #3/#5 in the NOTE asked whether the `STAND-DOWN` / `standDown` / `STAND_DOWN` collision
should be mass-renamed. **Owner decision: NO** — they are three *distinct layers* of the same concept,
and each is correct in its own layer. The explicit mapping:
| Spelling | Layer | Where it lives |
|---|---|---|
| `STAND-DOWN` | **grammar directive** (author-facing surface token) | this ADR §2a / §3; the pure-Kestrel turn the model emits |
| `standDown` | **typed `Action`** (the parsed outcome) | ADR-0012 / ADR-0029 `StandDownAction`; the terminal turn that crosses the determinism line |
| `STAND_DOWN` | **fail-closed sentinel** (harness terminal) | ADR-0012 the whole-turn fail-closed pass; dissolves into the `STAND-DOWN` directive on the file-handshake surface (§7) |
The one-way flow is `STAND-DOWN` (surface) → `parse()` → `standDown` (typed Action) → and, on an
unparseable whole turn, the `STAND_DOWN` sentinel is the fail-closed floor. Same concept, three layers;
the router and printer keep them distinct by construction. (Companion note in ADR-0032's amendment:
`watcher-GATE` vs `watcher-MANAGER` are likewise two roles; **bare "watcher" is deprecated in new
docs**.)
## Context
ADR-0013 (a) drew the principle sharply: *the agent's act at a Wake is not a tool-call schema or a
JSON blob — it is a Kestrel document, which is decision, record, and executable in one object.* Yet
the shipped harness does not honor that. Today `liveAgent` and the file-handshake brain reply with a
**JSON envelope** that *wraps* the Kestrel:
```jsonc
// the current wire shape (src/session/harness/prompt.ts — parseTurn / parseAuthoringReply):
{ "journal": "…", "actions": [ { "kind": "supersede", "document": "PLAN …\n WHEN …\n DO …" } ] }
// … or, in the ADR-0029 authoring loop, a mutually-exclusive view request:
{ "requestView": { "view": "VIEW probe budget 900\n gex 0dte", "reason": "…" } }
```
The Kestrel document is a **JSON-string-escaped payload inside the envelope**: every newline becomes
`\n`, every quote is escaped, and the whole multi-line plan is flattened into one string field. This
is a JSON tool-call wrapper around Kestrel — a **half-measure** against ADR-0013 (a). It has four
distinct costs, and the fourth is the sharpest:
1. **JSON scaffolding tokens every single turn** — `{"journal":"…","actions":[{"kind":"supersede",
"document":…}]}` is fixed overhead the model emits on every reply, dominating short turns (a
stand-down is mostly envelope).
2. **JSON-escaping of the Kestrel document** — the `\n`, the escaped quotes — costs *extra* tokens
(measured below: ~+5% on the document alone) **and** fragments the token stream in exactly the
tokenizer-hostile way the Rendering/percept design fights: `\n ` sequences and escaped
punctuation lex into more, smaller, less-natural tokens than the raw source does.
3. **Two grammars the model must satisfy at once** — a JSON grammar *and* the Kestrel grammar nested
inside a string. Each is a place to fail; the dry-run-1 collapse (30–47% of turns lost to
`WHEN … DO …` on one line, ADR-0030) happened *inside* the escaped string, where the model could
not see its own layout.
4. **The JSON envelope is its own failure mode.** dry-run-2 recorded **34 envelope non-emissions**
from Gemini — turns where the model produced usable content but never emitted the wrapping JSON
object the exact-`{…}` extractor required. **Dropping the envelope dissolves that failure class
entirely: there is no envelope to fail to emit.** A pure-Kestrel reply has *only* the Kestrel
grammar to satisfy, and the extractor is forgiving (strip a fence, else take the body) rather than
demanding a single well-formed JSON object.
The tension is the one every author-surface decision meets. The reply is deliberately
non-deterministic (judgment is the variable). But the record path must stay a pure function of Bus
bytes, `recordedAgent` replay must be byte-identical, the three m9i outcomes (authored / invalid /
provider-error) must stay distinct, and — the specific trap ADR-0029 §1 named — the surface must not
reintroduce the **phase ambiguity** between *register a standing View for future wakes* and
*re-materialize a lens now and re-author*. ADR-0029 used the JSON envelope (a distinct top-level
`requestView` key) precisely to dodge that ambiguity, and explicitly **rejected** "author a bare
`VIEW` document" (its Option B) because a bare `VIEW` cannot tell request from registration — "a
phase bug silently arms or silently loops." **Envelope-free must therefore solve that ambiguity
*in the grammar*, not merely inherit the envelope's dodge.** That is the real design work of this ADR.
## Decision
**The AgentTurn IS a Kestrel document. The agent emits pure Kestrel as its whole reply; the driver
parses it with the Kestrel parser and routes the outcome on statement-kind plus two new turn-level
directives. No JSON envelope, no string-escaping, one grammar.** This is not a new principle — it is
the *completion* of ADR-0013 (a), which the ADR-0029 envelope was a half-measure against.
### 1. The turn is a document; the outcome is what it contains
The reply is extracted (fail-open on framing, §3) and handed to the deterministic `parse()`
(`src/lang/parse.ts`). The outcome is determined by **what the document is**:
| The reply contains… | Outcome | Maps to |
|---|---|---|
| one or more `PLAN` statements (a whole module) | **author / supersede** the standing book | `SupersedeAction` |
| a `WAKE` statement (`DELIVER <view>` optional) | **schedule the next wake** (+ its standing View) | `ScheduleWakeAction` |
| a bare `VIEW` statement inside the document | a **standing** screen registration (future wakes) | part of the superseded document |
| a `SHOW VIEW …` directive (new, §2b) | a **view request** — materialize now, re-ask (ADR-0029 loop) | `AuthoringReply.requestView` |
| a `STAND-DOWN [reason]` directive (new, §2a) | **de-arm clean** (inventory rides its TP/EXIT) | `StandDownAction` |
| comments-only, or empty | **pass** (do nothing; standing Plans keep managing) | `{ actions: [] }` |
| unparseable Kestrel | **invalid** (repairable — guided error → repair-retry) | the fail-closed pass / repair loop |
Reasoning rides as `#` **comments** (ADR-0013 (b)); the harness lifts them into the turn's journal
(§2c). Multiple authored acts are multiple statements in the one module — the module *is* the
multi-action turn. There is no second serialization: the document the model wrote is the document the
Gate arms, `print(parse(document))` round-trips (ADR-0004), and the golden fixtures hold unchanged.
**The order-imperatives (`placeOrder` / `cancelOrder`) have no Kestrel statement form today** and are
the one place the JSON envelope carried something the grammar cannot yet say. An immediate order is
naturally a `PLAN` with a fire-now `DO` ticket (it already routes the *same* Gate as a fired Plan —
`src/session/agent.ts`), so `placeOrder` folds into the document with no new surface. `cancelOrder`
by resting-`ref` is the genuine residual; it needs either a third turn directive (`CANCEL <ref>`) or
a plan-scope `CANCEL-IF`. This is called out as **Open question 3** — it must not silently reintroduce
an imperative envelope.
### 2. The grammar extensions (the real design work)
Three things the current grammar (ADR-0001: `View`/`Wake`/`Plan`/`Grade` + `#` comments) cannot yet
express as a *turn*. Each is flagged as a grammar addition and carries a concrete proposal and a
recommendation; both keywords are top-level and lexically distinct from every existing statement head
(`VIEW`/`WAKE`/`PLAN`/`GRADE`/`POD`/`BOOK`/`USING`/`IMPORT`), so neither collides.
**(a) A turn-level STAND-DOWN / pass directive.** A `PLAN` is *authority*, not a stand-down; Kestrel
has no way to say "I stand aside this turn." Two dispositions are needed and they are different:
*stand down* (explicit de-arm, a chosen act, logged) and *pass* (do nothing, standing book keeps
managing). Proposal:
```kestrel
STAND-DOWN edgeless chop — no defined-risk setup; standing down
```
- `STAND-DOWN` is a **new top-level directive** (not a fifth statement kind — it authors nothing; the
statement algebra stays View/Wake/Plan/Grade, ADR-0001). Its reason is the **rest of the line**,
captured verbatim (a small lexer addition — rest-of-line capture, like a comment but *retained*),
mapping straight onto today's `StandDownAction.reason`.
- **Pass** needs *no* keyword: a reply that parses to an **empty module** (only `#` comments, or
nothing) is the legitimate pass — the natural "I looked, nothing to do." This keeps stand-down
(a deliberate de-arm) and pass (inaction) *distinct in the grammar*, which the current `{actions:[]}`
vs `{actions:[{standDown}]}` distinction already relies on.
- **Recommended:** `STAND-DOWN` with an inline rest-of-line reason (ergonomic, one line, mirrors the
JSON `reason`). Alternative considered: bare `STAND-DOWN` + a `#` reason comment (lexically cleaner
— the WHY is always a comment — but two lines for the commonest safe move). Recommend the inline
form; **confirm with owner.**
**(b) Disambiguate a view-REQUEST from a standing-VIEW registration (the crux — the thing ADR-0029
dodged).** ADR-0029 §1 rejected a bare `VIEW` document as the request *because a bare `VIEW` cannot
tell "materialize now and re-ask" from "register this View for future wakes"* (`DELIVER`,
`ScheduleWakeAction.view`) — and used a distinct JSON key (`requestView`) as the disambiguator.
Envelope-free re-supplies that disambiguator **as a keyword, not a JSON key**:
```kestrel
SHOW VIEW probe budget 900
gex 0dte
skew 25d
tape 5m
# need dealer-gamma + skew before I commit
```
- `SHOW` is a **new top-level directive** whose payload is an ordinary `ViewStatement` (the lexer
reads `SHOW`, then parses the remainder as a normal `VIEW …` statement — the pane grammar reused
**verbatim**, panes drawn from the one catalog, `print(parse(view))` round-trips). Its meaning is
fixed and single: *materialize this lens **now**, at the same frozen cutoff, and ask me again* — the
ADR-0029 bounded loop. The `reason` rides as the trailing `#` comment.
- A **bare `VIEW` statement** (inside a supersede document, or as a `WAKE`'s `DELIVER <view>`) keeps
its existing meaning: a **standing** screen selection for future wakes. The two are now
distinguished **lexically** — `SHOW VIEW` vs `VIEW` — exactly as ADR-0029 distinguished them with
`requestView:` vs a `VIEW` inside `supersede`. **The distinction ADR-0029 achieved with a JSON key,
this achieves with a grammar keyword.** The phase bug ADR-0029 feared (disambiguating by
document-kind + authoring-phase) does not arise: the driver routes on the *keyword*, present or
absent, never on phase. This is **not** ADR-0029's rejected Option B (a *bare* VIEW); it is Option
B's payload (a real `VIEW` document, for grammar fidelity) plus a distinct marker that restores the
disambiguation — the useful half of B with the anti-ambiguity guarantee re-added.
- **Recommended:** `SHOW` as the request marker (reads as "show me this now"; imperative, present
disposition). Alternatives considered: `REQUEST VIEW`, `PROBE VIEW`, `LOOK VIEW`. Recommend `SHOW`;
**confirm with owner** — this keyword is the load-bearing disambiguator and must be taught in the
profile.
**(c) Reasoning-as-comments for the journal — sufficient, with one honest dependency.** The journal
(the pre-hoc thesis + invalidation, a57.2) rides as `#` comment lines in the reply. The harness lifts
them into `AgentTurn.journal` at parse time (above the determinism line), and from there the journal
becomes a JOURNAL bus event and is stored in `CapturedTurns` **exactly as today** — provably pre-hoc
by `seq`, captured for grade, replayed byte-identically. This works **now**, because the journal
lives in the `AgentTurn` envelope (the sorted-key canonicalizer, `src/session/agent.ts`), *not* in the
printed Kestrel — so it does **not** depend on comments surviving `print(parse(x))`.
Formerly the honest caveat here was that the lexer *dropped* comments and the printer never emitted
them, so a `#` comment did not round-trip through the Kestrel document itself. **That gap is now
closed:** the byte-stable inline-comments workstream landed as **ADR-0033** — the lexer captures
`#` comments as trivia and the printer re-emits them byte-identically, so `print(parse(x))`
round-trips comments verbatim. The journal channel never depended on it (it is lifted into the turn
envelope before parse), but the ADR-0013 (b) promise of a comment that rides **with the armed Plan
document, byte-stably** (the WHY welded to the action, folded into the armed-document hash) is now
**delivered**, not deferred. **Open question 2 is resolved** (see below).
### 3. Extraction, parsing, and fail-closed
**Extraction is forgiving — this is why envelope-free is robust where exact-JSON was brittle.** Mirror
the shape of the current `extractJsonObject` but for prose→Kestrel: strip a fenced block if present
(```` ```kestrel `` / ` ``` ` ` ``` ` — or a bare ```` ``` ````), else take the whole reply body,
trimmed. There is no "first `{` to last `}`" fragility and no single-object requirement — the body
**is** the document. A model that emits its Kestrel with no fence, or inside a fence, or with a stray
trailing blank line, all extract cleanly. **This is precisely what removes the 34 Gemini non-emissions:
there is no envelope object whose absence is fatal.**
**Routing then reads the parse:**
1. Extract body (strip fence, trim).
2. Lift `#` comment lines → `journal` (join); strip them from the routing body.
3. First significant token is `STAND-DOWN` → **standDown** (reason = rest of line).
4. First significant token is `SHOW` → parse the following `VIEW` statement → **requestView**
(ADR-0029's non-terminal outcome — honored **only** under a viewshop-enabled config; under a
config that does not offer the move, an emitted `SHOW VIEW` is the **invalid** outcome, exactly as
ADR-0029 gates it today).
5. Body (sans comments) is empty → **pass**.
6. Otherwise `parse()` the body as a module:
- success → **supersede** with the document (a `WAKE` statement in it carries scheduleWake
semantics; the whole module supersedes);
- `parse()` throws → **invalid**, carrying the **repair-guiding parser error** (ADR-0030) →
repair-retry re-ask (ADR-0029's unified repair loop), and only the terminal fail-closed pass once
the budget is spent.
**The three m9i outcomes stay distinct, and so do standDown / pass / invalid.** *Authored* = a valid
supersede / scheduleWake / `STAND-DOWN` / pass / (well-formed) `SHOW VIEW`. *Invalid* = unparseable
Kestrel or an illegal `SHOW VIEW` under the baseline → the repairable outcome, never a fabricated
standDown, never a crash. *Provider-error* = the client raised (unchanged). Output is **never silently
repaired** — the parser stays exact and fail-closed (ADR-0004/0030); the loop repairs by re-asking the
model, not by loosening acceptance. This is the same fail-closed contract `parseTurn` /
`parseAuthoringReply` already enforce, re-expressed against the Kestrel surface instead of the JSON one.
### 4. Structured output is now inapplicable — state it plainly
A provider's structured-output / JSON-schema mode can constrain a *JSON* shape; it **cannot validate
Kestrel grammar** (Kestrel is an indentation-structured, context-sensitive surface language, not a
JSON schema). Envelope-free therefore means: **free-form text generation (`generateText`) + robust
Kestrel extraction + the deterministic `parse()` + repair-retry (ADR-0029/0030)** — *not* provider
schema validation. The recently-paused **structured-output-envelope** work (constraining the JSON
envelope with a provider schema) is **superseded** by this ADR: if the turn carries no JSON envelope,
there is no schema to enforce. That work is retained **only as a hypothetical fallback** should
envelope-free be rejected in review; it is not on the path if this ADR is accepted.
### 5. Enforcement: grammar-constrained decoding (a layered enhancement, not a dependency)
There are **two** places syntactic validity can be enforced, and §4's "structured output is
inapplicable" is precise only about the *JSON-schema* kind. A newer provider capability —
**grammar-constrained decoding** (OpenAI's non-JSON structured outputs: you supply a context-free
grammar and the model is constrained *at the token-sampling level* to only emit strings the grammar
accepts; the same capability exists in several open-source inference stacks) — is a *different* thing,
and it is the **natural enforcement layer for "the agent emits pure Kestrel."** A CFG for the Kestrel
turn surface, supplied at decode time, would mean the model **literally cannot emit invalid Kestrel**:
the `WHEN … DO …`-on-one-line collapse (the dry-run-1 #1 failure, 30–47% of turns) becomes
*unrepresentable at the sampling step*, not merely rejected after the fact. (The parallel research
spike `docs/research/2026-07-13-grammar-constrained-generation.md`, landing alongside this ADR,
investigates the mechanism and provider support; reference it for the details.)
The design treats this as a **layered enhancement, explicitly not a dependency**:
1. **It is the strongest realization of envelope-free.** Where a provider supports CFG constraint, the
*entire syntactic authoring-failure class disappears at decode time* — there is nothing to repair
for syntax, because nothing syntactically invalid can be sampled. Envelope-free is what makes this
*possible* (a single Kestrel grammar to constrain, rather than a JSON envelope wrapping an escaped,
un-constrainable string); grammar-constraint is what makes it *guaranteed*.
2. **The contract stands on its own without it.** The envelope-free contract (§1–§3) is defined by
**`generateText` + forgiving extraction + the deterministic `parse()` + repair-retry** — that is
the **baseline enforcement**, and it is complete by itself. Grammar-constrained decoding is an
*optional decode-time guarantee layered on top* where the provider supports it. If it is never
available anywhere, envelope-free still works exactly as specified; the repair-retry loop
(ADR-0029/0030) is the floor.
3. **Cross-provider asymmetry is an open question the design must absorb (Open q6).** If only *some*
providers can grammar-constrain (OpenAI and open-weight stacks: likely yes; Claude and Gemini: maybe
not, at time of writing), the harness must **degrade gracefully** — a CFG-constrained decode on the
providers that support it, and the same parse + repair-retry baseline on the ones that do not — so a
Kestrel grid stays **fair** across models. This interacts with the m9i controlled/optimized
division: whether decode-time constraint is *on* is a transport-side behavior above the determinism
line (like `cachePolicy`), so it can be a `ConfigId` axis and A/B'd, but it must never become an
unfair advantage baked into one model's baseline column. The honest framing: grammar-constraint
*removes a failure mode* for the providers that have it, and the telemetry (ADR-0030) will show the
validity gap between constrained and unconstrained arms — which is itself a finding, not a
contaminant, as long as the division is stamped and the boards never average.
4. **It eliminates only the SYNTACTIC failures — the SEMANTIC rules stay post-parse regardless.** A CFG
can guarantee the token stream *parses*; it cannot enforce Kestrel's context-sensitive **semantic**
invariants — price-must-resolve-to-an-anchor (never a bare mid), a `cross` may not be wrapped in
`held`, the provenance ceiling, budget/never-naked. Those are validated **after** parse
(`src/lang/validate.ts`) and a violation is still a repairable **invalid** outcome that surfaces the
guiding error and re-asks (ADR-0029/0030) — with or without grammar-constraint. So the repair-retry
loop never becomes dead code: grammar-constraint shrinks its job to *semantic* repair, but the loop
remains the enforcement path for everything the grammar cannot express. This is the clean split:
**syntax at decode time (where supported), semantics post-parse (always).**
### 6. Token analysis (measured with the repo cl100k oracle, `src/render/tokens.ts`, `tiktoken-cl100k`)
A representative turn — a multi-line `PLAN` (rider with `WHEN` / `DO` / `TP` / `EXIT`) plus a realistic
pre-hoc journal — rendered **both** ways and counted precisely:
| Turn | JSON envelope | pure Kestrel | saving |
|---|---:|---:|---:|
| representative rider + journal | **110 tok** | **95 tok** | **15 tok — 13.6% fewer** |
| stand-down + reason | 23 tok | 15 tok | 8 tok — **34.8% fewer** |
| large 9-clause plan + journal | 171 tok | 154 tok | 17 tok — 9.9% fewer |
| *(the escaping cost alone: the `document` string)* | 67 tok (JSON-escaped) | 64 tok (raw) | 3 tok — ~4.7% just from `\n`/quote escaping |
The saving is **largest on the smallest, most frequent replies** (a stand-down is ~35% envelope) and
settles at **~10–14%** on plan-bearing turns; the envelope is a fixed overhead that dominates short
turns. On top of the counted saving, envelope-free removes the **tokenizer-hostile fragmentation** of
`\n`-escaped source (the ~4.7% escaping tax is *token count*; the fragmentation of the stream into
smaller, less-natural tokens is an additional, harder-to-quantify cost the percept design already
fights). Every one of these is *output* tokens the model must generate — the attention axis
(`ev_per_ktoken`, ADR-0012 §5) reflects the saving directly.
### 7. Determinism, harness, prompt
- **Above the line, unchanged.** The Kestrel-document reply is the agent's choice — above the
determinism line (ADR-0011/0012). Only the **parsed terminal turn** crosses to the Bus.
`CapturedTurns` stores the `AgentTurn` (journal + typed actions) via the same sorted-key
canonicalizer as today — **not** the raw reply — so `recordedAgent` replays **byte-identically**
with zero new machinery. The envelope-free change touches only *how the terminal `AgentTurn` is
produced from the reply*, never what is captured or replayed.
- **A new versioned prompt profile.** The current DSL-taught profile (`authoring-v1`,
`src/session/harness/prompt.ts`) teaches *"reply with EXACTLY ONE JSON object"* and shows
`JSON.stringify` turns. Envelope-free needs a **new profile** — call it **`authoring-v2`** — that
teaches *"reply with a Kestrel document; your reasoning is `#` comments; stand down with
`STAND-DOWN`; do nothing by replying with only comments."* New bytes ⇒ new `promptHash` ⇒ new
`ConfigId` ⇒ a **distinct grid column** (ADR-0013 (d)). `authoring-v1` (JSON) stays **byte-frozen**
as the A/B control. The viewshop loop gets its sibling **`viewshop-v2`** (teaching `SHOW VIEW`
instead of `{requestView}`), extending `authoring-v2` exactly as `viewshop-v1` extends
`authoring-v1`; the pane menu interpolation and its catalog-hash discipline are unchanged.
- **The file-handshake external CLIs emit a Kestrel document too (symmetric).** The matched-interface
discipline (m9i.7) requires it: `turn-<ord>` becomes a **Kestrel document** the brain writes, parsed
by the *same* routing as `liveAgent`. The `STAND_DOWN` sentinel dissolves into the grammar — it is
just the `STAND-DOWN` directive now — one fewer special case. `first-party-liveagent`,
`claude-code`, `codex`, `opencode` all author the same Kestrel surface.
- **Interaction with ADR-0029 and ADR-0030.** ADR-0029's *outcome model* is fully preserved:
`RequestView` / `AuthoringReply` / the distinct-top-level-outcome-that-is-not-an-Action / the
above-the-line loop / the emergence log — all unchanged. Only the **wire surface** of the request
changes (`SHOW VIEW` Kestrel instead of `{requestView}` JSON), routing to the same
`AuthoringReply.requestView` variant. ADR-0030's error-cluster telemetry now measures Kestrel-surface
authoring errors directly (no longer errors *inside a JSON string*), which is a cleaner signal — the
model's layout is now visible to the model.
### 8. Migration and measurability
**Recommend a new profile A/B'd against the JSON-envelope baseline — not a straight replacement.**
`authoring-v2` (and `viewshop-v2`) run as **new grid columns** against the frozen `authoring-v1` /
`viewshop-v1` controls, split by `ConfigId` so the two never average (ADR-0013 (d), ADR-0030
discipline: two knobs, one measurement). The A/B measures the three claimed wins as facts, not
assumptions:
1. **Token saving** — the ~10–35% output reduction quantified in §6, per turn, on the real tape.
2. **Gemini non-emission elimination** — dry-run-2's 34 envelope non-emissions should go to
*structurally zero* (nothing to fail to emit). This is the strongest single reason to expect
envelope-free to *raise* usable-turn rate, and it is directly measurable.
3. **Authoring-quality delta** — does removing the JSON scaffolding *help or hurt* Plan validity? The
ADR-0030 error-cluster telemetry answers it: envelope-free removes the "collapsed inside the
escaped string" failure family but is a *new* surface the model must learn; the A/B tells us the
net. Graduate `authoring-v2` to the default **only** if it is net-positive; keep the envelope
profile registered as the control regardless. This keeps the controlled/optimized division honest.
## Consequences
- **ADR-0013 (a) is finally realized.** The agent authors Kestrel *as its whole turn*, not a JSON
tool-call wrapping Kestrel. The half-measure is gone.
- **A whole failure class disappears.** The JSON-envelope non-emission mode (dry-run-2: 34 Gemini
turns) is structurally impossible — there is no envelope. The extractor is forgiving where the
exact-`{…}` extractor was brittle.
- **Fewer output tokens every turn**, most on the most frequent (small) turns, plus the removal of
`\n`-escaped fragmentation — a direct, measured gain on the attention axis.
- **One grammar, not two.** The model satisfies only Kestrel; its layout is visible to it (not hidden
inside a JSON string), which should reduce the dry-run-1 layout-collapse family and sharpens the
ADR-0030 measurement.
- **A decode-time enforcement path opens up.** Because the turn is one Kestrel grammar (not a JSON
wrapper around an escaped string), it becomes *constrainable* by grammar-constrained decoding (§5)
on providers that support it — the strongest realization of envelope-free, making the syntactic
failure class unrepresentable at the sampling step. It stays a layered enhancement, not a
dependency: parse + repair-retry is the baseline everywhere, and semantic validation stays post-parse
regardless.
- **The determinism contract is untouched.** Captured turns, `SimRunId`, and `recordedAgent`
byte-identity all hold unchanged — only the reply→turn parse changes, above the line.
- **The grammar grows by two directives** (`STAND-DOWN`, `SHOW VIEW`) and a small lexer addition
(rest-of-line reason capture). Both are top-level and non-colliding. This is an ADR-0030 grammar
*proposal*, so it lands as a measured A/B arm, not a fiat change.
- **New surfaces to build (all additive, no determinism-core change):** the Kestrel-turn router
(replacing the JSON half of `parseTurn` / `parseAuthoringReply`), the forgiving Kestrel extractor,
the `STAND-DOWN` / `SHOW` grammar + lexer additions, the `authoring-v2` / `viewshop-v2` profiles,
the file-handshake Kestrel-turn variant, and the resolution of the `cancelOrder` residual (Open q3).
- **A dependency was surfaced, then closed.** The byte-stable inline-comments workstream (ADR-0013 (b),
now **ADR-0033**) is the thing that lets a `#` reason ride *with* an armed document; the journal
channel never waited on it, and the "WHY welded to the action" promise is now **delivered** —
comments round-trip byte-identically through `print(parse(x))`.
## Rejected alternatives
1. **Keep the JSON envelope; constrain it with provider JSON-schema structured output.** This is the
just-paused work. Rejected as the path: it entrenches the envelope's token cost, escaping
fragmentation, and two-grammar burden, and cannot validate the Kestrel *inside* the string anyway
(JSON schema validates the wrapper, not the payload). Retained only as the fallback if envelope-free
is rejected. Note this is a *different* capability from **grammar-constrained (CFG) decoding**
(§5), which this ADR *embraces* as the natural enforcement layer — CFG constraint applies to the
pure Kestrel surface directly, precisely because there is no JSON wrapper in the way.
2. **ADR-0029's Option B verbatim — a *bare* `VIEW` document as the request.** Still rejected, for
ADR-0029's own reason: a bare `VIEW` cannot disambiguate request from standing registration. This
ADR does **not** revive Option B; it adds the `SHOW` marker that removes exactly that ambiguity
(§2b) — Option B's grammar fidelity **plus** the disambiguation ADR-0029 required.
3. **Reintroduce a minimal JSON envelope for just `placeOrder`/`cancelOrder`.** Rejected: a "minimal"
envelope is still an envelope (scaffolding, escaping, a second grammar) and reopens the
non-emission failure mode for the turns that carry orders. Immediate orders are expressed as a
fire-now `PLAN`; the `cancelOrder`-by-`ref` residual is solved with a grammar directive, not a JSON
carve-out (Open q3).
4. **A straight replacement (retire `authoring-v1` immediately).** Rejected: it forfeits the ability
to *measure* whether envelope-free helps (token saving, non-emission elimination, authoring
quality) and violates the controlled/optimized separation. It must be a versioned A/B arm
(ADR-0030), graduated on evidence.
5. **Silently repair invalid Kestrel to make envelope-free "just work."** Rejected, non-negotiable:
the parser stays exact and fail-closed (ADR-0004/0011/0030). Envelope-free repairs by *re-asking*
the model with the guided error, never by loosening acceptance.
## Open questions (need owner input before implementation)
1. **The stand-down keyword and the request marker (the crux).** Recommended: `STAND-DOWN` (inline
rest-of-line reason) and `SHOW VIEW` (trailing `#` reason). Confirm the keywords and the reason
placement — they are taught in `authoring-v2` and are the load-bearing disambiguators.
2. **The journal / byte-stable-comments dependency — RESOLVED.** The journal-as-`#`-comment channel
(lifted into the turn envelope at parse, captured as today) is sufficient for v1, and the
comment-rides-with-the-armed-document promise (ADR-0013 (b)) is no longer deferred: the
byte-stable-comments workstream landed as **ADR-0033**, so a `#` comment survives
`print(parse(x))` byte-identically and folds into the armed-document hash.
3. **The `cancelOrder` residual.** A new `CANCEL <ref>` turn directive vs. expressing it as a
plan-scope `CANCEL-IF` vs. a narrow retained imperative. Recommend a `CANCEL <ref>` directive
(symmetric with `STAND-DOWN` / `SHOW`, no JSON). Confirm.
4. **Migration shape.** New A/B profile (recommended) vs. straight replacement. Confirm A/B, and the
graduation criterion (net-positive on token saving + non-emission rate + authoring validity).
5. **Whether `SHOW VIEW` should also be offered at mid-session `decide()` wakes**, or stay OPEN-only
as ADR-0029 §Scope has it for v1. This ADR follows ADR-0029: OPEN-only for v1.
6. **Grammar-constrained decoding across providers (§5).** If only some providers support CFG
constraint (OpenAI / open-weight: likely; Claude / Gemini: maybe not), confirm the graceful-degrade
posture — CFG decode where available, parse + repair-retry everywhere else — and whether
decode-time constraint is a `ConfigId` A/B axis (above the line, like `cachePolicy`). It must keep a
Kestrel grid fair: the constrained/unconstrained validity gap is a *stamped finding*, never an
unfair advantage folded into one model's baseline column. Tracked with the research spike
`docs/research/2026-07-13-grammar-constrained-generation.md`.