UNPKG

kestrel.markets

Version:

A typed, token-efficient language + runtime for agentic trading: agents author bounded plans, the runtime fires them at the tick. CLI + typed library + MCP server.

739 lines (709 loc) 59.6 kB
/** * # session/harness/ai-sdk-client — the BYOK Vercel-AI-SDK {@link LlmClient} (the ONLY provider file) * * The concrete, wire-touching `LlmClient` behind {@link liveAgent} (ADR-0013 (c): the Vercel AI SDK * is the default BYOK driver). This is the ONE module in the tree that imports the provider packages * (`ai`, `@ai-sdk/amazon-bedrock`, `@ai-sdk/anthropic`, `@ai-sdk/google`) — `../simulate.ts`, * `../agent.ts`, `../index.ts`, and `src/index.ts` never reach it, so the deterministic core / * node-light root stay provider-free (ADR-0013 Consequences). * * ## BYOK, scrubbed at capture (ADR-0013 (e)) * The key is supplied through the ENVIRONMENT only (`AWS_BEARER_TOKEN_BEDROCK` for Bedrock, * `ANTHROPIC_API_KEY` for the direct Anthropic path, `GEMINI_API_KEY`/`GOOGLE_GENERATIVE_AI_API_KEY`/ * `GOOGLE_API_KEY` for the Google/Gemini path) and injected into the provider at construction. * It NEVER enters {@link AgentConfig} (content-hashed into `ConfigId`), and the wire evidence this * client captures is run through {@link scrubSecrets} — with every credential env value AND the * provided secrets — BEFORE the bytes are attached to the completion, so no artifact can carry a * credential (an m9i acceptance criterion, met structurally rather than by convention). */ import { generateText, wrapLanguageModel, type LanguageModel, type LanguageModelMiddleware, type ModelMessage } from "ai"; import { createAmazonBedrock } from "@ai-sdk/amazon-bedrock"; import { createAnthropic } from "@ai-sdk/anthropic"; import { createGoogleGenerativeAI } from "@ai-sdk/google"; import { createOpenAICompatible } from "@ai-sdk/openai-compatible"; import { createOpenAI } from "@ai-sdk/openai"; import { createAzure } from "@ai-sdk/azure"; import { cacheTtlNeverReachesWire, type AgentConfig, type CacheTtl, type LlmClient, type LlmCompletion } from "../agent.ts"; import { CREDENTIAL_ENV_VARS, scrubSecrets } from "./prompt.ts"; import { FIREWORKS_PROVIDER_NAME, decodingMiddleware, defaultDecodingFor, type DecodingConstraint } from "./decoding.ts"; import { VERTEX_MAAS_PROVIDER_NAME, gcloudAdcTokenSource, requireVertexMaasProject, resolveVertexMaasLocation, vertexMaasBaseUrl, type AdcExecFn, type AdcTokenSource, } from "./vertex-maas.ts"; /** Which BYOK provider the client drives — the ENV name each reads its key from, never persisted: * `bedrock` → `AWS_BEARER_TOKEN_BEDROCK`; `anthropic` → `ANTHROPIC_API_KEY`; `google` → * `GEMINI_API_KEY` / `GOOGLE_GENERATIVE_AI_API_KEY` / `GOOGLE_API_KEY` (first present wins); * `gateway` → `AI_GATEWAY_API_KEY` (the Vercel AI Gateway, an OpenAI-compatible surface — the CASH * lane, billed per token, so reserved for lots of small cheap runs or models with no credit lane); * `local` → `LOCAL_LLM_BASE_URL` (a local mlx_lm.server / vLLM / Ollama serving open-weight watchers — * the ZERO-CASH, CFG-capable lane for the large watcher fleet; `apiKey` optional); * `openai` → `OPENAI_API_KEY` (the direct OpenAI API — a CASH lane; constrained by strict JSON Schema, * NOT GBNF, which OpenAI cannot take); * `azure` → `AZURE_OPENAI_ENDPOINT` + `AZURE_OPENAI_API_KEY` + `AZURE_OPENAI_API_VERSION` (the same GPT * models on pre-committed CREDITS — so a LARGE fan-out of a GPT watcher rides Azure, and only a SMALL * spot-check rides cash OpenAI; same strict-JSON-Schema constraint); * `fireworks` → `FIREWORKS_API_KEY` (an OpenAI-compatible surface for the BIG OPEN weights — DeepSeek, * Llama-405B, Qwen — a CASH lane, but the only hosted one that takes **GBNF** grammar-constrained * decoding, so it is where an open model gets the same emission floor the local lane gets); * `vertex-maas` → NO env key: an **ADC bearer** refreshed via `gcloud auth application-default * print-access-token` (Vertex Model Garden MaaS — DeepSeek V3.2 / Qwen3-Next-80B / Kimi K2 Thinking on * GCP CREDITS, over the OpenAI-compatible `locations/global` endpoint; see `./vertex-maas.ts` for the * region trap and the dead-ADC doctrine, kestrel-k8he). * * `codex-cli` is the ZERO-CASH ChatGPT-SUBSCRIPTION lane (`gpt-5.6-sol` and the other GPT variants). It is * NOT an AI-SDK provider and is NOT built here: no OpenAI-compatible HTTP endpoint accepts a ChatGPT * subscription OAuth token, so it is served by SHELLING OUT to the Codex CLI (`codex exec`) in * `./codex-cli-client.ts`. It is a member of this union so the whole harness can speak ONE provider * vocabulary; {@link harnessLlmClient} in `./harness-client.ts` is the dispatcher that routes it there, and * {@link buildModel} FAILS CLOSED if it ever reaches the SDK path. * * There is deliberately NO `subagent` member. A Claude Code subagent is ONE-SHOT — spawned, it works, it * returns text — so it can author a plan DOC but can never serve as a per-turn client inside the harness's * multi-turn loop. The `subagent` LANE in {@link roster} is therefore an AUTHORING lane only (see * `scripts/bench/author-cells.ts`), not a provider. Multi-turn watching still requires a real API endpoint. * * See {@link roster} (cost-tier policy) and {@link resolveLane} for which lane a given model routes to at * each run size, and `./decoding.ts` for which constrained-decoding encoding each lane can honor. */ export type HarnessProvider = | "bedrock" | "anthropic" | "google" | "gateway" | "local" | "codex-cli" | "openai" | "azure" | "fireworks" | "vertex-maas"; /** Fireworks' OpenAI-compatible inference endpoint. The lane that serves the BIG open weights AND * accepts a GBNF grammar in `response_format` (see `./decoding.ts`). */ export const FIREWORKS_BASE_URL = "https://api.fireworks.ai/inference/v1"; /** The Azure OpenAI `api-version` used when `AZURE_OPENAI_API_VERSION` is unset. Azure pins its request * surface per api-version; an older one may silently DROP the `pattern` keyword from a strict schema * (see the L4 note in `artifacts/grammar/watcher.schema.json`), which is why this is overridable. */ export const AZURE_API_VERSION_DEFAULT = "2024-10-21"; /** The Vercel AI Gateway OpenAI-compatible base URL (the CASH lane). Model ids ride the * provider-prefixed form, e.g. `anthropic/claude-haiku-4.5`, `openai/gpt-5.6-sol`. */ export const GATEWAY_BASE_URL = "https://ai-gateway.vercel.sh/v1"; /** The gateway skip-cache header. MUST be sent `true` on EVERY gateway request — else the AI Gateway may * serve n replicates ONE cached completion, fabricating a σ=0 (a contamination-class bug: a cache artifact * that masquerades as measured determinism). Set at gateway-client construction so no call site can forget * it. Matches the hosted adapter's convention (scripts/bench/hosted/gateway-lane.ts). */ export const GATEWAY_SKIP_CACHE_HEADER = "cf-aig-skip-cache"; /** The default local OpenAI-compatible base URL when `LOCAL_LLM_BASE_URL` is unset — the port an * `mlx_lm.server` / vLLM / Ollama-compatible server listens on for the zero-cash watcher fleet. */ export const LOCAL_BASE_URL_DEFAULT = "http://localhost:8080/v1"; /** The Vercel AI Gateway rejects `max_tokens` below this with HTTP 400; the gateway lane floors its * output ceiling here so a tiny watcher budget can't turn a live call into a 400 (VERIFIED LIVE). */ export const GATEWAY_MIN_OUTPUT_TOKENS = 16; /** A sane default reasoning budget (in output tokens) for a REASONING model on the gateway lane * (kestrel-a1f). Fable 5 is a thinking-always-on model: over the Vercel gateway's OpenAI-compatible * surface its reasoning/thinking tokens are drawn from the SAME `max_tokens` budget as the visible * reply. So a small authoring ceiling (the 1024 that shipped) is spent ENTIRELY on reasoning and the * visible plan text comes back EMPTY (finish_reason=length, no supersede). The gateway lane widens * the wire ceiling by the caller's `reasoningHeadroomTokens` so the requested VISIBLE-output budget * still survives after the model finishes thinking. This constant is a good default headroom for a * bounded authoring turn; callers pass it (or their own value) via {@link AiSdkClientOptions}. */ export const GATEWAY_REASONING_HEADROOM_TOKENS = 12000; /** The slice of the AI-SDK `generateText` result this client reads. Declared so the temperature-capability * path can be exercised offline through {@link AiSdkClientOptions.generate} (no network, no provider SDK). * `inputTokenDetails.cacheRead/cacheWriteTokens` are the AI SDK's flattened provider-native prompt-cache * counters (Anthropic `cache_read/cache_creation_input_tokens`, Bedrock `cacheRead/cacheWriteInputTokens`) — * the on-record source of the m9i.5 cache economics (kestrel-rul). */ export interface GenerateResult { readonly text: string; readonly reasoningText?: string; readonly usage: { readonly inputTokens?: number; readonly outputTokens?: number; readonly inputTokenDetails?: { readonly cacheReadTokens?: number; readonly cacheWriteTokens?: number; readonly noCacheTokens?: number; }; readonly outputTokenDetails?: { readonly reasoningTokens?: number }; }; readonly request?: unknown; readonly response?: { readonly id?: string; readonly modelId?: string; readonly headers?: unknown }; readonly providerMetadata?: unknown; readonly finishReason?: unknown; /** The AI SDK's call warnings. Read for ONE purpose: a provider that SILENTLY DROPS a setting reports it * here rather than throwing (see {@link droppedTemperature}). Two shapes exist across SDK generations: * the older `{ type: "unsupported-setting", setting }` and ai v7's `{ type: "unsupported", feature }` * (`src/logger/log-warnings.ts` formats the latter — the "The feature X is not supported" console line). */ readonly warnings?: readonly { readonly type?: string; readonly setting?: string; readonly feature?: string; readonly details?: string }[]; } /** Did the provider DROP `temperature` rather than honor it? The OpenAI reasoning models (the whole gpt-5 * family) do not accept `temperature`, and the AI SDK does NOT throw for them — it strips the field and * emits an `unsupported-setting` warning, so the call succeeds having silently used the model's default. * * That is a quiet lie in the evidence, and exactly the class of thing kestrel-gvx exists to stop: without * this check the captured wire record would report `temperature: 0` for a call whose temperature field * never went on the wire, and a reader would credit a greedy-decoding result to a run that never asked for * one. Detecting the warning lets `complete` record `"model-default"` — what ACTUALLY happened. * * (Distinct from {@link rejectsTemperature}, which handles the providers — the newest Bedrock Sonnet — that * ERROR on the field instead of dropping it. Same lie, two different provider behaviors.) * * BOTH warning shapes are matched: the older `unsupported-setting`/`setting` pair AND ai v7's * `unsupported`/`feature` pair. The installed ai v7 emits ONLY the latter — caught live on the azure lane * (azure-foundry-sweep v1, 2026-07-18): gpt-5.4 deployments dropped `temperature` with a v7-shape warning, * the old matcher missed it, and 24/24 rows' wire evidence claimed `temperature: 1` for a field that never * rode the wire — the exact kestrel-gvx lie this function exists to stop, inert for one shipped generation * because its fixture pinned the synthetic old shape rather than what the real SDK emits. */ export function droppedTemperature(r: GenerateResult): boolean { return (r.warnings ?? []).some( (w) => (w.type === "unsupported-setting" && w.setting === "temperature") || (w.type === "unsupported" && w.feature === "temperature"), ); } /** One AI-SDK conversation `ModelMessage` this client sends. Only `user`/`assistant` ride `messages`; * the cached SYSTEM prefix does NOT (AI SDK v7 rejects a leading `system`-role message in `messages`: * *"System messages are not allowed in the prompt or messages fields. Use the instructions option * instead"*) — it rides the top-level `system`/`instructions` field as an {@link AiSdkSystemMessage}. * `providerOptions` carries the provider-native prompt-cache breakpoint (Anthropic `cacheControl` / * Bedrock `cachePoint`) on the conversation message the harness marked cacheable. */ export interface AiSdkMessage { readonly role: "user" | "assistant"; readonly content: string; readonly providerOptions?: Record<string, unknown>; } /** The cached SYSTEM prefix as an AI-SDK `SystemModelMessage` — the shape the top-level * `system`/`instructions` option accepts (`Instructions = string | SystemModelMessage | SystemModelMessage[]` * in ai v7), so the byte-stable system prefix can carry the provider-native cache breakpoint WITHOUT being a * leading `system`-role message inside `messages` (which the v7 core rejects outright). Both the Bedrock and * Anthropic providers fold this into their top-level system field with the cache point attached. */ export interface AiSdkSystemMessage { readonly role: "system"; readonly content: string; readonly providerOptions?: Record<string, unknown>; } /** The `generateText` args this client passes (a subset). `temperature` is OMITTED for a model that rejects * the field outright — see {@link aiSdkLlmClient}'s capability-aware `complete`. `system` is ALWAYS set: a * plain string for the uncached path (and for Google, whose caching is implicit — no breakpoint), or an * {@link AiSdkSystemMessage} carrying the cache breakpoint when the byte-stable system prefix is cached. */ export interface GenerateArgs { readonly model: LanguageModel; readonly system?: string | AiSdkSystemMessage; readonly messages: readonly AiSdkMessage[]; readonly temperature?: number; readonly maxOutputTokens: number; readonly maxRetries: number; /** Top-level provider-native `providerOptions` for the whole request — currently the extended-thinking * config from {@link thinkingProviderOptions} (`{ anthropic: { thinking } }` / `{ bedrock: { reasoningConfig } }` * / `{ google: { thinkingConfig } }`). OMITTED (undefined) when `thinkingLevel:"none"`, so a no-thinking * request is byte-identical to before this knob was wired. Distinct from the per-MESSAGE `providerOptions` * on {@link AiSdkMessage}/{@link AiSdkSystemMessage}, which carry the prompt-cache breakpoint. */ readonly providerOptions?: Record<string, unknown>; } /** The provider-native prompt-cache breakpoint for `provider`, as AI-SDK `providerOptions` — or `undefined` * when the provider has NO explicit breakpoint mechanism. Anthropic marks an `ephemeral` `cacheControl` * block; Bedrock inserts a `cachePoint`; **Google/Gemini returns `undefined`** — Gemini has no per-message * cache-control primitive in this SDK version (implicit context caching is automatic and its cache-read * tokens still flow through {@link GenerateResult} usage; explicit `cachedContent` needs an out-of-band * cache-create call, so it is NOT wired here — we do not fake a breakpoint the provider cannot honor). * Applied to the last content part of the message it rides (the AI SDK falls back to message-level * `providerOptions` for the final part). * * `ttl` (kestrel-wa0j.1) opts the breakpoint into a non-default cache TTL. `undefined` ⇒ the provider * default and BYTE-IDENTICAL breakpoint objects to before the axis existed (no `ttl` key at all). Both * native lanes take it (verified against the installed SDKs: `@ai-sdk/anthropic` 4.0.12 * `cacheControl.ttl: "5m"|"1h"`; `@ai-sdk/amazon-bedrock` 5.0.17 `cachePoint.ttl: "5m"|"1h"`). A TTL on a * lane with NO breakpoint primitive THROWS — the ConfigId claims a TTL, and returning `undefined` would * silently drop a configured knob (fail-closed, never silently inert). */ export function cacheBreakpoint(provider: HarnessProvider, ttl?: CacheTtl): Record<string, unknown> | undefined { switch (provider) { case "bedrock": return { bedrock: { cachePoint: { type: "default", ...(ttl !== undefined ? { ttl } : {}) } } }; case "anthropic": return { anthropic: { cacheControl: { type: "ephemeral", ...(ttl !== undefined ? { ttl } : {}) } } }; case "google": case "gateway": case "local": case "openai": case "azure": case "fireworks": case "vertex-maas": case "codex-cli": // No explicit per-message cache-control primitive on these lanes: Gemini caches implicitly (no // breakpoint); the OpenAI-compatible surface caches automatically on a stable prefix; the Codex // CLI's caching is provider-internal. We do not fake a breakpoint the wire cannot honor — and a // configured TTL these lanes cannot express must FAIL, not silently ride a ConfigId column that // claims a cache TTL the provider never saw (kestrel-gvx: the evidence never lies about the wire). if (ttl !== undefined) { throw new Error( cacheTtlNeverReachesWire( ttl, `provider "${provider}" has no explicit prompt-cache breakpoint to carry a TTL`, `Use provider "anthropic" or "bedrock", or drop cacheTtl.`, ), ); } return undefined; } } /** The strategist/watcher reasoning-effort dial, mirrored from {@link AgentConfig.thinkingLevel} (it rides * the ConfigId, so it is part of run identity). `none` is the default EVERYWHERE (author-cells.ts included). */ export type ThinkingLevel = AgentConfig["thinkingLevel"]; /** LEVEL → extended-thinking budget, in *thinking* tokens (SEPARATE from the visible-reply `maxOutputTokens` * ceiling — see {@link aiSdkLlmClient}, which raises the wire ceiling ABOVE this so `max_tokens > budget`). * * `none` → 0 ⇒ thinking is DISABLED: NO `providerOptions.*.thinking` key rides the wire at all, so a `none` * request is BYTE-IDENTICAL to before this knob was wired (the additive-opt-in guarantee). `low` starts at * 1024 — the Anthropic API's HARD MINIMUM `budget_tokens` (a value below 1024 is rejected), so it is the * smallest budget that can legally enable thinking. The rest step up geometrically to give the reasoning-level * sweep (task#19) real separation without crossing the 32k soft-ceiling where Anthropic recommends streaming * to avoid long-request timeouts: `medium` 4096, `high` 12000 (matches {@link GATEWAY_REASONING_HEADROOM_TOKENS}, * the empirically-sufficient authoring headroom from kestrel-a1f), `max` 24000. */ export const THINKING_BUDGET: Record<ThinkingLevel, number> = { none: 0, low: 1024, medium: 4096, high: 12000, max: 24000, }; /** LEVEL → the OpenAI-surface `reasoning_effort` enum for the gpt/open lanes (`openai`/`azure`/`fireworks`), * which express effort as an ENUM, not a token budget (kestrel-0drf — this mapping used to be a documented * no-op, so every "gpt-5 at high thinking" row compared the seat against itself). * * The 5→3 granularity decision, explicit: `reasoning_effort` is only guaranteed THREE values across every * OpenAI reasoning generation (o-series: low/medium/high; gpt-5 adds `minimal`; newer snapshots add more), * so `max` CLAMPS to `high` — the strongest value every generation accepts. A newer-only top value * (`xhigh`) would turn the dial's top stop into a wire error on a model that would have honored `high` — * a knob that fails exactly where it should bite hardest. `none` never reaches this table: no key rides * the wire at all (the additive-opt-in guarantee), so a `none` request stays byte-identical to before the * dial was wired. (The codex-cli lane has its own 5→5 mapping — `codexReasoningEffort` in * `./codex-cli-client.ts` — because the CLI's dial does reach `ultra`.) */ export const REASONING_EFFORT: Record<Exclude<ThinkingLevel, "none">, "low" | "medium" | "high"> = { low: "low", medium: "medium", high: "high", max: "high", // clamp — see the doc above }; /** Map a {@link ThinkingLevel} to the provider-native extended-thinking/effort `providerOptions` for * `provider`, or `undefined` when NO thinking config should ride the wire — i.e. `none` (thinking disabled ⇒ * byte-identical to today), OR a lane with no native per-request primitive (`gateway`/`local`: the * OpenAI-compatible surface shares one `max_tokens` budget and is handled by the separate gateway * `reasoningHeadroomTokens` mechanism; `codex-cli`: the subprocess lane maps the SAME dial onto the CLI's * `model_reasoning_effort` flag in `./codex-cli-client.ts`, never through providerOptions). The wired lanes * each take a different key, VERIFIED against the installed SDKs (`@ai-sdk/anthropic`+`@ai-sdk/google` * 4.0.12, `@ai-sdk/amazon-bedrock` 5.0.17, `@ai-sdk/openai`+`@ai-sdk/azure` 4.0.14, * `@ai-sdk/openai-compatible` 3.0.9): * - anthropic → `{ anthropic: { thinking: { type: "enabled", budgetTokens } } }` (AnthropicProviderOptions.thinking) * - bedrock → `{ bedrock: { reasoningConfig: { type: "enabled", budgetTokens } } }` (BedrockProviderOptions.reasoningConfig) * - google → `{ google: { thinkingConfig: { thinkingBudget } } }` (GoogleGenerativeAIProviderOptions.thinkingConfig) * - openai/azure → `{ openai: { reasoningEffort } }` (both serve @ai-sdk/openai's Chat Completions model, * which parses providerOptions under the `openai` key — azure.chat included — and maps `reasoningEffort` * onto the `reasoning_effort` request field; see {@link REASONING_EFFORT} for the 5→3 clamp) * - fireworks → `{ fireworks: { reasoningEffort } }` (the openai-compatible model reads providerOptions * under its provider name and maps `reasoningEffort` → `reasoning_effort` on the body) */ export function thinkingProviderOptions( provider: HarnessProvider, level: ThinkingLevel, ): Record<string, unknown> | undefined { if (level === "none") return undefined; // thinking disabled — no key on the wire (byte-identical to today) const budgetTokens = THINKING_BUDGET[level]; switch (provider) { case "anthropic": return { anthropic: { thinking: { type: "enabled", budgetTokens } } }; case "bedrock": return { bedrock: { reasoningConfig: { type: "enabled", budgetTokens } } }; case "google": return { google: { thinkingConfig: { thinkingBudget: budgetTokens } } }; case "gateway": case "local": case "codex-cli": case "vertex-maas": // No native per-request thinking-budget primitive on these lanes — thinkingLevel is a no-op here (the // gateway lane uses `reasoningHeadroomTokens` instead; codex-cli maps effort in ./codex-cli-client.ts). // vertex-maas: the Model Garden OpenAI-compatible surface documents no `reasoning_effort` field — // UNVERIFIED against the live endpoint, so nothing is sent rather than a field the wire may drop // silently (kestrel-gvx: the evidence never claims a knob the provider never saw). Follow-up if the // MaaS surface is probed to honor it. return undefined; case "openai": case "azure": // The OpenAI reasoning family expresses effort as `reasoning_effort` (an enum, not a token budget). // Both lanes serve it through @ai-sdk/openai's Chat Completions model (the azure provider constructs // OpenAIChatLanguageModel with provider "azure.chat"), which parses providerOptions under the // `openai` key and puts `reasoningEffort` on the body as `reasoning_effort` (verified @ai-sdk/openai // 4.0.14). This was a documented no-op until kestrel-0drf — every "gpt at thinkingLevel X" row ran // at the provider-default effort, comparing the seat against itself. return { openai: { reasoningEffort: REASONING_EFFORT[level] } }; case "fireworks": // Fireworks' OpenAI-compatible surface takes the same `reasoning_effort` request field; the // openai-compatible chat model reads providerOptions under its provider name (`fireworks`) and maps // `reasoningEffort` onto the body (verified @ai-sdk/openai-compatible 3.0.9). Same clamped mapping. return { [FIREWORKS_PROVIDER_NAME]: { reasoningEffort: REASONING_EFFORT[level] } }; } } /** The injectable provider call — the real one wraps `generateText`; a test passes a stand-in. */ export type GenerateFn = (args: GenerateArgs) => Promise<GenerateResult>; /** Does this provider error mean the model REJECTS the `temperature` field outright, rather than a transient * failure? The newest Bedrock Sonnet generation (`us.anthropic.claude-sonnet-5`) raises "`temperature` is * deprecated for this model" even at 0. Matched narrowly so ANY other error still fails the cell explicitly * (m9i.1 — never a blanket retry that masks a real provider failure). */ export function rejectsTemperature(e: unknown): boolean { const msg = (e instanceof Error ? e.message : String(e)).toLowerCase(); return msg.includes("temperature") && /deprecat|not support|unsupported|not allowed|unexpected|invalid|unknown/.test(msg); } export interface AiSdkClientOptions { readonly provider: HarnessProvider; /** The provider-native model id (e.g. a Bedrock inference-profile id, or an Anthropic model id). */ readonly modelId: string; /** AWS region (Bedrock: defaults to `AWS_REGION`/`AWS_DEFAULT_REGION`, else `us-east-1`). On * `vertex-maas` this is the LOCATION override and only `"global"` is legal — any other value THROWS * the clear region-trap error at construction (us-central1 rejects all MaaS slugs; never a silent * rewrite — see `./vertex-maas.ts`). Ignored on every other lane. */ readonly region?: string; /** A hard output-token ceiling so a runaway reply can't blow the budget (default 2048). */ readonly maxOutputTokens?: number; /** GATEWAY-ONLY reasoning headroom, in output tokens (default 0 — no change for any other lane, and * no change for gateway callers that don't set it). For a REASONING model reached via the Vercel * gateway (Fable 5: thinking always on), the OpenAI-compatible `max_tokens` budget is shared between * the reasoning/thinking trace and the visible reply — so a small `maxOutputTokens` is consumed by * reasoning alone and the visible text returns EMPTY (kestrel-a1f). Setting this WIDENS the wire * ceiling to `maxOutputTokens + reasoningHeadroomTokens`, reserving `maxOutputTokens` for the visible * output after the model finishes thinking. Ignored off the gateway lane (Bedrock/Anthropic/Google * separate their thinking budget natively; local carries no reasoning budget here). See * {@link GATEWAY_REASONING_HEADROOM_TOKENS} for a sane default a bounded authoring turn can pass. */ readonly reasoningHeadroomTokens?: number; /** Extra secret strings to scrub from wire evidence, beyond the credential env vars (belt-and-suspenders). */ readonly secrets?: readonly string[]; /** Capture scrubbed wire evidence onto each completion (default true). */ readonly captureWire?: boolean; /** Test seam ONLY: a stand-in for the provider `generateText` (defaults to the real one). Exists so the * temperature-capability logic can be exercised offline; NEVER set on a production client. */ readonly generate?: GenerateFn; /** * CONSTRAINED DECODING. `"auto"` (the default) gives each lane the encoding it can honor — GBNF on * `fireworks`, strict JSON Schema on `openai`/`azure`, nothing on the lanes with no lever (see * `./decoding.ts`). `"off"` sends an unconstrained request. An explicit {@link DecodingConstraint} * overrides both (a specialized, frame-kernel-bound grammar, say). * * The default is `"auto"` and not `"off"` on purpose: a runner that adds a lane should get the emission * floor the benchmark's claims depend on WITHOUT having to know that it must ask for it. Forgetting to * ask would not fail loudly — it would just quietly produce a worse α and be misread as judgment. */ readonly decoding?: DecodingConstraint | "auto" | "off"; /** TEST SEAM ONLY: a stand-in `fetch` for the provider. The offline lane tests pass one that CAPTURES * the serialized request body (proving the grammar/schema really is attached to the outbound payload) * and returns a canned response — so the assertion is made against the real provider-package code path * with ZERO network calls. NEVER set on a production client. */ readonly fetch?: typeof globalThis.fetch; /** `vertex-maas` ONLY test seam: a stand-in for the `gcloud auth application-default * print-access-token` subprocess the ADC token source spawns (same pattern as `codexExec`). * NEVER set on a production client. */ readonly adcExec?: AdcExecFn; /** `vertex-maas` ONLY: a pre-built ADC token source (test seam / shared cache across clients). * Defaults to `gcloudAdcTokenSource({ exec: adcExec })`. */ readonly adcTokenSource?: AdcTokenSource; } /** Gather the concrete credential values present in the environment (never logged, only fed to the * scrubber) plus any explicitly-provided secrets, so ANY occurrence in wire evidence is redacted by * identity regardless of framing. */ function secretsToScrub(extra: readonly string[] = []): string[] { const env: string[] = []; for (const name of CREDENTIAL_ENV_VARS) { const v = process.env[name]; if (typeof v === "string" && v.trim().length >= 6) env.push(v); } return [...env, ...extra]; } /** The Google/Gemini API key, read from the environment ONLY (never persisted, scrubbed at capture). The * SDK's own default env name is `GOOGLE_GENERATIVE_AI_API_KEY`; we also honor `GEMINI_API_KEY` and * `GOOGLE_API_KEY` (the overnight `.dev.vars` convention) so the same key material works whatever the shell * calls it — first present, non-trivial value wins. Returns `undefined` when none is set (the provider then * falls back to its own default lookup, still env-only). */ function googleApiKeyFromEnv(): string | undefined { for (const name of ["GEMINI_API_KEY", "GOOGLE_GENERATIVE_AI_API_KEY", "GOOGLE_API_KEY"] as const) { const v = process.env[name]; if (typeof v === "string" && v.trim().length >= 6) return v; } return undefined; } /** The constraint this client will actually send: the caller's explicit one, or the lane's default under * `"auto"`, or none under `"off"`. Exported so a runner can LOG what it is about to constrain with (and * so the offline tests can assert the routing without constructing a provider). */ export function resolveDecoding(opts: AiSdkClientOptions): DecodingConstraint | undefined { const d = opts.decoding ?? "auto"; if (d === "off") return undefined; if (d === "auto") return defaultDecodingFor(opts.provider); return d; } /** The AI SDK's `LanguageModel` is `string | LanguageModelV2|V3|V4` — the string form is the registry * shorthand (`"openai/gpt-5"`), which this harness never uses (it constructs providers explicitly). Narrow * it away so the model can be handed to `wrapLanguageModel`, which takes only the object form. */ type LanguageModelObject = Exclude<LanguageModel, string>; /** The base (unconstrained) model for a lane. Split out from {@link buildModel} so the constraint * wrapper is applied in exactly ONE place and cannot be forgotten on a new lane. */ function buildBaseModel(opts: AiSdkClientOptions): LanguageModelObject { if (opts.provider === "codex-cli") { // FAIL-CLOSED. `codex-cli` is a SUBPROCESS lane (`./codex-cli-client.ts`), not an AI-SDK provider — there // is no OpenAI-compatible endpoint that accepts a ChatGPT-subscription OAuth token. Reaching here means a // caller bypassed the `harnessLlmClient` dispatcher; silently constructing an SDK model would send the // call to a CASH gateway instead of the FREE subscription — precisely the cost regression this lane // exists to prevent. Route through `harnessLlmClient` (`./harness-client.ts`) instead. throw new Error( `provider "codex-cli" is not an AI-SDK provider — it is the ChatGPT-subscription SUBPROCESS lane. ` + `Construct it via harnessLlmClient() from ./harness-client.ts (which routes to codexCliClient), ` + `never aiSdkLlmClient() (fail-closed: a silent SDK fallback would bill CASH on the gateway).`, ); } if (opts.provider === "bedrock") { const region = opts.region ?? process.env.AWS_REGION ?? process.env.AWS_DEFAULT_REGION ?? "us-east-1"; // apiKey defaults to AWS_BEARER_TOKEN_BEDROCK from the environment (never passed as a literal here). const bedrock = createAmazonBedrock({ region }); return bedrock(opts.modelId); } if (opts.provider === "google") { // Google/Gemini path. The key is read from the environment (never a config literal) and scrubbed from // wire evidence like every other credential. We pass it explicitly ONLY because the harness accepts // several env aliases the SDK's default lookup (GOOGLE_GENERATIVE_AI_API_KEY) does not — the VALUE is // still env-sourced and never persisted. Absent ⇒ let the provider do its own (env-only) default lookup. const apiKey = googleApiKeyFromEnv(); const google = createGoogleGenerativeAI(apiKey !== undefined ? { apiKey } : {}); return google(opts.modelId); } if (opts.provider === "gateway") { // Vercel AI Gateway — an OpenAI-compatible surface (the CASH lane). The key is env-only // (`AI_GATEWAY_API_KEY`, scrubbed at capture like every credential); absent ⇒ the provider sends // no Authorization header and the gateway 401s honestly (never a config literal). Model ids ride // the provider-prefixed form (e.g. `anthropic/claude-haiku-4.5`). The min-16 output floor is // enforced by {@link aiSdkLlmClient}, not here (buildModel carries no per-call budget). const apiKey = process.env.AI_GATEWAY_API_KEY; const gateway = createOpenAICompatible({ name: "gateway", baseURL: GATEWAY_BASE_URL, ...(apiKey !== undefined ? { apiKey } : {}), // cf-aig-skip-cache: true on EVERY gateway request (NON-OPTIONAL). The Vercel/Cloudflare AI Gateway // caches by request body; WITHOUT this header, n replicates of the same frame can silently collapse // into ONE cached completion — a contamination-class bug that FABRICATES a zero-variance (σ=0) result // that reads as measured determinism but is a cache artifact. Setting it here (at construction) means // it rides every call the gateway client ever makes and no call site can forget it. Mirrors the // hosted eval adapter's `gatewayHeaders` convention (scripts/bench/hosted/gateway-lane.ts, GATEWAY_ // SKIP_CACHE_HEADER). Any other lane is unaffected (this header is gateway-only). headers: { [GATEWAY_SKIP_CACHE_HEADER]: "true" }, }); return gateway(opts.modelId); } if (opts.provider === "local") { // Local OpenAI-compatible server (mlx_lm.server / vLLM / Ollama) — the ZERO-CASH, CFG-capable lane // for the open-weight watcher fleet. Base URL from `LOCAL_LLM_BASE_URL` (default localhost:8080/v1); // apiKey is optional (a local server usually ignores it) — a dummy keeps the SDK happy if one leaks in. const baseURL = process.env.LOCAL_LLM_BASE_URL ?? LOCAL_BASE_URL_DEFAULT; const apiKey = process.env.LOCAL_LLM_API_KEY ?? "local"; const local = createOpenAICompatible({ name: "local", baseURL, apiKey, ...(opts.fetch !== undefined ? { fetch: opts.fetch } : {}) }); return local(opts.modelId); } if (opts.provider === "openai") { // Direct OpenAI — the CASH lane. `apiKey` defaults to OPENAI_API_KEY from the ENVIRONMENT (never a // config literal, never persisted, scrubbed from wire evidence like every other credential). Absent ⇒ // the SDK raises its own "API key is missing" at call time, which is the honest failure. // // `.chat()` — CHAT COMPLETIONS, explicitly, not the (default) Responses API. Three reasons: // 1. The strict schema rides `response_format.json_schema` on Chat Completions but `text.format` on // Responses. Pinning Chat Completions gives all three new lanes (openai / azure / fireworks) ONE // wire contract, so `./decoding.ts` has one thing to attach and the tests one thing to assert. // 2. Azure serves Chat Completions on every api-version; its Responses surface is api-version-gated // and 404s on an older one — and the SAME schema must reach both lanes or the comparison is moot. // 3. The watcher is text-in/text-out with a schema. Nothing in Responses (statefulness, built-in // tools, reasoning-item plumbing) is used here, so its surface area is pure risk. const openai = createOpenAI({ ...(opts.fetch !== undefined ? { fetch: opts.fetch } : {}) }); return openai.chat(opts.modelId); } if (opts.provider === "azure") { // Azure OpenAI — the same GPT models on pre-committed CREDITS (so a LARGE fan-out routes here and only // a SMALL spot-check pays cash on the direct OpenAI lane — see `./roster.ts`). // // `AZURE_OPENAI_ENDPOINT` is the resource endpoint (e.g. https://my-resource.openai.azure.com). We pass // it as `baseURL` rather than deriving a `resourceName`, because a private/proxied endpoint is common // and a resourceName cannot express one. `opts.modelId` is the **DEPLOYMENT NAME**, not the OpenAI model // id — on Azure the deployment is the addressable unit, and `useDeploymentBasedUrls` puts it in the path. const endpoint = process.env.AZURE_OPENAI_ENDPOINT; const apiVersion = process.env.AZURE_OPENAI_API_VERSION ?? AZURE_API_VERSION_DEFAULT; const apiKey = process.env.AZURE_OPENAI_API_KEY; const azure = createAzure({ ...(endpoint !== undefined && endpoint.trim() !== "" ? { baseURL: `${endpoint.replace(/\/+$/, "")}/openai` } : {}), ...(apiKey !== undefined ? { apiKey } : {}), apiVersion, useDeploymentBasedUrls: true, ...(opts.fetch !== undefined ? { fetch: opts.fetch } : {}), }); return azure.chat(opts.modelId); // Chat Completions — see the `openai` branch for why, in full } if (opts.provider === "vertex-maas") { // Vertex Model Garden MaaS — DeepSeek V3.2 / Qwen3-Next-80B / Kimi K2 Thinking on GCP CREDITS, over // the OpenAI-compatible endpoint (kestrel-k8he; live probe wf_e58255fd-e01). Two fail-closed guards // fire HERE, at construction, before anything can reach the wire: // - PROJECT: the endpoint URL embeds the GCP project id; absent env project ⇒ throw naming the // env vars (a blank project could only 404 opaquely). // - THE REGION TRAP: only `locations/global` serves MaaS — us-central1 rejects every slug. A // configured region (opts.region or a GOOGLE_VERTEX_LOCATION/GOOGLE_CLOUD_LOCATION in the // shell) that is not "global" throws the clear region error; NEVER a silent rewrite to global. const project = requireVertexMaasProject(); const location = resolveVertexMaasLocation(opts.region); // AUTH IS AN ADC BEARER, NOT AN API KEY. Every outbound call gets `Authorization: Bearer <token>` // attached by this fetch wrapper, with the token refreshed via `gcloud auth application-default // print-access-token` (cached ~45 min; a DEAD ADC throws AdcAuthError naming the exact // `gcloud auth application-default login` reauth command — surface the login, never silently skip // the lane). The wrapper sets the header LAST so it rides the request whatever headers the SDK // built; the injected test fetch (if any) sits UNDERNEATH the wrapper, so the offline fixtures // exercise the real bearer-attachment path (delete this attachment ⇒ the fixture goes red). const tokenSource = opts.adcTokenSource ?? gcloudAdcTokenSource({ ...(opts.adcExec !== undefined ? { exec: opts.adcExec } : {}) }); const baseFetch = opts.fetch ?? globalThis.fetch; const fetchWithAdcBearer = (async (input: Parameters<typeof globalThis.fetch>[0], init?: RequestInit) => { const token = await tokenSource.token(); const headers = new Headers(init?.headers); headers.set("authorization", `Bearer ${token}`); return baseFetch(input, { ...init, headers }); }) as typeof globalThis.fetch; const maas = createOpenAICompatible({ name: VERTEX_MAAS_PROVIDER_NAME, baseURL: vertexMaasBaseUrl(project, location), fetch: fetchWithAdcBearer, }); return maas(opts.modelId); } if (opts.provider === "fireworks") { // Fireworks — an OpenAI-compatible surface for the BIG OPEN weights (DeepSeek V3/R1, Llama-405B, // Qwen3-235B, Kimi K2). A CASH lane, but the ONLY hosted one that takes a **GBNF grammar** in // `response_format`, so an open model gets the same emission floor the local lane gets (bead // kestrel-y31). The grammar is attached by the decoding middleware in `buildModel`, not here. const apiKey = process.env.FIREWORKS_API_KEY; const fireworks = createOpenAICompatible({ name: FIREWORKS_PROVIDER_NAME, baseURL: FIREWORKS_BASE_URL, ...(apiKey !== undefined ? { apiKey } : {}), ...(opts.fetch !== undefined ? { fetch: opts.fetch } : {}), }); return fireworks(opts.modelId); } // Direct Anthropic path — apiKey defaults to ANTHROPIC_API_KEY from the environment. const anthropic = createAnthropic({}); return anthropic(opts.modelId); } /** * The lane's model, with its constrained-decoding constraint WRAPPED AROUND IT. The wrap (rather than a * per-call argument) is deliberate: the constraint then rides every call the model makes, so no call site * — present or future — can forget it and quietly hand back an unconstrained run that looks like a * judgment result (bead kestrel-y31). */ function buildModel(opts: AiSdkClientOptions): LanguageModel { // Fail-closed: never construct a client with no model — an empty/whitespace id is an unresolved // model, and silently proceeding would send a request that either errors on the wire or, worse, // gets a provider default, corrupting the run's model identity (kestrel-rul.1). The friendly-name → // inference-profile MAPPING lives in `./roster.ts` (resolveBedrockModelId); this is the last-line // structural guard so a mis-resolution can never reach the wire as a blank id. if (opts.modelId.trim() === "") { throw new Error(`empty model id for provider "${opts.provider}" — refusing to construct a client with no model (fail-closed, kestrel-rul.1).`); } const base = buildBaseModel(opts); const constraint = resolveDecoding(opts); if (constraint === undefined) return base; // The middleware is typed structurally in `./decoding.ts` (which imports NO provider package); bridge it // to the SDK's `LanguageModelMiddleware` here, in the provider-quarantine file (the ONE impure module). const middleware = decodingMiddleware(constraint) as unknown as LanguageModelMiddleware; return wrapLanguageModel({ model: base, middleware }); } /** * Construct the BYOK AI-SDK {@link LlmClient}. Retries, silent fallbacks, and middleware repair are * OFF by default (ADR-0013 (c) / m9i.1: the harness must never quietly turn one config into another). * Each `complete` returns the reply text, the raw reasoning trace (free evidence), provider-native * usage, and — unless disabled — the SCRUBBED wire evidence. */ export function aiSdkLlmClient(opts: AiSdkClientOptions): LlmClient { // vertex-maas: the ADC token source is hoisted OUT of buildModel so every token it ever issues can be // fed to scrubSecrets at capture time. An ADC bearer lives in NO env var, so the CREDENTIAL_ENV_VARS // scrub alone would never catch it — this is the same belt-and-suspenders that keeps every other // credential out of wire evidence (ADR-0013 (e)), extended to a credential that is minted per call. const adcSource = opts.provider === "vertex-maas" ? (opts.adcTokenSource ?? gcloudAdcTokenSource({ ...(opts.adcExec !== undefined ? { exec: opts.adcExec } : {}) })) : undefined; const model = buildModel(adcSource !== undefined ? { ...opts, adcTokenSource: adcSource } : opts); // The gateway lane floors its output ceiling at 16 — the Vercel AI Gateway 400s on `max_tokens<16` // (VERIFIED LIVE). Other lanes keep the caller's exact budget (or the 2048 default). const requestedMaxOutputTokens = opts.maxOutputTokens ?? 2048; // The gateway lane floors at 16 (the gateway 400s below that) AND, for a reasoning model, ADDS the // caller's reasoning headroom on top: over the OpenAI-compatible surface the thinking trace and the // visible reply share one `max_tokens` budget, so without headroom a small ceiling is burned entirely // on reasoning and the plan text returns empty (kestrel-a1f). Headroom is gateway-only and defaults to // 0, so every other lane — and any gateway caller that doesn't opt in — keeps its exact prior budget. const reasoningHeadroom = opts.provider === "gateway" ? (opts.reasoningHeadroomTokens ?? 0) : 0; const maxOutputTokens = opts.provider === "gateway" ? Math.max(GATEWAY_MIN_OUTPUT_TOKENS, requestedMaxOutputTokens) + reasoningHeadroom : requestedMaxOutputTokens; const captureWire = opts.captureWire ?? true; const secrets = secretsToScrub(opts.secrets); const generate: GenerateFn = opts.generate ?? (async ({ system, messages, temperature, maxOutputTokens: callMaxOutputTokens, maxRetries, providerOptions }) => { const r = await generateText({ model, // `system` is either a plain string or an {@link AiSdkSystemMessage} carrying the cache breakpoint; // the ai v7 top-level `system`/`instructions` option accepts both (`Instructions = string | // SystemModelMessage | SystemModelMessage[]`). The `providerOptions` record is looser than the SDK's // `ProviderOptions`, so it is bridged here in the provider-quarantine file (the ONE impure module). ...(system !== undefined ? { system: system as unknown as string } : {}), // AI-SDK `ModelMessage` is a role-discriminated union whose content type depends on the role; the // client's uniform `{ role, content: string, providerOptions? }` shape is a structural member of it, // bridged here in the provider-quarantine file (the ONE impure module) so the rest of the client // depends only on the narrow `AiSdkMessage` shape. messages: messages as unknown as ModelMessage[], ...(temperature !== undefined ? { temperature } : {}), // The per-CALL ceiling (raised above the thinking budget when thinking is enabled — see `complete`); // for a no-thinking request it equals the closure `maxOutputTokens`, so the wire is unchanged. maxOutputTokens: callMaxOutputTokens, maxRetries, // Top-level extended-thinking config, or ABSENT for `thinkingLevel:"none"` (byte-identical to today). // The record is looser than the SDK's `ProviderOptions`; bridged here in the provider-quarantine file. ...(providerOptions !== undefined ? { providerOptions: providerOptions as never } : {}), }); // The SDK's rich generic result is a structural superset of the slice this client reads; bridge it to // {@link GenerateResult} here in the provider-quarantine file (the ONE impure module) so the rest of the // client — and the injectable test seam — depend only on the narrow shape. return r as unknown as GenerateResult; }); return { async complete(req): Promise<LlmCompletion> { // Translate the harness's cache signals to provider-native controls (kestrel-rul). The growing // conversation rides `messages` (user/assistant only); a marked message carries the cache breakpoint. // The cached SYSTEM prefix rides the TOP-LEVEL `system`/`instructions` field as a SystemModelMessage // carrying the breakpoint — NOT a leading system-role message in `messages`, which the ai v7 core // rejects outright ("System messages are not allowed in the prompt or messages fields. Use the // instructions option instead"); that rejection is exactly what fell `conversation-cached` back to a // 100% provider-error run on live Bedrock (dry-run-1). Both the Bedrock (cachePoint) and Anthropic // (cacheControl) providers fold a system message's providerOptions into their top-level system field // with the cache point attached. When a provider has no breakpoint (Google: `bp === undefined`), the // system stays a plain string and no per-message providerOptions is added — Gemini caches implicitly, // and its cache-read tokens still surface through usage. Uncached policies also keep the plain string. // `req.cacheTtl` (kestrel-wa0j.1) rides every breakpoint this request marks; absent ⇒ the provider // default (byte-identical), and a TTL on a lane with no breakpoint primitive throws (fail-closed). const bp = cacheBreakpoint(opts.provider, req.cacheTtl); const convo: AiSdkMessage[] = req.messages.map((m) => ({ role: m.role, content: m.content, ...(m.cache === true && bp !== undefined ? { providerOptions: bp } : {}), })); const cacheSystem = req.cacheSystem === true; const system: string | AiSdkSystemMessage = cacheSystem && bp !== undefined ? { role: "system", content: req.system, providerOptions: bp } : req.system; // Wire the strategist/watcher reasoning-effort dial (`thinkingLevel`) to the provider-native // extended-thinking/effort config (the dead-knob fix). `none` ⇒ `thinking === undefined` ⇒ NO // `providerOptions.*` key and the ceiling is untouched, so the request is byte-identical to before // this knob was wired; only an explicit `low`+ opts in. The gpt/open lanes (openai/azure/fireworks) // ride the same seam as `reasoning_effort` (kestrel-0drf); only gateway/local/codex-cli get // `undefined` here (gateway uses `reasoningHeadroomTokens`; codex-cli maps the dial onto its CLI // flag in ./codex-cli-client.ts — see {@link thinkingProviderOptions}). const thinking = thinkingProviderOptions(opts.provider, req.sampling.t