agentfootprint
Version:
The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.
350 lines • 18.6 kB
TypeScript
/**
* auditExport — tamper-evident audit bundle (backlog #20, compliance
* wedge item 2; pairs with #19's `otelObservability` GenAI spans).
*
* Consumes the typed `agentfootprint.*` event stream and accumulates an
* append-only, HASH-CHAINED record log: every record carries the SHA-256
* of its own canonical serialization plus the hash of the previous
* record. Flipping a single byte anywhere in an exported bundle makes
* `verifyAuditBundle` name the exact record that broke. The log serves
* the automatic event recording EU AI Act Art. 12 asks of high-risk
* systems (the events the system logged, in order). The tamper evidence
* — showing the log is unmodified since capture — is this library's
* addition: Art. 12 does not require it.
*
* Pattern: Observability strategy (one purpose — chain accumulation)
* + pure offline verifier.
* Role: Outer ring (Hexagonal). Attach via
* `agent.enable.observability({ strategy: auditExport() })`.
* Emits: nothing — terminal sink.
*
* ## What lands in the chain
*
* One record per typed event, in dispatch order: decisions
* (`agent.route_decided`, `composition.route_decided` incl. decide()
* evidence), tool calls (`stream.tool_start/_end`), validation
* rejections (#9), permission verdicts and halts, credential lifecycle,
* costs, errors, skill/memory/context activity. Each new `meta.runId`
* is anchored by a GENESIS record (`audit.genesis`) carrying the runId,
* the agent identity, and library versions — runs chain back-to-back in
* one log, so silently DROPPING a whole run breaks the chain too.
*
* A call a paused batch never dispatched (9.113.0) lands as its two
* brackets, each carrying `notDispatched: { pausedCall: { toolCallId,
* toolName } }` and neither carrying `error` — identifiers only. Bounded mode
* checks each field by its declared shape; verbatim mode sanitizes the payload.
* The chained record therefore says the call never ran — neither that it
* succeeded nor that it failed; the bounded `result` (`[type: string]`) never
* had to say it.
*
* High-volume content deltas (`stream.token`, `stream.thinking_delta`)
* are excluded by default (`includeTokenEvents: true` to include).
*
* ## Record / bundle schema
*
* ```
* AuditRecord = { seq, timestamp, eventType, payload, meta, prevHash, hash }
* hash = SHA-256 hex over canonicalJson(record minus `hash`)
* prevHash = previous record's `hash` (ZERO_HASH at chain start)
* AuditBundle = { header, records, finalHash }
* header = { format, hashAlgorithm, canonicalization, chainHead,
* firstSeq, recordCount, exportedAt, library }
* ```
*
* Canonicalization is `afp-cjson/1` (see `lib/canonicalJson.ts` — those
* rules ARE the contract; the header names them so independent
* verifiers can re-implement byte-exactly).
*
* ## Persistence + long runs
*
* Persistence is the CONSUMER's job — the bundle is plain JSON
* (`JSON.stringify(strategy.bundle())`, store anywhere). For long runs,
* `drain()` returns the records accumulated since the last drain while
* keeping the chain intact ACROSS drains: each segment's
* `header.chainHead` equals the previous segment's `finalHash`, so
* `verifyAuditBundle([seg1, seg2, ...])` re-verifies the concatenation
* end-to-end.
*
* ## PII discipline (mirrors #19's otelObservability)
*
* Payloads enter records through a bounding layer: every registered event
* has a shape that names every field of its payload (`auditShapes.ts`, total
* over the event registry and exhaustive by type, built from the combinators
* in `auditBounding.ts`). In the default `payloadMode: 'bounded'` a record
* payload keeps the run's STRUCTURE and never its CONTENT — every string or
* value the run composed from what it read or produced, whoever wrote it —
* except what is named below under "What a bounded record still carries".
* Bounding is CLOSED by default: a field a shape does not name, a value that
* is not the declared shape, and every field of an event type the registry
* does not declare (a consumer's own `emit`), is a marker. Some of what that
* bounds:
*
* - tool args → `'[keys: …]'` (top-level key NAMES only);
* `tool_end`'s `changedArgKeys` is names already
* - tool results → `'[type: …]'` (typeof only) — `result` and,
* when a rule changed it, `modelResult` alike
* - an attached script's program (`turn_end.scripts.shown[].text`)
* → `'[N chars]'`; checked labels, ids, counts and
* rendering-boundary flags retain their shapes
* - an evaluation's `explanation` → `'[N chars]'` (a judge's reason can
* quote the answer it graded)
* - userPrompt / LLM content / thinking blocks / history
* → `'[N chars]'` / `'[N messages]'` markers
* - content PREVIEWS (`contentSummary` on context/memory events,
* `rawContent`, `resultSummary`, `droppedSummaries`) → markers
* (for short content a preview IS the content; `contentHash`
* stays — it links identical content without echoing it)
* - error MESSAGE strings (`error`, `errorMessage`, `lastError`,
* `rawOutput`) → `'[N chars]'` (messages can echo values)
* - free-form Records (`questionPayload`, `resumeInput`, risk/eval
* `evidence`, memory `scoreEvidence`) → `'[keys: …]'`
* - a typed answer's limits and negatives on `turn_end` (`answerCoverage`,
* `answerNegatives`) → the same shape, every sentence and value a
* `'[N chars]'` marker: a coverage item's `what` / `why` / `short`, a
* declared period's instants, an in-progress item's prose, an assumed
* `value`, a negative's `statement` and `about`. The tool and argument
* names, call ids, counts, kinds, codes and verdicts stay; a field the
* shape does not name is a marker too, so one added later is bounded
* until it is classified
* - every other event whose content sits beside structure keeps its
* shape, each content leaf a marker: a tool's absence, coverage
* and described-result words and data (`tools.absent`,
* `tools.coverage_declared`, `tools.semantics_declared`), the evidence
* gate's values, an external ground's value, the message text a routing
* match quotes (`map.engaged`, `map.parked`, `skill.turn_routed`,
* `context.evaluated`), a validation issue's quoted value, parser and
* fallback messages and previews, a tool's progress payload and a
* check-in's evidence pack, a person's note, a middleware's reason or
* question, the model's own words (`skill.step_skipped`, `agent.handoff`),
* the opt-in assembled system prompt, a retrieved passage's heading, an
* integrity finding's witnessed values and message, a tool effect's
* reason, a pause's own reason, an artifact's label and refusal detail, a
* fail-fast reason, a check's `rationale` and `tellLLM`, a fallback's or a
* credential failure's message, and a router's own reason
*
* What the shapes keep is checked structure: a number, boolean or string
* must arrive as that kind, and a closed enum must be one of its declared
* words. Every container is classified field by field, even if its declared
* type contains only counts. Unknown children and wrong kinds are markers.
* Strings are capped at 256 characters and lists at 32 items. This retains
* ids, names, kinds, codes, telemetry tags, verdicts, counts, library times
* and the words the app declared up front (labels, descriptions, step notes,
* hints, display names, rule ids). The type walker and runtime damage sweep
* check both halves; `auditKeptByContract.ts` names the exceptions below.
*
* What a bounded record still carries, by contract:
*
* - each record's `meta`: run and stage ids, paths, turn and iteration
* indices and — when the caller named them — `sessionId`, `principal`
* and `tenant`, the who of an audit record;
* - decision evidence as the engine summarized it: `decide()` evidence on
* `composition.route_decided` and a skill guard's conditions on
* `context.evaluated` — rule labels, operators, thresholds and a summary
* of each value a rule read. It is redaction-aware (under an agent's
* `redact` a selected key is the placeholder); otherwise the summary
* shows the value;
* - a policy's telemetry `reason` tag, and the reason word an app attaches
* to a lifecycle event it emits itself (`skill.deactivated`,
* `tools.offered`, `permission.gate_closed`): the app's own words, as
* written — an app that writes a run's value into one carries it;
* - a `tryInsteadTool` name, only once this strategy has seen that tool run.
*
* Sanitizing caps a string's length; it does not hide what the string says.
*
* `payloadMode: 'verbatim'` embeds full payloads (still
* JSON-sanitized). For Art. 12 completeness on an access-controlled
* store that is often the point — but the bundle then carries prompts,
* tool args/results and model output. Treat it as PII-bearing. An agent
* built with `Agent.create({ redact })` serves every event before any
* strategy sees it, so this one receives — and chains — the placeholder
* wherever the policy selected a value (a selected key at any depth of a
* payload; a whole payload whose event NAME `emitPatterns` selects). The
* policy selects by name, never by content: with no policy, or under a name
* it does not select, the value arrives as it was.
*
* ## Tamper-EVIDENT, not tamper-PROOF (honest threat model)
*
* The chain proves INTERNAL consistency: any partial modification —
* edit, insert, delete, reorder, drop-a-run — is detected and named.
* It does NOT prove provenance: an adversary holding the only copy can
* recompute every hash from the mutation onward and present a
* self-consistent forgery. For non-repudiation, anchor `finalHash`
* externally as part of your retention process (write-once/WORM store,
* signed log, RFC 3161 timestamping, or simply a second party) — then
* a whole-suffix recomputation no longer matches the anchor.
*
* ## Runtime requirements
*
* Hashing uses `node:crypto` (`createHash('sha256')`) — zero new
* dependencies, imported lazily at first use (same gating as the
* optional vendor SDKs in this folder, so merely importing this module
* stays browser-safe). `auditExport` and `verifyAuditBundle` therefore
* run anywhere `node:crypto` exists: Node ≥ 20, Bun, Deno,
* edge runtimes with Node compat (e.g. Cloudflare `nodejs_compat`).
* In a browser there is no SYNC SHA-256 (WebCrypto is async-only), so
* both throw a descriptive error — verify server-side, or re-implement
* verification from the documented contract (it is pure: recompute
* SHA-256 over `afp-cjson/1` canonicalization and walk the chain).
*
* @example Capture → export → verify
* ```ts
* import { auditExport, verifyAuditBundle } from 'agentfootprint/observe';
*
* const audit = auditExport({ agent: 'loan-officer' });
* const stop = agent.enable.observability({ strategy: audit });
* await agent.run({ message: 'assess application A-17' });
* stop();
*
* const bundle = audit.bundle(); // JSON-serializable
* await fs.writeFile('run.audit.json', JSON.stringify(bundle));
*
* const check = verifyAuditBundle(bundle); // offline — no agent needed
* // check.valid === true; tamper with one byte → { valid: false, brokenAt: <seq> }
* ```
*/
import { CANONICAL_JSON_VERSION } from '../../lib/canonicalJson.js';
import type { ObservabilityStrategy } from '../../strategies/types.js';
import { type BoundRule } from './auditBounding.js';
/** SHA-256 of "nothing" — the `prevHash` of the first record in a
* chain and the `chainHead` of a chain's first segment. */
export declare const AUDIT_ZERO_HASH: string;
/** `eventType` of the per-run genesis record. Deliberately OUTSIDE the
* `agentfootprint.*` registry namespace — it is a chain-level record,
* not a dispatched event (#20 ships zero new typed events). */
export declare const AUDIT_GENESIS_EVENT_TYPE = "audit.genesis";
/** Format identifier carried on every bundle header. */
export declare const AUDIT_BUNDLE_FORMAT = "agentfootprint.audit/1";
/**
* One link of the hash chain.
*
* `hash` = SHA-256 hex over `canonicalJson` of the record WITHOUT the
* `hash` field (i.e. `{ seq, timestamp, eventType, payload, meta,
* prevHash }` — canonical key order makes field order irrelevant).
* Because the preimage is "everything but `hash`", ADDING a field to a
* record is detected exactly like mutating one.
*/
export interface AuditRecord {
/** 0-based position in the chain (monotonic across drains). */
readonly seq: number;
/** Wall-clock ms of the source event (`meta.wallClockMs`). */
readonly timestamp: number;
/** Registry event name verbatim, or {@link AUDIT_GENESIS_EVENT_TYPE}. */
readonly eventType: string;
/** Bounded / sanitized event payload (see module PII docs). */
readonly payload: unknown;
/** Sanitized event meta (runId, runtimeStageId, paths, indices). */
readonly meta: {
readonly runId: string;
} & Readonly<Record<string, unknown>>;
/** `hash` of the previous record ({@link AUDIT_ZERO_HASH} at chain start). */
readonly prevHash: string;
/** SHA-256 hex of this record's canonical preimage. */
readonly hash: string;
}
export interface AuditBundleHeader {
readonly format: typeof AUDIT_BUNDLE_FORMAT;
readonly hashAlgorithm: 'sha-256';
readonly canonicalization: typeof CANONICAL_JSON_VERSION;
/** `prevHash` of `records[0]` — {@link AUDIT_ZERO_HASH} for the first
* segment, the previous segment's `finalHash` after a `drain()`. */
readonly chainHead: string;
/** `seq` of `records[0]` (continues across drains). */
readonly firstSeq: number;
readonly recordCount: number;
/** Wall-clock ms when `bundle()` / `drain()` produced this export. */
readonly exportedAt: number;
readonly library: {
readonly name: 'agentfootprint';
readonly version: string;
};
}
/** JSON-serializable export of the chain (or one drained segment). */
export interface AuditBundle {
readonly header: AuditBundleHeader;
readonly records: readonly AuditRecord[];
/** `hash` of the last record (= `chainHead` when `records` is empty).
* The next drained segment's `chainHead` equals this value. */
readonly finalHash: string;
}
export interface AuditVerifyResult {
readonly valid: boolean;
/** Records whose hashes were recomputed and matched. */
readonly recordsChecked: number;
/** `seq` of the first record that fails (or the expected seq at the
* failure point, when the stored seq itself was tampered). */
readonly brokenAt?: number;
/** Human-readable cause — names the failed check. */
readonly reason?: string;
}
export interface AuditExportOptions {
/** Agent identity recorded in every run's genesis record (service /
* agent name as your compliance review knows it). */
readonly agent?: string;
/**
* `'bounded'` (default) — payloads pass the PII bounding layer (see
* module docs). `'verbatim'` — full payloads, JSON-sanitized only.
*
* @remarks Verbatim bundles carry prompts, tool args/results and
* model output. Treat the store as PII-bearing; an agent applies no
* redaction policy unless built with `Agent.create({ redact })`, which
* serves every event (and so this bundle) through it.
*/
readonly payloadMode?: 'bounded' | 'verbatim';
/** Include `stream.token` / `stream.thinking_delta` events (high
* volume; content still bounded under `payloadMode: 'bounded'`).
* Default `false`. */
readonly includeTokenEvents?: boolean;
/** Extra version pins for the genesis record (your app, model
* config revision, policy bundle hash, …). */
readonly versions?: Readonly<Record<string, string>>;
}
/** The strategy returned by {@link auditExport}. */
export interface AuditExportStrategy extends ObservabilityStrategy {
/** Snapshot the retained records WITHOUT draining. Safe mid-run. */
bundle(): AuditBundle;
/** Return the records accumulated since the last drain and clear
* them from memory. Chain state persists — consecutive drained
* segments re-verify end-to-end via `verifyAuditBundle([...])`. */
drain(): AuditBundle;
/** Records currently retained (since last drain). */
recordCount(): number;
}
/**
* The content field names the bounded mode replaces, across every event type.
* Pinned against the agent's redaction vocabulary
* (`redaction/conversation.ts` · `conversationRedaction`): a field this table
* calls content is a name the conversation travels under, and the test fails
* when the two disagree.
*
* @internal
*/
export declare function boundedContentFieldNames(): readonly string[];
/**
* The rules `BOUND_FIELDS` declares, by event type and field — what the pin
* holds every event's shape to.
*
* @internal
*/
export declare function boundedFieldRules(): Readonly<Record<string, Readonly<Record<string, BoundRule>>>>;
/** The bounded content leaves, pinned against the redaction vocabulary's value-kind rule.
* @internal */
export declare function boundedContentLeaves(): readonly string[];
export declare function auditExport(opts?: AuditExportOptions): AuditExportStrategy;
/**
* Recompute the hash chain of a bundle (or of consecutive drained
* segments, in order) and report the exact record where integrity
* breaks. Pure function over JSON data — runs offline, long after the
* run, with no agent and no strategy instance.
*
* Checks, in order, per segment:
* 1. header format / algorithm / canonicalization are supported
* 2. `recordCount` matches `records.length`
* 3. segment continuity (`chainHead`/`firstSeq` extend the previous
* segment's `finalHash`/seq range)
* 4. per record: `seq` is contiguous, `prevHash` links the previous
* record, and SHA-256 over the canonical preimage (the record
* minus `hash` — so ADDED fields are caught too) matches `hash`
* 5. `finalHash` equals the last record's hash
*/
export declare function verifyAuditBundle(input: AuditBundle | readonly AuditBundle[]): AuditVerifyResult;
//# sourceMappingURL=audit.d.ts.map