agentfootprint
Version:
The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.
175 lines (174 loc) • 9.46 kB
TypeScript
/**
* gate — the check itself, and the sentences it says.
*
* Pattern: resolve-once/ask-many (the `ResolvedOutputEnforcement` shape) plus
* an authored frame around untrusted text (the `buildCorrectiveTurn`
* shape, and for the same reason).
* Role: core/ layer. This is the module a reader should start from.
* Emits: N/A — the Route decider and the recheck stage emit; this file only
* computes verdicts and builds strings.
*
* ## WHAT THIS IS
*
* Every number, identifier and name in the model's final answer must appear in
* a tool result the run can point at. If one does not, the model TYPED it
* rather than read it. That is the whole claim.
*
* ## WHAT IT IS NOT — read this before trusting it
*
* **It is a fabrication detector, not a correctness judge.** It catches values
* that came from nowhere. It cannot catch a FALSE CLAIM ASSEMBLED FROM REAL
* VALUES: "fc1/3 is healthy" when the data says the port is down uses entirely
* grounded tokens — `fc1/3` is in the evidence, "healthy" is a word — and this
* check passes it without a murmur. So will "the outage started at 08:15" when
* 08:15 is a timestamp from a different port. Anyone who reads this as a
* hallucination check will trust it for the thing it provably cannot do.
*
* **It cannot catch MIS-REFERRAL either — an answer that is true of the WRONG
* THING.** The gate judges VALUES against evidence, and every value in
* "that machine has no backup record" can be perfectly grounded while the
* machine the sentence is about was never the one the user asked about. That
* is a recorded failure, not a hypothetical: a model resolved "that machine"
* out of its own earlier prose, called the lookup tool with a truncated job
* name, and got a real "nothing found" back — which the gate then grounded,
* correctly, in a real tool result. The referent was bound wrong one seam
* earlier, at the ARGUMENT, and that is the defect the choice-seam check owns
* (`src/integrity/unsupported-argument`, armed by `Tool.argumentsFrom`). This
* check is not the place to fix it: by the time an answer exists, the wrong
* lookup has already been served as evidence.
*
* It is also deliberately incomplete in the other direction: the extractor is
* conservative (see `extract.ts`), so small numbers and all-letters names pass
* unexamined. A missed fabrication is a miss; a false accusation costs a real
* turn and can refuse a good answer, so the bias points the way it does.
*
* **And it does not ask WHEN.** A value read four turns ago for a different
* question is as grounded here as one read a second ago — which is a measured
* failure, not a hypothetical (see `../../../integrity/prior-turn-evidence`).
* That fact is reported beside this check, never folded into it: the gate's
* job is "was this read at all", and answering a second question through the
* same verdict would make one dial control two decisions.
*
* ## Why the check is DETERMINISTIC
*
* No model call, no embedding, no judge. The library's thesis is that
* structure lets a smaller model perform like a bigger one — so a guard that
* needed a BIGGER model to police the small one would invert the whole value
* proposition, and would fail exactly where the small model is deployed
* (offline, cheap, fast). Set membership over normalized tokens is the entire
* mechanism, it costs microseconds, and it is the same on every run.
*/
import { MAX_REPORTED_VALUES } from './limits.js';
import type { StagedRefsMatch } from '../stagedRefs.js';
import type { EvidenceCorpus } from './evidenceIndex.js';
import { EVIDENCE_CHECK_FRAME_PREFIX } from './frames.js';
import type { FigureCheck } from './figures.js';
import type { EvidencePosture, EvidenceVerdict, NamesAndNumbersOptions, ResolvedEvidenceGate, UnsupportedValue } from './types.js';
export { EVIDENCE_CHECK_FRAME_PREFIX };
export { MAX_REPORTED_VALUES };
/** Clip a value for display without letting it pretend to be complete — the
* spelling every verdict reports a value under (`UnsupportedValue.value`). */
export declare function clip(v: string): string;
/**
* Validate the caller's options once, at build time, into the config the chart
* carries. Refusals name the option and the fix — nothing here is discovered
* at run time.
*/
export declare function resolveEvidenceGate(opts?: NamesAndNumbersOptions): ResolvedEvidenceGate;
/**
* Judge one answer.
*
* `exempt` is checked BEFORE the evidence: a value the user supplied is not a
* fabrication whether or not a tool ever echoed it back.
*
* The same pass reports WHEN each grounded value was read (`grounding`) — the
* time axis the corpus gained in 9.83.0. Three buckets, and an exempt value is
* in none of them: exemption is a statement about who SUPPLIED a value, and a
* value the app or the person put in front of the model was never something
* the run had to go and look up. Counting one as tool evidence — of any turn —
* would answer a question nobody asked.
*/
export declare function checkAnswer(answer: string, args: {
readonly gate: ResolvedEvidenceGate;
readonly evidence: EvidenceCorpus;
readonly exempt: ReadonlySet<string>;
/**
* Under `.time()` (step T7): the lookup forms of the time spellings the
* library DERIVED from this turn's readings (`evidenceIndex.ts` ·
* `derivedFormsOf`). A value no tool result carried that is found here
* lands in `derived`, not `unsupported`. Absent → the verdict it always was.
*/
readonly derived?: ReadonlySet<string>;
/**
* Under the figures dial (`figures.ts` · `figureCheckOf`): the numbers the
* answer writes wearing a unit, and the derivation question over the
* numbers the results carry. A NUMBER no result carried that is a
* declared derivation of them lands in `computed`, not `unsupported`.
* Absent → the verdict it always was.
*/
readonly figures?: FigureCheck;
/**
* The answer is TYPED (`.outputSchema()`): read by its values, its object
* keys — the schema's field names — never judged (`extract.ts` ·
* `AnswerReadOptions`). Absent → keys are read too.
*/
readonly typed?: boolean;
}): EvidenceVerdict;
/** Render the flagged values for a human or a model: `` `x` (shape) ``. */
export declare function describeValues(values: readonly UnsupportedValue[]): string;
/**
* The two messages a flagged answer adds to the conversation: the answer
* itself, then the correction.
*
* The failed answer goes back in for the reason the schema retry puts it back:
* nothing else writes an answering turn into `history`, so a correction sent
* alone would arrive at a model that cannot see what it said.
*
* The frame is AUTHORED and comes first; the quoted values come last and
* nothing is written after them. They are the model's own tokens rather than a
* third party's, so the risk is small — but the rule that the library's words
* come first and untrusted text never gets the last line is the same rule the
* compaction frame and the schema frame follow, and a rule with an exception
* is not a rule.
*
* ## The scope this sentence used to claim, and no longer does (9.83.0)
*
* Both this frame and {@link evidenceRefusalSentence} said the flagged values
* "appear in NO tool result FROM THIS TURN". The index they are written from
* has never been turn-scoped — it walks every `role: 'tool'` turn in
* `scope.history` — so the sentence named a boundary the check did not honour,
* in the two places the claim is read by a model and by an operator. It was
* also needlessly weak: a flagged value appears in no tool result the run can
* point at AT ALL, which is a stronger and true thing to say. So the words
* changed to the check's real reach. The recency fact the old wording gestured
* at is now reported where it can be measured — `prior-turn-evidence`, off by
* default — instead of asserted here where it could not be.
*
* `stagedRefs` (optional) is the staged-refs join computed by the recheck
* stage: when this turn holds placed artifact tickets a served `wants` tool
* can spend, the frame names the refs and the spender by their declared names
* — "call the tool that provides it" without saying WHICH tool over WHICH ref
* leaves the model to head-math again. The clause sits INSIDE the authored
* frame (ref ids and tool names are declarations, not model output), so the
* quoted values still come last and `isLibraryAuthoredTurn`'s prefix match is
* untouched.
*/
export declare function buildEvidenceCorrection(failedAnswer: string, values: readonly UnsupportedValue[], stagedRefs?: StagedRefsMatch): readonly [{
role: 'assistant';
content: string;
}, {
role: 'user';
content: string;
}];
/**
* The refusal sentence `rails` hands the caller, and the warning `assist`
* prints. Names the values and says what would satisfy the check — a refusal
* that does not teach is just a failure.
*
* The values are the model's own words, so naming them leaks nothing the
* caller was not about to be handed anyway.
*/
export declare function evidenceRefusalSentence(values: readonly UnsupportedValue[], posture: EvidencePosture, revised: boolean,
/** The figures dial was on: a flagged NUMBER was also asked for a
* derivation (`figures.ts`) and matched none — say so. */
figures?: boolean): string;