UNPKG

agentfootprint

Version:

The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.

87 lines (86 loc) • 4.43 kB
/** * findings/judge — a calibrated SECOND SOURCE on the ledger (9.104.0). * * Pattern: a pure question builder (`judgeQuestions`) + one async writer * (`judgeResult`) that spends a `Classifier` call per tool result * and files what came back through `recordFindings` — the ONE * writer of `AgentState.findingsLedger` stays the one writer. * Role: core/ layer. The dispatch loop (`stages/toolCalls.ts`) calls * `judgeResult` right after a tool result is committed to * `toolResults`, and only under `.findings({ judge })`; an agent * with no judge never reaches this file. * * WHAT IS ASKED. Intent is first-person: the judge is never asked why the * model called a tool. It is asked what a RESULT is worth for the * PROPOSITION the model declared before the call (`BasisRow.proposition`), * or — when the call declared none — for the user's question; the row says * which (`against`). Two questions, ids fixed so a bench and a lens can read * them: `standing`, a choice over the four standings, and `tests_subject`, * the probability that the result tests the proposition / question at all * (the probe's `tests_proposition`, renamed because the subject may be the * question). * * WHAT IS WRITTEN. A `JudgmentRow` — source, provider model string, the * chosen standing, the whole distribution, the confidence, the noul, usage, * latency — or, when the call failed, a `JudgmentErrorRow` with the * provider's status and message and the latency spent. Never a guessed * standing. The model's own standing (`StandingRow`) is untouched: two * sources, two rows, and the disagreement is a fact of the record. * * WHAT IS SERVED. Nothing, in this packet. `serve.ts` reads * `foldLedger(...).standingOf` (the model's) alone — policy A on the design * page; the bench decides whether B–D are worth a change. */ import type { Classifier, ClassifyRequest } from '../../../classify/types.js'; import { type FindingsScope, type PreviousResult, type TurnStamp } from './ledger.js'; import { type BasisRow, type Standing } from './types.js'; /** The two question ids every judgment is asked under. */ export declare const JUDGE_QUESTION_IDS: { readonly standing: "standing"; readonly testsSubject: "tests_subject"; }; /** * The four standings as criteria for the `choice` question — the ledger's * own vocabulary (`findings/types.ts`), described to the judge in the * ledger's own terms so its `choice` is a `Standing` by construction. */ export declare const STANDING_CRITERIA: Readonly<Record<Standing, string>>; /** What the judge reads. `result` is the tool's text, clipped at `JUDGE_RESULT_CHARS`. */ export interface JudgeState { readonly proposition?: string; readonly predicts?: string; readonly question: string; readonly tool: string; readonly result: string; } /** The pure output: the request to send, what it judges against, and whether the result was cut. */ export interface JudgeQuestions { readonly request: ClassifyRequest; readonly against: 'proposition' | 'question'; readonly clipped: boolean; } /** * Build the classifier request for one landed result. Pure. `against` is * `'proposition'` exactly when the call's basis row carries one; otherwise * the user's question is the subject and the row will say so. The result * text is cut at `JUDGE_RESULT_CHARS` with the cut reported, never hidden. */ export declare function judgeQuestions(basis: BasisRow | undefined, userQuestion: string, result: { readonly toolName: string; readonly result: string; }): JudgeQuestions; /** The scope surface the judge needs: the ledger (to find the basis row) and the user's question. */ export interface JudgeScope extends FindingsScope { userMessage?: string; } /** * Ask the judge about ONE landed result and file the answer. Never throws: * a provider failure — a status, a network error, a malformed answer, an * abort — is a `JudgmentErrorRow` and the run continues, because the judge * is advisory and its absence must never cost the run its answer. */ export declare function judgeResult(scope: JudgeScope, judge: Classifier, entry: PreviousResult & { readonly toolName: string; }, iteration: number, signal?: AbortSignal, /** The honesty layers' turn stamp — passed only while an honesty layer is armed. */ stamp?: TurnStamp): Promise<void>;