agentfootprint
Version:
The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.
68 lines • 4.18 kB
JavaScript
"use strict";
/**
* Strategy arms — the SUBSTITUTION tier of the counterfactual engine.
*
* Pattern: sibling spec type, shared statistics.
* Role: answer "scorer A versus scorer B", "topK versus a re-ranker",
* "sliding-window versus token-budget" with the SAME seeded-rerun
* discipline the ablation tier uses — and without pretending a
* substitution is a removal.
*
* ## Why this is NOT a fifth arm on `AblationSpec`
*
* Ablation is REMOVAL. `AblationSpec` names inputs an agent is BUILT from and
* `applyAblations` filters them out; the ablated agent is strictly smaller than
* the original. Substitution is REPLACEMENT: the agent that re-ranks instead of
* taking top-K is not smaller, it is different. Three concrete costs decided
* this, and each one is a place a shared union would have produced a
* confident-looking verdict for an incoherent comparison:
*
* 1. **`bisectCulprits` would accept it.** The union feeds a delta-debugging
* search (`bisect.ts`) whose premise is that probes are SUBSETS of removals:
* it halves the candidate set, probes each half, and returns a "minimal
* culprit set". Substitutions do not compose that way — swapping the scorer
* and swapping the window at once is a THIRD arm, not a bigger removal — and
* the answer it would print ("minimal culprit set = {scorer swap}") is the
* wrong tier of claim entirely: an arm is an alternative configuration, not
* a culprit. A type that lets an incoherent comparison reach a verdict is
* worse than the limitation it removes.
* 2. **`rerunWithoutSources` would report it under `removed`.** Its result
* type, its `removableSources` vocabulary and every sentence it composes say
* "removing X". A substitution arriving through its `ignore` escape hatch
* would be described, in a field NAME, as something it is not.
* 3. **`applyAblations` has nowhere to put it.** It returns filtered
* `{tools, injections, memoryEntries}`. A substitution is not a filter of
* those inputs; it is a construction choice made before they exist.
*
* The rejected third option — a `'strategy'` arm that genuinely ABLATES, i.e.
* falls back to the library default — was rejected because "no retrieval at
* all" and "re-rank instead of top-K" are different experiments and only the
* second is what a researcher means. The manifest already treats `memories: []`
* as an arm of its own; an arm that silently meant "the default" would answer a
* question nobody asked while looking like it supported the one they did.
*
* So: sibling TYPE, shared STATISTICS. `AblationRunStats`, `SimilarityStats`,
* `resolveSamples`, `similarityStats`, `costStatsFrom`, `probeFlipped` and
* `defaultOutcomeComparator` are reused verbatim; nothing in `ablation.ts`,
* `bisect.ts`, `rerun.ts` or `cost.ts` changed, so the four existing spec arms
* behave identically (pinned by test/context-bisect/ablation-union-pinned).
*
* ## What the library can and cannot DO with an arm
*
* For a removal the library can perform the intervention: `applyAblations`
* filters the inputs. For a substitution it cannot — construction is the
* consumer's code, and no library can reach into it and replace a scorer. What
* it can do instead is DECLARE the arm and then VERIFY it took effect, because
* 9.41.0 made configuration self-describing: `agentfootprint.agent.run_configured`
* names the provider, model, react mode, window, each memory's strategy /
* retrieval / embedder, the graph's posture and classifier, and the evidence
* posture. {@link ArmFacets} is deliberately that same vocabulary, so a run's
* own manifest — not the experimenter's bookkeeping — says which arm it belonged
* to, and an arm whose runs contradict its declaration gets no verdict at all.
*
* An arm may ALSO carry `ablations`: removals ride inside an arm through the
* unchanged removal machinery (`applyArm` → `applyAblations`). Composition, not
* union — each type still means exactly one thing.
*/
Object.defineProperty(exports, "__esModule", { value: true });
//# sourceMappingURL=types.js.map