agentfootprint
Version:
The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.
28 lines • 1.51 kB
TypeScript
/**
* `compareStrategyArms` — the substitution counterfactual as ONE call.
*
* Take two or more named configurations, run each of them N seeded times
* through the consumer's factory, and say — honestly — which ones answer
* differently from the incumbent. The product loop over `probe.ts` +
* `verdict.ts`, the way `rerunWithoutSources` is the product loop over the
* ablation machinery.
*
* ## The order of operations, and why it is this order
*
* 1. VALIDATE first (`validate.ts`): every refusal that can be known from the
* declaration fires before a single model call.
* 2. RUN the baseline arm. Its answers are needed before the reference can be
* chosen, because a caller with no prior run gets the baseline arm's own
* seed-0 answer as the reference.
* 3. SCORE the baseline against the reference. This one probe does three jobs
* at once and pays for none of them twice: the STABILITY gate (zero flips),
* the NULL BAND (its similarity spread), and the baseline's own row in the
* readout.
* 4. RUN + SCORE each challenger, check its manifest, issue its verdict.
*
* Cost is exactly `samples × arms` runner calls, reported as `runsUsed`. There
* is no hidden extra probe: the incumbent is one of the arms, not a surcharge.
*/
import type { CompareStrategyArmsOptions, StrategyComparison } from './types.js';
export declare function compareStrategyArms(options: CompareStrategyArmsOptions): Promise<StrategyComparison>;
//# sourceMappingURL=compare.d.ts.map