@juspay/neurolink
Version:
The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002
57 lines (56 loc) • 2.95 kB
TypeScript
/**
* Relevance-driven context compaction.
*
* Every stage of the existing compactor is *positional*: prune protects the
* most recent 40K tokens, truncation drops the oldest half, summarization
* keeps a trailing ratio. None of them has any notion of what the current
* request is about, so a fifty-turn conversation that ends with "now rename
* that variable" keeps the forty turns about database migrations and drops
* nothing that matters less.
*
* A decision model changes the economics of asking. Latency is flat in
* question count — one question and four hundred questions cost the same
* round trip — so "is this message needed to answer the current request?" can
* be asked about *every* message at once, for about $0.00002.
*
* What it must never do is lose something the turn needed. The guards below
* are therefore deliberately conservative, and every one of them is a veto:
*
* - only plain user/assistant text is ever eligible; tool calls, tool
* results, summaries, pinned skills and system messages are untouchable,
* because dropping half of a tool-call pair produces a malformed request
* rather than a smaller one
* - the most recent messages are never eligible, whatever the model says
* - a message is dropped only on a CONFIDENT no, never on silence or a
* near-coin-flip
* - at most a bounded fraction of eligible messages may go in one pass
*
* @module context/contextDecision
*/
import type { ChatMessage, ContextRelevanceOptions, ContextRelevanceResult, DecisionCallerFn } from "../types/index.js";
/**
* Ask which earlier messages the current request still needs, and return the
* ones to drop.
*
* Returns `null` when nothing should change — no decision provider, a failed
* call, nothing eligible, or no confident drop. A null return is the signal
* to carry on with positional compaction exactly as before.
*/
export declare function selectIrrelevantMessages(messages: ChatMessage[], currentRequest: string, decide: DecisionCallerFn, options?: ContextRelevanceOptions): Promise<ContextRelevanceResult | null>;
/**
* Gate a generated summary before it replaces the messages it covers.
*
* Stage 3 accepts any non-empty string today, so a summarizer that returned
* an apology, a refusal, or a truncated fragment silently destroys the
* conversation it was meant to preserve — and the messages are gone by the
* time anyone reads the summary.
*
* Returns `true` when the summary may be used and `false` only on a confident
* rejection. Anything else — no decision provider, a failed call, an
* unanswered or uncertain verdict — returns `true`, because refusing a
* summary means falling through to truncation, which loses strictly more.
*/
export declare function summaryPreservesContext(summary: string, replacedMessages: ChatMessage[], decide: DecisionCallerFn, options?: {
timeoutMs?: number;
minConfidence?: number;
}): Promise<boolean>;