UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

57 lines (56 loc) • 2.95 kB
/** * Relevance-driven context compaction. * * Every stage of the existing compactor is *positional*: prune protects the * most recent 40K tokens, truncation drops the oldest half, summarization * keeps a trailing ratio. None of them has any notion of what the current * request is about, so a fifty-turn conversation that ends with "now rename * that variable" keeps the forty turns about database migrations and drops * nothing that matters less. * * A decision model changes the economics of asking. Latency is flat in * question count — one question and four hundred questions cost the same * round trip — so "is this message needed to answer the current request?" can * be asked about *every* message at once, for about $0.00002. * * What it must never do is lose something the turn needed. The guards below * are therefore deliberately conservative, and every one of them is a veto: * * - only plain user/assistant text is ever eligible; tool calls, tool * results, summaries, pinned skills and system messages are untouchable, * because dropping half of a tool-call pair produces a malformed request * rather than a smaller one * - the most recent messages are never eligible, whatever the model says * - a message is dropped only on a CONFIDENT no, never on silence or a * near-coin-flip * - at most a bounded fraction of eligible messages may go in one pass * * @module context/contextDecision */ import type { ChatMessage, ContextRelevanceOptions, ContextRelevanceResult, DecisionCallerFn } from "../types/index.js"; /** * Ask which earlier messages the current request still needs, and return the * ones to drop. * * Returns `null` when nothing should change — no decision provider, a failed * call, nothing eligible, or no confident drop. A null return is the signal * to carry on with positional compaction exactly as before. */ export declare function selectIrrelevantMessages(messages: ChatMessage[], currentRequest: string, decide: DecisionCallerFn, options?: ContextRelevanceOptions): Promise<ContextRelevanceResult | null>; /** * Gate a generated summary before it replaces the messages it covers. * * Stage 3 accepts any non-empty string today, so a summarizer that returned * an apology, a refusal, or a truncated fragment silently destroys the * conversation it was meant to preserve — and the messages are gone by the * time anyone reads the summary. * * Returns `true` when the summary may be used and `false` only on a confident * rejection. Anything else — no decision provider, a failed call, an * unanswered or uncertain verdict — returns `true`, because refusing a * summary means falling through to truncation, which loses strictly more. */ export declare function summaryPreservesContext(summary: string, replacedMessages: ChatMessage[], decide: DecisionCallerFn, options?: { timeoutMs?: number; minConfidence?: number; }): Promise<boolean>;