@juspay/neurolink
Version:
The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002
36 lines (35 loc) • 1.67 kB
TypeScript
/**
* Context Budget Checker
*
* Pre-generation validation that estimates total input token cost
* and compares against the model's available input space.
*
* This runs BEFORE every LLM call to prevent context overflow.
*/
import type { BudgetCheckResult, BudgetCheckParams } from "../types/index.js";
/** Default compaction threshold (80% of available input) */
export declare const DEFAULT_COMPACTION_THRESHOLD = 0.8;
/**
* Tokens the CONVERSATION HISTORY may occupy, i.e. the model's available input
* space minus everything that rides alongside it (system prompt, current
* prompt, tool definitions, file attachments).
*
* This is the number the compactor must target. Passing it the undeducted
* `availableInputTokens` made every stage gate compare history-only tokens
* against the WHOLE budget, so compaction only engaged once history alone
* exceeded the entire window — with a large MCP tool set a request could sit
* far over budget while the compactor reported "nothing to do" and fell through
* to emergency truncation.
*
* Returns 0 when the fixed overhead already exceeds the window; callers must
* treat that as unrecoverable rather than compacting to an empty history.
*/
export declare function resolveHistoryBudget(result: BudgetCheckResult, compactionThreshold?: number): number;
/**
* Check whether a request fits within the model's context budget.
*
* Estimates total input tokens from: system prompt + tool definitions +
* conversation history + current prompt + file attachments, and compares
* against available input space.
*/
export declare function checkContextBudget(params: BudgetCheckParams): BudgetCheckResult;