UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

36 lines (35 loc) • 1.67 kB
/** * Context Budget Checker * * Pre-generation validation that estimates total input token cost * and compares against the model's available input space. * * This runs BEFORE every LLM call to prevent context overflow. */ import type { BudgetCheckResult, BudgetCheckParams } from "../types/index.js"; /** Default compaction threshold (80% of available input) */ export declare const DEFAULT_COMPACTION_THRESHOLD = 0.8; /** * Tokens the CONVERSATION HISTORY may occupy, i.e. the model's available input * space minus everything that rides alongside it (system prompt, current * prompt, tool definitions, file attachments). * * This is the number the compactor must target. Passing it the undeducted * `availableInputTokens` made every stage gate compare history-only tokens * against the WHOLE budget, so compaction only engaged once history alone * exceeded the entire window — with a large MCP tool set a request could sit * far over budget while the compactor reported "nothing to do" and fell through * to emergency truncation. * * Returns 0 when the fixed overhead already exceeds the window; callers must * treat that as unrecoverable rather than compacting to an empty history. */ export declare function resolveHistoryBudget(result: BudgetCheckResult, compactionThreshold?: number): number; /** * Check whether a request fits within the model's context budget. * * Estimates total input tokens from: system prompt + tool definitions + * conversation history + current prompt + file attachments, and compares * against available input space. */ export declare function checkContextBudget(params: BudgetCheckParams): BudgetCheckResult;