@juspay/neurolink
Version:
The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002
40 lines (39 loc) • 2.21 kB
TypeScript
/**
* Decision limits — the one place a provider's `decisionLimits` are read and
* a state's size is estimated against them.
*
* `SystemOneDecisionProvider` refuses an over-limit request through these
* functions, and `NeuroLink.decisionLimits()` / `estimateDecisionStateTokens`
* expose the same functions to hosts, so what a host measures before the call
* is exactly what the pre-flight check measures inside it. Two estimators that
* drift apart would let a host size a state as fitting and then watch it be
* refused.
*
* @module utils/decisionLimits
*/
import type { DecisionLimits, DecisionLimitsReading, DecisionState, ProviderDescriptor } from "../types/index.js";
/**
* Estimate how many tokens a decision model will read from `state`.
*
* `estimateTokens` assumes ~4 characters per token. That holds for English
* and is several times too generous for other scripts on an English
* tokenizer, so when the limits declare a `nonAsciiTokensPerChar` rate,
* non-ASCII characters are counted at it instead. Digits, ASCII punctuation
* and astral characters (emoji) are each counted separately only when the
* limits declare a rate for that class — a tokenizer that reads each digit,
* and most punctuation, on its own makes a number-heavy or JSON-heavy state
* several times longer than the ~4-characters-per-token estimate suggests. A
* non-string state is serialized first, as it is on the wire.
*
* This is the estimator the pre-flight refusal uses: a state this reports at
* or under `maxStateTokens` is sent, one over it is refused before any
* network call.
*/
export declare function estimateDecisionStateTokens(state: DecisionState, limits?: Pick<DecisionLimits, "nonAsciiTokensPerChar" | "digitTokensPerChar" | "symbolTokensPerChar" | "astralTokensPerChar">): number;
/**
* Flatten a descriptor's `decisionLimits` for one model: the per-model entry
* overrides the base, field by field, and `advisory` becomes
* `enforcedLocally: false`. Returns null for a descriptor that declares no
* limits.
*/
export declare function resolveDecisionLimitsReading(descriptor: Pick<ProviderDescriptor, "name" | "decisionLimits">, model: string): DecisionLimitsReading | null;