UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

40 lines (39 loc) • 2.21 kB
/** * Decision limits — the one place a provider's `decisionLimits` are read and * a state's size is estimated against them. * * `SystemOneDecisionProvider` refuses an over-limit request through these * functions, and `NeuroLink.decisionLimits()` / `estimateDecisionStateTokens` * expose the same functions to hosts, so what a host measures before the call * is exactly what the pre-flight check measures inside it. Two estimators that * drift apart would let a host size a state as fitting and then watch it be * refused. * * @module utils/decisionLimits */ import type { DecisionLimits, DecisionLimitsReading, DecisionState, ProviderDescriptor } from "../types/index.js"; /** * Estimate how many tokens a decision model will read from `state`. * * `estimateTokens` assumes ~4 characters per token. That holds for English * and is several times too generous for other scripts on an English * tokenizer, so when the limits declare a `nonAsciiTokensPerChar` rate, * non-ASCII characters are counted at it instead. Digits, ASCII punctuation * and astral characters (emoji) are each counted separately only when the * limits declare a rate for that class — a tokenizer that reads each digit, * and most punctuation, on its own makes a number-heavy or JSON-heavy state * several times longer than the ~4-characters-per-token estimate suggests. A * non-string state is serialized first, as it is on the wire. * * This is the estimator the pre-flight refusal uses: a state this reports at * or under `maxStateTokens` is sent, one over it is refused before any * network call. */ export declare function estimateDecisionStateTokens(state: DecisionState, limits?: Pick<DecisionLimits, "nonAsciiTokensPerChar" | "digitTokensPerChar" | "symbolTokensPerChar" | "astralTokensPerChar">): number; /** * Flatten a descriptor's `decisionLimits` for one model: the per-model entry * overrides the base, field by field, and `advisory` becomes * `enforcedLocally: false`. Returns null for a descriptor that declares no * limits. */ export declare function resolveDecisionLimitsReading(descriptor: Pick<ProviderDescriptor, "name" | "decisionLimits">, model: string): DecisionLimitsReading | null;