UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

39 lines (38 loc) • 2.11 kB
/** * Request-scoped peer-sharing context and the counters the rate gates read. * * The grant serving a borrowed request has to be visible deep inside the routing * path — account selection filters on it, and settlement bills against it — but * threading it through every call site of a 9,000-line route module would touch * everything and be wrong the first time someone added a code path. An * `AsyncLocalStorage` scope is the narrower change: the gate establishes it once * per request, and anything downstream in the same async chain (including the * `.then()` that settles a finished stream) can ask for it. * * Requests without a grant — the node's own client on loopback — run with no * context at all, so `getShareContext()` returning `undefined` is the normal, * hot case and means "this is my own traffic". * * @module proxy/shareContext */ import type { ProxyShareRequestContext } from "../types/index.js"; /** Read the counters a policy evaluation needs, without mutating them. */ export declare function readShareCounters(grantId: string, now?: number): { requestsInLastMinute: number; inFlight: number; }; /** * Claim a concurrency slot and record the request against the rate window. * Returns the release function; it is idempotent because a request can end more * than one way (normal return, thrown error, client disconnect) and double * release would let a grant exceed its concurrency ceiling. */ export declare function acquireShareSlot(grantId: string, now?: number): () => void; /** Run `fn` with the grant visible to everything downstream. */ export declare function runWithShareContext<T>(context: ProxyShareRequestContext, fn: () => T): T; /** The grant serving the current request, or undefined for the node's own traffic. */ export declare function getShareContext(): ProxyShareRequestContext | undefined; /** True when the current request is borrowed rather than the node's own. */ export declare function isBorrowedRequest(): boolean; /** Drop all counters. Test isolation only. */ export declare function resetShareCountersForTests(): void;