UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

83 lines (82 loc) • 4.2 kB
/** * Anthropic rate-limit / quota header capture. * * Anthropic returns limit state on the response headers of every request: * `anthropic-ratelimit-unified-*` for subscription (OAuth) accounts, * `anthropic-ratelimit-{requests,tokens}-*` for API-key accounts. The NeuroLink * Claude proxy forwards those verbatim and adds `x-neurolink-*` for what only * it knows (which account served the request, pool headroom, whether the * numbers are live or a carried-over snapshot). * * None of it used to reach the SDK: `doGenerate` returned a hardcoded empty * header bag and the streaming loop never looked. The capture point here is the * `fetch` the Anthropic SDK is constructed with — it is invoked exactly once * per HTTP request on BOTH the streaming and non-streaming paths, so a single * wrapper covers everything without touching the SSE loop or switching the * non-streaming call to `.withResponse()`. * * Scoping is per-request via AsyncLocalStorage rather than a field on the * provider: a provider instance is shared across concurrent calls, so an * instance field would race and attribute one request's limits to another. * * @module providers/anthropic/rateLimitCapture */ import type { AnthropicRateLimitInfo, ClaudeLimitSnapshot } from "../../types/index.js"; /** * Parse both Anthropic rate-limit header families into a single shape. * * Which family is present depends on the account type, so every field is * optional and absence is normal rather than an error. */ export declare function parseAnthropicLimitHeaders(headers: Headers): AnthropicRateLimitInfo; /** * Build a snapshot from a response, or undefined when the response carries * neither Anthropic rate-limit headers nor NeuroLink proxy metadata. */ export declare function buildLimitSnapshot(headers: Headers, status: number, now?: number): ClaudeLimitSnapshot | undefined; /** * Wrap a fetch so every response's limit headers are captured into the * enclosing `withLimitCapture` scope. A no-op outside such a scope. * * Capture never alters the response and never throws — a parsing failure must * not be able to break a request that the provider would otherwise complete. */ export declare function wrapFetchWithLimitCapture(inner: typeof fetch): typeof fetch; /** * Run `body` in a capture scope and return its result alongside whatever limit * snapshot the underlying HTTP request(s) produced. */ export declare function withLimitCapture<T>(body: () => Promise<T>): Promise<{ result: T; snapshot?: ClaudeLimitSnapshot; }>; /** * Current scope's snapshot, if any. Lets a long-running loop (the streaming * path) read limits mid-flight without unwinding the scope. */ export declare function getCapturedLimitSnapshot(): ClaudeLimitSnapshot | undefined; /** Raw headers of the most recent captured response in this scope. */ export declare function getCapturedResponseHeaders(): Record<string, string> | undefined; /** Enter a capture scope without wrapping a single call — for streaming, where * the scope must outlive the function that opened it. */ export declare function runInLimitCaptureScope<T>(body: () => T): T; /** * Attach limit state to the currently active OTel span. * * Uses the active span rather than threading one down from the generation * layer: that layer is provider-agnostic and should not learn about Anthropic * quota headers just to record them. The active span during a turn is the one * already carrying `gen_ai.usage.*` and `neurolink.cost`, so "what did this * cost" and "how much is left" answer from the same trace. */ export declare function setLimitSpanAttributes(snapshot: ClaudeLimitSnapshot): void; /** * Emit one structured line per request describing remaining capacity. * * Leads with headroom ("how much is left") because that is the figure an * operator acts on; the raw utilization stays available on the snapshot for * anything computing against it. Escalates to WARN when the session window is * nearly spent or the provider has already flagged the account as * throttled/rejected. */ export declare function logClaudeLimitSnapshot(snapshot: ClaudeLimitSnapshot, model?: string, now?: number): void;