UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

74 lines (73 loc) • 3.4 kB
/** * Proxy response quota headers. * * The proxy already knows, per request, exactly how much subscription capacity * the serving account has left — it parses Anthropic's `anthropic-ratelimit-*` * headers to route on them (see `accountQuota.ts`). Historically none of that * reached the client: only the SSE path forwarded a small legacy allowlist, and * every JSON/error path dropped headers entirely. * * This module turns that state into response headers in two layers: * * 1. **Verbatim passthrough** of Anthropic's own `anthropic-ratelimit-*` and * `retry-after` headers. This is the load-bearing part: a proxied response * then looks byte-identical to a direct one, so a consumer needs exactly one * parser for both. * 2. **`x-neurolink-*`** for what only the proxy can know — which account * served the request, pool headroom, whether the numbers are live or stale, * and the derived "how much is left" percentages. * * Pure CPU, no I/O — safe on the hot path and directly unit-testable. * * @module proxy/quotaHeaders */ import type { AccountQuota, ProxyQuotaHeaderContext } from "../types/index.js"; /** * Convert a 0.0-1.0 utilization fraction into a whole-percent "left" figure. * * Anthropic publishes utilization (used), never remaining, for subscription * windows — there is no absolute message or token count to report, so the * honest derived form is a percentage. Clamped because a utilization above 1.0 * (overage) would otherwise produce a negative "left". */ export declare function utilizationToLeftPct(used: number): number; /** * Copy Anthropic's rate-limit headers verbatim from an upstream response. * * Covers both header families: the unified subscription windows * (`unified-5h-*`, `unified-7d-*`) and the legacy per-tier counters * (`requests-remaining`, `tokens-remaining`, ...) that API-key accounts get. * Which family is present depends on the serving account type, which is why * `x-neurolink-account-type` accompanies them. */ export declare function pickUpstreamRateLimitHeaders(headers: Headers | Record<string, string>): Record<string, string>; /** * Build the `x-neurolink-*` half of the contract from proxy-side state. * * Always emits `x-neurolink-quota-source` — even when it is "none". A consumer * that sees quota numbers with no provenance cannot tell a fresh reading from a * snapshot carried over from a previous request, and would happily log stale * capacity as current. */ export declare function buildQuotaResponseHeaders(context: ProxyQuotaHeaderContext, now?: number): Record<string, string>; /** * Full response header set: upstream verbatim + proxy-derived. * * Upstream headers are applied first so a `x-neurolink-*` key can never be * shadowed by an upstream one (they share no names today, but the ordering * makes the precedence explicit rather than incidental). */ export declare function buildProxyLimitHeaders(args: { upstreamHeaders?: Headers | Record<string, string>; context: ProxyQuotaHeaderContext; now?: number; }): Record<string, string>; /** Compute pool headroom from the runtime account states backing a request. */ export declare function summarizePoolHeadroom(entries: ReadonlyArray<{ coolingUntil?: number; quota?: AccountQuota; }>, now?: number): { available: number; cooling: number; bestSessionLeftPct?: number; };