@juspay/neurolink
Version:
The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002
74 lines (73 loc) • 3.4 kB
TypeScript
/**
* Proxy response quota headers.
*
* The proxy already knows, per request, exactly how much subscription capacity
* the serving account has left — it parses Anthropic's `anthropic-ratelimit-*`
* headers to route on them (see `accountQuota.ts`). Historically none of that
* reached the client: only the SSE path forwarded a small legacy allowlist, and
* every JSON/error path dropped headers entirely.
*
* This module turns that state into response headers in two layers:
*
* 1. **Verbatim passthrough** of Anthropic's own `anthropic-ratelimit-*` and
* `retry-after` headers. This is the load-bearing part: a proxied response
* then looks byte-identical to a direct one, so a consumer needs exactly one
* parser for both.
* 2. **`x-neurolink-*`** for what only the proxy can know — which account
* served the request, pool headroom, whether the numbers are live or stale,
* and the derived "how much is left" percentages.
*
* Pure CPU, no I/O — safe on the hot path and directly unit-testable.
*
* @module proxy/quotaHeaders
*/
import type { AccountQuota, ProxyQuotaHeaderContext } from "../types/index.js";
/**
* Convert a 0.0-1.0 utilization fraction into a whole-percent "left" figure.
*
* Anthropic publishes utilization (used), never remaining, for subscription
* windows — there is no absolute message or token count to report, so the
* honest derived form is a percentage. Clamped because a utilization above 1.0
* (overage) would otherwise produce a negative "left".
*/
export declare function utilizationToLeftPct(used: number): number;
/**
* Copy Anthropic's rate-limit headers verbatim from an upstream response.
*
* Covers both header families: the unified subscription windows
* (`unified-5h-*`, `unified-7d-*`) and the legacy per-tier counters
* (`requests-remaining`, `tokens-remaining`, ...) that API-key accounts get.
* Which family is present depends on the serving account type, which is why
* `x-neurolink-account-type` accompanies them.
*/
export declare function pickUpstreamRateLimitHeaders(headers: Headers | Record<string, string>): Record<string, string>;
/**
* Build the `x-neurolink-*` half of the contract from proxy-side state.
*
* Always emits `x-neurolink-quota-source` — even when it is "none". A consumer
* that sees quota numbers with no provenance cannot tell a fresh reading from a
* snapshot carried over from a previous request, and would happily log stale
* capacity as current.
*/
export declare function buildQuotaResponseHeaders(context: ProxyQuotaHeaderContext, now?: number): Record<string, string>;
/**
* Full response header set: upstream verbatim + proxy-derived.
*
* Upstream headers are applied first so a `x-neurolink-*` key can never be
* shadowed by an upstream one (they share no names today, but the ordering
* makes the precedence explicit rather than incidental).
*/
export declare function buildProxyLimitHeaders(args: {
upstreamHeaders?: Headers | Record<string, string>;
context: ProxyQuotaHeaderContext;
now?: number;
}): Record<string, string>;
/** Compute pool headroom from the runtime account states backing a request. */
export declare function summarizePoolHeadroom(entries: ReadonlyArray<{
coolingUntil?: number;
quota?: AccountQuota;
}>, now?: number): {
available: number;
cooling: number;
bestSessionLeftPct?: number;
};