UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

79 lines (78 loc) • 4.4 kB
/** * Anthropic Messages API fallback over the pooled Codex Responses transport. * * This module deliberately contains only wire-format conversion and buffered * SSE parsing. Account selection, OAuth, cooldowns, and quota persistence stay * in the native Codex proxy handler so fallback traffic follows the same pool * rules as a native Codex request. */ import type { ClaudeRequest, CodexFallbackResult, CodexFallbackStream, CodexReasoningEffort, CodexResponsesRequest } from "../types/index.js"; export declare class CodexFallbackResponseError extends Error { readonly status: number; readonly responseBody: string; readonly code?: string; readonly retryable?: boolean; constructor(status: number, responseBody: string); } /** A provider terminal event, preserved through the format adapter. */ export declare class CodexFallbackStreamError extends Error { readonly status: number; readonly code: string; readonly retryable?: boolean; readonly usage?: CodexFallbackResult["usage"]; constructor(payload: Record<string, unknown>, eventType: string, usage?: CodexFallbackResult["usage"]); } /** * Routing key that pins a request to the cache already holding its prefix. * * Anthropic caching is explicit — the client marks breakpoints and the provider * honours them. OpenAI caching is automatic: a prefix is reused only when the * request lands on infrastructure already holding it, and `prompt_cache_key` is * what pins a request to one. Without it this hop takes its chances on routing, * which is the worst case for a fallback that arrives in bursts separated by * long gaps. * * The key must therefore describe the *prefix*, not the caller. Keying it by * session id gives every session a private namespace, so N agents that share * one system prompt and tool set each pay that shared prefix in full rather * than paying it once — the cost is linear in fan-out width and invisible, * because each request looks individually well-behaved. Across 42 captured * fallback bodies the prefixes collapse to 10 distinct values, the largest * shared by 12 requests belonging to different sessions. * * So derive the key from the prefix itself: the instructions and the tool * declarations, which are exactly the leading bytes the upstream caches and * the only part stable across a conversation's turns. Identical prefixes then * share one cache across sessions and subagents, a prefix that genuinely * differs still gets its own key so nothing unrelated contends, and turns * within one session keep sharing a key because neither input changes between * them. * * A request carrying neither instructions nor tools has no prefix worth * pinning, and falls back to the session id so its turns at least stay * together. With neither available the field is omitted rather than filled * with something shared: a key common to every request would pin unrelated * prefixes onto one cache instead of separating them. * * The digest covers only content already sent upstream in `instructions` and * `tools`; no account, device or session identifier reaches the wire. */ export declare function codexPromptCacheKey(body: ClaudeRequest): string | undefined; /** Convert a Claude Messages request into the ChatGPT Codex Responses shape. */ export declare function convertClaudeRequestToCodex(body: ClaudeRequest, model: string, reasoningEffort?: CodexReasoningEffort): CodexResponsesRequest; /** * Parse a complete Codex Responses SSE stream before emitting Claude output. * * A missing terminal event, malformed JSON, terminal error, or empty response * is rejected. That makes it safe for the caller to try the next configured * fallback without ever replaying output already sent to a client. */ export declare function parseCodexFallbackSSE(sse: string): CodexFallbackResult; /** Consume and validate a native Codex response before producing Claude output. */ export declare function consumeCodexFallbackResponse(response: Response): Promise<CodexFallbackResult>; /** * Translate a Codex stream as events arrive. A completed tool call is emitted * once its arguments validate; text does not wait for response.completed. * The caller owns error framing and must never retry after emitting output. */ export declare function createCodexFallbackStream(response: Response, model: string): Promise<CodexFallbackStream>;