UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

39 lines (38 loc) • 1.65 kB
/** * Forwarding a borrowed request to a lender's proxy. * * The wire format is unchanged Anthropic Messages in both directions, so this is * a passthrough, not a translation: the lender's proxy speaks exactly what the * borrower's client already sent. That is what makes peer borrowing cheap * compared with the provider fallback chain, which has to reshape the request * for a different API. * * **Reading the refusal, not the status.** A lender's 429 can mean "your grant * is spent" or "the upstream throttled me"; those want opposite reactions from * the borrower. The distinction is carried in `x-neurolink-grant-reason`, so * that header — not the status code — decides how long the peer is parked. * * @module proxy/peerTransport */ import type { ProxyPeer, ProxyPeerAttempt, ProxyPeerCooldownReason } from "../types/index.js"; /** * Map a lender's refusal to how long the peer should be left alone. * * Anything not recognized is treated as an upstream problem rather than a grant * problem — the conservative reading, since it recovers soonest. */ export declare function peerReasonFromRefusal(grantReason: string | null): ProxyPeerCooldownReason; /** * Send one request to one peer. * * On success the upstream `Response` is handed back with only an idle deadline * wrapped around its body, so a stream keeps streaming — buffering it here * would add the whole generation time to time-to-first-token on a path that is * already a second hop. */ export declare function forwardToPeer(args: { peer: ProxyPeer; body: string; stream: boolean; signal?: AbortSignal; }): Promise<ProxyPeerAttempt>;