@juspay/neurolink
Version:
The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002
73 lines (72 loc) • 3.92 kB
TypeScript
/**
* Bedrock cross-region inference profile resolution.
*
* Most current Bedrock models cannot be invoked by their bare model id. Each
* model card carries a Regional Availability table with three columns, and for
* a great many model/region pairs the In-Region column is "No" — Claude Opus
* 4.5 is No in every region AWS lists, Sonnet 4.6 is Yes only in eu-west-2.
* Those models are reachable only through a cross-region inference profile: the
* same id with a geography prefix (`us.`, `eu.`, `au.`, `jp.`, `apac.`) or the
* worldwide `global.` prefix.
*
* Sending the bare id where In-Region is No fails with a ValidationException
* telling you to use an inference profile. This module detects that specific
* failure and works out which prefixed id to retry with.
*
* ## Why prefixes are derived rather than tabulated
*
* The obvious implementation — a table of model to supported prefixes — is the
* one to avoid. The prefix set is per-model, not per-region: from the same
* caller region `ap-northeast-1`, Claude Opus 4.5 accepts only `global.`,
* Sonnet 4.5 also accepts `jp.`, and Haiku 4.5 works bare. A static table
* would need a row per model per region and would be wrong the day a model
* ships. AWS itself moved this data out of a central page and onto each
* model's detail page for the same reason.
*
* So resolution is empirical: on the specific error, try the geography that
* covers the caller's region, then `global.`, and remember what worked.
*
* `bedrock:ListInferenceProfiles` would give an authoritative answer, but it
* is a control-plane call needing a separate SDK client and an IAM permission
* a caller with plain inference access will not necessarily hold. Deriving
* candidates needs neither, and costs at most two extra attempts once per
* model/region pair.
*/
/**
* Does this error mean "this model needs an inference profile in this region"?
*
* Deliberately narrow. A ValidationException covers many unrelated causes —
* a malformed body, an unknown parameter, a model the account has no access to
* — and retrying those with a different model id would turn one clear failure
* into two confusing ones. Both the exception name and the distinctive phrase
* must match.
*
* The phrase is matched loosely enough to survive AWS rewording the sentence
* around it, and the retry is a no-op when nothing resolves, so a miss here
* costs nothing beyond the original error surfacing unchanged — which is the
* behaviour without this module at all.
*/
export declare function isInferenceProfileRequiredError(error: unknown): boolean;
/**
* Candidate ids to retry, most specific first: the caller's own geography,
* then worldwide.
*
* Returns an empty array when the id already carries a prefix — a prefixed id
* that still fails is a real error, not something to re-prefix.
*/
export declare function inferenceProfileCandidates(modelId: string, region: string): string[];
/** The id that previously worked for this model in this region, if any. */
export declare function getResolvedModelId(modelId: string, region: string): string | undefined;
export declare function rememberResolvedModelId(modelId: string, region: string, resolvedId: string): void;
/** Exposed so tests can start from a known state. */
export declare function clearResolvedModelIds(): void;
/**
* Run `send` against the given model id, falling back to inference-profile
* ids if — and only if — Bedrock says the bare id needs one.
*
* A previously resolved id is used straight away. Any error other than the
* inference-profile one propagates untouched on the first attempt, so this
* cannot mask an unrelated failure or silently change behaviour for a caller
* whose bare ids already work.
*/
export declare function withInferenceProfileFallback<T>(modelId: string, region: string, send: (effectiveModelId: string) => Promise<T>): Promise<T>;