UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

90 lines (89 loc) • 5.37 kB
import type { AIProviderName } from "../constants/enums.js"; import type { OpenAICompatCatalogEntry, OpenAICompatChatRequest, OpenAICompatCredentials, OpenAICompatResponseFormat } from "../types/index.js"; import { OpenAIChatCompletionsProvider } from "./openaiChatCompletionsBase.js"; /** * Generic OpenAI-compatible provider driven entirely by an * OpenAICompatCatalogEntry. Replaces a hand-written subclass for any * provider whose only differences from its siblings are credentials, base * URL, model defaults, and error-classification rules — see * OPENAI_COMPAT_CATALOG in openaiCompatCatalog.ts for the entries. * * If a provider needs a real hook override (adjustRequestBody, * adjustBodyAfter400, getChatCompletionsURL, getAuthHeaders, ...) that isn't * expressible as one of the catalog's data-driven quirks (messageContentFormat, * responseFormatDowngrade, replayReasoningContent, * supportsStructuredOutputWithTools, ...) it does NOT belong in the catalog — * write a dedicated subclass instead (see azureOpenai.ts). * * The exception is a WIRE DIALECT: a vendor that speaks OpenAI for ordinary * chat but encodes one part of the request differently. Expressing that as * data (the catalog's `messageContentFormat` / `responseFormatDowngrade`) * keeps the provider a one-JSON-file entry instead of promoting it to a * hand-written subclass over a single incompatibility. * * Today that is Cloudflare Workers AI, whose OpenAI-compatible endpoint * accepts `messages[].content` only as a plain string — never the * content-parts array OpenAI allows, and never the `null` that OpenAI uses * on an assistant message carrying tool_calls. Its schema rejects both with * HTTP 400 ("Type mismatch of '/messages/N/content'"). That bites precisely * on the second turn of a tool call, so single-turn chat looks healthy while * every tool round-trip fails. Normalizing content to a string below is the * whole fix: tools, tool_choice, the `tool` role and `tool_calls` are all * accepted as-is. * * The same shape covers DeepSeek: its `/chat/completions` rejects * `response_format: { type: "json_schema" }` outright ("This response_format * type is unavailable now"), but accepts `{ type: "json_object" }`. The * catalog's `responseFormatDowngrade` quirk downgrades the request so * `generate({ schema })` keeps working, matching the * `supportsStructuredOutputs: false` behavior of the `@ai-sdk/openai-compatible` * provider this replaced. */ export declare class ConfiguredOpenAICompatProvider extends OpenAIChatCompletionsProvider { private readonly entry; constructor(entry: OpenAICompatCatalogEntry, modelName?: string, sdk?: unknown, credentials?: OpenAICompatCredentials); protected getProviderName(): AIProviderName; protected getDefaultModel(): string; protected getFallbackModelName(): string; protected getFallbackModels(): string[]; /** * See CatalogQuirks.authHeaderStyle. The base class's default is * `Authorization: Bearer <apiKey>`; "x-api-key" vendors (Reka's OpenAPI * spec declares exactly one security scheme, an apiKey in header * `X-Api-Key`) get that header instead, with no Authorization header sent * at all — mirrors AzureOpenAIProvider's own getAuthHeaders() override for * its `api-key` header. */ protected getAuthHeaders(): Record<string, string>; /** * The catalog's `capabilities.tools` is the vendor's own answer, probed on * the wire when the entry was written; the model registry (the base * default) knows nothing about Tier-2 models and answers "supported" for * every unknown id. A vendor that declares tools: false must never receive * a `tools` array — Mancer's free model rejects one with 400 — so the * declaration wins here and the registry is only consulted otherwise. * Like the other entry-reading overrides above, this runs only after * construction: BaseProvider merely closes over it. */ supportsTools(): boolean; /** * The base class drops `response_format` from any request carrying tools, * because a generic OpenAI-compatible backend may honour it over tool * calling and answer with final-shape JSON on step one instead of running * the agentic loop. For a catalog entry that is not a guess — the flag * comes from a live wire probe (`capabilities.structuredOutputWithTools`) * — so an entry that declares the combination supported sends both and * gets the object from the vendor's own constrained decoding in ONE * request, instead of falling back to a tool-free re-ask. * * Conservative by default: only an explicit `true` opts in. A vendor that * declares support but rejects the combination at runtime is still caught * by `isToolsSchemaConflictError` and retried without structured output, * so a stale flag costs one request rather than the turn. */ protected suppressResponseFormatWithTools(): boolean; protected adjustRequestBody(body: OpenAICompatChatRequest, modelId: string): OpenAICompatChatRequest; protected adjustResponseFormat(rf: OpenAICompatResponseFormat | undefined, modelId: string): OpenAICompatResponseFormat | undefined; protected replayReasoningContent(): boolean; protected formatProviderError(error: unknown): Error; }