UNPKG

@juspay/neurolink

Version:

The pipe layer of an AI nervous system: one interface connecting provider neurons to your application, across three inference types — generate, stream and decide. `decide` returns typed, calibrated judgements from a non-generative model (~400ms, ~$0.00002

660 lines (659 loc) • 35.5 kB
import type { ZodType } from "zod"; import { AIProviderName } from "../../constants/enums.js"; import { BaseProvider } from "../../core/baseProvider.js"; import type { EmbedInput, EnhancedGenerateResult, TextGenerationOptions, StreamOptions, StreamResult, VertexAnthropicMessage, ChatMessage, MinimalChatMessage } from "../../types/index.js"; import type { Schema, LanguageModel } from "../../types/index.js"; export declare function buildAnthropicHistoryMessages(conversationMessages: Array<ChatMessage | MinimalChatMessage>): VertexAnthropicMessage[]; export declare function appendUserMessage(messages: VertexAnthropicMessage[], content: VertexAnthropicMessage["content"]): void; /** * Recursively strip JSON-schema fields that Vertex Gemini's function-call AND * `responseSchema` (structured output) validators reject with 400 * INVALID_ARGUMENT. Vertex implements OpenAPI 3.0 Schema strictly and rejects * extension fields that the broader JSON Schema spec allows. The fields * stripped here have no semantic meaning for the model, so removing them is * safe for every caller. * * Fields removed: * - `additionalProperties` — extension; Vertex rejects on any nested object. * - `default` — Vertex rejects defaults on object/array-typed properties and * on properties that are also marked `required`. Safest to strip globally * because the model never inspects them. * - `$schema`, `$id`, `$ref`, `definitions`, `$defs` — JSON-Schema-meta * fields that Vertex doesn't recognise. * - `examples` — accepted by some Gemini variants but not 2.5-flash; strip * to avoid the model rejecting tool schemas under that path. * - `errorMessage` — emitted by `convertZodToJsonSchema` (which enables * zod-to-json-schema's `errorMessages: true`) for any field carrying a * custom message, e.g. `z.string().regex(re, { message })`. Vertex's * `response_schema` validator rejects it with `Unknown name "errorMessage" * … Cannot find field`, failing the whole structured-output request. * * Exported for deterministic unit testing of the sanitization contract. */ export declare function stripAdditionalPropertiesDeep(schema: Record<string, unknown> | undefined): void; /** * Resolve the effective Vertex region for a given model. * * Policy (matches the bugfixes-suite contract): * - Every Gemini model (`gemini-*`) is force-routed to the `global` endpoint * regardless of any caller-supplied region. Regional endpoints 404 for * Gemini 3.x previews and the regional/global behaviour for 2.x is * consistent enough that pinning all Gemini traffic to global is the * right safe default. The legacy `GLOBAL_LOCATION_MODELS` allowlist is * kept as a defence-in-depth fallback so any non-`gemini-` identifiers * that still need global (e.g. image-gen aliases) keep working. * - Non-Gemini models (Claude on Vertex, embeddings, custom models) keep * the caller-supplied region or fall back to env-derived defaults. * * @param modelName - The target model identifier. * @param configuredLocation - Caller-provided region (e.g. options.region). * Used as the fallback for non-Gemini models; ignored for Gemini. * @returns The region string to pass to the @google/genai client. */ export declare const resolveVertexLocation: (modelName: string | undefined, configuredLocation?: string) => string; export declare class GoogleVertexProvider extends BaseProvider { private projectId; private location; /** * Vertex AI Express Mode credentials. * * Vertex supports two authentication modes. The long-standing one pairs a * project and location with Application Default Credentials, which makes * the SDK mint an OAuth token through google-auth-library before every * request. Express Mode instead authenticates with an API key alone. * * Express is used only when an apiKey is supplied WITHOUT an explicit * project or location, so existing ADC callers — including those already * passing an apiKey alongside a project — keep exactly the behaviour they * have today. */ private expressApiKey?; /** * Optional endpoint override, mirroring AI Studio's * `credentials.googleAiStudio.baseURL`: per-request credential first, then * the environment, then unset so the SDK applies its own default. Blank * values count as unset so an empty override cannot clobber that default. */ private baseURL?; private registeredTools; private toolContext; private static modelConfigCache; private static modelConfigCacheTime; private static readonly CACHE_DURATION; private static readonly MAX_CACHE_SIZE; private static maxTokensCache; private static maxTokensCacheTime; constructor(modelName?: string, _providerName?: string, sdk?: unknown, region?: string, credentials?: Record<string, unknown>); protected getProviderName(): AIProviderName; protected getDefaultModel(): string; /** * Returns the Vercel AI SDK model instance for Google Vertex * Creates fresh model instances for each request */ protected getAISDKModel(): Promise<LanguageModel>; /** * Validate stream options */ private validateStreamOptionsOnly; /** * Preprocess file input before routing to the native SDKs. * * BaseProvider runs this via `buildMultimodalMessagesArray`, but Vertex * overrides both `generate()` and `executeStream()` to reach the native * @google/genai / @anthropic-ai/vertex-sdk clients directly, so neither * inherits it. Without this the file content never reaches the model and * the reply is an entirely plausible "no document is attached" — a silent * wrong answer rather than an error. * * #1258: only `generate()` used to call this, so the same document that * `generate()` read back correctly came back as "no documents attached" * through `stream()`. Sharing one method is what keeps the two paths from * drifting apart again. * * #1259: the alias fold has to happen *before* the `files` check, or * requests carrying only `audioFiles`/`videoFiles` look empty here and skip * preprocessing entirely. */ private preprocessNativeFileInput; protected executeStream(options: StreamOptions, _analysisSchema?: ZodType<unknown> | Schema<unknown>): Promise<StreamResult>; /** * Emit `stream:end` so the Pipeline B observability listener creates a * `model.generation` span for native Vertex stream traffic. Mirrors * `emitGenerationEnd` (used by `generate()`). */ private emitStreamEnd; /** * Create @google/genai client configured for Vertex AI */ private createVertexGenAIClient; /** Endpoint override: credential, then environment, then unset. */ private resolveBaseURL; /** * Express Mode key, if this provider should use it. * * Deliberately NOT read from GOOGLE_API_KEY: that variable is already set * by callers who also configure a project, and treating it as an Express * opt-in would silently switch their authentication mode. Express is opted * into per request, or through GOOGLE_VERTEX_API_KEY which exists only for * this purpose. */ private resolveExpressApiKey; /** * Convert one AI-SDK tool into an Anthropic (Claude-on-Vertex) tool * declaration. Single source for the pre-loop snapshot AND the mid-turn * discovery refresh — Anthropic validates input_schema as JSON Schema * draft 2020-12 and rejects OpenAPI-3 dialect (`nullable: true`), so this * uses the default JSON Schema target, matching the direct anthropic * provider. The Gemini paths keep "openApi3". */ private buildAnthropicToolDeclaration; /** * Mid-turn tool sync for the Claude-on-Vertex loops. Claude only calls tools * present in the request's `tools` array, so without this per-step refresh * a tool discovered via search_tools is unreachable for the rest of the * turn. Mutates the array in place (requestParams holds it by reference). */ private refreshAnthropicToolDeclarations; /** * Dispatch-miss recovery for the Claude-on-Vertex loops. */ private resolveAnthropicToolOnMiss; /** * Build the synthetic `final_result` tool used to force structured output * on Claude-on-Vertex. Anthropic has no native responseSchema, so a schema * is instead enforced by requiring the model to call this tool with * matching arguments. Single source for the non-streaming and streaming * paths — both previously built this object independently from their own * schema variable, byte-identical apart from that variable's name. */ private buildFinalResultTool; /** * Execute stream using native @google/genai SDK for Gemini 3 models on Vertex AI * This bypasses @ai-sdk/google-vertex to properly handle thought_signature */ private executeNativeGemini3Stream; /** * Resolve the text a native Vertex *generate* call actually sends, for * both `executeNativeGemini3Generate` and `executeNativeAnthropicGenerate`. * * The baseline here is the same `input.text || prompt || "Please * respond."` chain both call sites have always used — `input.text` wins * when set because the file preprocessors (`processUnifiedFilesArray`, * `processCSVFilesForNativeSDK`) append attached-file content to it while * `options.prompt` stays the pre-attachment snapshot; `prompt` is the * fallback for the legacy prompt-only `generateText()` convention, which * never sets `input` at all; the literal is the last resort when neither * is set. * * That `||` chain alone has a gap model middleware falls into: * `applyTurnParamsRewrite` writes every rewritten turn back onto * `options.input.text`, including an *emptied* one (a redaction). `||` * treats that empty string exactly like an unset `input.text` and falls * through to `options.prompt` — which middleware never touched, since it * is a plain snapshot taken before middleware ran — so a redaction a * middleware applied is silently discarded and the ORIGINAL text still * reaches Vertex. * * The fix is narrow: when `input.text` is *set* (not undefined) and * differs from `prompt`, something deliberately changed it since the * snapshot was taken — a middleware rewrite, or a file preprocessor's * attachment-content append — and that divergence itself is the signal to * trust `input.text` outright, empty string included. When the two agree * (the untouched, no-middleware case, including the STT-silence * `input.text === prompt === ""` placeholder) there is no divergence to * trust, so the original fallback chain still applies unchanged. */ private resolveNativeGenerateText; /** * Execute generate using native @google/genai SDK for Gemini 3 models on Vertex AI * This bypasses @ai-sdk/google-vertex to properly handle thought_signature */ private executeNativeGemini3Generate; /** * One-shot, tools-disabled model call used when a native Gemini agentic loop * is force-terminated by the step cap with no text produced. Lets the model * synthesize a final answer from the function results already in `contents` * instead of returning a canned placeholder (Bug 1, part b). * * Tools are disabled by OMITTING `config.tools` — the codebase's established * mechanism. `@google/genai`'s `FunctionCallingConfigMode.NONE` is documented * as equivalent to passing no function declarations, and `functionCallingConfig` * is not used anywhere in this codebase. When the structured-output * (`final_result`) pattern was active, a trailing instruction countermands the * earlier "you MUST call final_result" directive so the model answers in plain * text. Never throws — returns empty text so the caller falls back to the * graceful cap message (buildToolLoopCapMessage), guaranteeing no new failure * path. */ private synthesizeFinalAnswerWithoutTools; /** * Create native AnthropicVertex client for Claude models */ private createAnthropicVertexClient; /** * Execute stream using native @anthropic-ai/vertex-sdk for Claude models on Vertex AI * This bypasses @ai-sdk/google-vertex completely and uses Anthropic's native SDK */ /** * Extended thinking for Claude-on-Vertex. * * The direct Anthropic provider has always translated `thinkingConfig` into * the Messages API's `thinking` field (anthropic/client.ts). This path never * did — it built the request without one — so `thinkingConfig` was accepted * and silently ignored here: no error, `usage.reasoning` 0, no reasoning * content. Measured against the raw @anthropic-ai/vertex-sdk with the same * project, region, credentials and model, `thinking` is supported and * returns a real thinking block, so the capability was ours to send. * * `thinkingLevel` is deliberately not mapped to a budget: the option's * contract reserves `budgetTokens` for Anthropic models and `thinkingLevel` * for Gemini 3, and inventing a conversion here would be guessing at the * caller's intent. * * The budget must leave room inside `max_tokens` for an answer, so it is * clamped below the ceiling rather than passed through — an over-large * budget is a 400 from the API. * * `forcesToolChoice` (schema mode on a model that accepts a forced choice) * pairs with a forced `tool_choice:{type:"any"}` in the request built right * after this call — Anthropic hard-rejects `thinking` combined with a `tool_choice` * that forces tool use: a live 400, "Thinking may not be enabled when * tool_choice forces tool use." So `thinking` is omitted entirely in that * mode, preserving the prior (pre-thinking-fix) behaviour of silently * ignoring `thinkingConfig` for schema calls instead of hard-failing them — * mirrors the same guard already applied to the OAuth proxy path in * `src/lib/proxy/oauthFetch.ts` (`delete parsed.thinking` when * `tool_choice.type` is `"any"` or `"tool"`). The step/context-cap * finalization backstop reuses this same omission for free since it spreads * `...requestParams`. `tool_choice:{type:"none"}` (the separate * tools-disabled backstop, used only outside schema mode) does not force * tool use — it forces the opposite — so it is not gated here, consistent * with oauthFetch.ts leaving `"none"` alone too. */ private buildClaudeThinkingParam; private executeNativeAnthropicStream; /** * Execute generate using native @anthropic-ai/vertex-sdk for Claude models on Vertex AI */ private executeNativeAnthropicGenerate; /** * Process CSV files and append content to options.input.text * This ensures CSV data is available in the prompt for native Gemini 3 SDK calls * Returns a new options object with modified input (immutable pattern) */ private processCSVFilesForNativeSDK; /** * Override stream to handle image generation models * Image models don't support streaming, so we fall back to generate */ stream(optionsOrPrompt: StreamOptions | string): Promise<StreamResult>; /** * Override generate to route ALL models to native SDKs * No more @ai-sdk/google-vertex dependency */ generate(optionsOrPrompt: TextGenerationOptions | string): Promise<EnhancedGenerateResult | null>; /** * Invoke `options.onError` with the lifecycle payload shape consumers * (and `test:middleware`) expect. Mirrors {@link fireGenerateOnFinish}. */ private fireGenerateOnError; /** * Wrap a {@link StreamResult} so each text chunk drives `options.onChunk` * and the final yield drives `options.onFinish`. Pipeline A providers get * this for free via the AI SDK `wrapStream` middleware; native @google/genai * bypasses that wrapper, so native consumers need their lifecycle * callbacks invoked from here. * * `BaseProvider.stream()` always passes the result this method returns * through `wrapStreamWithLifecycleCallbacks` afterwards too (that wrapper * has "no early return when there are no callbacks" — see its own doc * comment) — so unconditionally firing here double-fires onChunk/onFinish * for every native Vertex stream. `isLifecycleStreamCallbacksOwnedByBaseProvider` * reads the SAME nested lifecycle config off the SAME `options` object * `wrapStreamWithLifecycleCallbacks` reads from (NeuroLink.applyStreamLifecycleMiddleware * stamps it before either method runs), so both sites agree on which one * actually fires without a second, Vertex-specific marker — and without * relying on a Symbol on the `StreamResult` itself, which would not * survive `withStreamModelFallback`'s rebuild of a plain forwarding object * between this method and `wrapStreamWithLifecycleCallbacks`. * * onError is deliberately NOT gated by that marker — same precedent as * `middleware/builtin/lifecycle.ts`'s `wrapStream` ("onError is NOT * covered by this flag: it keeps its own, independent dedup via * fireOnErrorOnce's shared fired-marker"). `fireOnErrorOnce` stamps the * error so `BaseProvider`'s own catch (which checks * `hasLifecycleErrorFired`) skips its own fire for the same failure. */ private wrapStreamResultWithLifecycle; /** * The text a native Vertex turn is built from before any model middleware * rewrite — `options.input.text` when the caller set it (the normal * `generate()`/`stream()` convention), else `options.prompt` (the legacy * prompt-only `generateText()` convention, where `input` is never * populated — see `generateTextInInstanceScope` in `src/lib/neurolink.ts`, * which validates `options.prompt` and never touches `options.input`). * * `buildTurnPromptForMiddleware` and `applyTurnParamsRewrite` both read * this so they agree on one baseline: the first so a prompt-only caller * shows middleware its real text instead of an empty turn, the second so * a middleware-rewritten turn is compared against what middleware actually * started from — not a baseline that ignores `prompt` entirely, which * would make a prompt-only call look "unedited" no matter what middleware * did to it. `StreamOptions` carries no top-level `prompt` field, hence * the `"prompt" in options` guard instead of a cast. */ private vertexTurnBaselineText; /** * Build the V3 entry-turn prompt middleware sees for a native call. * Native Vertex loops build their own multi-turn conversation internally * once they run — this is only the caller's system prompt + text input, * the one turn middleware is given visibility into before that loop * starts. * * The user turn is an array of text parts rather than a bare string. That * is the V3 prompt shape, and prompt-editing middleware is written against * it: one that spreads `content` into a longer array turns a string into * one element per character, and one that calls `.map` / `.filter` on it * throws. The AI Studio path already hands its middleware parts * (`geminiContentsToV3Prompt`); this makes Vertex agree with it. * * The text itself comes from `vertexTurnBaselineText`, not * `options.input?.text` alone — a prompt-only `generateText()` caller * never sets `input`, and without the `options.prompt` fallback middleware * would be shown an empty turn and have nothing of the caller's to inspect * or build on. */ private buildTurnPromptForMiddleware; /** Sampling params a native call already honors, carried onto the V3 view. */ private buildTurnV3Params; /** * Read a middleware-transformed prompt back into the two things a native * Vertex turn can carry: one system instruction and one user text. * * Every system message counts (joined with a blank line, the way * `v3PromptToGeminiContents` joins them for AI Studio) and so does every * user message, not just the last. A middleware that appends or prepends * context leaves the caller's own text in the prompt beside it; reading * only the last user message would send the addition without the question * it was meant to qualify, or drop it and send the question bare. Text * blocks — across messages and across the parts of one message — are joined * with a blank line for the same reason: they are separate blocks, and * gluing them runs the last word of one into the first of the next. * * `unmapped` counts what a native turn has no place for — non-text user * parts (files, images) and assistant or tool messages — so the caller can * say it was dropped rather than drop it silently. */ private readTurnFromPrompt; /** * Apply a middleware-transformed turn back onto the native call options. * * The system instruction and the user text are each written back only when * they differ from what middleware actually started from — `systemPrompt` * against `options.systemPrompt`, user text against * `vertexTurnBaselineText(options)`, not `options.input?.text` alone — so a * middleware that touches neither leaves both exactly as they were. An * edit is honoured in every direction — a changed, added or removed system * message, an appended or prepended user message, a user message emptied * outright (a redaction) — because every native generate/stream executor * resolves its wire text through `options.input.text` once this write-back * has run (`resolveNativeGenerateText` on the generate paths, which * special-cases exactly this divergence so an emptied `input.text` is not * mistaken for an unset one and does not fall back to the stale * `options.prompt` snapshot; a direct `options.input.text` read on the * stream paths, which have no such fallback to fall into). Comparing * against `options.input?.text` alone — ignoring `options.prompt` — would * make a prompt-only `generateText()` caller's untouched turn look * "changed" the first time middleware ran (nothing to compare `turn.text` * against but `""`), writing the real prompt into `input.text` as a * spurious edit on every such call even when no middleware did anything. * * Sampling params are written back the same way and can be removed as well * as changed: a middleware that drops `temperature` or `topP` from the * params must reach the wire as "not sent", not as the caller's original * value kept because the key went missing. * * `tools` is offered to middleware read-only — a middleware that rewrites * it gets a WARN rather than a silent drop, matching the OpenAI-compatible * streaming path's contract: native tool wiring is derived independently * of this V3 view, and re-deriving it here would diverge from it. */ private applyTurnParamsRewrite; /** * Strip the lifecycle middleware from the options handed to * `applyMiddlewareToModel` for a native Vertex call, when something else is * already firing the same callbacks. * * NeuroLink's `applyGenerateLifecycleMiddleware` / * `applyStreamLifecycleMiddleware` (neurolink.ts) mirror a caller's * top-level `onFinish` / `onError` / `onChunk` into * `options.middleware.middlewareConfig.lifecycle.config` — the exact same * callback references — before this provider sees `options`. Left enabled, * the built-in lifecycle middleware would fire them again from inside the * bridge, and with malformed payloads: it reads the legacy `textDelta` chunk * field and a bare `finishReason`, neither of which a V3 model produces. * * Who else fires them differs by path, so the exclusion does too: * * - generate: only `fireGenerateOnFinish` / `fireGenerateOnError`, which read * the TOP-LEVEL callbacks. The entry is stripped only when one is present. * A caller who configures `middlewareConfig.lifecycle` directly has no * other firer here, so that middleware applies exactly once, unmodified. * - stream: `BaseProvider.stream()` wraps every stream in * `wrapStreamWithLifecycleCallbacks`, which fires whatever sits in * `lifecycle.config` however it got there — top-level shorthand or direct * config. The entry is stripped whenever it exists, or a direct-config * caller would get every callback twice (and `onChunk` a third time, for * the `finish` part the middleware also reports). * * Only the `lifecycle` entry is disabled, so every other configured * middleware (guardrails, transformParams, custom hooks, analytics, ...) * applies normally. */ private excludeAutoLifecycleMiddleware; /** * Run the native Gemini generate loop and, when Vertex rejects an * over-constrained `responseSchema`, retry once without it. * * The retry sits inside the function `runNativeGenerateWithMiddleware` hands * to the model bridge rather than around the bridge, because it is * provider-internal recovery: model middleware must see one `doGenerate` * that succeeds. Retrying outside would show every middleware the failed * first call — the lifecycle middleware would report an `onError` for a * request that then succeeded, and `transformParams` would run twice. */ private executeNativeGemini3GenerateWithSchemaFallback; /** * Apply the configured middleware chain around a native Vertex generate * loop. `callNative` is `executeNativeAnthropicGenerate` or * `executeNativeGemini3Generate` — a self-contained black box that runs * its own multi-step tool loop and returns a fully-resolved * `EnhancedGenerateResult`. Wrapping happens around that call, not inside * it: `transformParams` can rewrite the entry turn before the loop starts, * and `wrapGenerate` can observe the result or replace it outright (a * guardrail that blocks the call entirely never reaches `callNative`, so * `nativeResult` stays unset and a settled result is synthesized purely * from what middleware produced, then run through `enhanceResult` like any * other result so `enableAnalytics` / `enableEvaluation` still apply). */ private runNativeGenerateWithMiddleware; /** * Apply the configured middleware chain around a native Vertex stream * loop. `callNative` is `executeNativeAnthropicStream` or * `executeNativeGemini3Stream`. Both are bridged through the same lazy V3 * `ReadableStream` (`nativeChunksToV3Stream` / `v3StreamToNativeChunks`) * regardless of whether the underlying loop is genuinely live (Anthropic) * or already fully resolved (Gemini3) by the time it returns — pulling one * item at a time is correct for both, and never turns a live stream into * collect-then-replay. * * A guardrail that blocks the call never invokes `doStream`, so * `nativeResult` stays unset; a `StreamResult` is synthesized instead, * with `usage` / `finishReason` resolving once the merged stream is fully * drained (the same late-resolving-getter contract * `preserveStreamResultAccessors` already exists for). */ private runNativeStreamWithMiddleware; /** * Re-apply getter-based accessor properties from a source StreamResult onto * a wrapper copy. Wrapper spreads (`{ ...result }`) invoke and SNAPSHOT * enumerable getters at wrap time — for background-loop streams (the native * Anthropic path) that resolve finishReason / structuredOutput / toolCalls * / usage only as the consumer drains, the snapshot is permanently * undefined/empty. Copying the accessor descriptors keeps the wrapped * result live. Results built from plain data properties (the buffered * Gemini paths, and the native Anthropic stream's own `usage` — a mutable * object reference rather than a getter, so a spread keeps its identity * and later mutations stay visible) have no matching getter here and pass * through untouched. * * The re-applied property also accepts a direct write: NeuroLink.stream() * backstops a missing `usage` from analytics once the outer stream drains * by assigning `mcpStreamOutcome.usage = resolvedUsage` on whatever result * object it was handed. Copying the bare getter-only descriptor verbatim * would turn that assignment into a TypeError ("Cannot set property usage * of #<Object> which has only a getter"), so each re-applied accessor is * rebuilt with a setter that remembers an override and prefers it over the * live source getter once one has been written. */ private preserveStreamResultAccessors; /** * Attach `gen_ai.usage.*` and `neurolink.cost` attributes to a span. * Pulled out so the generate / stream / image-gen paths share one * implementation, and so observability/tracing tests find consistent * attributes regardless of which native sub-route fulfilled the request. */ private attachUsageAndCostAttributes; /** * Emit `generation:end` so the Pipeline B observability listener creates * the corresponding `model.generation` span. Vertex bypasses the AI SDK * (and therefore the experimental_telemetry plumbing), so this hand-off * is the only way native Vertex calls show up in Langfuse / Pipeline B * exporters. Mirrors the Bedrock + Ollama pattern. */ private emitGenerationEnd; /** * Turn-clock-shaped adapter over the shared {@link buildLoopExitMessage}. * Every call site in this file holds a clock, so the mapping lives here * rather than at ten call sites; the decision itself is the shared one. */ private buildLoopExitMessage; protected formatProviderError(error: unknown): Error; /** * Memory-safe cache management for model configurations * Implements LRU eviction to prevent memory leaks in long-running processes */ private static evictLRUCacheEntries; /** * Access and refresh cache entry (moves to end for LRU) */ private static accessCacheEntry; /** * Memory-safe cached check for whether maxTokens should be set for the given model * Optimized for streaming performance with LRU eviction to prevent memory leaks */ private shouldSetMaxTokensCached; /** * Memory-safe check if model has maxTokens issues using configuration-based approach * This replaces hardcoded model-specific logic with configurable behavior * Includes LRU caching to avoid repeated configuration lookups during streaming */ private modelHasMaxTokensIssues; /** * Check if Anthropic models are available * @returns Promise<boolean> indicating if Anthropic support is available */ hasAnthropicSupport(): Promise<boolean>; /** * Register a tool with the AI provider * @param name The name of the tool * @param schema The Zod schema defining the tool's parameters * @param description A description of what the tool does * @param handler The function to execute when the tool is called */ registerTool(name: string, schema: ZodType<unknown>, description: string, handler: (params: Record<string, unknown>) => Promise<unknown>): void; /** * Set the context for tool execution * @param context The context to use for tool execution */ setToolContext(context: Record<string, unknown>): void; /** * Get the current tool execution context * @returns The current tool execution context */ getToolContext(): Record<string, unknown>; /** * Set the tool executor function for custom tool execution * This method is called by BaseProvider.setupToolExecutor() * @param executor Function to execute tools by name */ setToolExecutor(executor: (toolName: string, params: unknown) => Promise<unknown>): void; /** * Clear all static caches - useful for testing and memory cleanup * Public method to allow external cache management */ static clearCaches(): void; /** * Get cache statistics for monitoring and debugging */ static getCacheStats(): { modelConfigCacheSize: number; maxTokensCacheSize: number; maxCacheSize: number; cacheAge: { modelConfig: number; maxTokens: number; }; }; /** * Detect image MIME type from buffer */ private detectImageType; /** * Estimate token count from text (simple character-based estimation) */ private estimateTokenCount; /** * Build image parts for multimodal content */ /** * Overrides the BaseProvider's image generation method to implement it for Vertex AI. * Uses REST API approach with google-auth-library for authentication. * Supports PDF input for image generation with gemini-3-pro-image-preview (Nano Banana Pro). * @param options The generation options containing the prompt and optional PDF files. * @returns A promise that resolves to the generation result, including the image data. */ protected executeImageGeneration(options: TextGenerationOptions): Promise<EnhancedGenerateResult>; /** * Get model suggestions when a model is not found */ /** * Candidate model ids for a "model not available" error. * * This list is static. It cannot know what the caller's project and region * actually serve, so it must not claim to: it previously announced * "always available" and included the requested model itself, which * produced errors that recommended the very id that had just failed — * `gemini-3-pro-preview-11-2025` suggesting `gemini-3-pro-preview-11-2025`. * The requested model is now filtered out, and the wording says these are * ids to try rather than ids known to work. */ private getModelSuggestions; /** * Generate an embedding for `text` using Vertex via @google/genai. * * Replaces the previous `@ai-sdk/google-vertex` text embedding model * path. Without this, RAG indexing falls through to BaseProvider.embed() * which throws "Embedding generation is not supported by the vertex * provider", and `neurolink rag index --provider=vertex` fails even * though the SDK conceptually supports it. */ embed(input: string | EmbedInput, modelName?: string): Promise<number[]>; /** * Batch-embed an array of strings via Vertex @google/genai. * Mirrors {@link embed} but returns one vector per input string. */ embedMany(texts: string[], modelName?: string): Promise<number[][]>; } export default GoogleVertexProvider; export { GoogleVertexProvider as GoogleVertexAI };