ai
Version:
AI SDK by Vercel - build apps like ChatGPT, Claude, Gemini, and more with a single interface for any model using the Vercel AI Gateway or go direct to OpenAI, Anthropic, Google, or any other model provider.
293 lines (239 loc) • 13.6 kB
text/mdx
---
title: Secure URL Fetching
description: Protect server-side requests to URLs supplied by users, models, tools, and external services.
---
# Secure URL Fetching
Many providers return a **URL in their response body** — a generated image,
audio, or video to download, or a polling URL to check job status. The AI SDK
fetches these server-side and returns the result to your code. Because that URL
comes from an external service, a malicious or compromised provider (or anyone
able to tamper with the response) could point it at an internal address such as
a cloud-metadata endpoint (`http://169.254.169.254/…`), a private host
(`http://10.0.0.5/…`), or `localhost`.
The SDK validates response-supplied URLs before fetching them. These automatic
checks inspect the URL and literal IP addresses; they do not resolve DNS.
**DNS protection is opt-in.** The SDK uses your configured `fetch` or the
runtime's global `fetch`, including on Node.js. A public-looking hostname can
still resolve to a private address, and DNS rebinding is not blocked by the
default transport. For applications that fetch untrusted URLs, use
[`guarded-fetch`](#opt-in-dns-protection-on-nodejs) on Node.js and/or
restrict outbound traffic at the network layer. The core SDK does not depend
on Undici.
For authenticated task-status polling, providers can construct the first URL
from the configured API endpoint. The SDK trusts that configured origin for the
initial request, but manually follows and validates every redirect away from it.
MiniMax, Kling AI, and ByteDance video polling use this protected path.
## When to use a guarded transport
Use `guarded-fetch` on Node.js whenever your server fetches URLs supplied
by users, models, tools, or external services, unless the destinations are
controlled by your application or your fetch implementation or network egress
policy already provides equivalent SSRF protection. This includes prompt image
and file URLs, remote chat attachments, tool-result files, generated media
downloads, and user-configured remote MCP servers.
Authentication alone does not make a user-supplied URL trusted. A URL selected
by a model or extracted from a document is still untrusted input. Trust the
destination only when your application controls where the request can go. A URL
syntax check or a separate DNS lookup before calling fetch does not provide
connection-time DNS protection.
Configure the transport at the request's actual download point: provider
`fetch` settings apply to provider requests, while [core file
downloads](#core-file-downloads) use separate hooks. Configure built-in
[MCP transports](/docs/ai-sdk-core/mcp-tools#securing-remote-server-urls) through
`transport.fetch`; custom tools and transports need their own fetch
configuration. URLs passed directly to a model provider are fetched by that
service, outside your server's transport.
## What the SDK protects against
When the SDK fetches a URL taken from a provider response, it:
- **Rejects private, loopback, and link-local IP literals** — IPv4 (`10/8`,
`172.16/12`, `192.168/16`, `127/8`, `169.254/16`, CGNAT, multicast, …) and the
equivalent IPv6 ranges, plus `localhost`, `.local`, `.localhost`, and
`.internal` hostnames. Non-`http(s)` schemes are rejected too.
- **Re-validates every redirect hop** — a URL that passes but then redirects to
an internal address is blocked; the redirect is never followed blindly.
- **Rejects HTTPS downgrades on server-side downloads** — a redirect from HTTPS
to HTTP is blocked before requesting the insecure target, even when that target
matches `trustedOrigin`. Initial HTTP URLs and HTTP-to-HTTPS upgrades remain
supported.
- **Strips risky request headers** — proxy-forwarding, cloud-metadata, and
cookie headers are removed before the request.
- **Drops credentials across origins** — caller headers (`Authorization`,
`Cookie`, and provider-specific API-key headers alike) are not sent to a host
on a different origin than the provider's; a redirect that crosses origin
drops all of them except the user-agent.
A blocked URL surfaces as a `DownloadError`.
## Fetching untrusted URLs directly
For custom integrations using `-sdk/provider-utils`, use `fetchUntrustedUrl`
when the URL comes from a response or other untrusted input. It validates URLs
and redirects and withholds credentials and unknown custom headers on the first
request unless the URL matches a configured origin:
```ts
import { fetchUntrustedUrl } from '-sdk/provider-utils';
const response = await fetchUntrustedUrl({
url: result.downloadUrl,
headers: {
authorization: `Bearer ${apiKey}`,
accept: 'image/*',
},
credentialedOrigin: providerBaseURL,
});
```
Here, `Authorization` is sent only when the initial URL has the same origin as
`providerBaseURL`. Otherwise, only allowlisted metadata such as `Accept`,
`Range`, and `User-Agent` is sent. A cross-origin redirect drops all caller
headers except `User-Agent`, and credentials are never reattached on later hops.
`credentialedOrigin` must come from your configuration, never from the response
URL. It does not disable URL validation. For a configured private endpoint, pass
`trustedOrigin` as well; if `credentialedOrigin` is omitted, `fetchUntrustedUrl`
uses `trustedOrigin` as the origin allowed to receive credentials. Extra protocol
metadata can be allowed with `untrustedFirstHopHeaders: ['x-protocol-version']`.
Only list headers that are safe to disclose to any destination; use a configured
origin for credentials.
The existing `fetchWithValidatedRedirects` helper preserves its first-request
behavior: it sanitizes risky headers but still forwards authentication and
custom headers. Switch to `fetchUntrustedUrl` to opt into first-request
credential isolation. Existing calls to `getFromApi` also retain their behavior;
pass `credentialedOrigin` when fetching response-supplied URLs with credentials.
## Self-hosted and local endpoints
URLs that are same-origin with the provider endpoint **you configured** (e.g. a
custom `baseURL` pointing at a self-hosted or `localhost` deployment) are
exempt from the SDK's URL checks — they target exactly the host you told the SDK to
talk to. When using `createGuardedFetch`, also add that origin to its
`trustedOrigins` option to allow private addresses. This also applies to task-status polling. Any redirect off that origin
is still validated before the redirected request is sent.
## Opt-in DNS protection on Node.js
Install the optional transport:
```sh
npm install guarded-fetch@^0.3.0
```
Pass it to providers that fetch untrusted URLs:
```ts
import { createFal } from '@ai-sdk/fal';
import { createGuardedFetch } from 'guarded-fetch';
import { generateImage } from 'ai';
const fetch = createGuardedFetch();
const fal = createFal({ fetch });
try {
const result = await generateImage({
model: fal.image('fal-ai/flux/schnell'),
prompt: 'A blue cat in Kyoto',
});
console.log(result.images);
} finally {
await fetch.close();
}
```
The transport checks every IPv4 and IPv6 address returned by DNS inside the
connector. If any result falls in a range rejected by the SDK's URL validator,
it rejects the request. The connector uses those same checked results to open
the socket, preventing a second DNS resolution between checking and connecting.
It also validates literal-IP URLs, which do not invoke DNS lookup.
The guarded connector resolves A and AAAA records directly through DNS. It does
not use `/etc/hosts` or DNS search domains. Explicitly trusted origins use the
runtime's normal resolver instead.
`createGuardedFetch()` returns a fetch function with a `close(): Promise<void>`
method. Reuse one instance across requests and close it at application shutdown,
after consuming or cancelling all response bodies. The example closes it after
one operation. It uses its own Undici fetch and agents; global fetch wrappers,
framework fetch caching, and global dispatcher configuration do not apply.
Automatic redirects are disabled: `redirect: 'follow'` (including the fetch
default) is treated as `'error'`. The SDK's download helpers use `'manual'` and
validate each redirect hop. When calling the transport directly, use
`fetchUntrustedUrl({ url, fetch })` for the SDK's redirect handling and header
sanitization. The transport alone does not isolate credentials or limit response
sizes.
This package requires Node.js 20.19 or later. It is not intended for browsers,
Bun, Deno, Cloudflare Workers, or framework edge runtimes. Use a transport
supported by your runtime or enforce network egress restrictions there.
### Trusted private endpoints
When connecting to a configured private provider, explicitly allow its origin:
```ts
const baseURL = 'http://127.0.0.1:8000';
const fetch = createGuardedFetch({ trustedOrigins: [baseURL] });
```
Only exact origins (scheme, hostname, and port) match; paths in entries are
ignored. Every path on a trusted origin bypasses both URL target and DNS
validation. This option does not allow other subdomains, ports, or redirect
destinations. Never derive it from provider responses, model output, or user
input. The provider's `baseURL` alone does not configure the transport's trusted
origins.
### Core file downloads
A provider's `fetch` option applies to requests made by that provider. Core file
downloads have separate hooks that all accept the function returned by
`createDownload({ fetch })`: the `download` option of `transcribe` and
`experimental_generateVideo`, and the `experimental_download` option of
`generateText`, `streamText`, `experimental_decide`, and `experimental_extract`.
`experimental_decide` and `experimental_extract` reject remote file URLs in
`state` with `DownloadError` unless `experimental_download` is supplied. Inline
bytes, base64 content, and data URLs do not require a download hook. For untrusted
URLs, supply `createDownload({ fetch })` with a transport that validates DNS and
pins the connection to the validated addresses, such as `createGuardedFetch()`
on Node.js. A model's guarded `fetch` does not protect core state downloads.
Calling `createDownload()` without a custom `fetch` explicitly opts into the
runtime's global fetch and does not provide DNS protection.
Batch download requests include the call's `abortSignal`. `createDownload`
forwards it to the transport; custom download functions should do the same so
cancellation and timeouts stop in-flight downloads.
```ts
import { createOpenAI } from '@ai-sdk/openai';
import { createGuardedFetch } from 'guarded-fetch';
import { createDownload, generateText } from 'ai';
const fetch = createGuardedFetch();
const openai = createOpenAI({ fetch });
const download = createDownload({ fetch, maxBytes: 20 * 1024 * 1024 });
try {
const result = await generateText({
model: openai('gpt-4.1'),
messages: [
{
role: 'user',
content: [
{
type: 'file',
data: new URL('https://example.com/report.pdf'),
mediaType: 'application/pdf',
},
],
},
],
experimental_download: download,
});
console.log(result.text);
} finally {
await fetch.close();
}
```
`experimental_download` also handles generated file URLs returned by the model
(for example image URLs from an image-generation response), so the same hook
covers both prompt downloads and generated-file downloads. Without it, those
downloads use the runtime's global fetch.
Do not replace `globalThis.fetch` with the guarded transport: it rejects
automatic redirects and private addresses for every library in the process,
including framework-internal requests.
URLs passed through to a model are fetched by that service and are outside the
local transport's control. Custom tools and other application fetches also need
their own transport configuration.
### MCP OAuth discovery
MCP OAuth discovery is stricter than other SDK requests. On server runtimes,
including Node.js, it blocks untrusted authorization-server hosts, redirects,
token endpoints, and registration endpoints unless you explicitly supply a
custom fetch with connect-time DNS validation, such as `createGuardedFetch()`
passed through
[`transport.fetch`](/docs/ai-sdk-core/mcp-tools#securing-remote-server-urls).
The runtime's global fetch does not satisfy this requirement, even when passed
explicitly. The configured MCP server origin and browser window discovery
remain supported. Browser Web Workers and Service Workers without a DOM also
require an explicit custom fetch under this strict policy: worker globals alone
are not sufficient to distinguish them from server workers. Direct callers of
`fetchWithValidatedEndpoint` can opt into this policy with
`requireDnsValidation: true`; other download helpers retain their existing
runtime behavior.
## Restrict outbound egress at the network layer
Network egress controls are the broadest protection: they cover requests made by
all libraries and runtimes. Enforce them at the workload's network boundary,
such as an outbound proxy, firewall, or container/network policy. Block access
to private, loopback, link-local, and cloud-metadata destinations over both IPv4
and IPv6, except explicitly required internal services. An inbound web
application firewall does not control outbound requests.
The SDK's URL checks and an optional guarded transport provide additional
application-level controls. They cannot enforce your deployment's routing,
proxy behavior, or organization-specific definition of a private network.