ai/docs/03-ai-sdk-core/38-video-generation.mdx

AI SDK by Vercel - build apps like ChatGPT, Claude, Gemini, and more with a single interface for any model using the Vercel AI Gateway or go direct to OpenAI, Anthropic, Google, or any other model provider.

---
title: Video Generation
description: Learn how to generate videos with the AI SDK.
---

# Video Generation

<Note>
  Video generation is an experimental feature. The API may change in future
  versions.
</Note>

The AI SDK provides the [`experimental_generateVideo`](/docs/reference/ai-sdk-core/generate-video)
function to generate videos based on a given prompt using a video model.

```tsx
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A cat walking on a treadmill',
});
```

You can access the video data using the `base64` or `uint8Array` properties:

```tsx
const base64 = video.base64; // base64 video data
const uint8Array = video.uint8Array; // Uint8Array video data
```

## Settings

### Aspect Ratio

The aspect ratio is specified as a string in the format `{width}:{height}`.
Models only support a few aspect ratios, and the supported aspect ratios are different for each model and provider.

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A cat walking on a treadmill',
  aspectRatio: '16:9',
});
```

Some models also accept `'adaptive'`, which lets the provider derive the output
ratio from the input media instead of a fixed value. This is typically required
for image-to-video, video editing, and video extension, where the output
inherits the ratio of the input and an explicit ratio is rejected.

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: { image: firstFrame, text: 'A cat walking on a treadmill' },
  aspectRatio: 'adaptive',
});
```

### Resolution

The resolution is specified as a string in the format `{width}x{height}`.
Models only support specific resolutions, and the supported resolutions are different for each model and provider.

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A serene mountain landscape at sunset',
  resolution: '1280x720',
});
```

### Duration

Some video models support specifying the duration of the generated video in seconds.

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A timelapse of clouds moving across the sky',
  duration: 5,
});
```

### Frames Per Second (FPS)

Some video models allow you to specify the frames per second for the generated video.

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A hummingbird in slow motion',
  fps: 24,
});
```

### Audio Generation

Some video models can generate audio alongside the video. Use the `generateAudio` option to control this:

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A jazz band playing in a cozy club',
  generateAudio: true,
});
```

### Generating Multiple Videos

`experimental_generateVideo` supports generating multiple videos at once:

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { videos } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A rocket launching into space',
  n: 3, // number of videos to generate
});
```

<Note>
  `experimental_generateVideo` will automatically call the model as often as
  needed (in parallel) to generate the requested number of videos.
</Note>

Each video model has an internal limit on how many videos it can generate in a single API call. The AI SDK manages this automatically by batching requests appropriately when you request multiple videos using the `n` parameter. Most video models only support generating 1 video per call due to computational cost.

If needed, you can override this behavior using the `maxVideosPerCall` setting:

```tsx
const { videos } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A rocket launching into space',
  maxVideosPerCall: 2, // Override the default batch size
  n: 4, // Will make 2 calls of 2 videos each
});
```

### Image-to-Video Generation

Some video models support generating videos from an input image. You can provide an image using the prompt object:

```tsx highlight={"6-9"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: {
    image: 'https://example.com/my-image.png',
    text: 'Animate this image with gentle motion',
  },
});
```

You can also provide the image as a base64-encoded string or `Uint8Array`:

```tsx
const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: {
    image: imageBase64String, // or imageUint8Array
    text: 'Animate this image',
  },
});
```

### First and Last Frame

Some video models support first-last-frame generation, where you provide the
starting and/or ending frames of the video. Use the `frameImages` option to pass
role-tagged images in a provider-agnostic way:

```tsx highlight={"4-13"}
const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'The cat walks across the scene and transforms into a dog by the end',
  frameImages: [
    {
      image: 'https://example.com/first-frame.png',
      frameType: 'first_frame',
    },
    {
      image: 'https://example.com/last-frame.png',
      frameType: 'last_frame',
    },
  ],
});
```

### Reference Inputs

Some video models support reference-to-video generation, where you provide one or
more reference images or videos that the model incorporates into the generated video. Use the `inputReferences` option to pass
these inputs in a provider-agnostic way:

```tsx highlight={"4-7"}
const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'The two characters meet in a bustling market',
  inputReferences: [
    'https://example.com/character-1.png',
    'https://example.com/character-2.png',
  ],
});
```

For URL-based video references, use the object form with an explicit `mediaType`:

```tsx highlight={"4-9"}
const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'Match the motion in the reference clip',
  inputReferences: [
    {
      data: 'https://example.com/reference.mp4',
      mediaType: 'video/mp4',
    },
  ],
});
```

Providers route each reference by its media type (image vs. video) and emit a
warning when a reference kind is unsupported (for example, providers that accept
only image references warn and ignore a video reference).

### Providing a Seed

You can provide a seed to the `experimental_generateVideo` function to control the output of the video generation process.
If supported by the model, the same seed will always produce the same video.

```tsx highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A cat walking on a treadmill',
  seed: 1234567890,
});
```

### Provider-specific Settings

Video models often have provider- or even model-specific settings.
You can pass such settings to the `experimental_generateVideo` function
using the `providerOptions` parameter. The options for the provider
become request body properties.

```tsx highlight={"8-10"}
import { experimental_generateVideo as generateVideo } from 'ai';
import { fal } from '@ai-sdk/fal';

const { video } = await generateVideo({
  model: fal.video('luma-dream-machine/ray-2'),
  prompt: 'A cat walking on a treadmill',
  aspectRatio: '16:9',
  providerOptions: {
    fal: { loop: true, motionStrength: 0.8 },
  },
});
```

### Abort Signals and Timeouts

`experimental_generateVideo` accepts an optional `abortSignal` parameter of
type [`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal)
that you can use to abort the video generation process or set a timeout.

```ts highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A cat walking on a treadmill',
  abortSignal: AbortSignal.timeout(60000), // Abort after 60 seconds
});
```

<Note>
  Video generation typically takes longer than image generation. Consider using
  longer timeouts (60 seconds or more) depending on the model and video length.
</Note>

### Polling

Video generation is an asynchronous process that can take several minutes to complete. The SDK automatically polls the provider to check if the video is ready. You can configure the polling behavior:

```tsx highlight={"8-11"}
import { experimental_generateVideo as generateVideo } from 'ai';
import { fal } from '@ai-sdk/fal';

const { video } = await generateVideo({
  model: fal.video('luma-dream-machine/ray-2'),
  prompt: 'A cinematic timelapse of a city from dawn to dusk',
  duration: 10,
  poll: {
    intervalMs: 5000, // Check every 5 seconds (default)
    timeoutMs: 600000, // Timeout after 10 minutes (default)
  },
});
```

For durable workflows that provide their own sleep primitive, pass it as
`poll.delay`. The custom delay is used for both polling intervals and webhook
timeouts.

### Webhooks

For models with native webhook support, pass a `webhook` factory that returns a
public URL and a promise that resolves when your application receives the
webhook request:

```tsx highlight={"8-15"}
import { fal } from '@ai-sdk/fal';
import { experimental_generateVideo as generateVideo } from 'ai';
import { createWebhook } from './create-webhook';

const { video } = await generateVideo({
  model: fal.video('luma-dream-machine/ray-2'),
  prompt: 'A cinematic timelapse of a city from dawn to dusk',
  poll: {
    timeoutMs: 600000, // Wait up to 10 minutes for the webhook
  },
  webhook: async () => {
    const { url, received } = await createWebhook();
    return { url, received };
  },
});
```

`createWebhook` is an application-specific helper that registers a webhook
listener before returning. Its `received` promise must resolve with the request
`headers` and `body`. The SDK sends `url` to the provider, waits for `received`,
and then retrieves the completed video.

You can provide `poll` together with `webhook`. For models with native webhook
support, `poll.timeoutMs` limits how long the SDK waits for the notification.
If the model does not support webhooks, the SDK falls back to polling with the
provided interval and timeout.

### Custom Headers

`experimental_generateVideo` accepts an optional `headers` parameter of type `Record<string, string>`
that you can use to add custom headers to the video generation request.

```ts highlight={"7"}
import { experimental_generateVideo as generateVideo } from 'ai';
__PROVIDER_IMPORT__;

const { video } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A cat walking on a treadmill',
  headers: { 'X-Custom-Header': 'custom-value' },
});
```

### Warnings

If the model returns warnings, e.g. for unsupported parameters, they will be available in the `warnings` property of the response.

```tsx
const { video, warnings } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A cat walking on a treadmill',
});
```

### Additional Provider-specific Metadata

Some providers expose additional metadata for the result overall or per video.

```tsx
const prompt = 'A cat walking on a treadmill';

const { video, providerMetadata } = await generateVideo({
  model: fal.video('luma-dream-machine/ray-2'),
  prompt,
});

// Access provider-specific metadata
const videoMetadata = providerMetadata.fal?.videos[0];
console.log({
  duration: videoMetadata?.duration,
  fps: videoMetadata?.fps,
  width: videoMetadata?.width,
  height: videoMetadata?.height,
});
```

The outer key of the returned `providerMetadata` is the provider name. The inner values are the metadata. A `videos` key is typically present in the metadata and is an array with the same length as the top level `videos` key.

When generating multiple videos with `n > 1`, you can also access per-call metadata through the `responses` array:

```tsx
const { videos, responses } = await generateVideo({
  model: __VIDEO_MODEL__,
  prompt: 'A rocket launching into space',
  n: 5, // May require multiple API calls
});

// Access metadata from each individual API call
for (const response of responses) {
  console.log({
    timestamp: response.timestamp,
    modelId: response.modelId,
    // Per-call provider metadata (lossless)
    providerMetadata: response.providerMetadata,
  });
}
```

### Error Handling

When `experimental_generateVideo` cannot generate a valid video, it throws a [`AI_NoVideoGeneratedError`](/docs/reference/ai-sdk-errors/ai-no-video-generated-error).

This error occurs when the AI provider fails to generate a video. It can arise due to the following reasons:

- The model failed to generate a response
- The model generated a response that could not be parsed

The error preserves the following information to help you log the issue:

- `responses`: Metadata about the video model responses, including timestamp, model, and headers.
- `cause`: The cause of the error. You can use this for more detailed error handling

```ts
import {
  experimental_generateVideo as generateVideo,
  NoVideoGeneratedError,
} from 'ai';

try {
  await generateVideo({ model, prompt });
} catch (error) {
  if (NoVideoGeneratedError.isInstance(error)) {
    console.log('NoVideoGeneratedError');
    console.log('Cause:', error.cause);
    console.log('Responses:', error.responses);
  }
}
```

## Video Models

| Provider                                                                | Model                       | Features                                                                      |
| ----------------------------------------------------------------------- | --------------------------- | ----------------------------------------------------------------------------- |
| [FAL](/providers/ai-sdk-providers/fal#video-models)                     | `luma-dream-machine/ray-2`  | Text-to-video, image-to-video                                                 |
| [FAL](/providers/ai-sdk-providers/fal#video-models)                     | `minimax-video`             | Text-to-video                                                                 |
| [Google](/providers/ai-sdk-providers/google#video-models)               | `veo-2.0-generate-001`      | Text-to-video, up to 4 videos per call                                        |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#video-models) | `veo-3.1-generate-001`      | Text-to-video, audio generation                                               |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#video-models) | `veo-3.1-fast-generate-001` | Text-to-video, audio generation                                               |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#video-models) | `veo-3.0-generate-001`      | Text-to-video, audio generation                                               |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#video-models) | `veo-3.0-fast-generate-001` | Text-to-video, audio generation                                               |
| [Google Vertex](/providers/ai-sdk-providers/google-vertex#video-models) | `veo-2.0-generate-001`      | Text-to-video, up to 4 videos per call                                        |
| [Kling AI](/providers/ai-sdk-providers/klingai#video-models)            | `kling-v2.6-t2v`            | Text-to-video                                                                 |
| [Kling AI](/providers/ai-sdk-providers/klingai#video-models)            | `kling-v2.6-i2v`            | Image-to-video                                                                |
| [Kling AI](/providers/ai-sdk-providers/klingai#video-models)            | `kling-v2.6-motion-control` | Motion control                                                                |
| [Replicate](/providers/ai-sdk-providers/replicate#video-models)         | `minimax/video-01`          | Text-to-video                                                                 |
| [xAI](/providers/ai-sdk-providers/xai#video-models)                     | `grok-imagine-video`        | Text-to-video, image-to-video, editing, extension, R2V                        |
| [xAI](/providers/ai-sdk-providers/xai#video-models)                     | `grok-imagine-video-1.5`    | Text-to-video, image-to-video, editing, extension, R2V (with reference audio) |

Above are a small subset of the video models supported by the AI SDK providers. For more, see the respective provider documentation.