UNPKG

ai

Version:

AI SDK by Vercel - build apps like ChatGPT, Claude, Gemini, and more with a single interface for any model using the Vercel AI Gateway or go direct to OpenAI, Anthropic, Google, or any other model provider.

939 lines (803 loc) • 33.5 kB
--- title: Chatbot Tool Usage description: Learn how to use tools with the useChat hook. --- # Chatbot Tool Usage With [`useChat`](/docs/reference/ai-sdk-ui/use-chat) and [`streamText`](/docs/reference/ai-sdk-core/stream-text), you can use tools in your chatbot application. The AI SDK supports three tool execution patterns in this context: 1. Automatically executed server-side tools 2. Automatically executed client-side tools 3. Tools that require user interaction, such as confirmation dialogs The flow is as follows: 1. The user enters a message in the chat UI. 1. The message is sent to the API route. 1. In your server-side route, the language model generates tool calls during the `streamText` call. 1. All tool calls are forwarded to the client. 1. Server-side tools are executed using their `execute` method and their results are forwarded to the client. 1. Client-side tools that should be automatically executed are handled with the `onToolCall` callback. You must call `addToolOutput` to provide the tool result. 1. Client-side tools that require user interactions can be displayed in the UI. The tool calls and results are available as tool invocation parts in the `parts` property of the last assistant message. 1. When the user interaction is done, `addToolOutput` can be used to add the tool result to the chat. 1. The chat can be configured to automatically submit when all tool results are available using `sendAutomaticallyWhen`. This triggers another iteration of this flow. The tool calls and tool executions are integrated into the assistant message as typed tool parts. A tool part is at first a tool call, and then it becomes a tool result when the tool is executed. The tool result contains all information about the tool call as well as the result of the tool execution. <Note> Tool result submission can be configured using the `sendAutomaticallyWhen` option. You can use the `lastAssistantMessageIsCompleteWithToolCalls` helper to automatically submit when all tool results are available. This simplifies the client-side code while still allowing full control when needed. </Note> ## Example In this example, we'll use three tools: - `getWeatherInformation`: An automatically executed server-side tool that returns the weather in a given city. - `askForConfirmation`: A user-interaction client-side tool that asks the user for confirmation. - `getLocation`: An automatically executed client-side tool that returns a random city. ### API route ```tsx filename='app/api/chat/route.ts' import { convertToModelMessages, createUIMessageStreamResponse, streamText, toUIMessageStream, UIMessage, } from 'ai'; __PROVIDER_IMPORT__; import { z } from 'zod'; // Allow streaming responses up to 30 seconds export const maxDuration = 30; export async function POST(req: Request) { const { messages }: { messages: UIMessage[] } = await req.json(); const result = streamText({ model: __MODEL__, messages: await convertToModelMessages(messages), tools: { // server-side tool with execute function: getWeatherInformation: { description: 'show the weather in a given city to the user', inputSchema: z.object({ city: z.string() }), execute: async ({}: { city: string }) => { const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy']; return weatherOptions[ Math.floor(Math.random() * weatherOptions.length) ]; }, }, // client-side tool that starts user interaction: askForConfirmation: { description: 'Ask the user for confirmation.', inputSchema: z.object({ message: z.string().describe('The message to ask for confirmation.'), }), }, // client-side tool that is automatically executed on the client: getLocation: { description: 'Get the user location. Always ask for confirmation before using this tool.', inputSchema: z.object({}), }, }, }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }), }); } ``` ### Client-side page The client-side page uses the `useChat` hook to create a chatbot application with real-time message streaming. Tool calls are displayed in the chat UI as typed tool parts. Please make sure to render the messages using the `parts` property of the message. There are three things worth mentioning: 1. The [`onToolCall`](/docs/reference/ai-sdk-ui/use-chat#on-tool-call) callback is used to handle client-side tools that should be automatically executed. In this example, the `getLocation` tool is a client-side tool that returns a random city. You call `addToolOutput` to provide the result (without `await` to avoid potential deadlocks). <Note> Always check `if (toolCall.dynamic)` first in your `onToolCall` handler. Without this check, TypeScript will throw an error like: `Type 'string' is not assignable to type '"toolName1" | "toolName2"'` when you try to use `toolCall.toolName` in `addToolOutput`. </Note> 2. The [`sendAutomaticallyWhen`](/docs/reference/ai-sdk-ui/use-chat#send-automatically-when) option with `lastAssistantMessageIsCompleteWithToolCalls` helper automatically submits when all tool results are available. 3. The `parts` array of assistant messages contains tool parts with typed names like `tool-askForConfirmation`. The client-side tool `askForConfirmation` is displayed in the UI. It asks the user for confirmation and displays the result once the user confirms or denies the execution. The result is added to the chat using `addToolOutput` with the `tool` parameter for type safety. Typed tool parts also include the `approval-requested`, `approval-responded`, and `output-denied` states. Include these states when handling `part.state` exhaustively, even when a tool does not require approval. See [Tool execution approval](#tool-execution-approval) for a complete approval UI. ```tsx filename='app/page.tsx' highlight="6,11,16,19-23,25-26,28-33,51,66-70,77-81" 'use client'; import { useChat } from '@ai-sdk/react'; import { DefaultChatTransport, lastAssistantMessageIsCompleteWithToolCalls, } from 'ai'; import { useState } from 'react'; export default function Chat() { const { messages, sendMessage, addToolOutput } = useChat({ transport: new DefaultChatTransport({ api: '/api/chat', }), sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls, // run client-side tools that are automatically executed: async onToolCall({ toolCall }) { // Check if it's a dynamic tool first for proper type narrowing if (toolCall.dynamic) { return; } if (toolCall.toolName === 'getLocation') { const cities = ['New York', 'Los Angeles', 'Chicago', 'San Francisco']; // No await - avoids potential deadlocks addToolOutput({ tool: 'getLocation', toolCallId: toolCall.toolCallId, output: cities[Math.floor(Math.random() * cities.length)], }); } }, }); const [input, setInput] = useState(''); return ( <> {messages?.map(message => ( <div key={message.id}> <strong>{`${message.role}: `}</strong> {message.parts.map(part => { switch (part.type) { // render text parts as simple text: case 'text': return part.text; // for tool parts, use the typed tool part names: case 'tool-askForConfirmation': { const callId = part.toolCallId; switch (part.state) { case 'input-streaming': return ( <div key={callId}>Loading confirmation request...</div> ); case 'input-available': return ( <div key={callId}> {part.input.message} <div> <button onClick={() => addToolOutput({ tool: 'askForConfirmation', toolCallId: callId, output: 'Yes, confirmed.', }) } > Yes </button> <button onClick={() => addToolOutput({ tool: 'askForConfirmation', toolCallId: callId, output: 'No, denied', }) } > No </button> </div> </div> ); case 'approval-requested': return <div key={callId}>Approval requested.</div>; case 'approval-responded': return <div key={callId}>Approval response received.</div>; case 'output-available': return ( <div key={callId}>Confirmation result: {part.output}</div> ); case 'output-error': return <div key={callId}>Error: {part.errorText}</div>; case 'output-denied': return <div key={callId}>Tool call denied.</div>; } break; } case 'tool-getLocation': { const callId = part.toolCallId; switch (part.state) { case 'input-streaming': return ( <div key={callId}>Preparing location request...</div> ); case 'input-available': return <div key={callId}>Getting location...</div>; case 'approval-requested': return <div key={callId}>Approval requested.</div>; case 'approval-responded': return <div key={callId}>Approval response received.</div>; case 'output-available': return <div key={callId}>Location: {part.output}</div>; case 'output-error': return ( <div key={callId}> Error getting location: {part.errorText} </div> ); case 'output-denied': return <div key={callId}>Location request denied.</div>; } break; } case 'tool-getWeatherInformation': { const callId = part.toolCallId; switch (part.state) { // example of pre-rendering streaming tool inputs: case 'input-streaming': return ( <pre key={callId}>{JSON.stringify(part, null, 2)}</pre> ); case 'input-available': return ( <div key={callId}> Getting weather information for {part.input.city}... </div> ); case 'approval-requested': return <div key={callId}>Approval requested.</div>; case 'approval-responded': return <div key={callId}>Approval response received.</div>; case 'output-available': return ( <div key={callId}> Weather in {part.input.city}: {part.output} </div> ); case 'output-error': return ( <div key={callId}> Error getting weather for {part.input.city}:{' '} {part.errorText} </div> ); case 'output-denied': return <div key={callId}>Weather request denied.</div>; } break; } } })} <br /> </div> ))} <form onSubmit={e => { e.preventDefault(); if (input.trim()) { sendMessage({ text: input }); setInput(''); } }} > <input value={input} onChange={e => setInput(e.target.value)} /> </form> </> ); } ``` ### Error handling Sometimes an error may occur during client-side tool execution. Use the `addToolOutput` method with a `state` of `output-error` and `errorText` value instead of `output` to record the error. ```tsx filename='app/page.tsx' highlight="19,36-41" 'use client'; import { useChat } from '@ai-sdk/react'; import { DefaultChatTransport, lastAssistantMessageIsCompleteWithToolCalls, } from 'ai'; import { useState } from 'react'; export default function Chat() { const { messages, sendMessage, addToolOutput } = useChat({ transport: new DefaultChatTransport({ api: '/api/chat', }), sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls, // run client-side tools that are automatically executed: async onToolCall({ toolCall }) { // Check if it's a dynamic tool first for proper type narrowing if (toolCall.dynamic) { return; } if (toolCall.toolName === 'getLocation') { try { const location = await getLocation(); // No await - avoids potential deadlocks addToolOutput({ tool: 'getLocation', toolCallId: toolCall.toolCallId, output: location, }); } catch (err) { addToolOutput({ tool: 'getLocation', toolCallId: toolCall.toolCallId, state: 'output-error', errorText: 'Unable to get the user location', }); } } }, }); } ``` When rendering messages, use `isToolOutputErrorUIPart` to identify failed static and dynamic tool parts without checking the tool state directly: ```tsx import { isToolOutputErrorUIPart, type UIMessage } from 'ai'; function ToolError({ part }: { part: UIMessage['parts'][number] }) { if (!isToolOutputErrorUIPart(part)) { return null; } return <div role="alert">{part.errorText}</div>; } ``` ## Tool Execution Approval Tool execution approval lets you require user confirmation before a server-side tool runs. Unlike [client-side tools](#example) that execute in the browser, tools with approval still execute on the server—but only after the user approves. Use tool execution approval when you want to: - Confirm sensitive operations (payments, deletions, external API calls) - Let users review tool inputs before execution - Add human oversight to automated workflows For tools that need to run in the browser (updating UI state, accessing browser APIs), use client-side tools instead. ### Server Setup Enable approval with `toolApproval` on `streamText`. The older `needsApproval` property on tools is deprecated. See [Tool Execution Approval](/docs/ai-sdk-core/tools-and-tool-calling#tool-execution-approval) for configuration options including dynamic approval based on input. ```tsx filename='app/api/chat/route.ts' import { createUIMessageStreamResponse, streamText, tool, toUIMessageStream, } from 'ai'; __PROVIDER_IMPORT__; import { z } from 'zod'; export async function POST(req: Request) { const { messages } = await req.json(); const result = streamText({ model: __MODEL__, messages, tools: { getWeather: tool({ description: 'Get the weather in a location', inputSchema: z.object({ city: z.string(), }), execute: async ({ city }) => { const weather = await fetchWeather(city); return weather; }, }), }, toolApproval: { getWeather: 'user-approval', }, }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }), }); } ``` ### Client-Side Approval UI When a tool requires manual approval, the tool part state is `approval-requested`. Automatic approvals and denials also flow through the same approval states, but they set `part.approval.isAutomatic === true`, so you can render the status without calling `addToolApprovalResponse`. Automatic approval decisions can also include `part.approval.reason`. ```tsx filename='app/page.tsx' 'use client'; import { useChat } from '@ai-sdk/react'; export default function Chat() { const { messages, addToolApprovalResponse } = useChat(); return ( <> {messages.map(message => ( <div key={message.id}> {message.parts.map(part => { if (part.type === 'tool-getWeather') { switch (part.state) { case 'approval-requested': { if (part.approval.isAutomatic) { return ( <div key={part.toolCallId}> Checking approval for {part.input.city}... </div> ); } return ( <div key={part.toolCallId}> <p>Get weather for {part.input.city}?</p> {part.approval.requestReason && ( <p>{part.approval.requestReason}</p> )} <button onClick={() => addToolApprovalResponse({ id: part.approval.id, approved: true, }) } > Approve </button> <button onClick={() => addToolApprovalResponse({ id: part.approval.id, approved: false, }) } > Deny </button> </div> ); } case 'approval-responded': return ( <div key={part.toolCallId}> Weather request for {part.input.city} was {part.approval.isAutomatic ? ' automatically' : ''}{' '} {part.approval.approved ? 'approved' : 'denied'}. {part.approval.reason ? ` Reason: ${part.approval.reason}` : ''} </div> ); case 'output-available': return ( <div key={part.toolCallId}> Weather in {part.input.city}: {part.output} </div> ); case 'output-denied': return ( <div key={part.toolCallId}> Weather request for {part.input.city} was denied. {part.approval.reason ? ` Reason: ${part.approval.reason}` : ''} </div> ); } } // Handle other part types... })} </div> ))} </> ); } ``` Call `addToolApprovalResponse` only for manual approvals. Automatic approval decisions already arrive in the UI stream as `approval-requested` and `approval-responded` states, and denied executions continue to `output-denied`. If you return a `reason` from an automatic approval or denial, it is available as `part.approval.reason`. For manual approval requests, the reason for requiring approval is available as `part.approval.requestReason`. It remains separate from an optional response reason supplied to `addToolApprovalResponse`. When a response reason is supplied with a denial, it is included in the `execution-denied` tool result that the model sees. When it is supplied with an approval for a tool executed by the AI SDK, it is recorded on the message part, but the SDK does not include it in the model prompt, modify the tool's input, or expose it through a dedicated execution option. To collect input from the user for a tool, use a client-side tool and `addToolOutput` instead. Approval request chunks can also include an `approvalDescriptor` with opaque application-specific metadata. UI message processing exposes it as `part.approval.descriptor` in the `approval-requested` state and preserves it in subsequent approval-bearing states, including `approval-responded`. This lets clients render or persist server-computed approval metadata without using it to determine whether the tool was approved. ### Securing Approvals for Sensitive Tools In the `useChat` pattern, the client sends the full message history to the server each turn. Without additional protection, a modified client could fabricate an approval response. For tools that perform sensitive operations, add `experimental_toolApprovalSecret` to your `streamText` call so the server cryptographically verifies that it issued the approval: ```tsx filename='app/api/chat/route.ts' highlight="6" const result = streamText({ model: __MODEL__, messages, tools: { deleteFile }, toolApproval: { deleteFile: 'user-approval' }, experimental_toolApprovalSecret: process.env.TOOL_APPROVAL_SECRET, }); ``` See [Security Considerations](/docs/agents/tool-approvals#security-considerations) for setup details. ### Auto-Submit After Approval <Note> If nothing happens after you approve a tool execution, make sure you either call `sendMessage` manually or configure `sendAutomaticallyWhen` on the `useChat` hook. </Note> Use `lastAssistantMessageIsCompleteWithApprovalResponses` to automatically continue the conversation after approvals: ```tsx import { useChat } from '@ai-sdk/react'; import { lastAssistantMessageIsCompleteWithApprovalResponses } from 'ai'; const { messages, addToolApprovalResponse } = useChat({ sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithApprovalResponses, }); ``` ## Dynamic Tools When using dynamic tools (tools with unknown types at compile time), the UI parts use a generic `dynamic-tool` type instead of specific tool types: ```tsx filename='app/page.tsx' { message.parts.map((part, index) => { switch (part.type) { // Static tools with specific (`tool-${toolName}`) types case 'tool-getWeatherInformation': return <WeatherDisplay part={part} />; // Dynamic tools use generic `dynamic-tool` type case 'dynamic-tool': return ( <div key={index}> <h4>Tool: {part.toolName}</h4> {part.state === 'input-streaming' && ( <pre>{JSON.stringify(part.input, null, 2)}</pre> )} {part.state === 'output-available' && ( <pre>{JSON.stringify(part.output, null, 2)}</pre> )} {part.state === 'output-error' && ( <div>Error: {part.errorText}</div> )} </div> ); } }); } ``` Dynamic tools are useful when integrating with: - MCP (Model Context Protocol) tools without schemas - User-defined functions loaded at runtime - External tool providers ## Tool call streaming Tool call streaming is **enabled by default** in AI SDK 5.0, allowing you to stream tool calls while they are being generated. This provides a better user experience by showing tool inputs as they are generated in real-time. ```tsx filename='app/api/chat/route.ts' export async function POST(req: Request) { const { messages }: { messages: UIMessage[] } = await req.json(); const result = streamText({ model: __MODEL__, messages: await convertToModelMessages(messages), // toolCallStreaming is enabled by default in v5 // ... }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }), }); } ``` With tool call streaming enabled, partial tool calls are streamed as part of the data stream. They are available through the `useChat` hook. The typed tool parts of assistant messages will also contain partial tool calls. You can use the `state` property of the tool part to render the correct UI. ```tsx filename='app/page.tsx' highlight="9,10" export default function Chat() { // ... return ( <> {messages?.map(message => ( <div key={message.id}> {message.parts.map(part => { switch (part.type) { case 'tool-askForConfirmation': case 'tool-getLocation': case 'tool-getWeatherInformation': switch (part.state) { case 'input-streaming': return <pre>{JSON.stringify(part.input, null, 2)}</pre>; case 'input-available': return <pre>{JSON.stringify(part.input, null, 2)}</pre>; case 'approval-requested': return <div>Approval requested.</div>; case 'approval-responded': return <div>Approval response received.</div>; case 'output-available': return <pre>{JSON.stringify(part.output, null, 2)}</pre>; case 'output-error': return <div>Error: {part.errorText}</div>; case 'output-denied': return <div>Tool call denied.</div>; } } })} </div> ))} </> ); } ``` ## Step start parts When you are using multi-step tool calls, the AI SDK will add step start parts to the assistant messages. If you want to display boundaries between tool calls, you can use the `step-start` parts as follows: ```tsx filename='app/page.tsx' // ... // where you render the message parts: message.parts.map((part, index) => { switch (part.type) { case 'step-start': // show step boundaries as horizontal lines: return index > 0 ? ( <div key={index} className="text-gray-500"> <hr className="my-2 border-gray-300" /> </div> ) : null; case 'text': // ... case 'tool-askForConfirmation': case 'tool-getLocation': case 'tool-getWeatherInformation': // ... } }); // ... ``` ## Server-side Multi-Step Calls You can also use multi-step calls on the server-side with `streamText`. This works when all invoked tools have an `execute` function on the server side. ```tsx filename='app/api/chat/route.ts' highlight="22-28,31" import { convertToModelMessages, createUIMessageStreamResponse, isStepCount, streamText, toUIMessageStream, UIMessage, } from 'ai'; __PROVIDER_IMPORT__; import { z } from 'zod'; export async function POST(req: Request) { const { messages }: { messages: UIMessage[] } = await req.json(); const result = streamText({ model: __MODEL__, messages: await convertToModelMessages(messages), tools: { getWeatherInformation: { description: 'show the weather in a given city to the user', inputSchema: z.object({ city: z.string() }), // tool has execute function: execute: async ({}: { city: string }) => { const weatherOptions = ['sunny', 'cloudy', 'rainy', 'snowy', 'windy']; return weatherOptions[ Math.floor(Math.random() * weatherOptions.length) ]; }, }, }, stopWhen: isStepCount(5), }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream }), }); } ``` ## Errors Language models can make errors when calling tools, and tools can fail while they run. By default, the client receives the same error text for a failed tool call as the model. This applies to tool execution errors and to invalid tool calls. The UI messages are sent back to the model on the next request, so they keep the text that the model's reasoning and answers are based on. <Note type="warning"> The error text of a failed tool call can contain details from your server, such as host names or file paths. The model receives this text and can repeat it in its answer. To keep such details away from the model and the client, catch the error in the tool's `execute` function and throw an error with a safe message. </Note> ```tsx const weatherTool = tool({ description: 'Get the weather in a location', inputSchema: z.object({ location: z.string() }), execute: async ({ location }) => { try { return await getWeather(location); } catch (error) { console.error(error); // the details stay on the server // the model and the client receive this message: throw new Error('The weather service is unavailable.'); } }, }); ``` The `execute` function does not run for an invalid tool call. Its error text contains the input that the model generated, the validation messages of the input schema, or the names of the available tools. Only [`redactToolErrors`](#redacting-tool-errors) replaces this text. Tool execution errors marked `providerExecuted: true` are also sent unchanged. String errors pass through; other error values are serialized with `JSON.stringify`. The original error data is needed when converting persisted UI messages back into model messages, for example to preserve Anthropic tool error codes. Harness runtime tool errors also retain their original messages. ### Redacting tool errors To replace the error text of failed tool calls before it is sent to the client, set `redactToolErrors` to `true`. The text then comes from the `onError` callback, which returns "An error occurred." by default: ```tsx const result = streamText({ // ... }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream, redactToolErrors: true, }), }); ``` When you pass your own `onError`, it handles failed tool calls as well, unless you set `redactToolErrors` to `false`. Tool execution errors marked `providerExecuted: true` are never redacted. Invalid tool calls of provider-executed tools always use `onError`. Redacting tool errors changes the message history. See [Tool error text and message history](#tool-error-text-and-message-history) before you enable it. ### Tool error text and message history During a [server-side multi-step call](#server-side-multi-step-calls), the model receives the full error text of a failed tool call. When tool errors are redacted, the UI receives a different text. The tool result that `convertToModelMessages` creates from the stored UI messages on the next request is then not the tool result that the model received. Some models bind their reasoning to the messages before it and reject a request when those messages have changed. For example, Anthropic models with [preserved thinking](/providers/ai-sdk-providers/anthropic#thinking-binding-controls) respond with a 400 error, and every later request with these messages fails the same way. The AI SDK logs a warning when a UI message stream sends a tool error text that differs from the text the model received and then sends reasoning from a later step. If you redact tool errors, use one of these options to avoid the rejected request: - Set `sendReasoning: false` so that the UI messages contain no reasoning to send back to the model. - With Anthropic models that use adaptive thinking, set `thinking.blockBinding.prefixMismatchBehavior` to `drop_block`. The API then drops the reasoning that no longer matches instead of rejecting the request. With both options, the model reads the redacted error text on later requests, not the text it received when the tool call failed. ### Stream errors Errors that are not tied to a tool call, such as a failed request to the model provider, never reach the model. They are masked for security reasons and show up as "An error occurred." in the UI. To customize these errors, pass `onError` to `toUIMessageStream`: ```tsx export function errorHandler(error: unknown) { if (error == null) { return 'unknown error'; } if (typeof error === 'string') { return error; } if (error instanceof Error) { // matches the error text that the model receives for a failed tool call return error.toString(); } return JSON.stringify(error); } ``` ```tsx const result = streamText({ // ... }); return createUIMessageStreamResponse({ stream: toUIMessageStream({ stream: result.stream, onError: errorHandler, }), }); ``` A custom `onError` also handles failed tool calls, so this handler returns the same error text that the model receives. Set `redactToolErrors: false` if your `onError` should handle stream errors only. In case you are using `createUIMessageResponse`, you can use the `onError` function when calling `toUIMessageResponse`: ```tsx const response = createUIMessageResponse({ // ... async execute(dataStream) { // ... }, onError: error => `Custom error: ${error.message}`, }); ```