UNPKG

agents

Version:

A home for your AI agents

646 lines (524 loc) 23.6 kB
# Human in the Loop Human-in-the-loop (HITL) patterns allow agents to pause execution and wait for human approval, confirmation, or input before proceeding. This is essential for compliance, safety, and oversight in agentic systems. ## Overview ### Why Human in the Loop? - **Compliance**: Regulatory requirements may mandate human approval for certain actions - **Safety**: High-stakes operations (payments, deletions, external communications) need oversight - **Quality**: Human review catches errors AI might miss - **Trust**: Users feel more confident when they can approve critical actions ### Common Use Cases | Use Case | Example | | ------------------- | ---------------------------------------- | | Financial approvals | Expense reports, payment processing | | Content moderation | Publishing, email sending | | Data operations | Bulk deletions, exports | | AI tool execution | Confirming LLM tool calls before running | | Access control | Granting permissions, role changes | ## Choosing an Approach Agents SDK supports multiple human-in-the-loop patterns. Choose based on your use case: | Use Case | Pattern | Best For | Example | | ---------------------- | --------------------- | -------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | Long-running workflows | Workflow Approval | Multi-step processes, durable approval gates | [examples/workflows/](https://github.com/cloudflare/agents/tree/main/examples/workflows) | | AIChatAgent tools | `needsApproval` | Chat-based tool calls with `@cloudflare/ai-chat` | [guides/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/guides/human-in-the-loop) | | OpenAI Agents SDK | `needsApproval` | Using OpenAI's agent SDK with conditional approval | [openai-sdk/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/openai-sdk/human-in-the-loop) | | Client-side tools | `onToolCall` | Tools that need browser APIs or user interaction | Pattern below | | Stateless servers | Stateless Elicitation | Current MCP tools requesting structured input | [examples/mcp-elicitation-mrtr/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation-mrtr) | | Legacy servers | Legacy Elicitation | Existing sessionful MCP deployments | [examples/mcp-elicitation/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation) | ### Decision Guide ``` Is this part of a multi-step workflow? ├── Yes Use Workflow Approval (waitForApproval) └── No Are you building an MCP server? ├── Yes Use MCP Elicitation (elicitInput) └── No Is this an AI chat interaction? ├── Yes Does the tool need browser APIs? ├── Yes Use onToolCall (client-side execution) └── No Use needsApproval (server-side with approval) └── No Use State + WebSocket for simple confirmations ``` ## Workflow-Based Approval For durable, multi-step processes, use Cloudflare Workflows with the `waitForApproval()` helper. The workflow pauses until a human approves or rejects. ### Basic Pattern ```typescript import { Agent, AgentWorkflow, callable } from "agents"; import type { AgentWorkflowEvent, AgentWorkflowStep } from "agents"; // Workflow that pauses for approval export class ExpenseWorkflow extends AgentWorkflow< ExpenseAgent, ExpenseParams > { async run(event: AgentWorkflowEvent<ExpenseParams>, step: AgentWorkflowStep) { const expense = event.payload; // Step 1: Validate the expense const validated = await step.do("validate", async () => { return validateExpense(expense); }); // Step 2: Wait for manager approval await this.reportProgress({ step: "approval", status: "pending", message: `Awaiting approval for $${expense.amount}` }); // This pauses the workflow until approved/rejected const approval = await this.waitForApproval<{ approvedBy: string }>(step, { timeout: "7 days" }); console.log(`Approved by: ${approval.approvedBy}`); // Step 3: Process the approved expense const result = await step.do("process", async () => { return processExpense(validated); }); await step.reportComplete(result); return result; } } ``` ### Agent Methods for Approval The agent provides methods to approve or reject waiting workflows: ```typescript export class ExpenseAgent extends Agent<Env, ExpenseState> { initialState: ExpenseState = { pendingApprovals: [], status: "idle" }; // Approve a waiting workflow @callable() async approve(workflowId: string, approvedBy: string): Promise<void> { await this.approveWorkflow(workflowId, { reason: "Expense approved", metadata: { approvedBy, approvedAt: Date.now() } }); // Update state to reflect approval this.setState({ ...this.state, pendingApprovals: this.state.pendingApprovals.filter( (p) => p.workflowId !== workflowId ) }); } // Reject a waiting workflow @callable() async reject(workflowId: string, reason: string): Promise<void> { await this.rejectWorkflow(workflowId, { reason }); this.setState({ ...this.state, pendingApprovals: this.state.pendingApprovals.filter( (p) => p.workflowId !== workflowId ) }); } // Track workflow progress async onWorkflowProgress( workflowName: string, workflowId: string, progress: unknown ): Promise<void> { const p = progress as { step: string; status: string }; if (p.step === "approval" && p.status === "pending") { // Add to pending approvals list this.setState({ ...this.state, pendingApprovals: [ ...this.state.pendingApprovals, { workflowId, requestedAt: Date.now() } ] }); } } } ``` ### Timeout Handling Set timeouts to prevent workflows from waiting indefinitely: ```typescript const approval = await this.waitForApproval(step, { timeout: "7 days" // or "1 hour", "30 minutes", etc. }); ``` If the timeout expires, the workflow continues without approval data. Handle this case: ```typescript const approval = await this.waitForApproval<{ approvedBy: string }>(step, { timeout: "24 hours" }); if (!approval) { // Timeout expired - escalate or auto-reject await step.reportError("Approval timeout - escalating to manager"); throw new Error("Approval timeout"); } ``` For more details, see [Workflows Integration](./workflows.md). ## AI Tool Approval with `needsApproval` When building AI chat agents, you often want humans to approve certain tool calls before execution. The AI SDK's `needsApproval` option pauses tool execution until the user approves or rejects. ### Server Define tools with `needsApproval` to require human confirmation: ```typescript import { AIChatAgent } from "@cloudflare/ai-chat"; import { createWorkersAI } from "workers-ai-provider"; import { streamText, tool, convertToModelMessages, stepCountIs } from "ai"; import { z } from "zod"; export class MyAgent extends AIChatAgent { async onChatMessage() { const workersai = createWorkersAI({ binding: this.env.AI }); const result = streamText({ model: workersai("@cf/moonshotai/kimi-k2.7-code"), messages: await convertToModelMessages(this.messages), tools: { // Tool with conditional approval processPayment: tool({ description: "Process a payment", inputSchema: z.object({ amount: z.number(), recipient: z.string() }), // Approval required for amounts over $100 needsApproval: async ({ amount }) => amount > 100, execute: async ({ amount, recipient }) => { return await chargeCard(amount, recipient); } }), // Tool that always requires approval deleteAccount: tool({ description: "Delete a user account", inputSchema: z.object({ userId: z.string() }), needsApproval: true, execute: async ({ userId }) => { return await deleteUser(userId); } }), // Tool that executes automatically (no approval) getWeather: tool({ description: "Get weather for a city", inputSchema: z.object({ city: z.string() }), execute: async ({ city }) => fetchWeather(city) }) }, stopWhen: stepCountIs(5) }); return result.toUIMessageStreamResponse(); } } ``` The `inputSchema` accepts the AI SDK's flexible schema format, so you are not limited to Zod. You can also use Valibot, a Standard JSON Schema-compatible schema, or a raw JSON Schema wrapped with `jsonSchema()` from `ai`. See [Use Valibot or another schema library](./agent-tools.md#use-valibot-or-another-schema-library) for details. ### Client Handle approval requests with `addToolApprovalResponse`: ```tsx import { useAgent } from "agents/react"; import { useAgentChat } from "@cloudflare/ai-chat/react"; import { isToolUIPart, getToolName } from "ai"; function Chat() { const agent = useAgent({ agent: "MyAgent" }); const { messages, sendMessage, addToolApprovalResponse } = useAgentChat({ agent }); return ( <div> {messages.map((message) => ( <div key={message.id}> {message.parts?.map((part, i) => { if (part.type === "text") { return <p key={i}>{part.text}</p>; } if (isToolUIPart(part)) { // Tool waiting for approval if ("approval" in part && part.state === "approval-requested") { const approvalId = part.approval?.id; return ( <div key={part.toolCallId} className="approval-card"> <p> Approve <strong>{getToolName(part)}</strong> with{" "} {JSON.stringify(part.input)}? </p> <button onClick={() => addToolApprovalResponse({ id: approvalId, approved: true }) } > Approve </button> <button onClick={() => addToolApprovalResponse({ id: approvalId, approved: false }) } > Reject </button> </div> ); } // Tool was denied if (part.state === "output-denied") { return ( <div key={part.toolCallId}>{getToolName(part)}: Denied</div> ); } // Tool completed if (part.state === "output-available") { return ( <div key={part.toolCallId}> {getToolName(part)}: {JSON.stringify(part.output)} </div> ); } } return null; })} </div> ))} </div> ); } ``` ### Custom denial messages with `addToolOutput` When a user rejects a tool, `addToolApprovalResponse({ id, approved: false })` sets the tool state to `output-denied` with a generic "Tool execution denied." message. If you need to give the LLM a more specific reason for the denial, use `addToolOutput` with `state: "output-error"` instead: ```tsx const { addToolOutput } = useAgentChat({ agent }); // Reject with a custom error message addToolOutput({ toolCallId: part.toolCallId, state: "output-error", errorText: "User declined: insufficient budget for this quarter" }); ``` This sends a `tool_result` to the LLM with your custom error text, so it can respond appropriately (e.g. suggest an alternative, ask clarifying questions). The `addToolOutput` function also works for tools in `approval-requested` or `approval-responded` states, not just `input-available`. `addToolApprovalResponse` (with `approved: false`) auto-continues the conversation when `autoContinueAfterToolResult` is enabled (the default), so the LLM sees the denial and can respond naturally. `addToolOutput` with `state: "output-error"` does **not** auto-continue — it gives you full control over what happens next. If you want the LLM to respond to the error, call `sendMessage()` afterward. See the complete example: [guides/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/guides/human-in-the-loop) ### Surviving restarts while waiting for a human A Durable Object can be evicted at any time (a deploy, an inactivity timeout, a resource limit), including while a turn is paused on an approval prompt or a client-side tool call. Durable [`chatRecovery`](./chat-agents.md#stream-recovery) is always enabled. The SDK recognizes that such a turn is _waiting on the human_, not stuck, and does **not** seal it: the no-progress window, attempt cap, `maxRecoveryWork`, and `shouldKeepRecovering` are all suspended while the interaction is pending. Recovery parks the turn instead of failing it, and the user's eventual approval or `tool_result` resumes the conversation through the normal continuation path. A user who takes minutes to respond to a prompt that was interrupted by a deploy therefore does not see a spurious "session interrupted" error. This protection applies to interactions only the client can resolve — `approval-requested` parts and `input-available` parts for client-side tools (those without a server `execute`). A server tool whose `execute()` was killed mid-flight is a genuine orphan and recovers through the normal transcript-repair path instead. ## Client-Side Tool Execution with `onToolCall` For tools that need browser APIs (geolocation, camera, clipboard) or user interaction, define the tool on the server without an `execute` function and handle it on the client with `onToolCall`: ### Server ```typescript export class MyAgent extends AIChatAgent { async onChatMessage() { const workersai = createWorkersAI({ binding: this.env.AI }); const result = streamText({ model: workersai("@cf/moonshotai/kimi-k2.7-code"), messages: await convertToModelMessages(this.messages), tools: { // No execute function - client handles via onToolCall getUserLocation: tool({ description: "Get the user's current location from their browser", inputSchema: z.object({}) }) }, stopWhen: stepCountIs(3) }); return result.toUIMessageStreamResponse(); } } ``` ### Client ```tsx const { messages, sendMessage } = useAgentChat({ agent, onToolCall: async ({ toolCall, addToolOutput }) => { if (toolCall.toolName === "getUserLocation") { const position = await new Promise((resolve, reject) => { navigator.geolocation.getCurrentPosition(resolve, reject); }); addToolOutput({ toolCallId: toolCall.toolCallId, output: { lat: position.coords.latitude, lng: position.coords.longitude } }); } } }); ``` The server receives the tool output via `CF_AGENT_TOOL_RESULT` and can auto-continue the conversation when `stopWhen` allows another step, letting the LLM respond to the location data in the same turn. ### OpenAI Agents SDK Pattern When using the [OpenAI Agents SDK](https://openai.github.io/openai-agents-js/), use the `needsApproval` function for conditional approval: ```typescript import { Agent as CloudflareAgent } from "agents"; import { Agent as OpenAIAgent, tool, run } from "@openai/agents"; import { z } from "zod"; export class WeatherAgent extends CloudflareAgent<Env> { async processQuery(query: string) { const weatherTool = tool({ name: "get_weather", description: "Get weather for a location", parameters: z.object({ location: z.string() }), // Conditional approval - only for certain locations needsApproval: async (_context, { location }) => { return location === "San Francisco"; // Require approval for SF }, execute: async ({ location }) => { const conditions = ["sunny", "cloudy", "rainy"]; return conditions[Math.floor(Math.random() * conditions.length)]; } }); const openaiAgent = new OpenAIAgent({ name: "Weather assistant", instructions: "Help the user check the weather.", tools: [weatherTool] }); return run(openaiAgent, query); } } ``` See the complete example: [openai-sdk/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/openai-sdk/human-in-the-loop) ### MCP Elicitation **Stateless Elicitation** uses multi-round-trip requests (MRTR). A handler returns `inputRequired(...)`; the client gathers the requested input and retries with SDK-managed state. No Worker remains suspended while the user responds. ```typescript import { McpServer, acceptedContent, inputRequired } from "@modelcontextprotocol/server"; import { createMcpHandler } from "agents/mcp/server"; import { z } from "zod"; function createServer() { const server = new McpServer({ name: "my-server", version: "1.0.0" }); server.registerTool( "ask-name", { inputSchema: z.object({}) }, async (_args, context) => { const answer = acceptedContent( context.mcpReq.inputResponses, "name", z.object({ name: z.string() }) ); if (!answer) { return inputRequired({ inputRequests: { name: inputRequired.elicit({ message: "What is your name?", requestedSchema: { type: "object", properties: { name: { type: "string" } }, required: ["name"] } }) } }); } return { content: [{ type: "text", text: `Hello ${answer.name}` }] }; } ); return server; } export default { fetch(request, env, ctx) { return createMcpHandler(createServer, { legacy: "reject" })( request, env, ctx ); } } satisfies ExportedHandler; ``` The MCP client renders the JSON Schema form and the original operation remains pending from the application's perspective while the SDK completes the rounds. See the [Stateless Elicitation example](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation-mrtr). **Legacy Elicitation** in existing Legacy deployments uses pushed `elicitation/create` requests over a stateful transport. See the explicitly legacy [examples/mcp-elicitation/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation) example when retaining `McpAgent` or `createLegacyMcpHandler` with `WorkerTransport`. ## State Patterns for Approvals Track pending approvals in agent state for UI rendering and persistence: ```typescript type PendingApproval = { id: string; workflowId?: string; type: "expense" | "publish" | "delete"; description: string; amount?: number; requestedBy: string; requestedAt: number; expiresAt?: number; }; type ApprovalRecord = { id: string; approvalId: string; decision: "approved" | "rejected"; decidedBy: string; decidedAt: number; reason?: string; }; type ApprovalState = { pending: PendingApproval[]; history: ApprovalRecord[]; }; ``` ### Multi-Approver Patterns For sensitive operations requiring multiple approvers: ```typescript type MultiApproval = { id: string; requiredApprovals: number; // e.g., 2 currentApprovals: Array<{ userId: string; approvedAt: number; }>; rejections: Array<{ userId: string; rejectedAt: number; reason: string; }>; }; @callable() async approveMulti(approvalId: string, userId: string): Promise<boolean> { const approval = this.state.pending.find(p => p.id === approvalId); if (!approval) throw new Error("Approval not found"); // Add this user's approval approval.currentApprovals.push({ userId, approvedAt: Date.now() }); // Check if we have enough approvals if (approval.currentApprovals.length >= approval.requiredApprovals) { // Execute the approved action await this.executeApprovedAction(approval); return true; } this.setState({ ...this.state }); return false; // Still waiting for more approvals } ``` ## Timeouts and Escalation ### Setting Approval Timeouts ```typescript const approval = await this.waitForApproval(step, { timeout: "24 hours" }); ``` ### Escalation with Scheduling Use `schedule()` to set up escalation reminders: ```typescript @callable() async submitForApproval(request: ApprovalRequest): Promise<string> { const approvalId = crypto.randomUUID(); // Add to pending this.setState({ ...this.state, pending: [...this.state.pending, { id: approvalId, ...request }] }); // Schedule reminder after 4 hours await this.schedule( Date.now() + 4 * 60 * 60 * 1000, "sendReminder", { approvalId } ); // Schedule escalation after 24 hours await this.schedule( Date.now() + 24 * 60 * 60 * 1000, "escalateApproval", { approvalId } ); return approvalId; } ``` ## Complete Examples | Pattern | Location | Description | | --------------------- | -------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | | Workflow approval | [examples/workflows/](https://github.com/cloudflare/agents/tree/main/examples/workflows) | Multi-step task processing with approval gate | | AIChatAgent tools | [guides/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/guides/human-in-the-loop) | Chat tool approval with needsApproval + onToolCall | | OpenAI Agents SDK | [openai-sdk/human-in-the-loop/](https://github.com/cloudflare/agents/tree/main/openai-sdk/human-in-the-loop) | Conditional tool approval with modal | | Stateless Elicitation | [examples/mcp-elicitation-mrtr/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation-mrtr) | Stateless multi-round input | | Legacy Elicitation | [examples/mcp-elicitation/](https://github.com/cloudflare/agents/tree/main/examples/mcp-elicitation) | Stateful pushed input requests | For detailed API documentation, see: - [Workflows](./workflows.md) - `waitForApproval()`, `approveWorkflow()`, `rejectWorkflow()` - [MCP Servers](./mcp-servers.md) - `inputRequired()` and legacy `elicitInput()` - [Callable Methods](./callable-methods.md) - `@callable()` decorator for approval endpoints