@flatfile/improv/README.md

A powerful TypeScript library for building AI agents with multi-threaded conversations, tool execution, and event handling capabilities

# @flatfile/improv

A powerful TypeScript library for building AI agents with multi-threaded conversations, tool execution, and event handling capabilities. Built on top of AWS Bedrock (Claude) with support for advanced features like attachments, tool calling, and event streaming.

## Core Features

- 🤖 **AI Agent Management**: Create and manage AI agents with knowledge bases, instructions, and memory
- 🧵 **Multi-threaded Conversations**: Handle parallel conversation threads with state management
- 🛠️ **Tool Integration**: Define and execute custom tools with type-safe parameters using Zod schemas
- 📎 **Attachment Support**: Handle various types of attachments (documents, images, videos)
- 🔄 **Event Streaming**: Built-in event system for real-time monitoring and response handling
- 💾 **Memory Management**: Track and persist conversation history and agent state
- 🔧 **AWS Bedrock Integration**: Native support for AWS Bedrock (Claude) with optimized prompting

## Table of Contents

- [Quick Start](#quick-start)
- [Core Components](#core-components)
  - [Agent](#agent)
  - [Thread](#thread)
  - [Tool](#tool)
  - [Message](#message)
- [Event System](#event-system)
- [Best Practices](#best-practices)
- [License](#license)
- [Contributing](#contributing)
- [Follow-up Messages](#follow-up-messages)
- [State Management](#state-management)
  - [Agent State](#agent-state)
  - [Thread State](#thread-state)
  - [Tool State](#tool-state)
  - [Evaluator System](#evaluator-system)
- [State Management Patterns](#state-management-patterns)
  - [Three-Keyed Lock Pattern](#three-keyed-lock-pattern)
- [AWS Bedrock Integration](#aws-bedrock-integration)
- [Model Drivers](#model-drivers)
- [Tool Decorators](#tool-decorators)
- [Streaming Support](#streaming-support)
- [Reasoning Capabilities](#reasoning-capabilities)

## Quick Start

```typescript
import { Agent, Tool, BedrockThreadDriver } from '@flatfile/improv';
import { z } from 'zod';

// Create a custom tool
const calculatorTool = new Tool({
  name: 'calculator',
  description: 'Performs basic arithmetic operations',
  parameters: z.object({
    operation: z.enum(['add', 'subtract', 'multiply', 'divide']),
    a: z.number(),
    b: z.number(),
  }),
  executeFn: async (args) => {
    const { operation, a, b } = args;
    switch (operation) {
      case 'add': return a + b;
      case 'subtract': return a - b;
      case 'multiply': return a * b;
      case 'divide': return a / b;
    }
  }
});

// Initialize the Bedrock driver
const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-haiku-20240307-v1:0',
  temperature: 0.7,
});

// Create an agent
const agent = new Agent({
  knowledge: [
    { fact: 'The agent can perform basic arithmetic operations.' }
  ],
  instructions: [
    { instruction: 'Use the calculator tool for arithmetic operations.', priority: 1 }
  ],
  tools: [calculatorTool],
  driver,
});

// Create and use a thread
const thread = agent.createThread({
  prompt: 'What is 25 multiplied by 4?',
  onResponse: async (message) => {
    console.log('Agent response:', message.content);
  }
});

// Send the thread
await thread.send();

// Stream the response
const stream = await thread.stream();
for await (const text of stream) {
  process.stdout.write(text); // Print each chunk as it arrives
}
```

## Reasoning Capabilities

Improv supports advanced reasoning capabilities through the `reasoning_config` option in the thread driver. This allows the AI to perform step-by-step reasoning before providing a final answer.

```typescript
import { Agent, BedrockThreadDriver } from '@flatfile/improv';

const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-7-sonnet-20250219-v1:0',
  temperature: 1,
  reasoning_config: {
    budget_tokens: 1024,
    type: 'enabled',
  },
});

const agent = new Agent({
  driver,
});

const thread = agent.createThread({
  systemPrompt: 'You are a helpful assistant that can answer questions about the world.',
  prompt: 'How many people will live in the world in 2040?',
});

const result = await thread.send();

console.log(result.last());
```

This example enables the AI to work through its reasoning process with a token budget of 1024 tokens before providing a final answer about population projections.

## Core Components

### Agent

The main agent class that manages knowledge, instructions, tools, and conversation threads.

```typescript
const agent = new Agent({
  knowledge?: AgentKnowledge[],  // Array of facts with optional source and timestamp
  instructions?: AgentInstruction[],  // Array of prioritized instructions
  memory?: AgentMemory[],  // Array of stored thread histories
  systemPrompt?: string,  // Base system prompt
  tools?: Tool[],  // Array of available tools
  driver: ThreadDriver,  // Thread driver implementation
  evaluators?: Evaluator[]  // Array of evaluators for response processing
});
```

### Thread

Manages a single conversation thread with message history and tool execution.

```typescript
const thread = new Thread({
  messages?: Message[],  // Array of conversation messages
  tools?: Tool[],  // Array of available tools
  driver: ThreadDriver,  // Thread driver implementation
  toolChoice?: 'auto' | 'any',  // Tool selection mode
  maxSteps?: number  // Maximum number of tool execution steps
});
```

### Tool

Define custom tools that the agent can use during conversations.

```typescript
const tool = new Tool({
  name: string,  // Tool name
  description: string,  // Tool description
  parameters: z.ZodTypeAny,  // Zod schema for parameter validation
  followUpMessage?: string,  // Optional message to guide response evaluation
  executeFn: (args: Record<string, any>, toolCall: ToolCall) => Promise<any>  // Tool execution function
});
```

### Message

Represents a single message in a conversation thread.

```typescript
const message = new Message({
  content?: string,  // Message content
  role: 'system' | 'user' | 'assistant' | 'tool',  // Message role
  toolCalls?: ToolCall[],  // Array of tool calls
  toolResults?: ToolResult[],  // Array of tool results
  attachments?: Attachment[],  // Array of attachments
  cache?: boolean  // Whether to cache the message
});
```

## Event System

The library uses an event-driven architecture. All major components extend `EventSource`, allowing you to listen for various events:

```typescript
// Agent events
agent.on('agent.thread-added', ({ agent, thread }) => {});
agent.on('agent.thread-removed', ({ agent, thread }) => {});
agent.on('agent.knowledge-added', ({ agent, knowledge }) => {});
agent.on('agent.instruction-added', ({ agent, instruction }) => {});

// Thread events
thread.on('thread.response', ({ thread, message }) => {});
thread.on('thread.max_steps_reached', ({ thread, steps }) => {});

// Tool events
tool.on('tool.execution.started', ({ tool, name, args }) => {});
tool.on('tool.execution.completed', ({ tool, name, args, result }) => {});
tool.on('tool.execution.failed', ({ tool, name, args, error }) => {});
```

## Best Practices

1. **Tool Design**
   - Keep tools atomic and focused on a single responsibility
   - Use Zod schemas for robust parameter validation
   - Implement proper error handling in tool execution
   - Use follow-up messages to guide response evaluation

2. **Thread Management**
   - Set appropriate `maxSteps` to prevent infinite loops (default: 30)
   - Use `onResponse` handlers for processing responses
   - Clean up threads using `closeThread()` when done
   - Monitor thread events for debugging and logging

3. **Agent Management**
   - Prioritize instructions using the priority field
   - Use knowledge facts to provide context
   - Implement evaluators for complex workflows
   - Leverage the memory system for persistence

4. **Event Handling**
   - Subscribe to relevant events for monitoring and logging
   - Use event data for analytics and debugging
   - Implement proper error handling in event listeners
   - Forward events with appropriate context

## License

MIT

## Contributing

Contributions are welcome! Please read our contributing guidelines for details.

## Follow-up Messages

Tools can include follow-up messages that guide the AI's evaluation of tool responses. This is particularly useful for:

- Providing context for tool results
- Guiding the AI's interpretation of data
- Maintaining consistent response patterns
- Suggesting next steps or actions

```typescript
const tool = new Tool({
  name: 'dataAnalyzer',
  description: 'Analyzes data and returns insights',
  parameters: z.object({
    data: z.array(z.any()),
    metrics: z.array(z.string())
  }),
  followUpMessage: `Review the analysis results:
1. What are the key insights from the data?
2. Are there any concerning patterns?
3. What actions should be taken based on these results?`,
  executeFn: async (args) => {
    // Tool implementation
  }
});
```

## State Management

The library provides several mechanisms for managing state:

### Agent State
- Knowledge base for storing facts
- Prioritized instructions for behavior guidance
- Memory system for storing thread histories
- System prompt for base context

### Thread State
- Message history tracking
- Tool execution state
- Maximum step limits
- Response handlers

### Tool State
- Parameter validation
- Execution tracking
- Result processing
- Event emission

### Evaluator System

Evaluators provide a way to process and validate agent responses:

```typescript
const evaluator: Evaluator = async ({ thread, agent }, complete) => {
  // Process the thread response
  const lastMessage = thread.last();
  
  if (lastMessage?.content.includes('done')) {
    complete();  // Signal completion
  } else {
    // Continue processing
    thread.send(new Message({
      content: 'Please continue with the task...'
    }));
  }
};

const agent = new Agent({
  // ... other options ...
  evaluators: [evaluator]
});
```

Evaluators can:
- Process agent responses
- Trigger additional actions
- Control conversation flow
- Validate results

## State Management Patterns

### Three-Keyed Lock Pattern

The three-keyed lock pattern is a state management pattern that ensures controlled flow through tool execution, evaluation, and completion phases. It's implemented as a reusable evaluator:

```typescript
import { threeKeyedLockEvaluator } from '@flatfile/improv';

const agent = new Agent({
  // ... other options ...
  evaluators: [
    threeKeyedLockEvaluator({
      evalPrompt: "Are there other items to process? If not, say 'done'",
      exitPrompt: "Please provide a final summary of all actions taken."
    })
  ]
});
```

The pattern works through three distinct states:

```mermaid
stateDiagram-v2
    [*] --> ToolExecution
    
    state "Tool Execution" as ToolExecution {
        [*] --> Running
        Running --> Complete
        Complete --> [*]
    }
    
    state "Evaluation" as Evaluation {
        [*] --> CheckMore
        CheckMore --> [*]
    }
    
    state "Summary" as Summary {
        [*] --> Summarize
        Summarize --> [*]
    }
    
    ToolExecution --> Evaluation: Non-tool response
    Evaluation --> ToolExecution: Tool called
    Evaluation --> Summary: No more items
    Summary --> [*]: Complete

    note right of ToolExecution
        isEvaluatingTools = true
        Handles tool execution
    end note
    
    note right of Evaluation
        isEvaluatingTools = false
        nextMessageIsSummary = false
        Checks for more work
    end note
    
    note right of Summary
        nextMessageIsSummary = true
        Gets final summary
    end note
```

The evaluator manages these states through:

1. **Tool Execution State**
   - Tracks when tools are being executed
   - Resets state when new tools are called
   - Handles multiple tool executions

2. **Evaluation State**
   - Triggered after tool completion
   - Prompts for more items to process
   - Can return to tool execution if needed

3. **Summary State**
   - Final state before completion
   - Gathers summary of actions
   - Signals completion when done

Key features:
- Automatic state transitions
- Event-based flow control
- Clean event listener management
- Configurable prompts
- Support for multiple tool executions

Example usage with custom prompts:

```typescript
const workflowAgent = new Agent({
  // ... agent configuration ...
  evaluators: [
    threeKeyedLockEvaluator({
      evalPrompt: "Review the results. Should we process more items?",
      exitPrompt: "Provide a detailed summary of all processed items."
    })
  ]
});

// The evaluator will automatically:
// 1. Let tools execute freely
// 2. After each tool completion, check if more processing is needed
// 3. When no more items need processing, request a final summary
// 4. Complete the evaluation after receiving the summary
```

This pattern is particularly useful for:
- Processing multiple items sequentially
- Workflows requiring validation between steps
- Tasks with dynamic tool usage
- Operations requiring final summaries

## AWS Bedrock Integration

The library uses AWS Bedrock (Claude) as its default LLM provider. Configure your AWS credentials:

```typescript
// Required environment variables
process.env.AWS_ACCESS_KEY_ID = 'your-access-key';
process.env.AWS_SECRET_ACCESS_KEY = 'your-secret-key';
process.env.AWS_REGION = 'your-region';

// Initialize the driver
const driver = new BedrockThreadDriver({
  model: 'anthropic.claude-3-haiku-20240307-v1:0',  // Default model
  temperature?: number,  // Default: 0.7
  maxTokens?: number,  // Default: 4096
  cache?: boolean  // Default: false
});
```

## Model Drivers

Improv supports multiple LLM providers through dedicated thread drivers:

### Available Drivers

| Driver | Provider | Documentation |
|--------|----------|---------------|
| `BedrockThreadDriver` | AWS Bedrock (Claude) | [Bedrock Driver Documentation](src/model.drivers/bedrock.driver.md) |
| `OpenAIThreadDriver` | OpenAI | [OpenAI Driver Documentation](src/model.drivers/openai.driver.md) |
| `CohereThreadDriver` | Cohere | [Cohere Driver Documentation](src/model.drivers/cohere.driver.md) |
| `GeminiThreadDriver` | Google Gemini | [Gemini Driver Documentation](src/model.drivers/gemini.driver.md) |
| `CerebrasThreadDriver` | Cerebras | [Cerebras Driver Documentation](src/model.drivers/cerebras.driver.md) |

Each driver provides a consistent interface while supporting model-specific features:

```typescript
// OpenAI example
import { OpenAIThreadDriver } from '@flatfile/improv';

const driver = new OpenAIThreadDriver({
  model: 'gpt-4o',
  apiKey: process.env.OPENAI_API_KEY,
  temperature: 0.7
});

// Cohere example
import { CohereThreadDriver } from '@flatfile/improv';

const driver = new CohereThreadDriver({
  model: 'command-r-plus',
  apiKey: process.env.COHERE_API_KEY
});

// Gemini example
import { GeminiThreadDriver } from '@flatfile/improv';

const driver = new GeminiThreadDriver({
  model: 'gemini-1.5-pro',
  apiKey: process.env.GOOGLE_API_KEY
});

// Cerebras example
import { CerebrasThreadDriver } from '@flatfile/improv';

const driver = new CerebrasThreadDriver({
  model: 'llama-4-scout-17b-16e-instruct',
  apiKey: process.env.CEREBRAS_API_KEY
});
```

Refer to each driver's documentation for available models and specific configuration options.

## Tool Decorators

The library provides decorators for creating tools directly on agent classes:

```typescript
class CustomAgent extends Agent {
  @ToolName("sampleData")
  @ToolDescription("Sample the original data with the mapping program")
  private async sampleData(
    @ToolParam("count", "Number of records to sample", z.number())
    count: number,
    @ToolParam("seed", "Random seed", z.number().optional())
    seed?: number
  ): Promise<any> {
    return { count, seed };
  }
}
```

This provides:
- Type-safe tool definitions
- Automatic parameter validation
- Clean method-based tools
- Integrated error handling

## Streaming Support

The library provides built-in support for streaming responses from the AI model. Keys features:
- Real-time text chunks as they're generated
- Automatic message management in the thread
- Event emission for stream progress
- Error handling and recovery
- Compatible with all thread features including tool calls

```typescript
const thread = agent.createThread({
  prompt: 'What is 25 multiplied by 4?',
});

const stream = await thread.stream();

for await (const text of stream) {
  process.stdout.write(text);
}

// The final response is also available in the thread
console.log(thread.last()?.content);
```