contaigents/TOOL_ARCHITECTURE.md

Modular AI Content Ecosystem with Audio Generation

# Tool Architecture for ChatService

This document describes the tool architecture added to the ChatService, enabling AI assistants to read files and understand project context.

## Overview

The tool architecture provides a structured way for AI assistants to interact with the file system and project environment. It includes:

- **File reading capabilities** for text, markdown, and code files
- **File writing capabilities** with create, rewrite, and partial write operations
- **Project structure awareness** through file tree context
- **Backup system** for safe file modifications
- **Extensible tool system** for adding new capabilities
- **Automatic tool call parsing and execution**

## Architecture Components

### 1. BaseTool (Abstract Class)
- Defines the interface for all tools
- Provides parameter validation
- Standardizes tool definitions and execution

### 2. FileReadTool
- Reads various file types (.txt, .md, .mdx, .js, .ts, .json, etc.)
- Provides file metadata (size, lines, encoding)
- Validates file extensions for security

### 3. FileWriteTool
- Supports three operation types: create, rewrite, and partial write
- **Create**: Creates new files (prevents overwriting existing files)
- **Rewrite**: Completely replaces file content with backup option
- **Partial**: Updates specific line ranges with precise control
- Automatic backup creation for safety
- Validates file extensions and parameters

### 4. AudioGenerationTool
- Generates audio files from text using AI providers (Gemini TTS)
- Supports multiple voices and providers
- Secure file path resolution with directory creation
- Integration with existing AudioGenerator utility
- Comprehensive error handling and validation

### 5. ToolManager
- Manages tool registration and execution
- Generates file tree context for system prompts
- Parses tool calls from LLM responses
- Provides tool usage instructions

### 6. Enhanced ChatService
- Integrates tool architecture seamlessly
- Includes file tree context in conversations
- Automatically executes tool calls in responses
- Maintains conversational flow with tool results

## Usage

### Tool Call Format
AI assistants use tools by including XML-formatted calls in their responses with unique IDs:

```xml
<tool_call name="tool_name" id="1">
  <parameter_name>parameter_value</parameter_name>
</tool_call>
```

**Examples:**
```xml
<tool_call name="read_file" id="1">
  <file_path>README.md</file_path>
</tool_call>

<tool_call name="write_file" id="2">
  <operation_type>create</operation_type>
  <file_path>new-doc.md</file_path>
  <content># New Document</content>
</tool_call>

<tool_call name="write_file" id="3">
  <operation_type>partial</operation_type>
  <file_path>doc.md</file_path>
  <start_line>5</start_line>
  <end_line>7</end_line>
  <content>Updated content</content>
</tool_call>
```

**For complex content with special characters, use CDATA:**
```xml
<tool_call name="write_file" id="4">
  <operation_type>create</operation_type>
  <file_path>complex.md</file_path>
  <content><![CDATA[
# Complex Content
With "quotes", [brackets], {braces}
JSON: {"key": "value"}
And multiline content!
]]></content>
</tool_call>
```

**Audio generation example:**
```xml
<tool_call name="audio_generation" id="5">
  <text>Hello, this is a demonstration of the audio generation tool.</text>
  <output_path>audio/demo.wav</output_path>
  <voice>zephyr</voice>
  <provider>gemini</provider>
</tool_call>
```

### Tool Response Format

When tools are executed, they return structured XML responses that match the tool call IDs:

```xml
<tool_response id="1">
  <tool_call name="read_file">
    <file_path>README.md</file_path>
  </tool_call>
  <result>
    <content>
      # Project Title
      This is the project README content...
    </content>
  </result>
</tool_response>
```

**For tool execution errors:**
```xml
<tool_response id="2">
  <tool_call name="read_file">
    <file_path>nonexistent.md</file_path>
  </tool_call>
  <result>
    <error>File not found: nonexistent.md</error>
  </result>
</tool_response>
```

### Why XML Format

The XML format offers several advantages:

1. **Reliable Parsing**: XML has well-defined parsing rules that handle edge cases robustly
2. **CDATA Support**: Complex content with quotes, brackets, and special characters can be wrapped in CDATA sections
3. **Multiline Content**: XML naturally handles multiline content without escaping issues
4. **Better Error Recovery**: XML parsers provide clear error messages and recovery mechanisms
5. **Cleaner Syntax**: More readable and easier for LLMs to generate correctly
6. **Extensible**: Can easily add attributes or nested structures if needed in the future

### System Prompt Enhancement
The ChatService automatically includes:
- Current project file tree structure
- Available tool descriptions and parameters
- Usage instructions and examples

### File Tree Context
The AI receives a visual representation of the project structure:
```
📝 README.md
📁 cli/
  📁 src/
    📁 services/
      📁 tools/
    📁 tests/
📁 frontend/
📁 docs/
```

## Supported File Types

The FileReadTool supports reading:
- Text files (.txt)
- Markdown files (.md, .mdx)
- Code files (.js, .ts, .jsx, .tsx, .py, .java, .cpp, .c, .h, .go, .rs, .php, .rb, .sh)
- Configuration files (.json, .yaml, .yml)
- Web files (.html, .css, .scss, .xml)
- SQL files (.sql)

## Testing

### Tool Architecture Test
```bash
npm run build
node --loader ts-node/esm src/index.ts test-tools
```

### File Write Tool Test
```bash
node --loader ts-node/esm src/index.ts test-file-write
```

### Combined Tools Workflow Test
```bash
node --loader ts-node/esm src/index.ts test-combined-tools
```

### Chat with Tools Demo
```bash
node --loader ts-node/esm src/index.ts demo-chat-tools
```

## Example Interactions

### Reading Project Files
**User:** "What is this project about? Please read the README.md file."

**AI Response:** "I'll read the README.md file to understand the project.

[TOOL:read_file:{"file_path":"README.md"}]

Based on the README.md file, this is Contaigents, an open-source platform for creating and managing content using locally running AI agents..."

### Creating New Files
**User:** "Create a new documentation file for the API endpoints."

**AI Response:** "I'll create a new API documentation file for you.

[TOOL:write_file:{"operation_type":"create","file_path":"docs/api-endpoints.md","content":"# API Endpoints\n\n## Overview\nThis document describes the available API endpoints.\n\n## Authentication\n...\n"}]

I've created a new API documentation file at docs/api-endpoints.md with a basic structure for documenting your API endpoints."

### Updating Existing Files
**User:** "Update the README.md file to add a new section about deployment."

**AI Response:** "I'll read the current README.md file first, then add the deployment section.

[TOOL:read_file:{"file_path":"README.md"}]

Now I'll add the deployment section at the end:

[TOOL:write_file:{"operation_type":"partial","file_path":"README.md","content":"\n## 🚀 Deployment\n\n### Production Deployment\n- Build the project: `npm run build`\n- Deploy to your preferred platform\n- Configure environment variables\n\n### Docker Deployment\n```bash\ndocker build -t contaigents .\ndocker run -p 3000:3000 contaigents\n```","start_line":80,"end_line":80}]

I've added a comprehensive deployment section to your README.md file with instructions for both production and Docker deployments."

### Understanding Project Structure
**User:** "What files are available in this project?"

**AI Response:** "Based on the project structure, I can see several key directories and files:

- README.md - Main project documentation
- cli/ - Command-line interface
- frontend/ - Web interface
- docs/ - Documentation site
- ..."

## Extending the Tool System

### Adding New Tools

1. Create a new tool class extending `BaseTool`:
```typescript
export class MyCustomTool extends BaseTool {
  getName(): string {
    return 'my_tool';
  }
  
  getDescription(): string {
    return 'Description of what this tool does';
  }
  
  getParameters(): ToolParameter[] {
    return [
      {
        name: 'param1',
        type: 'string',
        description: 'Parameter description',
        required: true
      }
    ];
  }
  
  async execute(parameters: Record<string, any>): Promise<ToolResult> {
    // Tool implementation
  }
}
```

2. Register the tool in `ToolManager`:
```typescript
const customTool = new MyCustomTool();
toolManager.registerTool(customTool);
```

## Security Considerations

- File reading is restricted to supported file types
- Path traversal protection through relative path validation
- No write operations in the basic file tool
- Tool execution is sandboxed within the defined parameters

## FileWriteTool Operations

### Create Operation
Creates a new file. Fails if the file already exists to prevent accidental overwrites.

```javascript
{
  operation_type: 'create',
  file_path: 'docs/new-guide.md',
  content: '# New Guide\n\nThis is a new guide...'
}
```

### Rewrite Operation
Completely replaces the content of an existing file. Creates a backup by default.

```javascript
{
  operation_type: 'rewrite',
  file_path: 'docs/existing-file.md',
  content: '# Updated Content\n\nCompletely new content...',
  create_backup: true
}
```

### Partial Operation
Updates specific line ranges in a file. Provides precise control over modifications.

```javascript
{
  operation_type: 'partial',
  file_path: 'docs/document.md',
  content: '## Updated Section\nNew content for lines 5-7',
  start_line: 5,
  end_line: 7,
  create_backup: true
}
```

## Future Enhancements

Planned tool additions:
- Directory operations (create, list, delete)
- Git integration tools
- Search and grep tools
- Code analysis tools
- Project metadata extraction tools
- File template generation tools