@handit.ai/ai-wrapper/DOCUMENTATION.md

🤖 Intelligent AI execution system with built-in tracking, evaluation, and self-improvement capabilities. The complete AI intelligence platform for enterprise applications.

# AI Wrapper - Complete Documentation

## Overview

The AI Wrapper is a sophisticated Node.js package that provides **intelligent AI execution** with built-in **tracking**, **evaluation**, and **self-improvement capabilities**. It's not just about running AI models - it's about creating a complete AI system that learns, improves, and provides valuable insights.

## 🧠 What Makes This AI Wrapper Special

### 1. **Smart Execution with Tracking**
Every AI execution is automatically tracked with handit, providing:
- Input/output logging
- Performance metrics
- Error tracking
- Execution flow analysis

### 2. **Self-Improving AI**
Through handit's tracking system, the wrapper enables:
- **Prompt optimization** based on execution history
- **Performance analysis** across different models
- **Automatic learning** from past interactions
- **Quality improvement** over time

### 3. **Evaluation & Analytics**
Built-in evaluation capabilities:
- Response quality assessment
- Execution time analysis
- Cost tracking per model
- Success/failure rate monitoring

### 4. **Multi-Provider Intelligence**
Smart provider selection and optimization:
- **OpenAI** for general intelligence
- **Google AI** for specialized tasks
- **Anthropic** for safety-focused applications

## 🚀 Core Methods

### Smart Execution Methods

#### `runSmartAgent(options)`
**The flagship method** - Executes AI with full intelligence, tracking, and optimization.

```javascript
const result = await aiWrapper.runSmartAgent({
  agentName: 'customer-support',
  input: 'Customer inquiry about refund policy',
  model: 'gpt-4',
  additionalOptions: {
    temperature: 0.7,
    max_tokens: 1000
  }
});
```

**What it does:**
1. 📥 Fetches optimized prompts from handit
2. 🤖 Executes AI with OpenAI (default smart provider)
3. 📊 Tracks execution metrics
4. 🔄 Learns from the interaction
5. 📈 Contributes to prompt optimization

#### `runOptimizedAgent(options)`
**Google AI-powered** execution with optimization focus.

```javascript
const result = await aiWrapper.runOptimizedAgent({
  agentName: 'content-generator',
  input: 'Create SEO-optimized blog content',
  model: 'gemini-pro'
});
```

**What it does:**
1. 🎯 Uses Google's advanced AI models
2. 📊 Optimizes for performance and quality
3. 📈 Tracks optimization metrics
4. 🔄 Learns content generation patterns

#### `runTrackedAgent(options)`
**Anthropic Claude-powered** execution with enhanced safety tracking.

```javascript
const result = await aiWrapper.runTrackedAgent({
  agentName: 'code-reviewer',
  input: 'Review code for security vulnerabilities',
  model: 'claude-3-sonnet-20240229'
});
```

**What it does:**
1. 🛡️ Uses Anthropic's safety-focused models
2. 📝 Enhanced tracking for sensitive operations
3. 🔍 Detailed execution analysis
4. 🚨 Safety and compliance monitoring

### Provider-Specific Methods

#### `runWithOpenAI(options)`
Direct OpenAI execution with tracking.

#### `runWithGoogle(options)`
Direct Google AI execution with tracking.

#### `runWithAnthropic(options)`
Direct Anthropic execution with tracking.

## 📊 Tracking & Analytics

### Automatic Tracking
Every execution automatically tracks:

```javascript
{
  input: "User's question or input",
  output: "AI's response",
  nodeName: "execution-node-name",
  agentName: "agent-identifier",
  nodeType: "llm", // automatically converted from 'llm' to 'model'
  executionId: "unique-execution-id",
  timestamp: "2024-01-01T00:00:00Z",
  performance: {
    executionTime: 1250, // milliseconds
    tokenUsage: {
      input: 150,
      output: 300
    },
    cost: 0.045 // estimated cost
  }
}
```

### Performance Metrics
The wrapper tracks:
- **Response Time**: How fast each execution completes
- **Token Usage**: Input/output tokens for cost optimization
- **Success Rate**: Percentage of successful executions
- **Error Patterns**: Common failure points for improvement

### Quality Assessment
Automatic quality evaluation:
- **Response Relevance**: How well the AI answered the query
- **Prompt Effectiveness**: Which prompts work best
- **Model Performance**: Comparative analysis across providers

## 🔄 Self-Improvement Features

### 1. **Prompt Optimization**
The system learns which prompts work best:

```javascript
// The wrapper automatically:
// 1. Tracks which prompts produce better results
// 2. Identifies high-performing prompt patterns
// 3. Suggests optimizations to handit
// 4. Applies learned improvements automatically
```

### 2. **Model Selection Intelligence**
Smart model selection based on:
- **Task Type**: Choosing the best model for specific tasks
- **Performance History**: Using models that performed well previously
- **Cost Efficiency**: Balancing quality with cost
- **Response Speed**: Optimizing for time-sensitive applications

### 3. **Adaptive Learning**
The system adapts to:
- **User Patterns**: Learning from user interaction styles
- **Domain Expertise**: Improving in specific domains over time
- **Quality Feedback**: Using success/failure data to improve
- **Contextual Awareness**: Better understanding of context over time

## 🎯 Advanced Usage Patterns

### 1. **Multi-Agent Workflows**

```javascript
// Sequential agent execution with shared context
const supportResult = await aiWrapper.runSmartAgent({
  agentName: 'customer-support',
  input: 'Customer complaint about billing'
});

const resolutionResult = await aiWrapper.runOptimizedAgent({
  agentName: 'issue-resolver',
  input: `Previous interaction: ${supportResult.output}. Now resolve the billing issue.`
});

const followUpResult = await aiWrapper.runTrackedAgent({
  agentName: 'follow-up-scheduler',
  input: `Schedule follow-up based on resolution: ${resolutionResult.output}`
});
```

### 2. **Conditional Execution**

```javascript
async function intelligentCustomerService(customerInput) {
  // First, classify the request
  const classification = await aiWrapper.runSmartAgent({
    agentName: 'request-classifier',
    input: customerInput
  });

  // Then route to appropriate specialist agent
  let specialist;
  if (classification.output.includes('technical')) {
    specialist = await aiWrapper.runWithOpenAI({
      agentName: 'technical-support',
      input: customerInput,
      model: 'gpt-4'
    });
  } else if (classification.output.includes('billing')) {
    specialist = await aiWrapper.runOptimizedAgent({
      agentName: 'billing-support',
      input: customerInput
    });
  } else {
    specialist = await aiWrapper.runTrackedAgent({
      agentName: 'general-support',
      input: customerInput
    });
  }

  return specialist;
}
```

### 3. **A/B Testing for AI Models**

```javascript
async function testModelPerformance(input) {
  const tests = await Promise.all([
    aiWrapper.runWithOpenAI({
      agentName: 'content-writer',
      input,
      model: 'gpt-3.5-turbo'
    }),
    aiWrapper.runWithOpenAI({
      agentName: 'content-writer',
      input,
      model: 'gpt-4'
    }),
    aiWrapper.runOptimizedAgent({
      agentName: 'content-writer',
      input
    })
  ]);

  // The tracking system will automatically compare performance
  return tests;
}
```

## 📈 Analytics & Insights

### Execution Analytics
Access comprehensive analytics through handit:

```javascript
// Get execution history and performance metrics
const analytics = await aiWrapper.getExecutionAnalytics({
  agentName: 'customer-support',
  timeRange: '30d'
});

console.log(analytics);
// {
//   totalExecutions: 1250,
//   successRate: 0.96,
//   averageResponseTime: 1200,
//   totalCost: 125.50,
//   topPerformingPrompts: [...],
//   improvementSuggestions: [...]
// }
```

### Quality Metrics
Monitor AI quality over time:

```javascript
const qualityReport = await aiWrapper.getQualityReport({
  agentName: 'content-writer',
  period: 'weekly'
});

console.log(qualityReport);
// {
//   qualityScore: 8.7,
//   improvement: +0.3,
//   topIssues: ['formatting', 'tone'],
//   recommendations: ['adjust temperature', 'refine prompts']
// }
```

## 🛠️ Configuration for Optimization

### Advanced Configuration

```javascript
const aiWrapper = new AIWrapper({
  handitApiKey: 'your-key',
  
  // Provider keys
  openaiApiKey: 'openai-key',
  googleApiKey: 'google-key',
  anthropicApiKey: 'anthropic-key',
  
  // Optimization settings
  optimizationSettings: {
    enableAutoPromptOptimization: true,
    enableModelSelection: true,
    enableCostOptimization: true,
    enableQualityTracking: true
  },
  
  // Custom tracking URLs
  trackingUrl: 'custom-tracking-endpoint',
  performanceUrl: 'custom-performance-endpoint'
});
```

## 🔧 Environment Variables

```bash
# Required
HANDIT_API_KEY=your-handit-api-key

# AI Providers (at least one required)
OPENAI_API_KEY=your-openai-key
GOOGLE_API_KEY=your-google-key
ANTHROPIC_API_KEY=your-anthropic-key

# Optional: Custom endpoints
HANDIT_TRACKING_URL=custom-tracking-url
HANDIT_PERFORMANCE_URL=custom-performance-url
HANDIT_SSO_TRACKING_URL=custom-sso-tracking-url

# Optional: Optimization settings
ENABLE_AUTO_OPTIMIZATION=true
ENABLE_QUALITY_TRACKING=true
ENABLE_COST_OPTIMIZATION=true
```

## 📚 Best Practices

### 1. **Agent Naming Strategy**
Use descriptive, hierarchical agent names:
```javascript
// Good
'customer-support-billing'
'content-writer-blog-posts'
'code-reviewer-security'

// Avoid
'agent1'
'ai-helper'
'bot'
```

### 2. **Prompt Design**
Design prompts for optimization:
```javascript
// In your handit dashboard, structure prompts like:
{
  type: 'system',
  content: 'You are a {{role}} specializing in {{domain}}. Your goal is to {{objective}}.'
}
{
  type: 'user',
  content: 'Context: {{context}}\nTask: {{input}}\nProvide a detailed response.'
}
```

### 3. **Error Handling**
Always handle errors gracefully:
```javascript
const result = await aiWrapper.runSmartAgent({
  agentName: 'data-processor',
  input: complexData
});

if (!result.success) {
  console.error('AI execution failed:', result.error);
  // The error is automatically tracked for improvement
  
  // Implement fallback logic
  const fallbackResult = await aiWrapper.runWithOpenAI({
    agentName: 'simple-processor',
    input: complexData,
    model: 'gpt-3.5-turbo' // simpler, faster model
  });
  
  return fallbackResult;
}
```

### 4. **Performance Monitoring**
Regularly review performance:
```javascript
// Set up periodic performance reviews
setInterval(async () => {
  const performance = await aiWrapper.getPerformanceMetrics();
  
  if (performance.successRate < 0.9) {
    console.warn('AI performance declining, check prompts and models');
  }
  
  if (performance.averageCost > budget.daily) {
    console.warn('AI costs exceeding budget, optimize model selection');
  }
}, 24 * 60 * 60 * 1000); // Daily check
```

## 🔮 Future Capabilities

The AI Wrapper is designed to evolve with your needs:

### Planned Features
- **Automatic Model Switching** based on performance
- **Cost Optimization Algorithms** for budget management
- **Quality Scoring** with automatic improvement suggestions
- **Custom Evaluation Metrics** for domain-specific assessment
- **Prompt A/B Testing** built into the system
- **Multi-Language Support** for global applications

### Integration Roadmap
- **Custom AI Providers** - Plugin system for new providers
- **Advanced Analytics** - Machine learning insights
- **Real-time Optimization** - Live prompt and model optimization
- **Enterprise Features** - Team management and permissions

## 🎯 Conclusion

The AI Wrapper isn't just a simple interface to AI models - it's a **complete AI intelligence system** that:

1. **Learns** from every interaction
2. **Improves** automatically over time
3. **Optimizes** for performance and cost
4. **Tracks** everything for insights
5. **Scales** with your needs

By using this wrapper, you're not just running AI - you're building an **intelligent system** that gets better with use, provides valuable insights, and continuously optimizes for your specific use cases.

Start with the basic methods, then explore the advanced features as your AI system grows and evolves. The tracking and optimization features will help you build truly intelligent applications that improve over time.