@tehreet/conduit/README.md

LLM API gateway with intelligent routing, robust process management, and health monitoring

<div align="center">
  <img src="https://i.imgur.com/GJrd7tU.png" alt="Conduit Logo" width="200" />
  
  # Conduit
  
  **Intelligent LLM API Gateway & Node.js Library**
  
  [![npm version](https://img.shields.io/npm/v/@tehreet/conduit.svg)](https://www.npmjs.com/package/@tehreet/conduit)
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  [![Node.js Version](https://img.shields.io/node/v/@tehreet/conduit.svg)](https://nodejs.org)
  [![CI/CD](https://github.com/vibe-coders-only/conduit/actions/workflows/ci.yml/badge.svg)](https://github.com/vibe-coders-only/conduit/actions)
  
  *A smart proxy layer with advanced routing, monitoring, and programmatic API*
</div>

---

## 🚀 Quick Start

### CLI Usage
```bash
# Install globally
npm install -g @tehreet/conduit

# Start server
conduit start

# Make requests
curl -X POST http://localhost:8111/v1/messages \
  -H "Content-Type: application/json" \
  -d '{"model": "claude-3-5-sonnet-20241022", "messages": [{"role": "user", "content": "Hello!"}]}'
```

### Library Usage
```javascript
import { ConduitServer, ConduitRouter } from '@tehreet/conduit';

// Programmatic server
const server = new ConduitServer(config);
await server.start();

// Standalone routing
const router = new ConduitRouter(config);
const decision = await router.route(context);
```

## ✨ Features

### 🧠 **Intelligent Routing**
- **Content Analysis**: Automatic complexity detection, language identification, topic extraction
- **Priority-Based Rules**: Custom conditions with confidence scoring
- **Context Awareness**: Synapse IDE integration with project/agent context
- **Token-Based Selection**: Optimal model selection based on request size

### 🔧 **Dual Architecture**
- **CLI Tool**: Traditional command-line interface with daemon mode
- **Node.js Library**: 10x faster programmatic API with full TypeScript support
- **100% Backward Compatibility**: Existing integrations continue to work

### 📊 **Monitoring & Analytics**
- **Usage Tracking**: Request, token, and cost analytics with project/agent breakdown
- **Health Monitoring**: Real-time health checks for server, plugins, and storage
- **Metrics Collection**: Integration with Prometheus, StatsD, and custom endpoints
- **Performance Caching**: Intelligent token counting cache and batch processing

### 🔌 **Extensible Architecture**
- **Plugin System**: Custom routing logic with lifecycle hooks
- **Custom Evaluators**: Extensible condition evaluation for routing rules
- **Storage Abstraction**: Memory, file, and database storage options
- **Event-Driven**: EventEmitter-based architecture for real-time updates

## 📦 Installation

### Global CLI
```bash
npm install -g @tehreet/conduit
```

### Library Dependency
```bash
npm install @tehreet/conduit
```

### Development
```bash
git clone https://github.com/vibe-coders-only/conduit.git
cd conduit
npm install
npm run build
```

## 🏗️ Architecture

Conduit provides both CLI and library interfaces:

```
┌─────────────────┐    ┌─────────────────┐
│   CLI Interface │    │ Library Interface│
│   (Traditional) │    │  (Programmatic) │
└─────────┬───────┘    └─────────┬───────┘
          │                      │
          └──────────┬───────────┘
                     │
        ┌─────────────────────────┐
        │    ConduitServer       │
        │   (Event-Driven)       │
        └─────────┬───────────────┘
                  │
        ┌─────────────────────────┐
        │    ConduitRouter       │
        │  (Advanced Routing)     │
        └─────────┬───────────────┘
                  │
    ┌─────────────┼─────────────┐
    │             │             │
┌───▼───┐  ┌─────▼─────┐  ┌───▼────┐
│Plugins│  │ Providers │  │Monitor │
│System │  │Integration│  │& Track │
└───────┘  └───────────┘  └────────┘
```

## 🔧 API Reference

### Core Classes

#### ConduitServer
```typescript
import { ConduitServer, ConduitConfig } from '@tehreet/conduit';

const config: ConduitConfig = {
  server: { port: 8111 },
  Router: { default: 'claude-3-5-sonnet-20241022' },
  Providers: [/* ... */]
};

const server = new ConduitServer(config);

// Lifecycle management
await server.start();
await server.stop();

// Request routing
const decision = await server.route(context);

// Configuration updates
await server.updateConfig(newConfig);

// Health status
const health = await server.getHealth();
```

#### ConduitRouter
```typescript
import { ConduitRouter, RoutingContext } from '@tehreet/conduit';

const router = new ConduitRouter(config);
await router.initialize();

const context: RoutingContext = {
  request: { body: { messages: [...] } },
  tokenCount: 1500,
  config,
  env: process.env,
  synapseContext: { projectId: 'proj-123' }
};

const decision = await router.route(context);
// { selectedModel: 'claude-3-5-haiku-20241022', confidence: 0.85, reason: 'Simple request' }
```

### Advanced Routing

#### Custom Routing Rules
```typescript
import { registerRoutingRule, AdvancedRoutingRule } from '@tehreet/conduit';

const customRule: AdvancedRoutingRule = {
  id: 'coding-requests',
  name: 'Coding Requests',
  priority: 90,
  conditions: [
    { type: 'content', field: 'codeBlocks', operator: '>', value: 0 },
    { type: 'token_count', operator: '<', value: 50000 }
  ],
  model: 'claude-3-5-sonnet-20241022',
  confidence: 0.9,
  enabled: true
};

registerRoutingRule(customRule);
```

#### Custom Evaluators
```typescript
import { registerCustomEvaluator } from '@tehreet/conduit';

registerCustomEvaluator('is_urgent', (condition, context) => {
  return context.request.headers['x-priority'] === 'urgent';
});

// Use in routing rules
const urgentRule = {
  id: 'urgent-requests',
  conditions: [{ type: 'custom', field: 'is_urgent', operator: '=', value: true }],
  model: 'claude-3-5-sonnet-20241022',
  priority: 100
};
```

### Monitoring & Analytics

#### Usage Tracking
```typescript
import { UsageTracker } from '@tehreet/conduit';

const tracker = new UsageTracker({
  enabled: true,
  storage: { type: 'file', path: './usage.json' },
  metrics: { enabled: true, provider: 'prometheus' }
});

await tracker.initialize();

// Track usage
await tracker.trackUsage({
  id: 'req-123',
  timestamp: new Date(),
  model: 'claude-3-5-sonnet-20241022',
  tokenCount: 1500,
  cost: 0.075,
  routingReason: 'Complex request'
});

// Get statistics
const stats = await tracker.getUsageStats({
  timeRange: { start: new Date('2024-01-01'), end: new Date() }
});
```

#### Health Monitoring
```typescript
import { HealthMonitor } from '@tehreet/conduit';

const monitor = new HealthMonitor();

// Custom health check
monitor.registerCheck('database', async () => {
  const isHealthy = await checkDatabase();
  return {
    healthy: isHealthy,
    message: isHealthy ? 'Database connected' : 'Database unavailable',
    timestamp: new Date()
  };
});

monitor.start(30000); // Check every 30 seconds
const status = await monitor.getStatus();
```

### Token Counter with Caching

#### Enhanced Token Counting
```typescript
import { tokenCounter } from '@tehreet/conduit';

// Single request
const result = await tokenCounter.countTokens('Hello world', 'claude-3-5-sonnet-20241022');
// { count: 2, method: 'tiktoken' }

// Batch processing
const requests = [
  { id: 'req1', text: 'Hello', model: 'claude-3-5-sonnet-20241022' },
  { id: 'req2', text: 'How are you?', model: 'claude-3-5-sonnet-20241022' }
];

const results = await tokenCounter.batchCountTokens(requests);
// [{ id: 'req1', count: 1, method: 'tiktoken' }, ...]

// Cache statistics
const stats = tokenCounter.getCacheStats();
// { size: 150, hitRate: 0.85, oldestEntry: Date, newestEntry: Date }
```

### Synapse Integration

#### Context-Aware Routing
```typescript
import { SynapseIntegration } from '@tehreet/conduit';

const integration = new SynapseIntegration(config);
await integration.initialize();

// Route with Synapse context
const result = await integration.routeClaudeRequest(
  ['--model', 'claude-3-5-sonnet-20241022', 'Hello'],
  {
    SYNAPSE_PROJECT_ID: 'proj-123',
    SYNAPSE_AGENT_ID: 'agent-456',
    SYNAPSE_PROJECT_MODEL_CONFIG: JSON.stringify({
      enabled: true,
      defaultModel: 'anthropic/claude-3-5-haiku-20241022'
    })
  }
);
```

## ⚙️ Configuration

### Basic Configuration
```json
{
  "server": {
    "port": 8111,
    "host": "127.0.0.1"
  },
  "Router": {
    "enabled": true,
    "default": "claude-3-5-sonnet-20241022",
    "longContext": "claude-3-5-sonnet-20241022",
    "background": "claude-3-5-haiku-20241022",
    "think": "claude-3-5-sonnet-20241022"
  },
  "Providers": [
    {
      "name": "anthropic",
      "api_base_url": "https://api.anthropic.com",
      "api_key": "${ANTHROPIC_API_KEY}",
      "models": ["claude-3-5-sonnet-20241022", "claude-3-5-haiku-20241022"]
    }
  ],
  "monitoring": {
    "enabled": true,
    "usage": {
      "enabled": true,
      "storage": "memory",
      "retention": 30
    },
    "health": {
      "enabled": true,
      "interval": 30
    },
    "metrics": {
      "enabled": false,
      "provider": "custom"
    }
  }
}
```

### Environment Variables
```bash
# Server
CONDUIT_PORT=8111
CONDUIT_HOST=127.0.0.1

# API Keys
ANTHROPIC_API_KEY=your-key-here
OPENAI_API_KEY=your-key-here

# Monitoring
CONDUIT_MONITORING_ENABLED=true
CONDUIT_METRICS_PROVIDER=prometheus

# Synapse Integration
SYNAPSE_PROJECT_ID=proj-123
SYNAPSE_AGENT_ID=agent-456
```

## 🧪 Testing

Run the comprehensive test suite:

```bash
# Build and test
npm run build
npm test

# Integration tests
node test-phase3.js  # Synapse integration
node test-phase5.js  # Enhanced features

# Performance tests
./test-wrapper-features.sh
```

## 🚀 Performance

### Benchmarks
- **10x faster** than CLI spawning for library usage
- **50% lower** memory usage with intelligent caching
- **Batch processing** optimizes multiple token count requests
- **Smart routing** ensures optimal model selection

### Caching Benefits
- Token counting cache reduces redundant calculations
- Content analysis cache for repeated requests
- Model configuration cache for faster routing
- Automatic cache cleanup and size management

## 📈 Production Deployment

### PM2 Configuration
```javascript
// ecosystem.config.js
module.exports = {
  apps: [{
    name: 'conduit',
    script: 'dist/cli.js',
    args: 'start --daemon',
    instances: 1,
    exec_mode: 'fork',
    env: {
      NODE_ENV: 'production',
      CONDUIT_PORT: 8111
    }
  }]
};
```

### Docker
```dockerfile
FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
RUN npm run build
EXPOSE 8111
CMD ["npm", "start"]
```

### Health Checks
```bash
# Basic health
curl http://localhost:8111/health

# Detailed status
curl http://localhost:8111/alive

# Readiness probe
curl http://localhost:8111/ready
```

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Commit changes: `git commit -m 'feat: add amazing feature'`
4. Push to branch: `git push origin feature/amazing-feature`
5. Open a Pull Request

### Development Setup
```bash
git clone https://github.com/vibe-coders-only/conduit.git
cd conduit
npm install
npm run build
npm run dev
```

## 📄 License

MIT License - see [LICENSE](LICENSE) for details.

## 🔗 Links

- **NPM Package**: [@tehreet/conduit](https://www.npmjs.com/package/@tehreet/conduit)
- **GitHub**: [vibe-coders-only/conduit](https://github.com/vibe-coders-only/conduit)
- **Issues**: [Bug Reports](https://github.com/vibe-coders-only/conduit/issues)
- **Discussions**: [Community](https://github.com/vibe-coders-only/conduit/discussions)

---

<div align="center">
  <strong>Transform your LLM integration with intelligent routing and monitoring</strong>
</div>