@tehreet/conduit/docs/conduit-feature-changes.md

LLM API gateway with intelligent routing, robust process management, and health monitoring

# Conduit Feature Changes Implementation Plan

## Overview

This document outlines the specific changes needed to transform Conduit from a CLI-focused tool into a scalable Node.js library with enhanced Synapse integration capabilities, while maintaining full backward compatibility.

## Phase 1: Core Library Infrastructure

### 1.1 Build System Changes

**Update package.json**
- Add dual build system (library + CLI)
- Add TypeScript declaration generation
- Update exports for library usage
- Add new build scripts

**Files to modify:**
- `package.json` - Add library build configuration
- `tsconfig.json` - Update for dual build
- Create `tsconfig.lib.json` - Library-specific TypeScript config

**New build structure:**
```
dist/
├── cli.js              # Bundled CLI (existing)
├── tiktoken_bg.wasm    # WASM file (existing)
└── lib/                # New library build
    ├── index.js
    ├── index.d.ts
    ├── server.js
    ├── server.d.ts
    ├── router.js
    ├── router.d.ts
    └── plugins.js
```

### 1.2 Enhanced Library Exports

**Create new files:**
- `src/lib.ts` - Main library entry point with all exports
- `src/types/index.ts` - Comprehensive type definitions

**Update existing files:**
- `src/index.ts` - Re-export library functions
- Add proper TypeScript exports throughout codebase

### 1.3 Type System Enhancements

**New interfaces to add:**
- `ConduitConfig` - Main configuration interface
- `ServerConfig` - Server-specific configuration
- `RouterConfig` - Router configuration
- `RoutingContext` - Request routing context
- `RoutingDecision` - Routing decision result
- `PluginConfig` - Plugin configuration
- `UsageData` - Usage tracking data
- `HealthStatus` - Health monitoring status

## Phase 2: Enhanced API Design

### 2.1 Server Management

**Create new ConduitServer class:**
- `src/server/ConduitServer.ts` - Enhanced server with programmatic API
- Replace current server with class-based approach
- Add lifecycle management (start/stop)
- Add health monitoring
- Add configuration updates

**Key methods:**
- `start()` - Start server programmatically
- `stop()` - Graceful shutdown
- `route(context)` - Route single request
- `getHealth()` - Get health status
- `updateConfig(config)` - Update configuration

### 2.2 Standalone Router

**Create new ConduitRouter class:**
- `src/router/ConduitRouter.ts` - Standalone routing engine
- Extract routing logic from current middleware
- Add plugin integration
- Add provider management

**Key methods:**
- `initialize()` - Initialize router
- `route(context)` - Make routing decision
- `updateConfig(config)` - Update router configuration

### 2.3 Plugin System Enhancement

**Update existing plugin files:**
- `src/plugins/plugin-interface.ts` - Enhanced plugin interface
- `src/plugins/plugin-manager.ts` - Improved plugin management
- Add lifecycle hooks (onLoad, onUnload)
- Add routing hooks (beforeRouting, afterRouting)
- Add health monitoring for plugins

**New plugin capabilities:**
- Inline plugins (not just file-based)
- NPM-based plugins
- Configuration validation
- Health status reporting

## Phase 3: Synapse Integration Layer

### 3.1 Synapse Integration Module

**Create new files:**
- `src/integrations/synapse.ts` - Synapse-specific integration
- `src/integrations/index.ts` - Integration exports

**Key features:**
- Enhanced context extraction from environment
- Synapse-specific routing logic
- Usage tracking for Synapse projects
- Agent and project configuration support

### 3.2 Configuration Management

**Create new files:**
- `src/config/ConfigManager.ts` - Configuration management
- `src/config/index.ts` - Configuration exports

**Update existing files:**
- `src/utils/config-presets.ts` - Enhance with validation
- `src/utils/context-extractor.ts` - Improve context extraction

**Key features:**
- Programmatic configuration updates
- Configuration validation
- Preset management
- Deep merging of configurations

## Phase 4: Usage and Monitoring

### 4.1 Usage Tracking System

**Create new files:**
- `src/monitoring/UsageTracker.ts` - Usage tracking
- `src/monitoring/UsageStorage.ts` - Usage data storage
- `src/monitoring/MetricsCollector.ts` - Metrics collection
- `src/monitoring/index.ts` - Monitoring exports

**Key features:**
- Track routing decisions
- Cost calculation
- Project-based usage stats
- Time-based usage reports

### 4.2 Health Monitoring

**Create new files:**
- `src/monitoring/HealthMonitor.ts` - Health monitoring system
- `src/monitoring/HealthCheck.ts` - Health check interface

**Key features:**
- Server health checks
- Router health checks
- Plugin health checks
- Storage health checks
- Automatic health monitoring

## Phase 5: Enhanced Core Features

### 5.1 Token Counter Improvements

**Update existing files:**
- `src/utils/token-counter.ts` - Enhance with async support
- Add batch token counting
- Add model-specific token counting
- Add caching for performance

### 5.2 Routing Enhancements

**Update existing files:**
- `src/utils/router.ts` - Enhance with new features
- Add custom routing rules
- Add condition-based routing
- Add priority-based routing
- Add confidence scoring

### 5.3 Context Extraction

**Update existing files:**
- `src/utils/context-extractor.ts` - Enhanced context extraction
- Add more environment variable support
- Add context validation
- Add context transformation

## Implementation Priority

### High Priority (Immediate)
1. **Build System Setup** - Enable dual build (library + CLI)
2. **Type Definitions** - Add comprehensive TypeScript types
3. **ConduitServer Class** - Programmatic server management
4. **ConduitRouter Class** - Standalone routing engine

### Medium Priority (Week 2)
1. **Enhanced Plugin System** - Lifecycle hooks and health monitoring
2. **Synapse Integration** - Dedicated integration layer
3. **Configuration Management** - Programmatic configuration updates
4. **Usage Tracking** - Basic usage tracking system

### Lower Priority (Week 3-4)
1. **Advanced Monitoring** - Health checks and metrics
2. **Enhanced Context Extraction** - More environment variables
3. **Performance Optimizations** - Caching and batching
4. **Advanced Routing Rules** - Custom conditions and priorities

## Backward Compatibility Requirements

### Must Maintain
1. **CLI Interface** - All existing CLI commands and flags
2. **Configuration Files** - Existing JSON configuration format
3. **Plugin System** - Existing plugin interface
4. **HTTP API** - Existing server endpoints
5. **Environment Variables** - All current environment variable support

### Can Enhance
1. **Error Handling** - Improve error messages and handling
2. **Performance** - Optimize for library usage
3. **Logging** - Add structured logging options
4. **Configuration** - Add validation and better defaults

## Files to Create

### New Core Files
- `src/lib.ts` - Main library entry point
- `src/types/index.ts` - Type definitions
- `src/server/ConduitServer.ts` - Enhanced server class
- `src/router/ConduitRouter.ts` - Standalone router class
- `src/config/ConfigManager.ts` - Configuration management
- `src/integrations/synapse.ts` - Synapse integration
- `src/monitoring/UsageTracker.ts` - Usage tracking
- `src/monitoring/HealthMonitor.ts` - Health monitoring
- `tsconfig.lib.json` - Library build configuration

### New Support Files
- `src/integrations/index.ts` - Integration exports
- `src/config/index.ts` - Configuration exports
- `src/monitoring/index.ts` - Monitoring exports
- `src/monitoring/UsageStorage.ts` - Usage storage
- `src/monitoring/MetricsCollector.ts` - Metrics collection
- `src/monitoring/HealthCheck.ts` - Health check interface

## Files to Modify

### Core Files
- `package.json` - Add library build and exports
- `tsconfig.json` - Update for dual build
- `src/index.ts` - Re-export library functions
- `src/server.ts` - Integrate with ConduitServer class
- `src/serverWrapper.ts` - Enhance with new features

### Utility Files
- `src/utils/router.ts` - Enhance routing logic
- `src/utils/token-counter.ts` - Add async support
- `src/utils/context-extractor.ts` - Enhanced context extraction
- `src/utils/config-presets.ts` - Add validation
- `src/plugins/plugin-interface.ts` - Enhanced interface
- `src/plugins/plugin-manager.ts` - Improved management

### Build Files
- `.github/workflows/ci.yml` - Update for dual build
- `.github/workflows/release.yml` - Update for library publishing

## Success Criteria

### Phase 1 Complete
- ✅ Dual build system working (CLI + library)
- ✅ TypeScript declarations generated
- ✅ Library can be imported and used programmatically
- ✅ CLI continues to work as before

### Phase 2 Complete
- ✅ ConduitServer class can start/stop programmatically
- ✅ ConduitRouter can route requests independently
- ✅ Plugin system supports lifecycle hooks
- ✅ All existing functionality preserved

### Phase 3 Complete
- ✅ Synapse integration working with enhanced context
- ✅ Configuration can be updated programmatically
- ✅ Usage tracking captures routing decisions
- ✅ Health monitoring reports system status

### Final Success
- ✅ 100% backward compatibility maintained
- ✅ Library usage 10x faster than CLI spawning
- ✅ Memory usage 50% lower than CLI spawning
- ✅ 99.9% error handling coverage
- ✅ Comprehensive documentation and examples

## Testing Requirements

### Unit Tests
- ConduitServer lifecycle management
- ConduitRouter routing decisions
- Plugin system functionality
- Configuration management
- Usage tracking accuracy

### Integration Tests
- Synapse integration scenarios
- CLI compatibility tests
- Server startup/shutdown tests
- Plugin loading/unloading tests
- Configuration validation tests

### Performance Tests
- Library vs CLI performance comparison
- Memory usage benchmarks
- Concurrent request handling
- Plugin execution overhead
- Token counting performance

This plan provides a clear roadmap for transforming Conduit into a scalable library while maintaining full backward compatibility and adding powerful new features for Synapse integration.