UNPKG

@tehreet/conduit

Version:

LLM API gateway with intelligent routing, robust process management, and health monitoring

1,343 lines (1,161 loc) 40.8 kB
# Conduit-Synapse Integration Implementation Guide ## Overview This document provides comprehensive step-by-step instructions for integrating Conduit's LLM routing capabilities into Synapse. The integration enables intelligent model selection, usage tracking, and cost optimization while maintaining a seamless user experience. ## Table of Contents 1. [Architecture Overview](#architecture-overview) 2. [Prerequisites](#prerequisites) 3. [Phase 1: Core Integration](#phase-1-core-integration) 4. [Phase 2: UI Implementation](#phase-2-ui-implementation) 5. [Phase 3: Database Schema Updates](#phase-3-database-schema-updates) 6. [Phase 4: Service Layer Updates](#phase-4-service-layer-updates) 7. [Phase 5: Testing & Validation](#phase-5-testing--validation) 8. [Migration Strategy](#migration-strategy) 9. [Code Examples](#code-examples) ## Architecture Overview ``` ┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐ │ Synapse UI │────▶│ Synapse Core │────▶│ Conduit │ │ (Model Config) │ │ (Env Variables) │ │ (LLM Routing) │ └─────────────────┘ └──────────────────┘ └─────────────────┘ │ │ ▼ ▼ ┌──────────────────┐ ┌─────────────────┐ │ Supabase DB │ │ Claude CLI │ │ (Usage Tracking) │ │ (Wrapped) │ └──────────────────┘ └─────────────────┘ ``` ## Prerequisites 1. Conduit installed and configured on the system 2. Synapse development environment set up 3. Access to Supabase project 4. Node.js 18+ and npm installed ## Phase 1: Core Integration ### Step 1.1: Install Conduit in Synapse Add Conduit as a dependency to Synapse: ```bash cd ~/synapse npm install --save @tehreet/conduit@latest ``` ### Step 1.2: Create Conduit Service in Synapse Create a new service file: `src/main/services/conduit-service.ts` ```typescript import { spawn, ChildProcess } from 'child_process'; import { EventEmitter } from 'events'; import * as path from 'path'; import * as fs from 'fs'; import { app } from 'electron'; import type { SentryService } from './sentry-service'; import type { EnhancedAgentConfig } from '../../shared/types/agent-management'; export interface ConduitMetadata { routingDecision: { model: string; source: string; tokenCount: number; timestamp: string; }; projectContext?: { projectId: string; agentId?: string; }; usage?: { inputTokens: number; outputTokens: number; cost: number; }; } export class ConduitService extends EventEmitter { private conduitProcess: ChildProcess | null = null; private conduitBinaryPath: string; private isRunning: boolean = false; constructor(private sentryService: SentryService) { super(); // Find conduit-claude binary const userHome = process.env.HOME || process.env.USERPROFILE || ''; this.conduitBinaryPath = path.join(userHome, '.conduit', 'bin', 'conduit-claude'); // Ensure Conduit is installed this.ensureConduitInstalled(); } private async ensureConduitInstalled(): Promise<void> { if (!fs.existsSync(this.conduitBinaryPath)) { console.log('[ConduitService] Installing Conduit wrapper...'); try { // Run conduit install-wrapper command const { execSync } = require('child_process'); execSync('npx @tehreet/conduit install-wrapper', { stdio: 'inherit' }); } catch (error) { this.sentryService.captureException(error as Error, { context: 'ConduitService.ensureConduitInstalled' }); throw new Error('Failed to install Conduit wrapper'); } } } async startConduitServer(): Promise<void> { if (this.isRunning) { console.log('[ConduitService] Conduit server already running'); return; } try { const { execSync } = require('child_process'); execSync('npx @tehreet/conduit start', { stdio: 'inherit' }); this.isRunning = true; console.log('[ConduitService] Conduit server started'); } catch (error) { this.sentryService.captureException(error as Error, { context: 'ConduitService.startConduitServer' }); throw error; } } async stopConduitServer(): Promise<void> { if (!this.isRunning) { return; } try { const { execSync } = require('child_process'); execSync('npx @tehreet/conduit stop', { stdio: 'inherit' }); this.isRunning = false; console.log('[ConduitService] Conduit server stopped'); } catch (error) { this.sentryService.captureException(error as Error, { context: 'ConduitService.stopConduitServer' }); } } async executeClaude( args: string[], projectId: string, agentConfig?: EnhancedAgentConfig ): Promise<ChildProcess> { // Prepare environment variables for Conduit const env = { ...process.env, SYNAPSE_PROJECT_ID: projectId, SYNAPSE_PROJECT_MODEL_CONFIG: JSON.stringify({ enabled: true, defaultModel: agentConfig?.model || 'claude-3-5-sonnet-20241022' }) }; if (agentConfig) { env.SYNAPSE_AGENT_ID = agentConfig.id; env.SYNAPSE_AGENT_TYPE = agentConfig.persona_type; env.SYNAPSE_AGENT_MODEL_CONFIG = JSON.stringify({ useProjectDefaults: agentConfig.use_project_model ?? true, overrideModel: agentConfig.custom_model_id }); } // Add telemetry endpoint if configured const appConfig = this.getAppConfig(); if (appConfig.telemetryEnabled) { env.SYNAPSE_TELEMETRY_ENDPOINT = `http://localhost:${appConfig.port}/api/telemetry`; } // Spawn Claude through Conduit wrapper const claudeProcess = spawn(this.conduitBinaryPath, args, { env, stdio: ['pipe', 'pipe', 'pipe'] }); // Listen for metadata in NDJSON stream this.setupMetadataListener(claudeProcess, projectId, agentConfig?.id); return claudeProcess; } private setupMetadataListener( process: ChildProcess, projectId: string, agentId?: string ): void { let buffer = ''; process.stdout?.on('data', (chunk) => { buffer += chunk.toString(); const lines = buffer.split('\n'); buffer = lines.pop() || ''; for (const line of lines) { if (line.trim()) { try { const data = JSON.parse(line); if (data.type === 'conduit_metadata') { this.handleConduitMetadata(data.data, projectId, agentId); } } catch (error) { // Not JSON or not metadata, ignore } } } }); } private handleConduitMetadata( metadata: ConduitMetadata, projectId: string, agentId?: string ): void { console.log('[ConduitService] Received routing metadata:', metadata); // Emit events for UI updates this.emit('routing-decision', { projectId, agentId, ...metadata.routingDecision }); // Track usage if available if (metadata.usage) { this.emit('usage-tracked', { projectId, agentId, ...metadata.usage }); } } private getAppConfig(): any { // Get Synapse app configuration const userDataPath = app.getPath('userData'); const configPath = path.join(userDataPath, 'config.json'); try { if (fs.existsSync(configPath)) { return JSON.parse(fs.readFileSync(configPath, 'utf-8')); } } catch (error) { console.error('[ConduitService] Failed to read app config:', error); } return { telemetryEnabled: true, port: 3001 }; } } ``` ### Step 1.3: Update Session Service to Use Conduit Modify `src/main/services/session-service.ts` to use ConduitService: ```typescript // Add to imports import { ConduitService } from './conduit-service'; // Add to SessionService class private conduitService: ConduitService; // Update constructor constructor( supabaseService: SupabaseService, sentryService: SentryService, conduitService: ConduitService // Add this parameter ) { this.conduitService = conduitService; // ... existing code } // Update createClaudeProcess method private async createClaudeProcess( sessionId: string, agentId: string, message: string ): Promise<ChildProcess> { const session = this.sessions.get(sessionId); if (!session) { throw new Error('Session not found'); } const agent = await this.getAgentConfig(agentId); // Prepare Claude CLI arguments const args = ['--message', message]; // Add model if specified if (agent.custom_model_id && !agent.use_project_model) { args.push('--model', agent.custom_model_id); } // Use Conduit to execute Claude const claudeProcess = await this.conduitService.executeClaude( args, session.projectId, agent ); // Set up output handling this.setupOutputHandling(claudeProcess, sessionId); return claudeProcess; } ``` ## Phase 2: UI Implementation ### Step 2.1: Add Model Configuration UI Create a new component: `src/renderer/components/ModelConfiguration.tsx` ```tsx import React, { useState, useEffect } from 'react'; import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; import { Label } from '@/components/ui/label'; import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select'; import { Switch } from '@/components/ui/switch'; import { Button } from '@/components/ui/button'; import { Alert, AlertDescription } from '@/components/ui/alert'; import { InfoIcon, SaveIcon } from 'lucide-react'; interface ModelConfig { enabled: boolean; defaultModel: string; routingRules?: Array<{ name: string; condition: string; model: string; }>; } interface ModelConfigurationProps { projectId: string; agentId?: string; onSave: (config: ModelConfig) => Promise<void>; } const AVAILABLE_MODELS = [ { id: 'claude-3-5-sonnet-20241022', name: 'Claude 3.5 Sonnet', category: 'balanced' }, { id: 'claude-3-5-haiku-20241022', name: 'Claude 3.5 Haiku', category: 'fast' }, { id: 'claude-3-opus-20240229', name: 'Claude 3 Opus', category: 'powerful' }, ]; export const ModelConfiguration: React.FC<ModelConfigurationProps> = ({ projectId, agentId, onSave }) => { const [config, setConfig] = useState<ModelConfig>({ enabled: true, defaultModel: 'claude-3-5-sonnet-20241022' }); const [saving, setSaving] = useState(false); const [useProjectDefaults, setUseProjectDefaults] = useState(true); useEffect(() => { // Load existing configuration loadConfiguration(); }, [projectId, agentId]); const loadConfiguration = async () => { try { const response = await window.api.getModelConfiguration(projectId, agentId); if (response.success && response.config) { setConfig(response.config); if (agentId && response.useProjectDefaults !== undefined) { setUseProjectDefaults(response.useProjectDefaults); } } } catch (error) { console.error('Failed to load model configuration:', error); } }; const handleSave = async () => { setSaving(true); try { await onSave({ ...config, useProjectDefaults: agentId ? useProjectDefaults : undefined }); } finally { setSaving(false); } }; const isAgentConfig = !!agentId; return ( <Card> <CardHeader> <CardTitle> {isAgentConfig ? 'Agent Model Configuration' : 'Project Model Configuration'} </CardTitle> </CardHeader> <CardContent className="space-y-6"> {/* Enable/Disable Routing */} <div className="flex items-center justify-between"> <div className="space-y-0.5"> <Label htmlFor="routing-enabled">Enable Intelligent Routing</Label> <p className="text-sm text-muted-foreground"> Use Conduit to automatically select the best model based on context </p> </div> <Switch id="routing-enabled" checked={config.enabled} onCheckedChange={(enabled) => setConfig({ ...config, enabled })} disabled={isAgentConfig && useProjectDefaults} /> </div> {/* Use Project Defaults (Agent only) */} {isAgentConfig && ( <div className="flex items-center justify-between"> <div className="space-y-0.5"> <Label htmlFor="use-project-defaults">Use Project Defaults</Label> <p className="text-sm text-muted-foreground"> Inherit model settings from the project configuration </p> </div> <Switch id="use-project-defaults" checked={useProjectDefaults} onCheckedChange={setUseProjectDefaults} /> </div> )} {/* Model Selection */} {(!isAgentConfig || !useProjectDefaults) && ( <div className="space-y-2"> <Label htmlFor="default-model">Default Model</Label> <Select value={config.defaultModel} onValueChange={(model) => setConfig({ ...config, defaultModel: model })} disabled={!config.enabled} > <SelectTrigger id="default-model"> <SelectValue placeholder="Select a model" /> </SelectTrigger> <SelectContent> {AVAILABLE_MODELS.map((model) => ( <SelectItem key={model.id} value={model.id}> <div className="flex items-center justify-between w-full"> <span>{model.name}</span> <span className="text-xs text-muted-foreground ml-2"> {model.category} </span> </div> </SelectItem> ))} </SelectContent> </Select> </div> )} {/* Routing Rules Preview */} {config.enabled && !useProjectDefaults && ( <Alert> <InfoIcon className="h-4 w-4" /> <AlertDescription> <strong>Active Routing Rules:</strong> <ul className="mt-2 space-y-1 text-sm"> <li>• Long context (&gt;60k tokens) → Claude 3.5 Sonnet</li> <li>• Background tasks → Claude 3.5 Haiku</li> <li>• Complex reasoning → Claude 3 Opus</li> <li>• Default → {AVAILABLE_MODELS.find(m => m.id === config.defaultModel)?.name}</li> </ul> </AlertDescription> </Alert> )} {/* Save Button */} <Button onClick={handleSave} disabled={saving || (isAgentConfig && useProjectDefaults)} className="w-full" > <SaveIcon className="h-4 w-4 mr-2" /> {saving ? 'Saving...' : 'Save Configuration'} </Button> </CardContent> </Card> ); }; ``` ### Step 2.2: Add Usage Analytics Dashboard Create `src/renderer/components/UsageAnalytics.tsx`: ```tsx import React, { useState, useEffect } from 'react'; import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'; import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'; import { Progress } from '@/components/ui/progress'; import { BarChart, Bar, LineChart, Line, PieChart, Pie, Cell, XAxis, YAxis, CartesianGrid, Tooltip, ResponsiveContainer } from 'recharts'; import { formatDistanceToNow } from 'date-fns'; interface UsageData { modelUsage: Array<{ model: string; requests: number; tokens: number; cost: number; }>; costOverTime: Array<{ date: string; cost: number; }>; routingDecisions: Array<{ source: string; count: number; }>; recentRequests: Array<{ timestamp: string; model: string; tokens: number; cost: number; agentName: string; }>; } export const UsageAnalytics: React.FC<{ projectId: string }> = ({ projectId }) => { const [usageData, setUsageData] = useState<UsageData | null>(null); const [loading, setLoading] = useState(true); const [timeRange, setTimeRange] = useState<'day' | 'week' | 'month'>('week'); useEffect(() => { loadUsageData(); const interval = setInterval(loadUsageData, 30000); // Refresh every 30s return () => clearInterval(interval); }, [projectId, timeRange]); const loadUsageData = async () => { try { const response = await window.api.getUsageAnalytics(projectId, timeRange); if (response.success) { setUsageData(response.data); } } catch (error) { console.error('Failed to load usage data:', error); } finally { setLoading(false); } }; if (loading) { return <div>Loading usage data...</div>; } if (!usageData) { return <div>No usage data available</div>; } const COLORS = ['#8884d8', '#82ca9d', '#ffc658', '#ff6b6b']; const totalCost = usageData.modelUsage.reduce((sum, item) => sum + item.cost, 0); const totalTokens = usageData.modelUsage.reduce((sum, item) => sum + item.tokens, 0); return ( <div className="space-y-6"> {/* Summary Cards */} <div className="grid grid-cols-3 gap-4"> <Card> <CardHeader className="pb-2"> <CardTitle className="text-sm font-medium">Total Cost</CardTitle> </CardHeader> <CardContent> <div className="text-2xl font-bold">${totalCost.toFixed(4)}</div> <p className="text-xs text-muted-foreground">This {timeRange}</p> </CardContent> </Card> <Card> <CardHeader className="pb-2"> <CardTitle className="text-sm font-medium">Total Tokens</CardTitle> </CardHeader> <CardContent> <div className="text-2xl font-bold">{totalTokens.toLocaleString()}</div> <p className="text-xs text-muted-foreground">Processed</p> </CardContent> </Card> <Card> <CardHeader className="pb-2"> <CardTitle className="text-sm font-medium">Avg Cost/Request</CardTitle> </CardHeader> <CardContent> <div className="text-2xl font-bold"> ${(totalCost / Math.max(1, usageData.recentRequests.length)).toFixed(4)} </div> <p className="text-xs text-muted-foreground">Per request</p> </CardContent> </Card> </div> {/* Detailed Analytics */} <Tabs defaultValue="models" className="space-y-4"> <TabsList className="grid w-full grid-cols-4"> <TabsTrigger value="models">Model Usage</TabsTrigger> <TabsTrigger value="costs">Cost Trends</TabsTrigger> <TabsTrigger value="routing">Routing Sources</TabsTrigger> <TabsTrigger value="recent">Recent Activity</TabsTrigger> </TabsList> <TabsContent value="models" className="space-y-4"> <Card> <CardHeader> <CardTitle>Model Usage Distribution</CardTitle> </CardHeader> <CardContent> <ResponsiveContainer width="100%" height={300}> <BarChart data={usageData.modelUsage}> <CartesianGrid strokeDasharray="3 3" /> <XAxis dataKey="model" /> <YAxis /> <Tooltip /> <Bar dataKey="requests" fill="#8884d8" name="Requests" /> <Bar dataKey="tokens" fill="#82ca9d" name="Tokens" /> </BarChart> </ResponsiveContainer> </CardContent> </Card> {/* Cost by Model */} <Card> <CardHeader> <CardTitle>Cost by Model</CardTitle> </CardHeader> <CardContent> <ResponsiveContainer width="100%" height={300}> <PieChart> <Pie data={usageData.modelUsage} dataKey="cost" nameKey="model" cx="50%" cy="50%" outerRadius={100} label={(entry) => `${entry.model}: $${entry.cost.toFixed(4)}`} > {usageData.modelUsage.map((entry, index) => ( <Cell key={`cell-${index}`} fill={COLORS[index % COLORS.length]} /> ))} </Pie> <Tooltip /> </PieChart> </ResponsiveContainer> </CardContent> </Card> </TabsContent> <TabsContent value="costs"> <Card> <CardHeader> <CardTitle>Cost Over Time</CardTitle> </CardHeader> <CardContent> <ResponsiveContainer width="100%" height={400}> <LineChart data={usageData.costOverTime}> <CartesianGrid strokeDasharray="3 3" /> <XAxis dataKey="date" /> <YAxis /> <Tooltip formatter={(value: number) => `$${value.toFixed(4)}`} /> <Line type="monotone" dataKey="cost" stroke="#8884d8" strokeWidth={2} /> </LineChart> </ResponsiveContainer> </CardContent> </Card> </TabsContent> <TabsContent value="routing"> <Card> <CardHeader> <CardTitle>Routing Decision Sources</CardTitle> </CardHeader> <CardContent> <ResponsiveContainer width="100%" height={400}> <PieChart> <Pie data={usageData.routingDecisions} dataKey="count" nameKey="source" cx="50%" cy="50%" outerRadius={120} label={(entry) => `${entry.source}: ${entry.count}`} > {usageData.routingDecisions.map((entry, index) => ( <Cell key={`cell-${index}`} fill={COLORS[index % COLORS.length]} /> ))} </Pie> <Tooltip /> </PieChart> </ResponsiveContainer> </CardContent> </Card> </TabsContent> <TabsContent value="recent"> <Card> <CardHeader> <CardTitle>Recent Requests</CardTitle> </CardHeader> <CardContent> <div className="space-y-2"> {usageData.recentRequests.slice(0, 10).map((request, idx) => ( <div key={idx} className="flex items-center justify-between p-3 border rounded-lg" > <div className="flex-1"> <div className="font-medium">{request.agentName}</div> <div className="text-sm text-muted-foreground"> {request.model} • {request.tokens.toLocaleString()} tokens </div> </div> <div className="text-right"> <div className="font-medium">${request.cost.toFixed(4)}</div> <div className="text-xs text-muted-foreground"> {formatDistanceToNow(new Date(request.timestamp), { addSuffix: true })} </div> </div> </div> ))} </div> </CardContent> </Card> </TabsContent> </Tabs> </div> ); }; ``` ## Phase 3: Database Schema Updates ### Step 3.1: Create Migration for Model Configuration Create `supabase/migrations/20250115_add_conduit_integration.sql`: ```sql -- Add model configuration columns to projects table ALTER TABLE projects ADD COLUMN model_config JSONB DEFAULT '{"enabled": true, "defaultModel": "claude-3-5-sonnet-20241022"}'::jsonb, ADD COLUMN routing_stats JSONB DEFAULT '{}'::jsonb; -- Add model configuration to agents table ALTER TABLE agents ADD COLUMN use_project_model BOOLEAN DEFAULT true, ADD COLUMN custom_model_id TEXT, ADD COLUMN model_stats JSONB DEFAULT '{}'::jsonb; -- Create usage tracking table CREATE TABLE IF NOT EXISTS usage_tracking ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE, agent_id UUID REFERENCES agents(id) ON DELETE SET NULL, model TEXT NOT NULL, input_tokens INTEGER NOT NULL, output_tokens INTEGER NOT NULL, total_tokens INTEGER GENERATED ALWAYS AS (input_tokens + output_tokens) STORED, cost DECIMAL(10, 6) NOT NULL, routing_source TEXT NOT NULL, metadata JSONB DEFAULT '{}'::jsonb, created_at TIMESTAMPTZ DEFAULT NOW(), -- Indexes for performance INDEX idx_usage_project_created (project_id, created_at DESC), INDEX idx_usage_agent_created (agent_id, created_at DESC), INDEX idx_usage_model (model), INDEX idx_usage_created (created_at DESC) ); -- Create routing decisions table for analytics CREATE TABLE IF NOT EXISTS routing_decisions ( id UUID PRIMARY KEY DEFAULT gen_random_uuid(), project_id UUID NOT NULL REFERENCES projects(id) ON DELETE CASCADE, agent_id UUID REFERENCES agents(id) ON DELETE SET NULL, requested_model TEXT, selected_model TEXT NOT NULL, routing_source TEXT NOT NULL, token_count INTEGER NOT NULL, decision_metadata JSONB DEFAULT '{}'::jsonb, created_at TIMESTAMPTZ DEFAULT NOW(), -- Indexes INDEX idx_routing_project_created (project_id, created_at DESC), INDEX idx_routing_source (routing_source) ); -- Create function to get usage statistics CREATE OR REPLACE FUNCTION get_usage_statistics( p_project_id UUID, p_time_range INTERVAL DEFAULT INTERVAL '7 days' ) RETURNS TABLE ( model TEXT, request_count BIGINT, total_tokens BIGINT, total_cost DECIMAL, avg_tokens_per_request NUMERIC ) AS $$ BEGIN RETURN QUERY SELECT u.model, COUNT(*)::BIGINT as request_count, SUM(u.total_tokens)::BIGINT as total_tokens, SUM(u.cost)::DECIMAL as total_cost, AVG(u.total_tokens)::NUMERIC as avg_tokens_per_request FROM usage_tracking u WHERE u.project_id = p_project_id AND u.created_at >= NOW() - p_time_range GROUP BY u.model ORDER BY request_count DESC; END; $$ LANGUAGE plpgsql; -- Create RLS policies ALTER TABLE usage_tracking ENABLE ROW LEVEL SECURITY; ALTER TABLE routing_decisions ENABLE ROW LEVEL SECURITY; -- Users can only see their own project's usage CREATE POLICY "Users can view own project usage" ON usage_tracking FOR SELECT USING ( project_id IN ( SELECT id FROM projects WHERE user_id = auth.uid() ) ); CREATE POLICY "Users can insert own project usage" ON usage_tracking FOR INSERT WITH CHECK ( project_id IN ( SELECT id FROM projects WHERE user_id = auth.uid() ) ); -- Similar policies for routing_decisions CREATE POLICY "Users can view own routing decisions" ON routing_decisions FOR SELECT USING ( project_id IN ( SELECT id FROM projects WHERE user_id = auth.uid() ) ); CREATE POLICY "Users can insert own routing decisions" ON routing_decisions FOR INSERT WITH CHECK ( project_id IN ( SELECT id FROM projects WHERE user_id = auth.uid() ) ); ``` ## Phase 4: Service Layer Updates ### Step 4.1: Update Main Process Modify `src/main/index.ts` to initialize Conduit: ```typescript // Add to imports import { ConduitService } from './services/conduit-service'; // In app.whenReady() app.whenReady().then(async () => { // ... existing initialization code ... // Initialize Conduit service const conduitService = new ConduitService(sentryService); // Start Conduit server await conduitService.startConduitServer(); // Pass to session service const sessionService = new SessionService( supabaseService, sentryService, conduitService ); // ... rest of initialization ... // Ensure Conduit stops on app quit app.on('before-quit', async () => { await conduitService.stopConduitServer(); }); }); ``` ### Step 4.2: Add IPC Handlers Add to `src/main/ipc-handlers.ts`: ```typescript // Model configuration handlers ipcMain.handle('get-model-configuration', async (event, projectId: string, agentId?: string) => { try { if (agentId) { // Get agent-specific configuration const agent = await supabaseService.getAgent(agentId); return { success: true, config: agent.model_config || { enabled: true, defaultModel: 'claude-3-5-sonnet-20241022' }, useProjectDefaults: agent.use_project_model }; } else { // Get project configuration const project = await supabaseService.getProject(projectId); return { success: true, config: project.model_config || { enabled: true, defaultModel: 'claude-3-5-sonnet-20241022' } }; } } catch (error) { console.error('Failed to get model configuration:', error); return { success: false, error: error.message }; } }); ipcMain.handle('save-model-configuration', async (event, projectId: string, agentId: string | undefined, config: any) => { try { if (agentId) { // Update agent configuration await supabaseService.updateAgent(agentId, { use_project_model: config.useProjectDefaults, custom_model_id: config.useProjectDefaults ? null : config.defaultModel, model_config: config.useProjectDefaults ? null : config }); } else { // Update project configuration await supabaseService.updateProject(projectId, { model_config: config }); } return { success: true }; } catch (error) { console.error('Failed to save model configuration:', error); return { success: false, error: error.message }; } }); // Usage analytics handlers ipcMain.handle('get-usage-analytics', async (event, projectId: string, timeRange: string) => { try { const interval = { 'day': '1 day', 'week': '7 days', 'month': '30 days' }[timeRange] || '7 days'; // Get model usage statistics const { data: modelUsage } = await supabaseService.client .rpc('get_usage_statistics', { p_project_id: projectId, p_time_range: interval }); // Get cost over time const { data: costData } = await supabaseService.client .from('usage_tracking') .select('cost, created_at') .eq('project_id', projectId) .gte('created_at', new Date(Date.now() - parseDuration(interval)).toISOString()) .order('created_at', { ascending: true }); // Get routing decision sources const { data: routingData } = await supabaseService.client .from('routing_decisions') .select('routing_source') .eq('project_id', projectId) .gte('created_at', new Date(Date.now() - parseDuration(interval)).toISOString()); // Get recent requests const { data: recentRequests } = await supabaseService.client .from('usage_tracking') .select('*, agents(name)') .eq('project_id', projectId) .order('created_at', { ascending: false }) .limit(50); // Process data for charts const costOverTime = processCostOverTime(costData); const routingDecisions = processRoutingDecisions(routingData); return { success: true, data: { modelUsage: modelUsage || [], costOverTime, routingDecisions, recentRequests: recentRequests?.map(r => ({ timestamp: r.created_at, model: r.model, tokens: r.total_tokens, cost: r.cost, agentName: r.agents?.name || 'Unknown' })) || [] } }; } catch (error) { console.error('Failed to get usage analytics:', error); return { success: false, error: error.message }; } }); // Listen for Conduit events conduitService.on('usage-tracked', async (data) => { try { // Save usage data to Supabase await supabaseService.client .from('usage_tracking') .insert({ project_id: data.projectId, agent_id: data.agentId, model: data.model, input_tokens: data.inputTokens, output_tokens: data.outputTokens, cost: data.cost, routing_source: data.source || 'conduit', metadata: { timestamp: data.timestamp, sessionId: data.sessionId } }); } catch (error) { console.error('Failed to track usage:', error); } }); conduitService.on('routing-decision', async (data) => { try { // Save routing decision for analytics await supabaseService.client .from('routing_decisions') .insert({ project_id: data.projectId, agent_id: data.agentId, selected_model: data.model, routing_source: data.source, token_count: data.tokenCount, decision_metadata: { timestamp: data.timestamp } }); } catch (error) { console.error('Failed to save routing decision:', error); } }); ``` ## Phase 5: Testing & Validation ### Step 5.1: Create Integration Tests Create `src/test/conduit-integration.test.ts`: ```typescript import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { ConduitService } from '../main/services/conduit-service'; import { spawn } from 'child_process'; import * as path from 'path'; describe('Conduit Integration', () => { let conduitService: ConduitService; let mockSentryService: any; beforeAll(async () => { mockSentryService = { captureException: jest.fn() }; conduitService = new ConduitService(mockSentryService); await conduitService.startConduitServer(); }); afterAll(async () => { await conduitService.stopConduitServer(); }); it('should wrap Claude CLI with environment variables', async () => { const args = ['--message', 'Test message']; const projectId = 'test-project-123'; const agentConfig = { id: 'test-agent-456', model: 'claude-3-5-haiku-20241022', use_project_model: false, custom_model_id: 'claude-3-5-haiku-20241022' }; const process = await conduitService.executeClaude(args, projectId, agentConfig); expect(process).toBeDefined(); expect(process.pid).toBeDefined(); // Verify environment variables were set const env = process.spawnargs.find(arg => arg.includes('SYNAPSE_PROJECT_ID')); expect(env).toBeDefined(); }); it('should handle metadata from Conduit', (done) => { const mockMetadata = { type: 'conduit_metadata', data: { routingDecision: { model: 'claude-3-5-sonnet-20241022', source: 'synapse-context', tokenCount: 150, timestamp: new Date().toISOString() } } }; conduitService.once('routing-decision', (data) => { expect(data.model).toBe('claude-3-5-sonnet-20241022'); expect(data.source).toBe('synapse-context'); expect(data.tokenCount).toBe(150); done(); }); // Simulate metadata reception conduitService['handleConduitMetadata'](mockMetadata.data, 'test-project', 'test-agent'); }); it('should track usage correctly', (done) => { const mockUsage = { inputTokens: 100, outputTokens: 200, cost: 0.0015 }; conduitService.once('usage-tracked', (data) => { expect(data.inputTokens).toBe(100); expect(data.outputTokens).toBe(200); expect(data.cost).toBe(0.0015); done(); }); // Simulate usage tracking conduitService['handleConduitMetadata']({ routingDecision: { model: 'test', source: 'test', tokenCount: 300, timestamp: '' }, usage: mockUsage }, 'test-project', 'test-agent'); }); }); ``` ### Step 5.2: Manual Testing Checklist ```markdown ## Conduit-Synapse Integration Testing Checklist ### Pre-Integration - [ ] Conduit installed globally or locally - [ ] Conduit server can start/stop successfully - [ ] Wrapper script installed at ~/.conduit/bin/conduit-claude ### Core Integration - [ ] Synapse can start Conduit server on launch - [ ] Synapse properly stops Conduit server on quit - [ ] Claude commands are routed through Conduit wrapper - [ ] Environment variables are passed correctly ### Model Configuration UI - [ ] Project model configuration saves correctly - [ ] Agent model configuration respects "use project defaults" - [ ] Model selection dropdown shows all available models - [ ] Routing enable/disable toggle works ### Usage Tracking - [ ] Usage data is saved to Supabase - [ ] Routing decisions are logged - [ ] Cost calculations are accurate - [ ] Analytics dashboard displays correct data ### Performance - [ ] No noticeable latency added to Claude responses - [ ] UI remains responsive during model selection - [ ] Analytics load quickly (<2 seconds) ### Error Handling - [ ] Graceful fallback if Conduit is unavailable - [ ] Clear error messages for configuration issues - [ ] No data loss on connection failures ``` ## Migration Strategy ### For Existing Users Since you mentioned you don't care about migrating users and can clear data: 1. **Backup Current Data** (optional) ```bash # Export current sessions/agents if needed npm run export-data ``` 2. **Clear Existing Data** ```sql -- Run in Supabase SQL editor TRUNCATE projects CASCADE; TRUNCATE agents CASCADE; TRUNCATE sessions CASCADE; ``` 3. **Run Migrations** ```bash cd ~/synapse npx supabase migration up ``` 4. **Fresh Start** - Launch updated Synapse - Create new project with model configuration - Create agents with desired settings ## Code Examples ### Example: Using Conduit from Synapse Code ```typescript // In any Synapse service that needs to call Claude const response = await conduitService.executeClaude( ['--message', userMessage, '--thinking'], projectId, agentConfig ); // Handle streaming response response.stdout.on('data', (chunk) => { // Process Claude's response const data = chunk.toString(); // ... handle NDJSON stream }); ``` ### Example: Custom Routing Plugin for Synapse Create `~/.conduit/plugins/synapse-custom-routing.js`: ```javascript class SynapseCustomRouting { constructor() { this.name = 'synapse-custom-routing'; this.version = '1.0.0'; } async customRouter(context) { // Access Synapse-specific context const { synapseContext } = context; // Custom routing logic based on agent type if (synapseContext?.agentType === 'coding') { return 'claude-3-5-sonnet-20241022'; } if (synapseContext?.agentType === 'research') { return 'claude-3-opus-20240229'; } // Let Conduit handle default routing return null; } } module.exports = SynapseCustomRouting; ``` ## Troubleshooting ### Common Issues and Solutions 1. **Conduit wrapper not found** ```bash # Reinstall wrapper cd ~/synapse npx @tehreet/conduit install-wrapper ``` 2. **Port conflicts** ```bash # Check if Conduit is already running npx @tehreet/conduit status # Stop if needed npx @tehreet/conduit stop ``` 3. **Environment variables not passing** - Check Electron's process spawning - Verify env object in executeClaude method - Enable debug logging: `export LOG=true` 4. **Usage tracking not working** - Verify Supabase RLS policies - Check network connectivity - Ensure migrations ran successfully ## Next Steps 1. Implement the code changes in phases 2. Test each phase thoroughly before moving to next 3. Monitor usage and costs after deployment 4. Iterate on routing rules based on actual usage patterns 5. Consider adding more advanced features: - Custom routing rules UI - Budget alerts - Team usage sharing - Export usage reports