sf-agent-framework/docs/02-architecture/README.md

AI Agent Orchestration Framework for Salesforce Development - Two-phase architecture with 70% context reduction

# System Architecture

## Overview

The SF-Agent Framework implements a sophisticated multi-layered architecture designed for enterprise-scale Salesforce development. This document provides a comprehensive view of the system architecture, component interactions, and data flow patterns.

## Architecture Documentation

- **[Architecture Diagrams](./architecture-diagrams.md)** - Complete Mermaid diagrams for all components
- **[Architecture Explanation](./architecture-explanation.md)** - Detailed explanation of how and why
- **[Data Flow Patterns](./data-flow.md)** - How data moves through the system

## High-Level System Architecture

```mermaid
graph TB
    subgraph "User Interface Layer"
        CLI[CLI Interface]
        WebUI[Web UI/Chat]
        IDE[IDE Plugins]
        API[REST API]
    end

    subgraph "Orchestration Layer"
        PM[Phase Manager<br/>128k/32k]
        WE[Workflow Engine]
        CM[Context Manager]
    end

    subgraph "Agent Layer"
        PA[Planning Agents<br/>128k context]
        DA[Development Agents<br/>32k context]
        QA[Quality Agents]
        SA[Specialty Agents]
    end

    subgraph "Processing Layer"
        DP[Document Processor]
        TE[Template Engine]
        TE2[Task Executor]
    end

    subgraph "Storage Layer"
        AR[Artifact Registry]
        DS[Document Store]
        MD[Metrics Database]
    end

    subgraph "Integration Layer"
        SF[Salesforce API]
        GIT[Git Integration]
        MCP[MCP Protocol]
    end

    CLI --> PM
    WebUI --> PM
    IDE --> PM
    API --> PM

    PM --> WE
    WE --> CM

    CM --> PA
    CM --> DA
    PA --> QA
    DA --> SA

    PA --> DP
    DA --> TE
    QA --> TE2

    DP --> AR
    TE --> DS
    TE2 --> MD

    AR --> SF
    DS --> GIT
    MD --> MCP

    style CLI fill:#e3f2fd
    style WebUI fill:#e3f2fd
    style IDE fill:#e3f2fd
    style API fill:#e3f2fd
    style PM fill:#fff9c4
    style PA fill:#e8f5e9
    style DA fill:#fce4ec
```

## Core Components

### 1. User Interface Layer

#### Web UI (Chat Interface)

- **Purpose**: Browser-based interaction with AI platforms
- **Components**:
  - Web bundle loader
  - Slash command processor
  - Session state manager
- **Integrations**: ChatGPT, Gemini, Claude

#### CLI Interface

- **Purpose**: Command-line operations for developers
- **Components**:
  - Command parser (`commander.js`)
  - Interactive prompts (`inquirer`)
  - Progress indicators (`ora`)
- **Features**:
  - Auto-completion
  - Command history
  - Batch operations

#### IDE Plugins

- **Purpose**: Native IDE integration
- **Supported IDEs**:
  - Cursor
  - Claude Code (Windsurf)
  - VS Code
  - IntelliJ IDEA
- **Features**:
  - Context awareness
  - Code generation
  - Real-time validation

#### API Layer

- **Purpose**: Programmatic access
- **Protocols**: REST, GraphQL (planned)
- **Authentication**: Token-based
- **Rate Limiting**: Configurable

### 2. Orchestration Layer

#### Phase Manager

- **Responsibilities**:
  - Track current phase (planning/development)
  - Manage context allocation
  - Handle phase transitions
  - Optimize token usage

```javascript
class PhaseManager {
  currentPhase: 'planning' | 'development'
  contextLimit: number
  contextUsed: number

  transition(newPhase) {
    // Save current state
    // Load new phase configuration
    // Adjust context limits
  }
}
```

#### Workflow Engine

- **Features**:
  - Sequential execution
  - Parallel processing
  - Conditional branching
  - User choice points
  - Validation gates

```yaml
workflow:
  phases:
    - planning:
        parallel: true
        tracks: [requirements, architecture]
    - development:
        sequential: true
        steps: [implement, test, deploy]
```

#### Context Manager

- **Purpose**: Intelligent context loading
- **Strategies**:
  - Lazy loading
  - Predictive caching
  - LRU eviction
  - Compression

### 3. Agent Layer

#### Planning Agents (128k context)

- `sf-product-manager`: Requirements and roadmap
- `sf-business-analyst`: Process analysis
- `sf-architect`: Solution design
- `sf-technical-architect`: Technical design
- `sf-data-architect`: Data modeling
- `sf-integration-architect`: Integration patterns
- `sf-security-architect`: Security design

#### Development Agents (32k context)

- `sf-developer`: Apex and LWC coding
- `sf-admin`: Configuration
- `sf-qa`: Testing
- `sf-devops-lead`: CI/CD
- `sf-release-manager`: Deployment

#### Quality Agents

- `sf-security`: Security scanning
- `sf-performance`: Performance optimization
- `sf-compliance`: Compliance checking

#### Specialty Agents

- `sf-omnistudio-architect`: OmniStudio
- `sf-mcp-architect`: MCP integration
- `sf-data-migration-specialist`: Data migration

### 4. Processing Layer

#### Document Processor

- **Capabilities**:
  - Markdown parsing
  - YAML processing
  - Template rendering
  - Content extraction

#### Template Engine

- **Features**:
  - Variable substitution
  - Conditional sections
  - Loop structures
  - LLM instructions

```yaml
template:
  variables:
    project_name: '{{project}}'
  sections:
    - condition: '{{has_integration}}'
      content: 'Integration requirements...'
```

#### Task Executor

- **Responsibilities**:
  - Task queuing
  - Dependency resolution
  - Parallel execution
  - Error handling

### 5. Storage Layer

#### Artifact Registry

- **Purpose**: Track all created artifacts
- **Schema**:

```javascript
{
  id: "AR-123",
  type: "document",
  path: "/docs/requirements.md",
  created_by: "sf-architect",
  created_at: "2025-08-11T10:00:00Z",
  hash: "sha256:...",
  dependencies: ["AR-100", "AR-101"]
}
```

#### Document Store

- **Structure**:

```
docs/
├── requirements/
├── architecture/
├── stories/
├── handoffs/
└── artifacts/
```

#### Metrics Database

- **Tracked Metrics**:
  - Context usage
  - Execution time
  - Success rate
  - Token consumption

### 6. Integration Layer

#### Salesforce API

- **Integrations**:
  - Metadata API
  - Tooling API
  - REST API
  - Bulk API
- **Operations**:
  - Deploy
  - Retrieve
  - Query
  - Execute

#### Git Integration

- **Features**:
  - Auto-commit
  - Branch management
  - PR creation
  - Merge strategies

#### MCP Protocol

- **Purpose**: Model Context Protocol
- **Features**:
  - Dynamic context loading
  - Cross-model compatibility
  - Streaming responses

## Data Flow Patterns

### 1. Planning Phase Flow

```
User Input → Orchestrator → Planning Agent → Template Engine → Document Store
     ↓                           ↓                  ↓              ↓
   Context ← Context Manager ← Dependencies ← Template ← Artifact Registry
```

### 2. Development Phase Flow

```
Story Queue → Development Agent → Code Generator → Salesforce API
      ↓              ↓                  ↓              ↓
   Context ← Lean Context ← Templates ← Validation ← Deployment
```

### 3. Handoff Flow

```
Source Agent → Handoff Protocol → Package Creation → Target Agent
       ↓              ↓                  ↓               ↓
   Artifacts → Validation → Registry → Notification → Acceptance
```

### 4. Web Bundle Flow

```
Framework → Bundle Builder → Single File → Web UI → Slash Commands
     ↓            ↓              ↓           ↓            ↓
  Agents → Dependencies → Compression → Upload → Execution
```

## State Management

### Global State

```javascript
{
  session: {
    id: "session-123",
    user: "developer@example.com",
    started: "2025-08-11T10:00:00Z"
  },
  phase: {
    current: "development",
    context_limit: 32000,
    context_used: 15000
  },
  workflow: {
    id: "salesforce-implementation",
    step: 3,
    total_steps: 7,
    state: "in_progress"
  },
  artifacts: {
    created: ["requirements.md", "architecture.md"],
    pending: ["story-001.md"],
    registry: Map<string, Artifact>
  }
}
```

### Agent State

```javascript
{
  agent: {
    id: "sf-developer",
    type: "development",
    mode: "lean",
    context: {
      loaded: ["story-001.md", "coding-standards.md"],
      available: 32000,
      used: 8000
    }
  }
}
```

## Communication Protocols

### Inter-Agent Communication

```javascript
// Message Format
{
  from: "sf-architect",
  to: "sf-developer",
  type: "handoff",
  payload: {
    artifacts: ["architecture.md"],
    instructions: "Implement user service",
    validation: ["unit-tests", "code-coverage"]
  },
  timestamp: "2025-08-11T10:00:00Z"
}
```

### CLI-to-Framework

```javascript
// Command Structure
{
  command: "workflow",
  action: "start",
  params: {
    workflow: "salesforce-implementation",
    interactive: true
  },
  context: {
    cwd: "/project",
    user: "developer"
  }
}
```

## Security Architecture

### Authentication

- **Methods**: Token, OAuth, SSO
- **Storage**: Encrypted credentials
- **Rotation**: Automatic token refresh

### Authorization

- **Model**: Role-Based Access Control (RBAC)
- **Levels**: Admin, Developer, Viewer
- **Enforcement**: Per-command, per-agent

### Data Protection

- **Encryption**: At rest and in transit
- **Sanitization**: PII removal
- **Audit**: All operations logged

## Performance Optimization

### Context Optimization

- **Strategies**:
  - Lazy loading
  - Incremental loading
  - Compression
  - Caching

### Parallel Processing

- **Areas**:
  - Multi-agent execution
  - Batch operations
  - Concurrent API calls

### Caching Strategy

```javascript
// Cache Layers
L1: Memory (in-process)
L2: File system
L3: Remote cache (Redis)
```

## Scalability Considerations

### Horizontal Scaling

- **Agent Distribution**: Multiple agent instances
- **Load Balancing**: Round-robin, least-loaded
- **Session Affinity**: Sticky sessions

### Vertical Scaling

- **Resource Allocation**: Dynamic context limits
- **Priority Queues**: Critical path optimization
- **Batch Processing**: Bulk operations

## Monitoring & Observability

### Metrics Collection

```javascript
{
  performance: {
    response_time: 250, // ms
    token_usage: 15000,
    cache_hit_rate: 0.85
  },
  reliability: {
    success_rate: 0.98,
    error_rate: 0.02,
    retry_count: 3
  },
  usage: {
    active_sessions: 42,
    daily_requests: 1500,
    unique_users: 25
  }
}
```

### Logging Strategy

- **Levels**: ERROR, WARN, INFO, DEBUG
- **Structured**: JSON format
- **Centralized**: Log aggregation
- **Retention**: 30 days default

### Health Checks

```javascript
GET /health
{
  status: "healthy",
  checks: {
    database: "ok",
    filesystem: "ok",
    memory: "ok",
    integrations: {
      salesforce: "ok",
      git: "ok"
    }
  }
}
```

## Deployment Architecture

### Local Development

```
Developer Machine
├── SF-Agent Framework
├── Project Code
├── Local Git
└── Salesforce Scratch Org
```

### CI/CD Pipeline

```
Git Push → CI Server → Tests → Build → Deploy → Salesforce Org
     ↓         ↓         ↓       ↓       ↓           ↓
  Webhook → Jenkins → Jest → Webpack → SFDX → Production
```

### Production Deployment

```
Load Balancer
    ├── Framework Instance 1
    ├── Framework Instance 2
    └── Framework Instance N
           ↓
    Shared Storage
    Shared Cache
    Metrics DB
```

## Future Architecture Enhancements

### Planned Features

1. **Microservices Architecture**: Decompose into services
2. **Event-Driven Architecture**: Event bus integration
3. **GraphQL API**: Flexible data queries
4. **WebSocket Support**: Real-time updates
5. **Distributed Caching**: Redis/Memcached
6. **Container Orchestration**: Kubernetes support

### Research Areas

1. **AI Model Integration**: Multiple LLM support
2. **Federated Learning**: Learn from usage patterns
3. **Quantum-Ready**: Future quantum integration
4. **Edge Computing**: Local processing capabilities

---

_Last Updated: 2025-08-11_
_Version: 4.0.0_