context-forge/.claude/docs/orchestration.md

AI orchestration platform with autonomous teams, enhancement planning, migration tools, 25+ slash commands, checkpoints & hooks. Multi-IDE: Claude, Cursor, Windsurf, Cline, Copilot

# Orchestration System Documentation

## Overview

The Context Forge Orchestration System enables autonomous AI agent teams to work on your project 24/7 without human intervention. By leveraging tmux session management and self-scheduling capabilities, teams of AI agents can collaborate, implement features, and maintain code quality continuously.

## Key Features

### 1. Autonomous Operation
- **24/7 Development**: Agents work continuously without human intervention
- **Self-Scheduling**: Agents schedule their own check-ins and manage workload
- **Git Discipline**: Automatic commits every 30 minutes prevent work loss
- **Error Recovery**: Agents can recover from failures and continue working

### 2. Team Hierarchy
The system implements a three-tier hierarchy to overcome context window limitations:

```
         Orchestrator (Strategic Oversight)
              /                \
    Project Manager         Project Manager
    /      |      \         /      |      \
Developer  QA   DevOps  Developer  QA   Researcher
```

### 3. Communication Models
- **Hub-and-Spoke**: All communication flows through Project Managers (default)
- **Hierarchical**: Follow organizational chart for communication
- **Mesh**: Direct communication between any agents (for smaller teams)

## Getting Started

### Prerequisites

1. **tmux**: Required for agent session management
   ```bash
   # macOS
   brew install tmux
   
   # Ubuntu/Debian
   sudo apt-get install tmux
   ```

2. **Git Repository**: Project must be initialized with git
   ```bash
   git init
   ```

3. **Context Forge Setup**: Project should have CLAUDE.md and PRPs
   ```bash
   context-forge init
   ```

### Basic Usage

Deploy a default team:
```bash
context-forge orchestrate
```

Deploy a small team:
```bash
context-forge orchestrate small
```

Deploy a large team:
```bash
context-forge orchestrate large
```

### Command Options

```bash
context-forge orchestrate [size] [options]

Arguments:
  size                    Team size: small, medium (default), or large

Options:
  -s, --strategy          Deployment strategy: big-bang, phased, or adaptive
  -c, --communication     Communication model: hub-and-spoke, hierarchical, or mesh
  --no-git               Disable git auto-commit
  --no-scheduling        Disable self-scheduling
  --commit-interval      Git commit interval in minutes (default: 30)
  --check-interval       Self-scheduling interval in minutes (default: 15)
```

## Team Configurations

### Small Team (4 agents)
Best for: Simple projects, prototypes, focused features
- 1 Orchestrator
- 1 Project Manager
- 2 Developers

### Medium Team (5 agents) - Default
Best for: Standard projects, balanced workload
- 1 Orchestrator
- 1 Project Manager
- 2 Developers
- 1 QA Engineer

### Large Team (11 agents)
Best for: Complex projects, enterprise applications
- 1 Orchestrator
- 2 Project Managers (Frontend & Backend)
- 4 Developers
- 2 QA Engineers
- 1 DevOps Engineer
- 1 Code Reviewer

## Agent Roles and Responsibilities

### Orchestrator
**Purpose**: High-level oversight and strategic decisions
- Monitor overall project progress
- Make architectural decisions
- Resolve cross-team dependencies
- Ensure quality standards
- Handle escalations

**Constraints**:
- Does not implement code directly
- Does not micromanage team members
- Focuses on strategic decisions

### Project Manager
**Purpose**: Team coordination and quality assurance
- Coordinate developer tasks
- Track progress and blockers
- Ensure code quality
- Report to orchestrator
- Enforce git discipline

**Key Behaviors**:
- Aggregates status from team members
- Identifies and escalates blockers
- Maintains exceptionally high standards

### Developer
**Purpose**: Feature implementation and technical execution
- Implement features according to PRPs
- Write comprehensive tests
- Follow coding standards
- Document code
- Collaborate with team

**Key Behaviors**:
- Commits code every 30 minutes
- Creates feature branches
- Writes tests before implementation

### QA Engineer
**Purpose**: Quality assurance and testing
- Create test plans
- Execute tests thoroughly
- Track and report bugs
- Verify fixes
- Ensure coverage targets

**Key Behaviors**:
- Tests all new features
- Maintains test documentation
- Verifies acceptance criteria

### DevOps Engineer
**Purpose**: Infrastructure and deployment
- Set up CI/CD pipelines
- Manage deployments
- Monitor infrastructure
- Ensure security
- Optimize performance

### Code Reviewer
**Purpose**: Code quality and standards
- Review pull requests
- Ensure code quality
- Check security issues
- Verify best practices
- Provide feedback

## Git Discipline

The orchestration system enforces strict git discipline to prevent work loss:

### Auto-Commit Protocol
- Commits happen automatically every 30 minutes
- Meaningful commit messages: "Progress: [task] - [description]"
- Feature branches for new work
- Tags for stable versions

### Branching Strategy
```
main
├── feature/authentication
├── feature/payment-integration
└── fix/user-validation
```

### Emergency Recovery
If work is lost, agents can recover using:
```bash
# Check recent commits
git log --oneline -10

# Recover from stash
git stash list
git stash pop
```

## Self-Scheduling System

Agents manage their own schedules to ensure continuous operation:

### Scheduling Intervals
- **Default**: 15 minutes
- **Minimum**: 5 minutes (high activity)
- **Maximum**: 60 minutes (low activity)
- **Adaptive**: Adjusts based on workload

### Schedule Script
Agents use a scheduling script to set their next check-in:
```bash
.claude/orchestration/schedule.sh 15 "Complete authentication feature"
```

### Recovery Strategies
- **Resume**: Continue from last checkpoint
- **Restart**: Begin fresh with current state
- **Escalate**: Request human intervention

## Communication Protocols

### Message Types
1. **Status Updates**: Regular progress reports
2. **Task Assignments**: New work allocation
3. **Questions**: Clarification requests
4. **Escalations**: Blocked or critical issues
5. **Completions**: Task finished notifications

### Message Format
```
STATUS [AGENT_NAME] [TIMESTAMP]
Completed: 
- Implemented user authentication
- Added test coverage
Current: Working on JWT integration
Blocked: None
ETA: 2 hours
```

### Escalation Criteria
Agents escalate when:
- Blocked for more than 10 minutes
- Critical error discovered
- Architecture decision needed
- Security vulnerability found

## Monitoring and Control

### View Active Sessions
```bash
# Attach to orchestration session
tmux attach -t cf-projectname

# List all windows
tmux list-windows -t cf-projectname
```

### Check Status
Use the slash command in Claude:
```
/orchestrate-status
```

Or check the status file:
```bash
cat .claude/orchestration/status.json
```

### Agent Logs
All agent conversations are logged:
```
.claude/orchestration/logs/
├── orchestrator_2024-01-15T10-30-00.log
├── pm-001_2024-01-15T10-30-00.log
└── dev-001_2024-01-15T10-30-00.log
```

### Performance Metrics
- Tasks completed per hour
- Code quality scores
- Test coverage percentages
- Blocker resolution time
- Commit frequency

## Slash Commands

### /orchestrate-project
Deploy a full orchestration team for the entire project.

### /orchestrate-feature
Deploy a focused team for specific feature implementation.

### /orchestrate-status
Check the current status of orchestration.

## Best Practices

### 1. Project Preparation
- Create comprehensive CLAUDE.md
- Generate PRPs for major features
- Set up test infrastructure
- Initialize git with proper .gitignore

### 2. Team Deployment
- Start with medium team for most projects
- Use phased deployment for complex projects
- Monitor first hour closely
- Adjust team size based on workload

### 3. Quality Control
- Review auto-commits regularly
- Set up CI/CD for automated testing
- Monitor code quality metrics
- Perform weekly code reviews

### 4. Intervention Guidelines
Only intervene when:
- Agents are blocked for extended periods
- Critical architectural decisions needed
- Security issues discovered
- Quality standards not met

## Troubleshooting

### Common Issues

#### tmux not found
```bash
# Install tmux first
brew install tmux  # macOS
```

#### Session already exists
```bash
# Kill existing session
tmux kill-session -t cf-projectname
```

#### Agent not responding
```bash
# Check agent window
tmux capture-pane -t cf-projectname:2 -p
```

#### Git conflicts
Agents handle most conflicts, but you can:
```bash
# Review conflicts
git status
git diff
```

### Debug Mode
Enable debug logging:
```bash
export DEBUG=true
context-forge orchestrate
```

## Advanced Configuration

### Custom Team Structure
Create a custom configuration file:
```json
{
  "teamStructure": {
    "orchestrator": {...},
    "projectManagers": [...],
    "developers": [...],
    "specialists": [...]
  }
}
```

### Integration with CI/CD
Agents can trigger CI/CD pipelines:
```yaml
# .github/workflows/agent-ci.yml
on:
  push:
    branches: [feature/*]
```

### Performance Tuning
Adjust for large codebases:
- Increase commit interval to 45-60 minutes
- Use adaptive scheduling
- Deploy specialized agents for specific areas

## Security Considerations

### Access Control
- Each agent needs Claude API access
- Use environment variables for sensitive data
- Implement branch protection rules

### Code Review
- All agent code should be reviewed
- Set up automated security scanning
- Monitor for sensitive data exposure

### Audit Trail
The system maintains comprehensive logs:
- All agent conversations
- Git commit history
- Task completion records
- Error and recovery logs

## Future Enhancements

### Planned Features
1. Multi-repository orchestration
2. Cross-project agent collaboration
3. Advanced performance analytics
4. Custom agent personalities
5. Integration with issue trackers

### Community Contributions
We welcome contributions to improve orchestration:
- Custom agent roles
- New communication models
- Performance optimizations
- Integration adapters

## Additional Documentation

- [Implementation Details](../../docs/orchestration/IMPLEMENTATION.md) - Technical architecture and design
- [Tutorial](../../docs/orchestration/tutorial.md) - Step-by-step guide to getting started

## Conclusion

The Context Forge Orchestration System transforms how AI assists in software development. By deploying autonomous agent teams, you can achieve continuous progress while maintaining high quality standards. The system's self-managing nature, combined with strict git discipline and comprehensive monitoring, ensures reliable and productive AI-powered development.

For questions or support, visit: https://github.com/context-forge/context-forge/issues