cmte/test/README.md

Design by Committee™ except it's just you and LLMs

# Committee Testing Suite

This directory contains test files for validating the functionality of Committee.

## Test Structure

The tests are organized into the following categories:

1. **End-to-End Workflow Tests**: Tests that demonstrate the full functionality of Committee through CLI commands
2. **Direct API Tests**: Tests that use the Committee API directly without going through the CLI
3. **Component Integration Tests**: Tests that verify the interaction between different components
4. **Unit Tests**: Tests for individual modules and functions

## Running Tests

You can run all tests with:

```bash
npm test
```

### Individual Test Categories

```bash
# Run the workflow test (CLI-based)
npm run test:workflow

# Run the direct API workflow test
npm run test:direct

# Run the component integration test
npm run test:integration

# Run other specific tests
npm run test:templates
npm run test:two-phase
npm run test:feedback
npm run test:components
npm run test:execution
npm run test:human-input
```

## Test Descriptions

### workflow-test.ts

A complete end-to-end test that:
- Creates a simple test workflow structure
- Executes the workflow using the CLI
- Handles human input requirements
- Verifies the workflow completes successfully

This test demonstrates how to use the Committee CLI to run workflows and validates that all CLI commands function as expected.

### direct-workflow-test.ts

A direct API test that:
- Creates a simple test workflow structure
- Uses the Committee API directly (bypassing the CLI)
- Demonstrates workflow execution, human input handling, and resuming
- Verifies the execution flow works correctly

This test is more reliable than the CLI test because it avoids any CLI-specific issues and works directly with the core API.

### component-integration-test.ts

Tests the integration between the core components:
- ComponentRegistry: Tests loading workflow, phases, sets, and tasks
- SetExecutor: Tests sequential and parallel execution of sets
- PhaseExecutor: Tests phase execution with sets
- WorkflowExecutor: Tests full workflow execution

This test verifies that all components interact correctly and that data flows properly between them.

## Mock Mode

The integration tests use a `mockTaskExecution` feature that simulates the execution of tasks without making actual LLM calls. This allows for fast, deterministic testing of the execution flow without incurring API costs or requiring internet connectivity.

## Creating New Tests

When creating new tests, follow these best practices:

1. Create test files in the appropriate directory
2. Use descriptive function names that explain what is being tested
3. Log test results clearly with success/failure indicators
4. Clean up any temporary files/directories created during testing
5. Handle errors gracefully

## Test Workflow Structure

The test workflows follow the standard Committee structure:

```
test/workflows/workflow-name/
├── tasks/                # Task definitions
├── sets/                # Task set definitions
├── phases/              # Phase definitions
├── workflow.yaml        # Workflow definition
└── README.md           # Optional documentation
```

All test outputs are saved to the `_output/` directory at the project root.