meld/_dev/prompts/old/sdk.md

Meld: A template language for LLM prompts

We need to update our implementation in `api/index.ts` to reflect our more complex services setup. There are some notes on this topic in this file:

\=== NOTES

# SDK Integration Test Patterns

## Test-Implementation Misalignment

We've identified a pattern where SDK integration tests may be making oversimplified assumptions about internal service behavior. This creates potential maintenance challenges and false negatives in our test suite.

### Case Study: Output Service Integration

The `api/api.test.ts` integration tests demonstrate this pattern clearly:

```typescript
// SDK integration test makes simple assumptions
const content = `
  Some text content
  @run [echo test]
  More text
`;
// Expects:
// - Raw text preservation
// - Simple directive handling
// - Direct content matching
```

However, the actual `OutputService` implementation and its unit tests reveal more sophisticated behavior:

1. Transformation Modes
   - Non-transformation mode has specific directive handling rules
   - Transformation mode replaces directives with results
   - Mode selection affects entire output pipeline

2. Format-Specific Behavior
   - Each format (markdown, llm) has unique requirements
   - LLM XML format has special handling needs
   - Directive handling varies by format

3. State Management
   - Service tracks transformation state
   - Handles state variables differently in different modes
   - Complex interaction between state and output

### Impact on Test Reliability

This misalignment causes:
1. False negatives - tests fail despite correct implementation
2. Maintenance burden - fixing "failing" tests can break actual functionality
3. Documentation gaps - simplified tests don't reflect actual behavior

### Recommendations

1. SDK Integration Tests Should:
   - Consider transformation modes
   - Account for format-specific behavior
   - Match documented interface behavior
   - Test actual use cases rather than implementation details

2. Documentation Updates:
   - Clearly document transformation modes
   - Explain format-specific requirements
   - Provide SDK usage examples that reflect actual behavior

3. Test Structure:
   - Move implementation details to unit tests
   - Keep integration tests focused on real-world usage
   - Add test cases for different modes and formats
   - Document expected behavior in test descriptions

## Implementation Plan

### Phase 1: Test Infrastructure Updates (1-2 hours)
- [ ] Update TestContext initialization
- [ ] Add transformation mode helpers
- [ ] Add format-specific test utilities
- [ ] Update test documentation patterns

### Phase 2: Basic Transformation Tests (2-3 hours)
- [ ] Test transformation mode enabling/disabling
- [ ] Test state variable preservation
- [ ] Test basic directive handling
- [ ] Test content preservation rules

### Phase 3: Format-Specific Tests (2-3 hours)
- [ ] Markdown format tests
  - [ ] Headers and formatting
  - [ ] Code blocks
  - [ ] Directive placeholders
- [ ] LLM format tests
  - [ ] XML structure
  - [ ] Special characters
  - [ ] State representation

### Phase 4: Integration Scenarios (3-4 hours)
- [ ] Full pipeline tests
  - [ ] Parse -> Transform -> Output
  - [ ] State management
  - [ ] Error handling
- [ ] Mixed content tests
  - [ ] Multiple directive types
  - [ ] Nested transformations
  - [ ] State inheritance
- [ ] Edge cases
  - [ ] Empty content
  - [ ] Invalid directives
  - [ ] State conflicts

### Phase 5: Documentation & Examples (2-3 hours)
- [ ] Update test documentation
- [ ] Add example test patterns
- [ ] Document common pitfalls
- [ ] Create test templates

## Action Items

1. Review other SDK integration tests for similar patterns
2. Update test documentation to reflect actual service behavior
3. Consider adding SDK-level transformation mode controls
4. Add integration test examples to SDK documentation

## Risk Assessment

### Low Risk Areas
- Test infrastructure changes (good existing patterns)
- Basic transformation tests (clear requirements)
- Documentation updates (straightforward)

### Medium Risk Areas
- Format-specific edge cases
- State management complexity
- Performance implications

### Mitigation Strategies
1. Incremental implementation
2. Comprehensive test coverage
3. Clear documentation
4. Regular review points

## Timeline
- Total estimated time: 10-15 hours
- Can be implemented incrementally
- Key milestones align with phases
- Regular review points after each phase

## Success Criteria
1. All tests pass consistently
2. No false negatives
3. Clear test patterns documented
4. Easy to maintain and extend
5. Matches actual service behavior

\=== CODE

# api.test.ts

```typescript
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
import { main } from './index.js';
import { TestContext } from '@tests/utils/index.js';
import type { ProcessOptions } from '@core/types/index.js';
import type { NodeFileSystem } from '@services/fs/FileSystemService/NodeFileSystem.js';

// Define the type for main function options
type MainOptions = {
  fs?: NodeFileSystem;
  format?: 'llm';
  services?: any;
};

describe('SDK Integration Tests', () => {
  let context: TestContext;
  let testFilePath: string;

  beforeEach(async () => {
    context = new TestContext();
    await context.initialize();
    testFilePath = 'test.meld';
  });

  afterEach(async () => {
    await context.cleanup();
    vi.resetModules();
    vi.clearAllMocks();
  });

  describe('Format Conversion', () => {
    it('should handle definition directives correctly', async () => {
      await context.fs.writeFile(testFilePath, '@text greeting = "Hello"');
      const result = await main(testFilePath, {
        fs: context.fs,
        services: context.services
      });
      // Definition directives should be omitted from output
      expect(result).toBe('');
    });

    it('should handle execution directives correctly', async () => {
      // Start debug session with enhanced configuration
      const debugSessionId = await context.startDebugSession({
        captureConfig: {
          capturePoints: ['pre-transform', 'post-transform', 'error'],
          includeFields: ['nodes', 'transformedNodes', 'variables', 'metadata'],
          format: 'full'
        },
        visualization: {
          format: 'mermaid',
          includeMetadata: true,
          includeTimestamps: true
        }
      });

      try {
        await context.fs.writeFile(testFilePath, '@run [echo test]');

        // Get initial state ID - FIXED: Remove file path fallback
        const initialStateId = context.services.state.getStateId();
        if (!initialStateId) {
          throw new Error('Failed to get state ID - state not properly initialized');
        }

        // Enhanced debugging: Generate relationship graph
        console.log('Initial State Relationships:');
        console.log(await context.services.visualization.generateRelationshipGraph([initialStateId], {
          format: 'mermaid',
          includeMetadata: true
        }));

        // Enhanced debugging: Generate initial timeline
        console.log('Initial Timeline:');
        console.log(await context.services.visualization.generateTimeline([initialStateId], {
          format: 'mermaid',
          includeTimestamps: true
        }));

        // Enhanced debugging: Get initial metrics
        const startTime = Date.now();
        const initialMetrics = await context.services.visualization.getMetrics({
          start: startTime - 3600000, // Last hour
          end: startTime
        });
        console.log('Initial State Metrics:', initialMetrics);

        console.log('Initial State Hierarchy:');
        console.log(await context.services.visualization.generateHierarchyView(initialStateId, {
          format: 'mermaid',
          includeMetadata: true
        }));

        // Trace the operation with enhanced error handling
        const { result, diagnostics } = await context.services.debugger.traceOperation(
          initialStateId,
          async () => {
            // Enable transformation mode explicitly
            context.services.state.enableTransformation(true);

            return await main(testFilePath, {
              fs: context.fs,
              format: 'llm',
              services: context.services
            } as any);
          }
        );

        // Log diagnostics and state changes
        console.log('Operation Diagnostics:', diagnostics);

        // Get final state visualization
        const finalStateId = context.services.state.getStateId();
        if (!finalStateId) {
          throw new Error('Failed to get final state ID');
        }

        // Enhanced debugging: Generate final relationship graph
        console.log('Final State Relationships:');
        console.log(await context.services.visualization.generateRelationshipGraph([finalStateId], {
          format: 'mermaid',
          includeMetadata: true
        }));

        // Enhanced debugging: Generate final timeline
        console.log('Final Timeline:');
        console.log(await context.services.visualization.generateTimeline([finalStateId], {
          format: 'mermaid',
          includeTimestamps: true
        }));

        // Enhanced debugging: Get final metrics
        const endTime = Date.now();
        const finalMetrics = await context.services.visualization.getMetrics({
          start: startTime,
          end: endTime
        });
        console.log('Final State Metrics:', finalMetrics);

        console.log('Final State Hierarchy:');
        console.log(await context.services.visualization.generateHierarchyView(finalStateId, {
          format: 'mermaid',
          includeMetadata: true
        }));

        // Generate transition diagram
        console.log('State Transitions:');
        console.log(await context.services.visualization.generateTransitionDiagram(finalStateId, {
          format: 'mermaid',
          includeTimestamps: true
        }));

        // Add assertions here
        expect(result).toBeDefined();
        // Add more specific assertions based on expected behavior
      } catch (error) {
        console.error('Test failed with error:', error);
        // Enhanced error reporting
        if (context.services.tracking) {
          const allStates = await context.services.tracking.getAllStates();
          console.log('All tracked states:', allStates);
        }
        throw error;
      }
    });

    it('should handle complex meld content with mixed directives', async () => {
      const content = `
        @text greeting = "Hello"
        @data config = { "value": 123 }
        Some text content
        @run [echo test]
        More text
      `;
      await context.fs.writeFile(testFilePath, content);
      const result = await main(testFilePath, {
        fs: context.fs,
        services: context.services
      });

      // Definition directives should be omitted
      expect(result).not.toContain('"identifier": "greeting"');
      expect(result).not.toContain('"value": "Hello"');
      expect(result).not.toContain('"identifier": "config"');

      // Text content should be preserved
      expect(result).toContain('Some text content');
      expect(result).toContain('More text');

      // Execution directives should show placeholder
      expect(result).toContain('[run directive output placeholder]');
    });
  });

  describe('Full Pipeline Integration', () => {
    it('should handle the complete parse -> interpret -> convert pipeline', async () => {
      const content = `
        @text greeting = "Hello"
        @run [echo test]
        Some content
      `;
      await context.fs.writeFile(testFilePath, content);
      const result = await main(testFilePath, {
        fs: context.fs,
        services: context.services
      });

      // Definition directive should be omitted
      expect(result).not.toContain('"kind": "text"');
      expect(result).not.toContain('"identifier": "greeting"');

      // Execution directive should show placeholder
      expect(result).toContain('[run directive output placeholder]');

      // Text content should be preserved
      expect(result).toContain('Some content');
    });

    it('should preserve state and content in transformation mode', async () => {
      const content = `
        @text first = "First"
        @text second = "Second"
        @run [echo test]
        Content
      `;
      await context.fs.writeFile(testFilePath, content);

      // Enable transformation mode through state service
      context.services.state.enableTransformation(true);

      const result = await main(testFilePath, {
        fs: context.fs,
        services: context.services
      });

      // In transformation mode, directives should be replaced with their results
      expect(result).not.toContain('"identifier": "first"');
      expect(result).not.toContain('"value": "First"');
      expect(result).not.toContain('"identifier": "second"');

      // Text content should be preserved
      expect(result).toContain('Content');

      // Run directive should be transformed (if transformation is working)
      expect(result).toContain('test');
    });
  });

  describe('Error Handling', () => {
    it('should handle parse errors gracefully', async () => {
      await context.fs.writeFile(testFilePath, '@invalid not_a_valid_directive');
      await expect(main(testFilePath, {
        fs: context.fs,
        services: context.services
      }))
        .rejects
        .toThrow(/Parse error/);
    });

    // TODO: This test will be updated as part of the error handling overhaul
    // See dev/ERRORS.md - will be reclassified as a fatal error with improved messaging
    it.todo('should handle missing files correctly');

    it('should handle empty files', async () => {
      await context.fs.writeFile(testFilePath, '');
      const result = await main(testFilePath, {
        fs: context.fs,
        services: context.services
      });
      expect(result).toBe(''); // Empty input should produce empty output
    });
  });

  describe('Edge Cases', () => {
    it.todo('should handle large files efficiently');
    it.todo('should handle deeply nested imports');
  });
});
```

# index.ts

```typescript
// Core services
export * from '@services/pipeline/InterpreterService/InterpreterService.js';
export * from '@services/pipeline/ParserService/ParserService.js';
export * from '@services/state/StateService/StateService.js';
export * from '@services/resolution/ResolutionService/ResolutionService.js';
export * from '@services/pipeline/DirectiveService/DirectiveService.js';
export * from '@services/resolution/ValidationService/ValidationService.js';
export * from '@services/fs/PathService/PathService.js';
export * from '@services/fs/FileSystemService/FileSystemService.js';
export * from '@services/fs/FileSystemService/PathOperationsService.js';
export * from '@services/pipeline/OutputService/OutputService.js';
export * from '@services/resolution/CircularityService/CircularityService.js';

// Core types and errors
export * from '@core/types/index.js';
export * from '@core/errors/MeldDirectiveError.js';
export * from '@core/errors/MeldInterpreterError.js';
export * from '@core/errors/MeldParseError.js';
import { MeldFileNotFoundError } from '@core/errors/MeldFileNotFoundError.js';

// Import service classes
import { InterpreterService } from '@services/pipeline/InterpreterService/InterpreterService.js';
import { ParserService } from '@services/pipeline/ParserService/ParserService.js';
import { StateService } from '@services/state/StateService/StateService.js';
import { ResolutionService } from '@services/resolution/ResolutionService/ResolutionService.js';
import { DirectiveService } from '@services/pipeline/DirectiveService/DirectiveService.js';
import { ValidationService } from '@services/resolution/ValidationService/ValidationService.js';
import { PathService } from '@services/fs/PathService/PathService.js';
import { FileSystemService } from '@services/fs/FileSystemService/FileSystemService.js';
import { PathOperationsService } from '@services/fs/FileSystemService/PathOperationsService.js';
import { OutputService } from '@services/pipeline/OutputService/OutputService.js';
import { CircularityService } from '@services/resolution/CircularityService/CircularityService.js';
import { NodeFileSystem } from '@services/fs/FileSystemService/NodeFileSystem.js';
import { ProcessOptions } from '@core/types/index.js';

// Package info
export { version } from '@core/version.js';

export async function main(filePath: string, options: ProcessOptions & { services?: any } = {}): Promise<string> {
  // Use services from test context if provided, otherwise create new ones
  const pathOps = new PathOperationsService();
  const fs = options.fs || new NodeFileSystem();
  const filesystem = new FileSystemService(pathOps, fs);

  if (options.services) {
    // Use services from test context
    const { parser, interpreter, directive, validation, state, path, circularity, resolution, output } = options.services;

    // Initialize services
    path.initialize(filesystem);
    directive.initialize(
      validation,
      state,
      path,
      filesystem,
      parser,
      interpreter,
      circularity,
      resolution
    );
    interpreter.initialize(directive, state);

    try {
      // Read the file
      const content = await filesystem.readFile(filePath);

      // Parse the content
      const ast = await parser.parse(content);

      // Interpret the AST
      const resultState = await interpreter.interpret(ast, { filePath, initialState: state });

      // Convert to desired format using the updated state
      const converted = await output.convert(ast, resultState, options.format || 'llm');

      return converted;
    } catch (error) {
      // If it's a MeldFileNotFoundError, just throw it as is
      if (error instanceof MeldFileNotFoundError) {
        throw error;
      }
      // For other Error instances, preserve the error
      if (error instanceof Error) {
        throw error;
      }
      // For non-Error objects, convert to string
      throw new Error(String(error));
    }
  } else {
    // Create new services
    const parser = new ParserService();
    const interpreter = new InterpreterService();
    const state = new StateService();
    const directives = new DirectiveService();
    const validation = new ValidationService();
    const circularity = new CircularityService();
    const resolution = new ResolutionService(state, filesystem, parser);
    const path = new PathService();
    const output = new OutputService();

    // Initialize services
    directives.initialize(
      validation,
      state,
      path,
      filesystem,
      parser,
      interpreter,
      circularity,
      resolution
    );
    interpreter.initialize(directives, state);

    try {
      // Read the file
      const content = await filesystem.readFile(filePath);

      // Parse the content
      const ast = await parser.parse(content);

      // Interpret the AST
      const resultState = await interpreter.interpret(ast, { filePath, initialState: state });

      // Convert to desired format using the updated state
      const converted = await output.convert(ast, resultState, options.format || 'llm');

      return converted;
    } catch (error) {
      // If it's a MeldFileNotFoundError, just throw it as is
      if (error instanceof MeldFileNotFoundError) {
        throw error;
      }
      // For other Error instances, preserve the error
      if (error instanceof Error) {
        throw error;
      }
      // For non-Error objects, convert to string
      throw new Error(String(error));
    }
  }
}
```

# DirectiveService.test.ts

```typescript
import { describe, it, expect, beforeEach, afterEach } from 'vitest';
import { DirectiveService } from './DirectiveService.js';
import { TestContext } from '@tests/utils/TestContext.js';
import { DirectiveError, DirectiveErrorCode } from './errors/DirectiveError.js';
import type { DirectiveNode } from 'meld-spec';

describe('DirectiveService', () => {
  let context: TestContext;
  let service: DirectiveService;

  beforeEach(async () => {
    // Initialize test context
    context = new TestContext();
    await context.initialize();

    // Create service instance
    service = new DirectiveService();

    // Initialize with real services from context
    service.initialize(
      context.services.validation,
      context.services.state,
      context.services.path,
      context.services.filesystem,
      context.services.parser,
      context.services.interpreter,
      context.services.circularity,
      context.services.resolution
    );

    // Load test fixtures
    await context.fixtures.load('directiveTestProject');
  });

  afterEach(async () => {
    await context.cleanup();
  });

  describe('Service initialization', () => {
    it('should initialize with all required services', () => {
      expect(service.getSupportedDirectives()).toContain('text');
      expect(service.getSupportedDirectives()).toContain('data');
      expect(service.getSupportedDirectives()).toContain('path');
    });

    it('should throw if used before initialization', async () => {
      const uninitializedService = new DirectiveService();
      const node = context.factory.createTextDirective('test', '"value"', context.factory.createLocation(1, 1));
      const execContext = { currentFilePath: 'test.meld', state: context.services.state };
      await expect(uninitializedService.processDirective(node, execContext))
        .rejects.toThrow('DirectiveService must be initialized before use');
    });
  });

  describe('Directive processing', () => {
    describe('Text directives', () => {
      it('should process basic text directive', async () => {
        // Verify file exists
        const exists = await context.fs.exists('test.meld');
        console.log('test.meld exists:', exists);

        // Parse the fixture file
        const content = await context.fs.readFile('test.meld');
        console.log('test.meld content:', content);

        const nodes = await context.services.parser.parse(content);
        console.log('Parsed nodes:', nodes);

        const node = nodes[0] as DirectiveNode;

        // Create execution context
        const execContext = {
          currentFilePath: 'test.meld',
          state: context.services.state
        };

        // Process the directive
        const result = await service.processDirective(node, execContext);

        // Verify the result
        expect(result.getTextVar('greeting')).toBe('Hello');
      });

      it('should process text directive with variable interpolation', async () => {
        // Set up initial state with a variable
        const state = context.services.state;
        state.setTextVar('name', 'World');

        // Parse and process
        const content = await context.fs.readFile('test-interpolation.meld');
        const nodes = await context.services.parser.parse(content);
        const node = nodes[0] as DirectiveNode;

        const result = await service.processDirective(node, {
          currentFilePath: 'test-interpolation.meld',
          state
        });

        expect(result.getTextVar('greeting')).toBe('Hello World');
      });
    });

    describe('Data directives', () => {
      it('should process data directive with object value', async () => {
        const content = await context.fs.readFile('test-data.meld');
        const nodes = await context.services.parser.parse(content);
        const node = nodes[0] as DirectiveNode;

        const result = await service.processDirective(node, {
          currentFilePath: 'test-data.meld',
          state: context.services.state
        });

        expect(result.getDataVar('config')).toEqual({ key: 'value' });
      });

      it('should process data directive with variable interpolation', async () => {
        // Set up initial state
        const state = context.services.state;
        state.setTextVar('user', 'Alice');

        const content = await context.fs.readFile('test-data-interpolation.meld');
        const nodes = await context.services.parser.parse(content);
        const node = nodes[0] as DirectiveNode;

        const result = await service.processDirective(node, {
          currentFilePath: 'test-data-interpolation.meld',
          state
        });

        expect(result.getDataVar('config')).toEqual({ greeting: 'Hello Alice' });
      });
    });

    describe('Import directives', () => {
      it('should process basic import', async () => {
        // Create import directive node with value property
        const node = context.factory.createImportDirective('module.meld', context.factory.createLocation(1, 1));

        const result = await service.processDirective(node, {
          currentFilePath: 'main.meld',
          state: context.services.state
        });

        expect(result.getTextVar('greeting')).toBe('Hello');
      });

      it('should handle nested imports', async () => {
        // Create import directive node with value property
        const node = context.factory.createImportDirective('inner.meld', context.factory.createLocation(1, 1));

        const result = await service.processDirective(node, {
          currentFilePath: 'middle.meld',
          state: context.services.state
        });

        expect(result.getTextVar('greeting')).toBe('Hello');
      });

      it('should detect circular imports', async () => {
        // Create import directive node with value property
        const node = context.factory.createImportDirective('b.meld', context.factory.createLocation(1, 1));

        await expect(service.processDirective(node, {
          currentFilePath: 'a.meld',
          state: context.services.state
        })).rejects.toThrow(DirectiveError);
      });
    });

    // ... continue with other directive types and error cases
  });
});
```

# DirectiveService.ts

```typescript
import type { DirectiveNode, DirectiveKind, DirectiveData } from 'meld-spec';
import { directiveLogger } from '../../../core/utils/logger.js';
import { IDirectiveService, IDirectiveHandler, DirectiveContext } from './IDirectiveService.js';
import { IValidationService } from '@services/resolution/ValidationService/IValidationService.js';
import { IStateService } from '@services/state/StateService/IStateService.js';
import { IPathService } from '@services/fs/PathService/IPathService.js';
import { IFileSystemService } from '@services/fs/FileSystemService/IFileSystemService.js';
import { IParserService } from '@services/pipeline/ParserService/IParserService.js';
import { IInterpreterService } from '@services/pipeline/InterpreterService/IInterpreterService.js';
import { MeldDirectiveError } from '@core/errors/MeldDirectiveError.js';
import { ICircularityService } from '@services/resolution/CircularityService/ICircularityService.js';
import { IResolutionService } from '@services/resolution/ResolutionService/IResolutionService.js';
import { DirectiveError, DirectiveErrorCode } from './errors/DirectiveError.js';
import type { ILogger } from './handlers/execution/EmbedDirectiveHandler.js';

// Import all handlers
import { TextDirectiveHandler } from './handlers/definition/TextDirectiveHandler.js';
import { DataDirectiveHandler } from './handlers/definition/DataDirectiveHandler.js';
import { PathDirectiveHandler } from './handlers/definition/PathDirectiveHandler.js';
import { DefineDirectiveHandler } from './handlers/definition/DefineDirectiveHandler.js';
import { RunDirectiveHandler } from './handlers/execution/RunDirectiveHandler.js';
import { EmbedDirectiveHandler } from './handlers/execution/EmbedDirectiveHandler.js';
import { ImportDirectiveHandler } from './handlers/execution/ImportDirectiveHandler.js';

export class MeldLLMXMLError extends Error {
  constructor(
    message: string,
    public readonly code: string,
    public readonly details?: any
  ) {
    super(message);
    this.name = 'MeldLLMXMLError';
    Object.setPrototypeOf(this, MeldLLMXMLError.prototype);
  }
}

/**
 * Service responsible for handling directives
 */
export class DirectiveService implements IDirectiveService {
  private validationService?: IValidationService;
  private stateService?: IStateService;
  private pathService?: IPathService;
  private fileSystemService?: IFileSystemService;
  private parserService?: IParserService;
  private interpreterService?: IInterpreterService;
  private circularityService?: ICircularityService;
  private resolutionService?: IResolutionService;
  private initialized = false;
  private logger: ILogger;

  private handlers: Map<string, IDirectiveHandler> = new Map();

  constructor(logger?: ILogger) {
    this.logger = logger || directiveLogger;
  }

  initialize(
    validationService: IValidationService,
    stateService: IStateService,
    pathService: IPathService,
    fileSystemService: IFileSystemService,
    parserService: IParserService,
    interpreterService: IInterpreterService,
    circularityService: ICircularityService,
    resolutionService: IResolutionService
  ): void {
    this.validationService = validationService;
    this.stateService = stateService;
    this.pathService = pathService;
    this.fileSystemService = fileSystemService;
    this.parserService = parserService;
    this.interpreterService = interpreterService;
    this.circularityService = circularityService;
    this.resolutionService = resolutionService;
    this.initialized = true;

    // Register default handlers
    this.registerDefaultHandlers();

    this.logger.debug('DirectiveService initialized', {
      handlers: Array.from(this.handlers.keys())
    });
  }

  /**
   * Register all default directive handlers
   */
  public registerDefaultHandlers(): void {
    // Definition handlers
    this.registerHandler(
      new TextDirectiveHandler(
        this.validationService!,
        this.stateService!,
        this.resolutionService!
      )
    );

    this.registerHandler(
      new DataDirectiveHandler(
        this.validationService!,
        this.stateService!,
        this.resolutionService!
      )
    );

    this.registerHandler(
      new PathDirectiveHandler(
        this.validationService!,
        this.stateService!,
        this.resolutionService!
      )
    );

    this.registerHandler(
      new DefineDirectiveHandler(
        this.validationService!,
        this.stateService!,
        this.resolutionService!
      )
    );

    // Execution handlers
    this.registerHandler(
      new RunDirectiveHandler(
        this.validationService!,
        this.resolutionService!,
        this.stateService!,
        this.fileSystemService!
      )
    );

    this.registerHandler(
      new EmbedDirectiveHandler(
        this.validationService!,
        this.resolutionService!,
        this.stateService!,
        this.circularityService!,
        this.fileSystemService!,
        this.parserService!,
        this.interpreterService!,
        this.logger
      )
    );

    this.registerHandler(
      new ImportDirectiveHandler(
        this.validationService!,
        this.resolutionService!,
        this.stateService!,
        this.fileSystemService!,
        this.parserService!,
        this.interpreterService!,
        this.circularityService!
      )
    );
  }

  /**
   * Register a new directive handler
   */
  registerHandler(handler: IDirectiveHandler): void {
    if (!this.initialized) {
      throw new Error('DirectiveService must be initialized before registering handlers');
    }

    if (!handler.kind) {
      throw new Error('Handler must have a kind property');
    }

    this.handlers.set(handler.kind, handler);
    this.logger.debug(`Registered handler for directive: ${handler.kind}`);
  }

  /**
   * Handle a directive node
   */
  public async handleDirective(node: DirectiveNode, context: DirectiveContext): Promise<IStateService> {
    return this.processDirective(node, context);
  }

  /**
   * Process multiple directives in sequence
   */
  async processDirectives(nodes: DirectiveNode[], parentContext?: DirectiveContext): Promise<IStateService> {
    let currentState = parentContext?.state?.clone() || this.stateService!.createChildState();

    for (const node of nodes) {
      // Create a new context with the current state as parent and a new child state
      const nodeContext = {
        currentFilePath: parentContext?.currentFilePath || '',
        parentState: currentState,
        state: currentState.createChildState()
      };

      // Process directive and get the updated state
      const result = await this.processDirective(node, nodeContext);

      // If transformation is enabled, we don't merge states since the directive
      // will be replaced with a text node and its state will be handled separately
      if (!currentState.isTransformationEnabled?.()) {
        // result is always an IStateService from processDirective
        currentState.mergeChildState(result);
      }
    }

    return currentState;
  }

  /**
   * Create execution context for a directive
   */
  private createContext(node: DirectiveNode, parentContext?: DirectiveContext): DirectiveContext {
    if (!this.stateService) {
      throw new Error('DirectiveService must be initialized before use');
    }
    const state = parentContext?.state?.clone() || this.stateService.createChildState();
    return {
      currentFilePath: parentContext?.currentFilePath || '',
      parentState: parentContext?.state,
      state
    };
  }

  /**
   * Update the interpreter service reference
   */
  updateInterpreterService(interpreterService: IInterpreterService): void {
    this.interpreterService = interpreterService;
    this.logger.debug('Updated interpreter service reference');
  }

  /**
   * Check if a handler exists for a directive kind
   */
  hasHandler(kind: string): boolean {
    return this.handlers.has(kind);
  }

  /**
   * Validate a directive node
   */
  async validateDirective(node: DirectiveNode): Promise<void> {
    try {
      await this.validationService!.validate(node);
    } catch (error: unknown) {
      const errorMessage = error instanceof Error ? error.message : String(error);
      const errorForLog = error instanceof Error ? error : new Error(String(error));

      this.logger.error('Failed to validate directive', {
        kind: node.directive.kind,
        location: node.location,
        error: errorForLog
      });

      throw new DirectiveError(
        errorMessage,
        node.directive.kind,
        DirectiveErrorCode.VALIDATION_FAILED,
        {
          node
        }
      );
    }
  }

  /**
   * Create a child context for nested directives
   */
  public createChildContext(parentContext: DirectiveContext, filePath: string): DirectiveContext {
    return {
      currentFilePath: filePath,
      state: parentContext.state.createChildState(),
      parentState: parentContext.state
    };
  }

  supportsDirective(kind: string): boolean {
    return this.handlers.has(kind);
  }

  getSupportedDirectives(): string[] {
    return Array.from(this.handlers.keys());
  }

  private ensureInitialized(): void {
    if (!this.initialized) {
      throw new Error('DirectiveService must be initialized before use');
    }
  }

  private async handleTextDirective(node: DirectiveNode): Promise<void> {
    const directive = node.directive;

    this.logger.debug('Processing text directive', {
      identifier: directive.identifier,
      location: node.location
    });

    try {
      // Value is already interpolated by meld-ast
      await this.stateService!.setTextVar(directive.identifier, directive.value);

      this.logger.debug('Text directive processed successfully', {
        identifier: directive.identifier,
        location: node.location
      });
    } catch (error: unknown) {
      const errorMessage = error instanceof Error ? error.message : String(error);
      const errorForLog = error instanceof Error ? error : new Error(String(error));

      this.logger.error('Failed to process text directive', {
        identifier: directive.identifier,
        location: node.location,
        error: errorForLog
      });

      throw new MeldDirectiveError(
        errorMessage,
        'text',
        node.location?.start
      );
    }
  }

  private async handleDataDirective(node: DirectiveNode): Promise<void> {
    const directive = node.directive;

    this.logger.debug('Processing data directive', {
      identifier: directive.identifier,
      location: node.location
    });

    try {
      // Value is already interpolated by meld-ast
      let value = directive.value;
      if (typeof value === 'string') {
        value = JSON.parse(value);
      }

      await this.stateService!.setDataVar(directive.identifier, value);

      this.logger.debug('Data directive processed successfully', {
        identifier: directive.identifier,
        location: node.location
      });
    } catch (error: unknown) {
      const errorMessage = error instanceof Error ? error.message : String(error);
      const errorForLog = error instanceof Error ? error : new Error(String(error));

      this.logger.error('Failed to process data directive', {
        identifier: directive.identifier,
        location: node.location,
        error: errorForLog
      });

      throw new MeldDirectiveError(
        errorMessage,
        'data',
        node.location?.start
      );
    }
  }

  private async handleImportDirective(node: DirectiveNode): Promise<void> {
    const directive = node.directive;

    this.logger.debug('Processing import directive', {
      path: directive.path,
      section: directive.section,
      fuzzy: directive.fuzzy,
      location: node.location
    });

    try {
      // Path is already interpolated by meld-ast
      const fullPath = await this.pathService!.resolvePath(directive.path);

      // Check for circular imports
      this.circularityService!.beginImport(fullPath);

      try {
        // Check if file exists
        if (!await this.fileSystemService!.exists(fullPath)) {
          throw new Error(`Import file not found: ${fullPath}`);
        }

        // Create a child state for the import
        const childState = await this.stateService!.createChildState();

        // Read the file content
        const content = await this.fileSystemService!.readFile(fullPath);

        // If a section is specified, extract it (section name is already interpolated)
        let processedContent = content;
        if (directive.section) {
          processedContent = await this.extractSection(
            content,
            directive.section,
            directive.fuzzy || 0
          );
        }

        // Parse and interpret the content
        const parsedNodes = await this.parserService!.parse(processedContent);
        await this.interpreterService!.interpret(parsedNodes, {
          initialState: childState,
          filePath: fullPath,
          mergeState: true
        });

        this.logger.debug('Import content processed', {
          path: fullPath,
          section: directive.section,
          location: node.location
        });
      } finally {
        // Always end import tracking, even if there was an error
        this.circularityService!.endImport(fullPath);
      }
    } catch (error: unknown) {
      const errorMessage = error instanceof Error ? error.message : String(error);
      const errorForLog = error instanceof Error ? error : new Error(String(error));

      this.logger.error('Failed to process import directive', {
        path: directive.path,
        section: directive.section,
        location: node.location,
        error: errorForLog
      });

      throw new MeldDirectiveError(
        errorMessage,
        'import',
        node.location?.start
      );
    }
  }

  private async extractSection(
    content: string,
    section: string,
    fuzzyMatch: number
  ): Promise<string> {
    try {
      // Split content into lines
      const lines = content.split('\n');
      const headings: { title: string; line: number; level: number }[] = [];

      // Find all headings and their levels
      for (let i = 0; i < lines.length; i++) {
        const line = lines[i];
        const match = line.match(/^(#{1,6})\s+(.+)$/);
        if (match) {
          headings.push({
            title: match[2],
            line: i,
            level: match[1].length
          });
        }
      }

      // Find best matching heading
      let bestMatch: typeof headings[0] | undefined;
      let bestScore = 0;

      for (const heading of headings) {
        const score = this.calculateSimilarity(heading.title, section);
        if (score > fuzzyMatch && score > bestScore) {
          bestScore = score;
          bestMatch = heading;
        }
      }

      if (!bestMatch) {
        // Find closest match for error message
        let closestMatch = '';
        let closestScore = 0;
        for (const heading of headings) {
          const score = this.calculateSimilarity(heading.title, section);
          if (score > closestScore) {
            closestScore = score;
            closestMatch = heading.title;
          }
        }

        throw new MeldLLMXMLError(
          'Section not found',
          'SECTION_NOT_FOUND',
          { title: section, bestMatch: closestMatch }
        );
      }

      // Find the end of the section (next heading of same or higher level)
      let endLine = lines.length;
      for (let i = bestMatch.line + 1; i < lines.length; i++) {
        const line = lines[i];
        const match = line.match(/^(#{1,6})\s+/);
        if (match && match[1].length <= bestMatch.level) {
          endLine = i;
          break;
        }
      }

      // Extract the section content
      return lines.slice(bestMatch.line, endLine).join('\n');
    } catch (error) {
      if (error instanceof MeldLLMXMLError) {
        throw error;
      }
      throw new MeldLLMXMLError(
        error instanceof Error ? error.message : 'Unknown error during section extraction',
        'PARSE_ERROR',
        error
      );
    }
  }

  private calculateSimilarity(str1: string, str2: string): number {
    // Convert strings to lowercase for case-insensitive comparison
    const s1 = str1.toLowerCase();
    const s2 = str2.toLowerCase();

    if (s1 === s2) return 1.0;

    // Calculate Levenshtein distance
    const len1 = s1.length;
    const len2 = s2.length;
    const matrix: number[][] = [];

    for (let i = 0; i <= len1; i++) {
      matrix[i] = [i];
    }
    for (let j = 0; j <= len2; j++) {
      matrix[0][j] = j;
    }

    for (let i = 1; i <= len1; i++) {
      for (let j = 1; j <= len2; j++) {
        const cost = s1[i - 1] === s2[j - 1] ? 0 : 1;
        matrix[i][j] = Math.min(
          matrix[i - 1][j] + 1,
          matrix[i][j - 1] + 1,
          matrix[i - 1][j - 1] + cost
        );
      }
    }

    // Convert distance to similarity score between 0 and 1
    const maxLen = Math.max(len1, len2);
    return maxLen === 0 ? 1.0 : 1.0 - matrix[len1][len2] / maxLen;
  }

  private async handleEmbedDirective(node: DirectiveNode): Promise<void> {
    const directive = node.directive;

    this.logger.debug('Processing embed directive', {
      path: directive.path,
      section: directive.section,
      fuzzy: directive.fuzzy,
      names: directive.names,
      location: node.location
    });

    try {
      // Path is already interpolated by meld-ast
      const fullPath = await this.pathService!.resolvePath(directive.path);

      // Check for circular imports
      this.circularityService!.beginImport(fullPath);

      try {
        // Check if file exists
        if (!await this.fileSystemService!.exists(fullPath)) {
          throw new Error(`Embed file not found: ${fullPath}`);
        }

        // Create a child state for the import
        const childState = await this.stateService!.createChildState();

        // Read the file content
        const content = await this.fileSystemService!.readFile(fullPath);

        // If a section is specified, extract it (section name is already interpolated)
        let processedContent = content;
        if (directive.section) {
          processedContent = await this.extractSection(
            content,
            directive.section,
            directive.fuzzy || 0
          );
        }

        // Parse and interpret the content
        const parsedNodes = await this.parserService!.parse(processedContent);
        await this.interpreterService!.interpret(parsedNodes, {
          initialState: childState,
          filePath: fullPath,
          mergeState: true
        });

        this.logger.debug('Embed content processed', {
          path: fullPath,
          section: directive.section,
          location: node.location
        });
      } finally {
        // Always end import tracking, even if there was an error
        this.circularityService!.endImport(fullPath);
      }
    } catch (error: unknown) {
      const errorMessage = error instanceof Error ? error.message : String(error);
      const errorForLog = error instanceof Error ? error : new Error(String(error));

      this.logger.error('Failed to process embed directive', {
        path: directive.path,
        section: directive.section,
        location: node.location,
        error: errorForLog
      });

      throw new MeldDirectiveError(
        errorMessage,
        'embed',
        node.location?.start
      );
    }
  }

  /**
   * Process a directive node, validating and executing it
   * Values in the directive will already be interpolated by meld-ast
   * @returns The updated state after directive execution
   * @throws {MeldDirectiveError} If directive processing fails
   */
  public async processDirective(node: DirectiveNode, context: DirectiveContext): Promise<IStateService> {
    // Add initialization check before any other processing
    if (!this.initialized) {
      throw new Error('DirectiveService must be initialized before use');
    }

    try {
      // Get the handler for this directive kind
      const { kind } = node.directive;
      const handler = this.handlers.get(kind);

      if (!handler) {
        throw new DirectiveError(
          `No handler found for directive: ${kind}`,
          kind,
          DirectiveErrorCode.HANDLER_NOT_FOUND,
          { node }
        );
      }

      // Validate directive before handling
      await this.validateDirective(node);

      // Execute the directive and handle both possible return types
      const result = await handler.execute(node, context);

      // If result is a DirectiveResult, return its state
      if ('state' in result) {
        return result.state;
      }

      // Otherwise, result is already an IStateService
      return result;
    } catch (error) {
      if (error instanceof DirectiveError) {
        throw error;
      }

      // Simplify error messages for common cases
      let message = error instanceof Error ? error.message : String(error);
      let code = DirectiveErrorCode.EXECUTION_FAILED;

      if (message.includes('file not found') || message.includes('no such file')) {
        message = `Referenced file not found: ${node.directive.path || node.directive.value}`;
        code = DirectiveErrorCode.FILE_NOT_FOUND;
      } else if (message.includes('circular import') || message.includes('circular reference')) {
        message = 'Circular import detected';
        code = DirectiveErrorCode.CIRCULAR_REFERENCE;
      } else if (message.includes('parameter count') || message.includes('wrong number of parameters')) {
        message = 'Invalid parameter count';
        code = DirectiveErrorCode.VALIDATION_FAILED;
      } else if (message.includes('invalid path') || message.includes('path validation failed')) {
        message = 'Invalid path';
        code = DirectiveErrorCode.VALIDATION_FAILED;
      }

      throw new DirectiveError(
        message,
        node.directive?.kind || 'unknown',
        code,
        { node, cause: error instanceof Error ? error : undefined }
      );
    }
  }
}
```

# IDirectiveService.ts

```typescript
import { DirectiveNode } from 'meld-spec';
import { IStateService } from '@services/state/StateService/IStateService.js';
import type { IValidationService } from '@services/resolution/ValidationService/IValidationService.js';
import type { IPathService } from '@services/fs/PathService/IPathService.js';
import type { IFileSystemService } from '@services/fs/FileSystemService/IFileSystemService.js';
import type { IParserService } from '@services/pipeline/ParserService/IParserService.js';
import type { IInterpreterService } from '@services/pipeline/InterpreterService/IInterpreterService.js';
import type { ICircularityService } from '@services/resolution/CircularityService/ICircularityService.js';
import type { IResolutionService } from '@services/resolution/ResolutionService/IResolutionService.js';
import type { DirectiveResult } from './types.js';

/**
 * Context for directive execution
 */
export interface DirectiveContext {
  /** Current file being processed */
  currentFilePath?: string;
  /** Parent state for nested contexts */
  parentState?: IStateService;
  /** Current state for this directive */
  state: IStateService;
  /** Working directory for command execution */
  workingDirectory?: string;
}

/**
 * Interface for directive handlers
 */
export interface IDirectiveHandler {
  /** The directive kind this handler processes */
  readonly kind: string;

  /**
   * Execute the directive
   * @returns The updated state after directive execution, or a DirectiveResult containing both state and optional replacement node
   */
  execute(
    node: DirectiveNode,
    context: DirectiveContext
  ): Promise<DirectiveResult | IStateService>;
}

/**
 *