meld-spec/README.md

Specification for the Meld scripting language

# meld-spec

## Types

The `meld-spec` package exports the following key types:

### Core Node Types
- `MeldNode` - Base interface for all AST nodes
- `DirectiveNode` - AST node for directives
- `TextNode` - AST node for text content
- `CodeFenceNode` - AST node for code fences
- `VariableNode` - AST node for variables
- `ErrorNode` - AST node for errors
- `TextVarNode` - AST node for text variables (${identifier})
- `DataVarNode` - AST node for data variables (#{identifier})
- `PathVarNode` - AST node for path variables ($identifier)
- `CommentNode` - AST node for comments

### Path Types
- `StructuredPath` - Enhanced path representation for file-based directives
  - `raw` - Original path string as written in the document
  - `normalized` - Normalized form of the path (e.g., with leading `./`)
  - `structured` - Parsed path components
    - `base` - Base directory or special path (e.g., `.`, `$HOMEPATH`)
    - `segments` - Array of path segments
    - `variables` - Map of variables used in the path
    - `cwd` - Optional boolean flag for path type:
      - `true` for paths without slashes (e.g., `file.md`, `${file_path}`)
      - `false` for paths with slashes (e.g., `path/to/file.md`)
      - Defaults to `false` when omitted

### Supporting Types
- `SourceLocation` - Location information for nodes
- `DirectiveData` - Base interface for directive data
- `DirectiveKind` - Union type of all directive kinds
- `MultiLineBlock` - Structure for multi-line content

### Command Types
- `CommandDefinition` - Definition of a command
- `CommandMetadata` - Metadata for commands
- `RiskLevel` - Risk level for commands ('high' | 'med' | 'low')

### Parser Types
- `Parser` - Interface that all Meld parser implementations must implement
- `ParserTestCase` - Structure for parser test cases

### Validation Types
- `ValidationError` - Structure for validation errors
- `ValidationContext` - Context for validation
- `ValidationResult` - Result of validation
- `Example` - Structure for test examples

## Updates

We've enhanced the error handling specification to better support both user-friendly errors and detailed debugging. The key changes are:

1. ErrorNode now includes an optional `debugDetails` field for implementation-specific error details
2. Standard error messages (in `error` field) remain unchanged and are used for test comparisons
3. When in debug mode, you should format errors as: `error + '\n' + debugDetails`

This means you can keep your detailed PEG.js (or other parser) error messages - just put them in `debugDetails` instead of the main error message. This gives users the best of both worlds: clean errors by default, with the option to see technical details when needed.

Example:
```typescript
// Your error node might look like:
{
  type: 'Error',
  error: 'Empty path in @import directive',  // Standard message from spec
  debugDetails: 'Expected \'[\' or [^#[\\]] but \']\' found'  // Your parser's details
}
```

The test suite will only check the `error` field, so you're free to include whatever debugging info is most helpful in `debugDetails`.

## Implementing the Specification

To implement the Meld specification, you'll need to:

1. Import the types and implement the `Parser` interface:
```typescript
import { Parser, MeldNode, DirectiveNode } from 'meld-spec';

class MyMeldParser implements Parser {
  parse(input: string): MeldNode[] {
    // Your implementation here
  }
}
```

2. Import and use the test cases:
```typescript
// Import specific test cases
import {
  dataTests,
  dataInvalidTests,
  textTests,
  textInvalidTests,
  runTests,
  runInvalidTests,
  embedTests,
  embedInvalidTests,
  // ... other test imports
} from 'meld-spec';

// Import validation utilities
import { validateSyntax } from 'meld-spec';

// Test your implementation
const parser = new MyMeldParser();
const result = validateSyntax({ parser }, dataTests);
expect(result.valid).toBe(true);
```

The specification tests define both valid and invalid cases for each feature. Your implementation should:
- Match the expected AST structure for valid cases
- Produce appropriate errors for invalid cases

## Error Handling

The Meld specification defines a standard approach for error handling across all implementations:

### Error Node Structure

```typescript
interface ErrorNode {
  type: 'Error';
  error: string;         // User-friendly error message
  debugDetails?: string; // Optional technical details for debug mode
  partialNode?: MeldNode;// Optional partial AST if available
}
```

### Error Handling Guidelines

1. **Standard Error Messages**
   - Implementations MUST use the standard error messages defined in the test files
   - Error messages should be user-friendly and descriptive
   - Examples:
     - "Empty path in @import directive"
     - "Data variables not allowed in import path"

2. **Debug Mode**
   - Implementations SHOULD support a debug mode
   - In debug mode, errors should include both standard message and technical details
   - Format: `error + '\n' + debugDetails`
   - Example:
     ```
     // Normal mode:
     Empty path in @import directive
     
     // Debug mode:
     Empty path in @import directive
     Expected '[' or [^#[\\]] but ']' found
     ```

3. **Implementation Details**
   - Debug details are implementation-specific
   - Can include parser errors, stack traces, or any helpful debugging information
   - The `debugDetails` field is optional but recommended
   - Debug information should be appended after the standard error message

4. **Comparison Behavior**
   - When comparing error nodes (e.g., in tests), only the `error` field is compared
   - The `debugDetails` field is ignored during comparisons
   - This allows implementations to provide rich debug information while maintaining compatibility

5. **Partial AST**
   - When possible, implementations should provide a partial AST in the `partialNode` field
   - This can help with error recovery and providing better error messages
   - The partial AST is optional and not used in error comparisons