meld-ast/CHANGELOG.md

AST parser for Meld

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [3.4.5]

### Improved
- Enhanced syntax highlighting support for double bracket embed content:
  - Added explicit `isTemplateContent: true` flag to AST structure for double bracket embed directives
  - Ensured the AST doesn't include a `path` property for template content to avoid confusion
  - Improved clarity of AST structure for syntax highlighters
  - Added test cases specifically verifying the AST structure for proper highlighting
  - Extended documentation on how template content is structured differently from path content

## [3.4.4]

### Added
- Enhanced testing for special path variables:
  - Verified correct parsing of `$.` and `$~` path prefixes
  - Confirmed proper AST structure for project and home path references
  - Validated path normalization for special variables
  - Added test coverage for special variables in both import and embed directives
  - Documented requirement for bracketed syntax with path segments (e.g., `[$./docs/file.md]`)
  - Documented that non-bracketed syntax like `@embed $./docs/file.md` doesn't work

### Fixed
- Fixed handling of variable references in double bracket embed content:
  - Now properly handles text variables (`{{variable}}`) in double bracket content
  - Correctly processes path variables (`$pathVariable`) in double bracket content
  - Ensures no invalid warnings are generated for variable references
  - Added comprehensive test coverage for variables in double bracket templates
  - Improved debugging and validation logic for embedded content

## [3.4.3]

### Fixed
- Turned off debug logging in production:
  - Disabled verbose console output that was affecting performance
  - Removed unnecessary logging during parsing operations
  - Improved overall parser performance

## [3.4.2]

### Fixed
- Fixed path variable handling in import directives:
  - Ensured consistent handling of path variables between @import and @embed directives
  - Removed incorrect section identifier support from @import directives
  - Fixed inconsistent handling of special path prefixes (`$.` and `$~`)
  - Added comprehensive test coverage for path variables in import directives
  - Aligned import directive parsing with embed directive for better consistency

## [3.4.1]

### Fixed
- Fixed array access in direct variable embedding:
  - Fixed `@embed {{list.0}}` so it properly handles array access using dot notation
  - Properly normalizes array access notation from dot notation to bracket notation internally
  - Maintains backwards compatibility with existing tests and AST structure
  - Now correctly resolves variable references with array/field access at runtime

## [3.4.0]

### Added
- Added named imports syntax:
  - Now supports selective imports with `@import [var1, var2] from [path.meld]`
  - Supports aliasing with `@import [var1, var2 as alias2] from [path.meld]`
  - Supports explicit wildcard imports with `@import [*] from [path.meld]`
  - Maintains backward compatibility with traditional imports `@import [path.meld]`
  - Enhanced AST structure with `imports` array containing name and alias information
  - Added comprehensive test coverage for named imports

## [3.3.0]

### Changed
- Changed array index notation from brackets to dot notation:
  - Now uses Ruby-style dot notation for numeric indices: `{{users.0}}` instead of `{{users[0]}}`
  - Avoids potential parsing issues with bracket notation
  - Added comprehensive test coverage for numeric field access with dot notation

## [3.2.0]

### Added
- Added URL support in path directives:
  - Now supports URLs beginning with `http://` or `https://` in `@import` and `@embed` paths
  - URLs are preserved in their original form during path normalization
  - Added `url: true` property to structured path objects for URL paths
  - Enhanced path validation to allow paths with slashes when they are URLs or use special variables
  - Improved error messages for invalid paths with clear guidance on proper format

## [3.1.1]

### Fixed
- Fixed directive handling in the grammar:
  - Only recognize directives (`@name`) when they appear at the start of a line
  - Properly handle `@` characters within regular text content
  - Improved line-start detection logic for more reliable parsing
- Fixed comment handling:
  - Properly parse comments (`>> comment`) at the beginning of lines
  - Ensured comments are correctly recognized in the AST
  - Prevented comment markers from being consumed as regular text

## [3.1.0]

### Added
- Added array access support in data variables:
  - Now supports square bracket notation in templates: `{{users[0]}}`, `{{data.items[0].name}}`
  - Handles numeric indices: `{{array[0]}}`
  - Handles string indices: `{{map["key"]}}`
  - Handles variable indices: `{{array[index]}}`
  - Supports mixing field and array access: `{{data.users[0].name}}`
  - Supports nested array access: `{{matrix[0][1]}}`
- Enhanced field access structure:
  - Field and array access elements now have a consistent structure
  - Added comprehensive test coverage for array access patterns

## [3.0.1]

### Fixed
- Fixed JSON array syntax in `@data` directives:
  - Fixed parsing of arrays of objects at the top level
  - Added support for trailing commas in arrays
  - Ensured nested arrays and objects work properly
  - Fixed documentation to match implementation

## [3.0.0]

### Added
- Implemented clear distinction between embed directive bracket types:
  - Single brackets (`@embed [path/to/file.md]`) are now strictly for paths
  - Double brackets (`@embed [[content]]`) are now strictly for content
  - Added validation to enforce correct syntax usage
  - Added helpful error messages for incorrect syntax
- Improved path variable handling in embed directives:
  - Path variables in single brackets (`@embed [$file_path]`) are now properly extracted
  - Path variables in double brackets (`@embed [[$file_path]]`) are treated as literal text
  - Direct variable embedding (`@embed {{variable}}`) is supported for cleaner syntax
  - Enhanced tests to verify both path and content handling
- Enhanced validation in `validateEmbedPath` and `validateEmbedContent` functions:
  - Added checks for multiline content and multiple words in paths
  - Improved detection of content that should use double brackets
  - Updated validation to provide more helpful error messages
- Added support for special path variables in path directives:
  - `$HOMEPATH` and `$~` for home directory paths
  - `$PROJECTPATH` and `$.` for project root paths
  - Proper parsing of path segments after special variables
- Added support for JSON array syntax in @data directives:
  - Now supports `@data var = [ { "some": "var" }, { "another": "var" } ]`
  - Handles nested arrays and objects
  - Supports trailing commas in arrays
- Added optional variable warning flag:
  - Added `variable_warning` boolean option (defaults to `false`)
  - When `true`, suppresses warnings for undefined variables in paths
  - When `false` (default), warnings are emitted for undefined variables

### Changed
- Multiple improvements to embed directive handling:
  - Clear distinction between paths (single brackets) and content (double brackets)
  - Improved property handling for path structures
  - Consistent handling of header levels in embed directives
- Updated test suite to verify bracket syntax distinctions:
  - Added tests for path variables in different bracket types
  - Added tests to ensure proper AST structure based on bracket type
- AST structure now provides clear distinction between paths and content:
  - Single brackets create a node with a `path` property
  - Double brackets create a node with a `content` property
- Improved path validation for import and embed directives:
  - Added context-aware path validation to handle path directives properly
  - Fixed handling of relative paths in test environments
  - Enhanced path structure with proper base and segments properties
- Improved debugging of path resolution issues

### Documentation
- Added comprehensive documentation:
  - Created new ADVICE.md with best practices for modifying PEG.js grammar
  - Documented lessons learned from debugging and fixing parser issues
  - Updated README with details on path directive special variables
  - Added documentation for variable warning configuration

## [2.1.0]
- Standardized field naming across all directives:
  - Changed the `id` field to `identifier` in `PathDirective` and `VarDirective`
  - Updated related tests to check for the `identifier` field
  - Improved codebase consistency for better maintainability

## [2.0.0]

### Changed
- **BREAKING CHANGE**: Unified variable syntax to improve consistency:
  - Text variables now use `{{variable}}` syntax (previously `${variable}`)
  - Data variables now use `{{variable}}` syntax (previously `#{variable}`)
  - Path variables still use `$variable` syntax (unchanged)
  - The parser distinguishes between text and data variables based on context
  - Simplifies the user experience by reducing the number of syntaxes to learn
- Improved multiline template support through `[[...]]` syntax:
  - Works with the new unified variable syntax
  - Provides consistent handling for both text and data multiline content

### Migration
- Replace all instances of `${variable}` with `{{variable}}` for text variables
- Replace all instances of `#{variable}` with `{{variable}}` for data variables
- No changes needed for path variables using `$variable` syntax
- No changes needed for multiline templates using `[[...]]` syntax

## [1.5.0]

### Added
- Support for multiline template literals using `[[ ... ]]` syntax:
  - Added new `MultilineTemplateLiteral` rule to the grammar
  - Enhanced `TextValue` to support multiline templates
  - Added variable interpolation support within multiline templates
  - Added comprehensive test coverage for the new syntax

## [1.4.0]

### Fixed
- Enhanced path parsing and validation:
  - Fixed path validation in EmbedDirective and DataValue rules
  - Improved handling of relative paths
  - Added better debug documentation
- Fixed "Assignment to constant variable" errors:
  - Created new variables instead of reassigning constants in EmbedDirective and DataValue rules
  - Improved code structure
- Fixed header level and section with header tests:
  - Removed cwd property when not needed in structured objects

## [1.3.3]

### Fixed
- Add a Comment case to the node validation switch statement in meld-ast to make sure it's included in validation
- Improved code fence parsing to better handle edge cases:
  - Fixed handling of empty code fences to include proper newlines
  - Fixed handling of mismatched backticks to properly reject invalid fences
  - Ensured consistent newline handling in preserved code fence content
  - Fixed CommonMark test cases for nested code fence handling

## [1.3.2]

Fix: removed vitest being imported in production code.

## [1.3.1]

### Fixed
- Fixed code fence parsing to properly handle nested fences according to CommonMark spec:
  - Equal backticks properly terminate the fence
  - Fewer backticks are treated as literal content
  - Empty code fences are allowed
  - Added comprehensive test coverage for all backtick count combinations (3-5)

## [1.3.0]

### Added
- Added support for code fences with 4 and 5 backticks
- Added proper nesting support for code fences:
  - 4-backtick fences can contain 3-backtick fences
  - 5-backtick fences can contain both 3 and 4-backtick fences
- Added `preserveCodeFences` parser option:
  - Defaults to `true` to maintain backward compatibility
  - When `true`, preserves code fence markers in content
  - When `false`, only includes content between fences

### Changed
- Changed default behavior of code fence parsing to preserve fence markers
- Updated grammar to support nested code fence detection
- Improved code fence content handling based on `preserveCodeFences` option

## [1.1.0]

### Fixed
- Fixed parser exports to properly support both ESM and CommonJS environments
- Fixed issue with duplicate exports in generated parser code
- Fixed TypeScript type resolution for parser imports

### Changed
- Moved generated parser files to separate `lib/grammar` directory
- Updated build process to preserve parser files during clean step
- Improved TypeScript configuration to properly handle parser files

### Added
- Added better error handling for parser initialization
- Added proper type definitions for parser exports
- Added support for both default and named exports in ESM

## [1.0.8]
- Fixed grammar file path resolution in installed packages:
  - Added environment detection to handle development and production paths
  - Fixed package root resolution in node_modules
  - Added detailed environment information in error messages
  - Improved debug logging with file existence checks
  - Added development/production path handling

## [1.0.5]

- Fixed ESM path resolution:
  - Fixed package root resolution in ESM environments using triple dirname
  - Added comprehensive debug logging for path resolution
  - Improved error messages to include package root information
  - Fixed grammar file loading in ESM contexts
  - Added detailed path resolution logging
  - Fixed incorrect path resolution in node_modules
- Fixed grammar file path resolution:
  - Switched to package root-relative paths for more reliable resolution
  - Added detailed debug logging for grammar file loading attempts
  - Improved error messages to show full resolved paths
  - Ensured grammar file is properly included in package distribution
  - Added fallback paths for different installation scenarios

## [1.0.4]

- Fixed peggy ESM/CJS interoperability issues:
  - Improved parser loading strategy to handle both ESM and CJS environments
  - Fixed peggy import handling to support both `require` and dynamic `import`
  - Added proper type definitions for peggy parser options
  - Enhanced error messages with more context
  - Added fallback mechanisms for different module systems
- Improved pre-built parser handling:
  - Added support for both ESM (.js) and CJS (.cjs) parser files
  - Prioritized pre-built parser loading over grammar compilation
  - Added better error handling and debug logging
  - Improved parser initialization error messages

## [1.0.3]

### Fixed
- Fixed grammar file path resolution:
  - Added support for multiple grammar file locations
  - Fixed path resolution in production environments
  - Improved error messages with attempted paths
- Resolved ESM/CJS compatibility issues:
  - Removed top-level await in parser loading
  - Fixed parser initialization in ESM context
  - Improved pre-built parser loading strategy
- Fixed type declaration generation:
  - Corrected mock parser implementation
  - Added proper peggy SyntaxError interface support
  - Fixed type assertions in test utilities
- Improved test framework compatibility:
  - Enhanced mock SyntaxError implementation
  - Added proper static properties to mock classes
  - Fixed type definitions in test utilities

## [1.0.1]

### Changed
- Simplified build configuration:
  - Removed separate dts build step
  - Unified build process through tsup
  - Improved source map generation
  - Cleaner package.json scripts
- Improved test framework:
  - Migrated from Jest to Vitest
  - Enhanced mock validation context implementation
  - Added better test framework compatibility

### Fixed
- Test framework compatibility issues
- Mock validation context implementation
- Jest to Vitest migration issues

## [1.0.0]

### Added
- Comprehensive error handling system:
  - `MeldAstError` class with rich context
  - Error codes for different failure types
  - Location tracking in errors
  - Error serialization support
  - Custom error handler support
  - Non-fatal error collection
- Robust parser initialization:
  - Environment-independent loading strategy
  - Reliable fallback mechanism
  - Clear error messages and debug logging
- Complete testing framework:
  - Type-safe node creation with `createMockNode`
  - Parser behavior testing with `createMockParser`
  - Validation testing with `createMockValidationContext`
  - Location testing with `createMockLocation`
  - Parse result type guard with `hasErrors`
- Comprehensive build system:
  - Grammar validation during build
  - Source map generation for all files
  - Build verification with extensive checks
  - Pre-built parser bundling
  - File presence verification
  - Parser export verification
- Full TypeScript support:
  - Strict types for parser options
  - Comprehensive error type definitions
  - Type tests and validation
  - JSDoc documentation
- Complete development documentation:
  - Project structure guide
  - Build process documentation
  - Development workflow guide
  - Debugging instructions
  - Common development tasks
  - Integration examples
  - Troubleshooting guide

### Changed
- Replaced environment-based parser loading with universal strategy
- Enhanced parser options with full validation and error handling
- Improved error context and type consistency
- Strengthened build process reliability
- Restructured project for better organization:
  - Colocated related functionality
  - Improved file organization
  - Better separation of concerns
- Streamlined package exports:
  - Direct import paths
  - Improved ESM/CommonJS interop
  - Better internal API access

### Fixed
- Parser initialization reliability
- Error handling consistency
- Build process validation
- Source map generation and linking
- Development environment setup
- Module resolution and interop
- Type definition organization

## [0.4.2]

### Changed
- Moved to colocated TypeScript type definitions
- Updated package exports to point to ESM/CJS-specific type definitions
- Removed separate types directory in favor of generating .d.ts files alongside compiled code

## [0.3.2]

### Changed
- Updated TypeScript configuration to use `"module": "NodeNext"` and `"moduleResolution": "NodeNext"`
- Updated to meld-spec@0.3.9 for improved module resolution and test exports
- Added comprehensive documentation for comment node support
- Improved build process documentation

### Fixed
- Fixed module resolution issues with ESM/CJS interop
- Ensured proper test file exports from meld-spec dependency

## [0.3.1]

### Changed
- Converted package to ESM format
- Updated build process to generate ESM-compatible parser
- Updated TypeScript configuration for proper ESM support
- Updated test suite to work with ESM imports
- Switched from local meld-spec import to npm published version

### Fixed
- Fixed grammar file path resolution in production builds
- Improved error handling for parser loading
- Fixed test suite compatibility with ESM modules

## [0.2.2]

### Fixed
- Fixed runtime error where grammar file wasn't being copied to dist directory
- Added proper error handling for grammar file loading

### Changed
- Removed unused experimental grammar files
- Streamlined build process to only copy necessary grammar file

## [0.2.1]
- Initial public release with basic functionality