mlld/CHANGELOG.md

mlld: a modular prompt scripting language

# Changelog

All notable changes to the mlld project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).


## [2.0.0-rc34]

### Added
- **Array Index Support in For Loops**: The `_key` pattern
now provides array indices when iterating over arrays
  - Arrays provide their indices as keys: `0`, `1`, `2`, etc.
  - Example: `/for @item in ["a", "b", "c"] => /show
"@item_key: @item"` outputs `0: a`, `1: b`, `2: c`
  - Objects continue to provide property names as keys
  - Enables consistent key access patterns across all
collection types

- **Dot Escape Sequence**: Added `\.` to escape sequences for
  literal dots in strings
  - Disambiguates between field access and string
concatenation
  - `@variable.field` - attempts to access the `field`
property
  - `@variable\.txt` - produces the string value followed by
`.txt`
  - Works in all string contexts: double quotes, backticks,
and templates
  - Example: `/output @content to "file-@num\.txt"` creates
`file-42.txt`

- **Metadata Shelf for Alligator Arrays**: Preserves LoadContentResult metadata when arrays pass through JavaScript functions
  - When `<*.md>` arrays are passed to JS functions like `slice()`, metadata (filename, frontmatter, etc.) is preserved
  - Enables patterns like: `/var @subset = @slice(@files, 0, 5)` followed by `/for @file in @subset => /show @file.filename`
  - Transparent to JS functions - they receive content strings as before
  - Fixes issue where `@file.filename` would fail after JS array operations

### Fixed
- **Missing Slash in For Actions**: Fixed syntax error on
line 18 of `llm/run/testing.mld` where `/show` was missing
its slash prefix

- **LoadContentResult Preservation in For Loops**: For loops now properly preserve LoadContentResult objects
  - `@file` in `/for @file in <*.md>` maintains its properties (filename, content, fm, etc.)
  - Field access like `@file.filename` works correctly in all for loop contexts

## [2.0.0-rc33]

### Added
- **Wildcard (*) Literal**: New wildcard literal that always evaluates to true in conditional contexts -- specifically useful as a catch-all in a multiple condition /when sequence in order to be more immediately understandable than '/when... true' 
  - Basic usage: `/when * => /show "Always executes"`
  - Default handler in when blocks: `/when [@condition => action, * => "default"]`
  - Catch-all pattern in exe functions: `/exe @handler() = when: [* => "default response"]`
  - Works with logical operators: `/when * && @check => action`
  - Evaluates to true in ternary expressions: `/var @result = * ? "yes" : "no"`
  - Follows Unix glob convention where `*` means "match anything"

### Fixed
- **Template Variable References**: Fixed parsing bug where tail modifier keywords (`with`, `pipeline`, `needs`, `as`, `trust`) were incorrectly interpreted inside template contexts
  - Created separate `TemplateVariableReference` pattern for template interpolation that doesn't check for tail modifiers
  - Keywords like "with" can now appear as literal text after variables in templates
  - Fixes: `/exe @claude(prompt,tools) = `@prompt with @tools`` now parses correctly
  - Affects backtick templates, double-colon templates, and double-quoted strings
  - Template variables should never have tail modifiers - those constructs only make sense in command contexts

- **Shell Escaping in /for Loops**: Fixed shell command escaping issues when iterating over arrays with special characters
  - Loop variables are now properly quoted when used in shell commands
  - Handles filenames with spaces, quotes, and other special characters correctly
  - Example: `/for @file in <*.md> => /run echo "@file"` now works with "file with spaces.md"

- **Nested Function Execution**: Fixed execution of nested functions in imported modules
  - Functions like `@module.category.function()` now execute correctly instead of returning string representations
  - Deeply nested module exports are now properly resolved as executable functions
  - Affects complex module structures with multiple levels of organization

## [2.0.0-rc32]

### Added
- **For Loop Iteration**: New `/for` directive for iteration over arrays and objects
  - Output form: `/for @item in @collection => action` - Executes action for each item
  - Collection form: `/var @results = for @item in @collection => expression` - Collects results into array
  - Array iteration: `/for @item in ["a", "b", "c"] => /show @item`
  - Object iteration: `/for @value in {"x": 1, "y": 2} => /show @value` 
  - Object key access: `@value_key` pattern provides access to keys when iterating objects
  - Works with all iterable values including globs: `/for @file in <*.md> => /show @file.filename`
  - Preserves variable type information throughout iteration for consistent behavior
  - Semantic token support in LSP for syntax highlighting
  - Compatible with pipelines and transformations

## [2.0.0-rc31]

### Added
- **Enhanced error display with source context**: Errors now show the exact source location with surrounding context and a visual pointer
  - Compiler-style error messages with line numbers and caret indicators pointing to the precise error location

- **Improved Error Pattern System**: Complete refactor of parse error enhancement for better performance and maintainability
  - Patterns are now pure functions that extract variables (no imports allowed)
  - Templates use `${VARIABLE}` placeholders for dynamic error messages
  - Build-time compilation: All patterns compile into single `parse-errors.generated.js` file
  - Convention-over-configuration pair of `pattern.js`, `error.md`, and `example.md` 
  - Build integration: `npm run build:errors` compiles all patterns

- **LSP Semantic Tokens Support**: Full semantic highlighting via Language Server Protocol
  - Context-aware highlighting for all template types (backtick, double-colon, triple-colon)
  - Proper interpolation detection based on template context (@var vs {{var}})
  - Command content interpolation with @variable support
  - Field access and array indexing highlighting (@user.profile.name, @items[0])
  - Embedded language region marking for editor syntax injection
  - Mixed array/object support - highlights mlld constructs within data structures
  - Operator highlighting for logical (&&, ||, !), comparison (==, !=, <, >), and ternary (? :)
  - Error recovery and graceful handling of partial ASTs
  - Performance optimizations with text caching
  - Available in VSCode and any LSP-compatible editor (Neovim, etc.)

- **Enhanced LSP Error Reporting**: Precise error locations and improved error messages
  - Errors now use exact start/end positions from parser's mlldErrorLocation data
  - Full-line highlighting when errors occur at the beginning of a line
  - Multi-line error messages display with proper formatting in VSCode
  - Parser error messages can be edited directly in the grammar files
  - Example error messages include all valid syntax patterns

## [2.0.0-rc30]

This release allows mlld to function as a logical router

### Added
- **Logical and Comparison Operators in Expressions**
  - New operators for `/var` assignments and `/when` conditions: `&&`, `||`, `==`, `!=`, `!`, `?`, `:`
  - Expression parsing with proper operator precedence: `@a && @b || @c` parses as `((@a && @b) || @c)`
  - Ternary conditional expressions: `/var @result = @test ? @trueVal : @falseVal`
  - Binary expressions with comparison: `/var @isEqual = @x == @y`, `/var @different = @a != @b`
  - Unary negation: `/var @opposite = !@condition`
  - Parentheses for explicit precedence: `/var @complex = (@a || @b) && (@c != @d)`
  - Full expression support in when conditions: `/when @tokens > 1000 && @mode == "production" => /show "High usage detected"`
  - Short-circuit evaluation: `&&` and `||` operators properly short-circuit for performance
  - Type coercion following mlld semantics: `"true" == true` → true, `null == undefined` → true
  - Comparison operators: `<`, `>`, `<=`, `>=` for numeric comparisons

- **Implicit When Actions**
  - Simplified syntax within `/when` blocks - directive prefix is now optional
  - Variable assignments: `/when @prod => @config = "production"` (no `/var` needed)
  - Function calls: `/when @ready => @setupDatabase()` (no `/run` needed)
  - Exec assignments: `/when @processing => @transform() = @processData(@input)` (no `/exe` needed)
  - Mixed implicit/explicit actions in blocks: `/when @cond => [@x = "value", /var @y = "other"]`

- **RHS When Expressions (Value-Returning)**
  - When expressions as values in `/var` assignments: `/var @greeting = when: [@time < 12 => "Good morning", @time < 18 => "Good afternoon", true => "Good evening"]`
  - When expressions in `/exe` definitions: `/exe @processData(type, data) = when: [@type == "json" => @jsonProcessor(@data), @type == "xml" => @xmlProcessor(@data), true => @genericProcessor(@data)]`
  - First-match semantics - returns the first matching condition's value
  - Returns `null` when no conditions match
  - Lazy evaluation in variables - re-evaluates on each access
  - Pipeline support: `/var @result = when: [...] | @uppercase`

- **Enhanced String Interpolation**
  - Fixed file reference interpolation in double-quoted strings: `"Content from <file.md> here"`
  - Consistent handling of both `@variable` and `<file.md>` interpolation patterns
  - Proper support for `wrapperType: 'doubleQuote'` in interpreter evaluation
  - Safety checks prevent empty value arrays from causing "missing value" errors

### Changed
- **Hybrid console.log behavior in JavaScript execution**
  - `console.log()` now always outputs to stdout for debugging visibility
  - When a function has an explicit return value, that value is stored in the variable
  - When a function has no return value but uses console.log, the console output becomes the result (backward compatibility)
  - This approach maintains compatibility with existing tests while providing better debugging experience
  - Example: `js { console.log("debug"); return "result" }` shows "debug" on stdout and stores "result"
  - Example: `js { console.log("output") }` shows "output" on stdout AND stores "output" as the result

### Fixed
- **Grammar and Parser Improvements**
  - Fixed CommandReference type mismatches between grammar output and TypeScript expectations
  - Added translation layer in evaluators to handle both legacy and new AST formats
  - Improved error recovery and backward compatibility for when directive patterns
  
- **Test Infrastructure Stability**
  - Updated test expectations to align with new console.log behavior
  - Fixed test cases that relied on specific output formatting
  - Resolved shadow environment test issues with variable interpolation in literal strings

## [2.0.0-rc28]

### Fixed
- **ImportResolver PathContext issue in ephemeral mode**
  - Fixed TypeError when running mlld scripts via `npx mlldx@latest` with local file paths
  - ImportResolver was not receiving PathContext when Environment.setEphemeralMode() recreated it
  - Ephemeral mode now properly passes PathContext to ImportResolver constructor
  - Enables relative imports to work correctly in ephemeral/CI environments

- **Double-colon syntax (`::...::`) now properly handles colons in content**
  - Fixed parser incorrectly terminating on single colons (`:`) inside double-colon templates
  - Grammar fix in `DoubleColonTextSegment` changed from `![:@<]` to `!("::" / "@" / "<")`
  - Affects all uses of double-colon syntax: `/var`, `/exe`, `/show`, data objects, etc.
  - Now correctly handles URLs (`https://example.com`), times (`3:30`), ratios (`16:9`), and other colon-containing content
  - Double-colon syntax works as complete alternative to backticks for templates with `@var` interpolation
  - Triple-colon syntax `:::...:::` continues to support `{{var}}` interpolation

### Changed
- **Renamed WhenSwitchForm to WhenMatchForm**
  - Grammar and types now use "WhenMatchForm" for the `/when @var: [...]` syntax
  - More accurate naming - this form executes actions for all matching conditions, not just the first
  - Updated subtype from `whenSwitch` to `whenMatch` throughout codebase for more accurate reflection of functionality

## [2.0.0-rc27]

### Added
- **Registry Direct Publishing for Module Updates**
  - Module owners can now publish updates directly without PR review
  - First module publish still requires PR for quality control
  - Automatic PR detection prevents duplicate submissions
  - Interactive version bump when conflicts occur
  - Auto-grant publish rights after first module is merged
  - API service live at registry-api.mlld.org for direct publishing

- **Version and Tag Support for Registry Modules**
  - Import specific versions: `@import { ... } from @author/module@1.0.0`
  - Semver range support: `@import { ... } from @author/module@^1.0.0`
  - Tag support: `@import { ... } from @author/module@beta`
  - Version resolution follows semver rules
  - Backward compatible - existing imports continue to work

### Fixed
- **Support for variables in /run code blocks**
  - Fixed regression where `/run js (@variable) {...}` syntax wasn't working
  - Variables can now be passed to code blocks: `/run js (@name, @data) { console.log(name, data) }`
  - Changed grammar to require `@variable` references (not bare identifiers) since `/run` executes immediately
  - Aligns with design principle: bare identifiers are for parameters in `/exe` definitions, `@` references are for existing variables
  - Works with all supported languages: `js`, `node`, `python`, `bash`, etc.
  - Variables are auto-unwrapped (LoadContentResult objects become their content strings)

## [2.0.0-rc26]

### Added
- **Auto-unwrapping of LoadContentResult objects in JavaScript/Node functions**
  - LoadContentResult objects (from `<file>` syntax) are now automatically unwrapped to their content strings when passed to JS/Node functions
  - Enables natural usage: `/run @processFile(<data.txt>)` - the function receives the file content as a string, not the LoadContentResult object
  - Also handles LoadContentResultArray from glob patterns: `<*.txt>` unwraps to an array of content strings
  - Maintains mlld's content-first philosophy where file content is the primary concern
  - Works with all JavaScript (`js`) and Node.js (`node`) executables

## [2.0.0-rc25]

### Added
- **Built-in @typeof() function for type introspection**
  - New transformer function that returns type information for any mlld variable
  - Syntax: `@typeof(@variable)` returns the variable's type (e.g., "simple-text", "primitive (number)", "object (3 properties)")
  - Includes source directive information: `@typeof(@myVar)` → "simple-text [from /var]"
  - Works with all variable types: simple-text, path, primitive, object, array, executable, pipeline-input
  - Can be used in pipelines: `@myVar | @typeof`
  - Available in both uppercase (@TYPEOF) and lowercase (@typeof) forms

## [2.0.0-rc24]

### Fixed
- **Inconsistent handling of LoadContentResult objects between /show and /output**
  - Fixed `/output` to match `/show` behavior when outputting variables containing `<file>` alligator syntax results
  - `/output @myfile` now outputs just the file content (not the full metadata object) when `@myfile` contains a LoadContentResult
  - Also handles arrays of LoadContentResult objects from glob patterns, concatenating their content with double newlines
  - Both commands now consistently treat the alligator syntax as accessing file content, not the full file object

## [2.0.0-rc23]

### Fixed
- **Namespace import structure for better ergonomics**
  - Namespace imports intelligently unwrap single-export modules
  - `/import @mlld/env as environment` now allows `@environment.get()` instead of requiring `@environment.env.get()`
  - Modules exporting a single main object matching common patterns (module name, 'main', 'default', 'exports') are automatically unwrapped
  - Multiple-export modules remain unchanged, preserving full namespace structure

- **Shadow environment preservation regression from rc22**
  - Fixed issue where shadow environments were lost when accessing imported executables through field access
  - rc22's manual reconstruction of ExecutableVariable from `__executable: true` objects was missing deserialization of captured shadow environments
  - Shadow environments (stored as objects during export) are now properly deserialized back to Maps
  - Captured shadow environments are correctly passed to code execution via `__capturedShadowEnvs` parameter
  - Functions like `@github.pr.review()` can now access their required shadow environment functions

- **Node.js executable path in test environments**
  - Fixed `mlld-wrapper.cjs` to use `process.execPath` instead of hardcoded 'node'
  - Fixed test utility to use `process.execPath` for cross-environment compatibility
  - Resolves "spawn node ENOENT" errors in environments where 'node' is not in PATH

## [2.0.0-rc22]

### Fixed
- **Nested executable field access in `/run` directives**
  - Fixed interpreter bug where `/run @github.pr.review(...)` and similar nested field access patterns failed
  - Handles both local ExecutableVariable objects and serialized `__executable: true` format from imports
  - Properly reconstructs executable metadata for imported modules with nested structure

## [2.0.0-rc21]

### Added
- **Environment variable management for CLI**
  - Added `--env` flag to load environment variables from a specific file
  - `mlld test` command automatically loads `.env` and `.env.test` files from the current directory
  - `mlldx` supports `--env` flag for ephemeral environments
- **Test isolation improvements**
  - Tests now run in isolated processes when multiple test files are executed
  - Prevents environment variable pollution between test modules
  - Shadow environment functions are properly cleaned up between tests
  - Added `--isolate` flag for explicit process isolation

### Changed
- **Test command environment handling**
  - Removed console output capture that was interfering with HTTP requests
  - Improved test result parsing from isolated subprocess output
  - Better error handling for test cleanup failures

### Fixed
- **Variable contamination between test modules**
  - Shadow environment variables no longer leak between test files
  - Each test gets a clean environment state
  - Process isolation ensures complete separation when running multiple tests

## [2.0.0-rc20]

### Added
- **Shadow environment preservation through imports**
  - Functions that use shadow environments now work correctly when imported from modules
  - Implements lexical scoping for shadow environments - functions retain access to their original shadow context
  - Supports both JavaScript and Node.js shadow environments

### Fixed
- **Shadow environment functions not accessible after import**
  - Previously, functions relying on shadow environment helpers would fail with "function not defined" errors
  - Shadow environments are now captured at function definition time and restored during execution
  - Enables proper module encapsulation with internal helper functions

## [2.0.0-rc19]

### Added
- **Async/await support in JavaScript executor**
  - JavaScript code blocks now automatically support `await` syntax
  - Detects `await` keyword and creates async functions transparently
  - Shadow environment functions work with async code

## [2.0.0-rc18]

### Fixed
- **Module import resolution for nested object structures**
  - Fixed bug where functions in deeply nested module exports appeared as strings instead of executables
  - ObjectReferenceResolver now recursively resolves VariableReference nodes in nested objects
  - Affects modules with 3+ level nesting like `@mlld/github` where `github.pr.view` was showing as `"@pr_view"` instead of `<function>`
  - Registry review workflow and all GitHub integrations now work properly
- **System variable export filtering**
  - Fixed module export filtering to properly exclude system variables using `metadata.isSystem`
  - Prevents namespace collisions when importing multiple modules with frontmatter
  - System variables like `@fm` are no longer incorrectly exported from modules

## [2.0.0-rc16]

### Changed
- **@input resolver no longer strips MLLD_ prefix** 
  - Environment variables with `MLLD_` prefix are now imported with their full names
  - What you set is what you get: `MLLD_GITHUB_TOKEN` imports as `MLLD_GITHUB_TOKEN`, not `GITHUB_TOKEN`

## [2.0.0-rc15]

### Added
- **mlldx command for ephemeral/CI environments**: New binary for serverless and CI use cases
  - `mlldx` runs with ephemeral mode enabled - all caching happens in memory only
  - No filesystem persistence for read-only containers and serverless functions
  - Auto-approves all imports, no interactive prompts that would hang CI/CD pipelines
  - Available via npx: `npx mlldx@latest script.mld` or installed globally
  - Ships from same package as mlld
  - Useful for GitHub Actions, Vercel functions, AWS Lambda, and other ephemeral environments

## [2.0.0-rc14]

### Fixed
- **Serverless environment support**: Fixed cache directory creation in read-only filesystems
  - Automatically uses `/tmp` for cache in serverless environments (Vercel, AWS Lambda)
  - Detects serverless by checking for `/var/task` path or environment variables
  - Enables mlld to run in read-only container environments

## [2.0.0-rc13]

### Added
- **Import auto-approval CLI flags**: New flags for non-interactive environments
  - `--risky-approve-all`, `--yolo`, `-y` flags to bypass import security prompts
  - Essential for serverless/CI environments where interactive prompts would hang
  - Enables registry review system to work in Vercel functions

### Fixed
- **mlld clean command cache clearing**: Enhanced to remove all cached imports
  - Now clears immutable import cache in `.mlld/cache/imports/` directory
  - Removes both content files and metadata (`.meta.json` files)
  - Fixes stale import cache issues when remote files are updated
- **Serverless environment support**: Fixed cache directory creation in read-only filesystems
  - Automatically uses `/tmp` for cache in serverless environments (Vercel, AWS Lambda)
  - Detects serverless by checking for `/var/task` path or environment variables
  - Enables mlld to run in read-only container environments

## [2.0.0-rc12]

### Fixed
- **URL-relative import resolution**: Fixed relative imports when running scripts from URLs
  - Scripts loaded from URLs (e.g., via `npx mlld@latest https://...`) can now use relative imports
  - `../modules/file.mld` correctly resolves to full URL when current file is a URL
  - Enables serverless execution of mlld scripts with local module dependencies
  - Fixes registry review system import resolution issues

## [2.0.0-rc11]

### Fixed
- **Import collision detection**: Fixed false positive collisions with system variables
  - System variables like frontmatter (`@fm`) no longer trigger import collision errors
  - Multiple modules with frontmatter can now be imported without conflicts
  - Collision detection now only applies to legitimate user-defined variables
  - Resolves registry review deployment issues caused by frontmatter variable conflicts

## [2.0.0-rc10]

### Added
- **URL execution support**: Run mlld scripts directly from URLs
  - Execute scripts from any HTTP/HTTPS URL: `mlld https://example.com/script.mld`
  - Useful for CI/CD pipelines: `npx mlld@latest https://raw.githubusercontent.com/user/repo/main/script.mld`
  - In-memory execution without temporary files
  - Automatic redirect handling (up to 5 redirects)
  - Configurable timeout and size limits via CLI options
- **mlld clean command**: New command for cleaning cached module metadata
  - `mlld clean <module...>` - Remove specific modules from lock file and cache
  - `mlld clean --all` - Clear all cached imports and force fresh resolution
  - `mlld clean --registry` - Clear only registry modules (preserving local modules)
  - `--verbose` flag for detailed output during cleaning operations
  - Helps resolve issues with stale cached module data preventing proper imports

### Fixed
- **Registry import system**: Complete overhaul of module import processing
  - Fixed registry imports returning empty objects instead of module exports
  - Unified import processing path for both local and registry imports
  - Added proper frontmatter extraction for registry resolver imports
  - Improved error handling with specific 404 detection and clear error messages
- **Registry URL validation**: Added publish-time verification
  - Verify generated URLs are publicly accessible before publishing
  - Check that published content matches recorded integrity hashes
  - Prevent broken modules from being published without detection
- **Lock file path handling**: Fixed CLI commands to use correct lock file location
  - Commands now properly read `mlld.lock.json` from project root instead of `.mlld/` subdirectory
  - Affects `mlld ls`, `mlld clean`, and other commands that manage module metadata

## [2.0.0-rc7]

### Fixed
- **Logger compatibility with serverless environments**: 
  - Fixed winston logger attempting to create logs directory in read-only filesystems
  - File transports are now conditionally added only when logs directory exists
  - Prevents ENOENT errors when running mlld in Vercel, AWS Lambda, and other serverless platforms

## [2.0.0-rc6]

### Added
- **Enhanced `/when` directive support**:
  - Variable function calls in when actions: `/when !@condition => /var @result = @function(@param)`
  - Non-existent fields now evaluate to falsy instead of throwing errors
  - Works in all when forms: simple (`@when @obj.missing => ...`), block, and with modifiers
- Updated module publishing flow

## [2.0.0-rc5]

### Changed
- **Variable Type System**: Complete refactor of how variables flow through mlld
  - Variables now preserve type information and metadata throughout evaluation
  - Type detection uses O(1) property checks instead of content inspection
  - Shadow environments (JS, Node, Python) receive rich type info via proxies
  
### Added
- **Bash Variable Adapter**: Clean adapter for bash/sh environments
  - Bash receives string values while other languages get full type information
  - Fixes JavaScript errors when bash tries to access helper functions
- **Type Introspection**: New methods for runtime type checking
  - `mlld.getType()`, `mlld.isVariable()`, `mlld.getMetadata()`

### Fixed
- ArrayVariable storing AST structure instead of evaluated values
- Empty string returns and JavaScript errors in bash/sh execution
- Overly broad type guards that matched any string array

### Removed
- Enhanced variable passing feature flag (now always enabled)
- Legacy factory functions `createRenamedContentArray` and `createLoadContentResultArray`

## [2.0.0-rc4]

### Added
- **File Reference Interpolation**: File references `<file.md>` can now be interpolated in strings and templates
  - Interpolate in backticks: `` `Content: <README.md>` ``
  - Interpolate in double quotes: `"Including <file.txt> here"`
  - Field access on files: `<package.json>.name`, `<data.json>.users[0].email`
  - Works with globs: `<*.md>.fm.title` gets all markdown titles
  - Special `<>` placeholder in 'as' clauses: `<*.md> as "# <>.filename"`
- **Condensed Pipe Syntax**: Both file references and variables support pipe transformations
  - File pipes: `<file.json>|@json|@xml` - load JSON and convert to XML
  - Variable pipes: `@data|@upper|@trim` - transform variable values
  - No spaces allowed in condensed syntax (use full `| @transform` in directives)
- **Variable Pipe Support**: Variables can now use pipes in interpolation contexts
  - In templates: `` `Data: @myvar|@json` ``
  - In quotes: `"Name: @user.name|@upper"`
  - Transforms can be built-in or imported from modules
- **Triple Colon Template Syntax**: New `:::...:::` syntax for `{{var}}` interpolation
  - Addresses the common case of needing backticks inside templates
  - Example: `:::Code example: `getData()` returns {{data}}:::`
  - Double colon `::...::` syntax now uses `@var` interpolation instead of `{{var}}`

### Changed
- **Template Interpolation Syntax**: Double colon `::...::` now uses `@var` interpolation instead of `{{var}}`
  - **Migration required**: Change `::Hello {{name}}::` to `:::Hello {{name}}:::`
  - Double colon templates can now include backticks: `::The `function()` returns @value::`
  - This change enables technical documentation with inline code examples
- **Removed Foreach Section Pattern**: The `foreach <@array # section>` syntax has been removed
  - Migration: Use `<*.md # section> as "template"` instead
  - The new file interpolation syntax completely supersedes this pattern
  - Simpler and more intuitive: direct glob + template in one expression

### Fixed
- Circular file references now emit warnings instead of errors
  - `<**/*.mld>` in an .mld file correctly returns all OTHER .mld files
  - Prevents infinite loops while allowing useful self-excluding patterns

### Changed
- **Reserved Variables Now Lowercase**: All built-in reserved variables have been converted to lowercase for consistency
  - `@NOW` → `@now` (current timestamp)
  - `@DEBUG` → `@debug` (debug information)
  - `@INPUT` → `@input` (stdin/environment access)
  - `@PROJECTPATH` → `@base` (project root directory)
- **Removed @. Alias**: The `@.` alias for project root has been removed; use `@base` instead
- **Simplified Naming**: Aligns with interpreter's `basePath` terminology and modern naming conventions

## [2.0.0-rc3]

### Added
- **Dev Mode**: Local module development support with automatic prefix mapping
  - `mlld mode dev` - Enable dev mode (persists in lock file)
  - `mlld dev status` - Show current mode and detected local modules
  - `mlld dev list` - List all local modules with their publish names
  - `--dev` flag for one-time dev mode override
  - `MLLD_DEV=true` environment variable support
  - Automatically maps `@author/module` imports to local files in `llm/modules/`
- **Mode Command**: Set mlld execution mode
  - `mlld mode dev/development` - Enable development mode
  - `mlld mode prod/production` - Enable production mode
  - `mlld mode user` - Default user mode
  - `mlld mode clear/reset` - Remove mode setting (defaults to user)
  - Mode stored in `mlld.lock.json` under `config.mode`
  - Future extensibility for security modes with different permissions
- **Alligator Syntax**: New syntax for file loading that eliminates bracket ambiguity
  - File loading: `<file.md>` replaces `[file.md]`
  - Section extraction: `<file.md # Section>` replaces `[file.md # Section]`
  - URL loading: `<https://example.com/file.md>` replaces `[https://example.com/file.md]`
  - Resolver paths: `<@./path>` and `<@PROJECTPATH/path>` replace bracketed versions
  - Square brackets `[...]` now exclusively mean arrays, removing all ambiguity
  - Clear visual distinction: angles `<>` load content, brackets `[]` define arrays
- **Glob Pattern Support**: Alligator syntax now supports glob patterns for loading multiple files
  - Glob patterns: `<*.md>`, `<**/*.ts>`, `<src/**/*.js>`
  - Returns array of LoadContentResult objects with metadata
  - Each file includes content and rich metadata properties
- **Rich Metadata for Loaded Content**: Files and URLs loaded with `<>` syntax now include metadata
  - **File Metadata**:
    - `content`: The file's text content (default when used as string)
    - `filename`: Just the filename (e.g., "README.md")
    - `relative`: Relative path from current directory
    - `absolute`: Full absolute path
    - `tokest`: Estimated token count based on file type (750/KB for text, 500/KB for code)
    - `tokens`: Exact token count using tiktoken (lazy-evaluated)
    - `fm`: Parsed frontmatter for markdown files (lazy-evaluated)
    - `json`: Parsed JSON for .json files (lazy-evaluated)
  - **URL Metadata** (additional properties for URLs):
    - `url`: The full URL
    - `domain`: Just the domain (e.g., "example.com")
    - `title`: Page title (extracted from HTML)
    - `description`: Meta description or og:description
    - `html`: Raw HTML content (for HTML pages)
    - `text`: Plain text extraction (HTML stripped)
    - `md`: Markdown version (same as content for HTML)
    - `headers`: Response headers object
    - `status`: HTTP status code
    - `contentType`: Content-Type header value
  - Access metadata with field syntax: `@file.filename`, `@url.domain`, `@page.title`, etc.
  - Smart object behavior: shows content when displayed, preserves metadata when stored
  - Note: Some metadata properties use lazy evaluation and may not be accessible in certain contexts due to issue #315
- **HTML to Markdown Conversion**: URLs returning HTML are automatically converted to clean Markdown
  - Uses Mozilla's Readability to extract article content (removes navigation, ads, sidebars)
  - Uses Turndown to convert the clean HTML to well-formatted Markdown
  - `/show <https://example.com/article>` displays the article as Markdown by default
  - Raw HTML still accessible via `@page.html` property (when #315 is resolved)

### Fixed
- Duplicate `--dev` case clause in ArgumentParser
- Property name consistency (`dev` vs `devMode`) across CLI interfaces

## [2.0.0]

Represents an overhaul and consolidation of all syntax. 

The `/` command approach creates clear disambiguiation between commands and variables/executables, while also setting the stage for using mlld in chat contexts. We are moving to a simple variable definition model with `/var` while allowing rich expression for different types based on the provided syntax.

### Updated Syntax:
- Directives: Changed from @ prefix to / prefix (e.g., @text → /var, @add → /show)
- Variable creation: Now requires @ prefix (e.g., /var @name = "value")
- Command syntax: Changed from [(command)] to {command} or "command" (single-line, non-shellscript)
- Code syntax: must use {...} for code blocks
- Unified /var: Replaced multiple directives (@text, @data) with single /var
- Renamed directives: @add → /show, @exec → /exe
- /output for file output
- Comments: Use >> for line start/end comments (but not in params/objects/templates)
- Template syntax: Changed from [[...]] to ::...:: to avoid array parsing ambiguity

### Updated Interpolation:
- Double quotes: Now support @variable interpolation
- Backticks: Primary template syntax with @variable interpolation
- Double colons: Template syntax for @-heavy content, uses {{variable}}
- Commands: Use @variable in both {...} and "..." forms

### Added:
- **Namespace Imports**: Import entire files or modules as namespaced objects
  - File imports: `/import [./file.mld]` creates namespace from filename (e.g., `@file`)
  - Custom alias: `/import [./file.mld] as myname` creates `@myname` namespace
  - Module imports: `/import @author/module` creates `@module` namespace
  - Access fields: `@namespace.field` to access imported variables
  - Replaces deprecated wildcard syntax `/import { * } from [file]`
- **Primitive Value Support**: Direct assignment of unquoted numbers, booleans, and null
  - Numbers: `/var @count = 42`, `/var @price = 19.99`
  - Booleans: `/var @active = true`, `/var @disabled = false`
  - Null: `/var @empty = null`
  - Type preservation: Primitives maintain their JavaScript types through the system
  - JavaScript semantics: Type coercion follows JavaScript rules (e.g., `"text" + 5 = "text5"`)
  - Exec invocation support: Primitive literals in function calls (e.g., `@add(@num, 8)`)
- **Built-in @now Variable**: New built-in variable for current timestamp
  - Returns ISO 8601 timestamp: `2024-01-15T10:30:00.000Z`
  - Available in all contexts where variables are allowed
  - Also available as `mlld_now()` function in JavaScript/Node shadow environments
- **@mlld/time Module**: Comprehensive date/time functionality replaces simple built-in time operations with full-featured module

## [1.4.11]

### Fixed
- Fixed pipeline operator converting JSON array strings to `[object Object]` (#272)
  - ExecInvocation nodes with pipelines are now handled correctly in data value evaluation
  - Functions are executed first, then their JSON string results are passed through the pipeline
  - Pipeline now preserves JSON array data as strings instead of mangling them
  - This fix ensures data can be properly passed between functions in a pipeline
- Fixed pipeline format feature to provide wrapped input to all pipeline stages
  - Previously only the first pipeline stage received wrapped input objects with `text`, `type`, and `data` properties
  - Now all stages consistently receive wrapped input, enabling format-aware processing throughout the pipeline
  - This allows subsequent pipeline stages to access parsed data (e.g., `input.csv` for CSV format)

## [1.4.10]
### Fixed
- Fixed parser failing on bracket characters (`[` or `]`) in JavaScript/code string literals (#273)
  - Code content within `[(...))]` blocks is now treated as opaque text
  - Enables string comparisons like `if (char === "[")` and array literals like `["[", "]"]`
  - Fixes regex patterns, JSON parsing, and other code using bracket characters

## [1.4.9]
### Fixed
- Fixed Node.js exec functions throwing ReferenceError when optional parameters are not provided
  - All declared parameters are now properly initialized in the execution context, even when undefined
  - Enables functions like `filterByFrontmatter(files, field, value)` to be called with just `(files, field)`
  - Affects both shadow environment (VM) and subprocess execution modes

## [1.4.8]
### Added
- **Pipeline-aware @debug**: The @debug variable now includes pipeline execution context when evaluated during pipeline operations
  - Shows current stage number and total stages in pipeline
  - Displays the command/transformer being executed
  - Includes input data details (type, length, preview)
  - Lists outputs from previous pipeline stages
  - Context is accessible in child environments via parent chain lookup

### Fixed
- Fixed `mlld setup` command throwing "Cannot read properties of null (reading 'config')" error when no mlld.lock.json exists
- Fixed pipeline `@data` variable evaluation returning null for complex pipeline expressions
  - `VariableReferenceWithTail` nodes now properly marked for lazy evaluation
  - Enables correct execution of expressions like `@data result = @input | @transformer1 | @transformer2`
- Fixed incorrect MlldCommandExecutionError constructor usage that caused "Cannot use 'in' operator" errors
  - Updated all error instantiations to use new signature with proper sourceLocation parameter
- Fixed Node.js shadow environment keeping process alive due to uncleaned timers
  - Added `cleanup()` method to NodeShadowEnvironment to clear timers and VM context
  - Environment cleanup is now called after CLI execution to ensure clean process exit
  - Prevents hanging processes when using setTimeout/setInterval in @exec node functions

## [1.4.7]

### Fixed
- #270 LocalResolver fails to resolve .mld.md files with 'Access denied' error

## [1.4.6]
### Added
- **Node shadow env support**
- Some resolver bugs

### Fixed
- @debug / @DEBUG wasn't working
- Created better naming clarity with prefix/resolver/registry distinction and refactor
- JS shadow env bug
- @data not allowing RHS @run 

### Documentation
- Lots of docs updates for resolvers
- Added missing alias and setup commands to cli help text

## [1.4.5]

### Added
- **mlld run Command**: Execute mlld scripts from a configured directory
  - Run scripts by name: `mlld run script-name` (without .mld extension)
  - List available scripts: `mlld run` (no arguments)
  - Script directory configured in `mlld.lock.json` via `mlld setup`
  - Default script directory: `llm/run/`
  - Helpful error messages showing available scripts when script not found

## [1.4.4]

### Added
- Check for reserved words when publishing

## [1.4.3]

### Added
- **mlld test Command**: New command for running mlld test suites
  - Discovers and runs `.test.mld` files in test directories
  - Supports custom test directories with `--test-dir` flag
  - Shows detailed test results with pass/fail status
  - Integrates with CI/CD workflows
- **Built-in Transformers**: Common data format transformers are now built into mlld
  - `@XML` / `@xml` - Convert content to SCREAMING_SNAKE_CASE XML using llmxml
  - `@JSON` / `@json` - Pretty-print JSON or convert markdown structures to JSON
  - `@CSV` / `@csv` - Convert JSON/markdown tables to CSV format
  - `@MD` / `@md` - Format markdown using prettier
  - Transformers can be chained in pipelines: `run [(cmd)] | @json | @csv`
  - Both uppercase (canonical) and lowercase (convenience) forms available
- **Smart Pipeline Parameter Handling**: Pipelines now intelligently pass data to multi-parameter functions
  - Single parameter functions continue to work as before (pass @INPUT as first param)
  - Multi-parameter functions with JSON input auto-destructure: `{"name": "Smith", "title": "Dr."}` → `@greet` maps to name="Smith", title="Dr."
  - Non-JSON input falls back to first parameter with empty strings for missing params
  - @INPUT variable available in each pipeline step with the piped data
- **Enhanced JavaScript Error Handling**: JavaScript/Node.js errors now properly integrate with mlld's error system
  - Error messages are preserved and shown in context
  - Stack traces included for debugging
  - Works in pipelines and shows full execution context
- **Namespace Imports**: Support for importing all variables from a file under a namespace alias (#264)
  - Import .mld files: `@import { * as utils } from "utils.mld"` - access as `{{utils.helper}}`
  - Import JSON files: `@import { * as config } from "config.json"` - access as `{{config.name}}`
  - Nested object access: `{{config.database.host}}` for deep properties
  - Works in templates with dot notation for clean, organized variable access

### Fixed
- Template executable property naming consistency (`template` vs `templateContent`)
- JavaScript return values now properly parsed from JSON (fixes falsy value handling in @when)
- Empty string parameter binding in pipelines
- Parameter binding when fewer arguments than parameters
- Pipeline syntax validation (only executables allowed after pipe operator)
- Module path resolution in built-in transformer imports
- isCommandVariable import in interpreter for executable variable handling
- **Template interpolation in foreach**: Fixed parameter interpolation in exec templates used with foreach - must use `{{param}}` syntax inside `[[...]]` templates
- **Shell parameter access**: Fixed exec functions with shell/sh commands to properly access parameters as environment variables using `$param` syntax
- **Array length property**: Removed incorrect test expectation for `.length` property on arrays (not implemented in mlld)
- **Grammar test expectations**: Fixed text directive test expecting undefined `meta.run` property for command execution
- **Shadow environment support for JavaScript**: Restored shadow environment functionality for `js` language
  - `js` execution uses in-process evaluation with direct function access
  - `node` execution uses subprocess isolation without shadow environment support
  - Shadow functions in `js` are synchronous for simple expressions, avoiding need for `await`
- **When directive comparisons**: Fixed `@when` with `first:` modifier to use value comparison instead of truthiness
  - `@when @var first: [...]` now compares `@var` value against each condition like switch syntax
  - Added string-boolean comparison: `"true"` matches `true`, `"false"` matches `false`
  - Consolidated comparison logic across all when variants for consistency
- **Pipeline parsing**: Fixed grammar to prevent pipelines from crossing line boundaries

### Changed
- **Template newline handling**: Moved newline stripping from interpreter to grammar level
  - Grammar now strips leading newline after `[[` and trailing newline before `]]`
  - These newlines are treated as formatting for readability, not content
  - More efficient and consistent than post-processing
  - Removed unused `normalizeTemplateContent()` function

### Documentation
- Added `docs/pipeline.md` - Comprehensive pipeline documentation
- Added `docs/transformers.md` - Built-in transformer reference
- Added `docs/security.md` - Security considerations for mlld usage
- Updated `docs/input-variables.md` with pipeline @INPUT documentation
- Updated `llms.txt` with pipeline and transformer information

## [1.4.2]

### Added
- Initial groundwork for pipeline support (full implementation in 1.4.3)

## [1.4.1]

### Added
- **VSCode Extension 0.3.0**: LSP implementation with autocomplete, syntax validation, hover info, go-to-definition
- **Markdown formatting**: prettier integration (default on, `--no-format` to disable)
- **Fuzzy path matching** for local files: case-insensitive and whitespace-flexible (`./my-file` finds `My File.mld`)

### Fixed
- `mlld language-server` command added to CLI
- Template normalization for leading/trailing newlines
- `variable.metadata` property access in add evaluator
- JavaScript/Node.js exec functions now support `return` statements - returned values are captured as JSON instead of requiring `console.log(JSON.stringify(...))`

## [1.4.0]
Added:
- **New Resolver Architecture** - Complete overhaul of how mlld loads files and modules:
  - Pluggable resolver system for extensible file/module loading
  - Built-in resolvers: TIME, DEBUG, INPUT, PROJECTPATH, LOCAL, GITHUB, HTTP, REGISTRY
  - Content type detection for proper handling of different file formats
  - Private module support via GitHub and local directory resolvers
  - JSON import support: `@import { key } from "./data.json"`
- **New CLI Commands**:
  - `mlld setup` - Interactive configuration wizard for resolvers and authentication
  - `mlld alias` - Create path aliases for module imports
  - `mlld auth` - GitHub authentication management (login/logout/status)
  - `mlld env` - Manage allowed environment variables
- **Private Modules**:
  - GitHub resolver for private repositories with secure authentication
  - Enhanced `mlld publish` with `--private` flag and custom `--path` support
  - Path aliases map prefixes to local paths (e.g., `@shared/` → `../shared-modules`)
  - Location-aware `mlld init` prompts to use configured module directories
- **Environment Variables**:
  - Access control via `mlld.lock.json` security settings
  - Import allowed variables through @INPUT: `@import { API_KEY } from @INPUT`
  - Manage with `mlld env allow/remove/list`
- **Developer Mode (`--dev` flag)**:
  - Test modules with their final import paths before publishing
  - Automatic fallback to local versions when modules aren't found in repositories
  - Smart error messages guide developers to use `@local/` imports or publish their modules
  - Detects uncommitted changes and suggests using dev mode for testing
- **Shadow Environments** for @exec: `@exec js = { helperA, helperB }`
  - Inject helper functions into JavaScript execution contexts
- **Negation Operator** for @when conditions:
  - `@when !@variable => @action`
  - Works with all @when forms (simple, switch, block)
- **mlld Stacktrace** - Shows execution context when errors occur:
  - Directive execution path with file:line locations
  - Failed imports show parse errors inline
  - Error display in bordered box
  - (More work on this intended)
- **Unified Executable Syntax** - Simplified and added @exec definitions:
  - Direct syntax without @run prefix: `@exec greet(name) = [(echo "Hello, @name!")]`
  - Template executables: `@exec greeting(name) = [[Hello {{name}}!]]` or `` `Hello @name!` ``
  - Section executables: `@exec getSection(file, section, newheader) = [@file # @section] as @newheader`
  - Resolver executables: `@exec fetch(path) = @resolver/api/@path` 
  - Code executables: `@exec calc(x) = js [(return x * 2)]` (drops @run requirement)
- **Configuration Updates**:
  - Global config moved to `~/.config/mlld/mlld.lock.json`
  - Resolver registry configuration with priority support
  - Secure token storage using keytar (system keychain)

Fixed:
- **@PROJECTPATH variable** - Now correctly resolves to project root directory
- **Import error messages** - Much clearer error messages for import failures
- **Content type detection** - Consistent handling of .mld, .json, and other file types
- Shadow environment functions in @exec now properly handle async/await
- Numeric parameters in @exec functions are now correctly converted from strings
- **@when directive grammar bug** - Fixed parsing of `@add @variable` inside @when actions (#258)
- **@run with template executables** - Fixed "nodes is not iterable" error when using @run with @exec template functions
- **Truthiness documentation** - Clarified that strings "false" and "0" are falsy in @when conditions (matching existing behavior)

Changed:
- **@text deprecation** - Parameterized templates must now use `@exec` instead of `@text`. Using `@text name(params)` now throws an error directing to use `@exec`

Breaking Changes:
- None expected, but this is a major architectural change. Please report any issues!

## [1.3.4]
Added:
- Made keytar installation optional for npx purposes

## [1.3.3]
I can't remember what I did for 1.3.3 and I forgot to add it to the changelog.

## [1.3.2]
Fixed:
- @when now can supports running exec invocations

## [1.3.1]
Added:
- @when now has full support for @output variants added in 1.3.0

## [1.3.0]
Added:
- File output: @output @variable to "path/to/file.ext"
- Stream output: @output @variable to stdout/stderr
- Environment variables: @output @variable to env or env:CUSTOM_NAME
- Format conversion: @output @variable to "file