nx-config2/CHANGELOG.md

Next-generation configuration management for Node.js - Zero dependencies, TypeScript-first, production-ready

# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [2.2.2] - 2024-12-XX

### Removed

- Removed unnecessary documentation and configuration files related to `athenix-parser` peer dependency resolution, as `nx-config2` does not depend on these packages

## [2.2.1] - 2024-12-XX

### Removed

- Removed unnecessary documentation and configuration files related to `athenix-parser` peer dependency resolution, as `nx-config2` does not depend on these packages

## [2.2.0] - 2024-12-XX

### Added

#### Remote Configuration Loading (NEW!)
- **`remoteConfig()`**: Initialize configuration from remote JSON sources using `nx-remote-json`
- **Multi-Backend Support**: Fetch configuration from disk, S3-compatible storage, or MongoDB
- **Environment Variable Resolution**: Mode, router, and key can be specified via `ENV.VARIABLE` patterns
- **Unified Pipeline**: Remote configs are processed through the standard `initConfig` pipeline
- **Type-Safe**: Full TypeScript support with generic type parameters
- **Integration**: Seamlessly integrates with existing validation, schema, and documentation features

## [2.1.0] - 2024-12-XX

### Added

#### Automatic JSON-like Parsing (NEW!)
- **Heuristic JSON Detection**: Automatic parsing of JSON-like strings in environment variables
- **Object Parsing**: Values starting with `{` and ending with `}` are automatically parsed as objects
- **Array Parsing**: Values starting with `[` and ending with `]` are automatically parsed as arrays
- **Graceful Fallback**: Invalid JSON returns the original string value (no errors)
- **Type Preservation**: Arrays of numbers, strings, and objects are properly typed

#### JSON File References in Environment Variables (NEW!)
- **File Reference Pattern**: Support for `{{path.json}}` syntax in environment variables
- **Automatic Loading**: JSON files are automatically loaded and parsed when referenced
- **Path Resolution**: Paths are resolved relative to config file or current working directory
- **Fallback Behavior**: Missing or invalid files return the original string value

#### Config Object File References (NEW!)
- **`_loadFile` Property**: Reserved property for loading JSON files directly from config objects
- **Object Replacement**: Objects with `_loadFile` are replaced with the file's contents
- **Array Support**: Files can contain either JSON objects or JSON arrays
- **Recursive Parsing**: Loaded content is recursively parsed, supporting nested ENV tokens and `_loadFile` references

#### Automatic Config Loading (NEW!)
- **`autoLoadConfig()`**: One-call function to automatically load configuration from environment variables
- **Default Config Map**: Pre-configured map covering common use cases (api, database, storage, logging)
- **Automatic Addition**: Additional `ALL_UPPERCASE` env vars automatically added as camelCase properties at root
- **JSON Parsing**: JSON-like env vars are automatically parsed
- **Beautified Output**: Config is sorted with default sections first, then alphabetically
- **Non-strict by Default**: Missing optional vars don't throw errors

#### Remote Configuration Loading (NEW!)
- **`remoteConfig()`**: Initialize configuration from remote JSON sources using `nx-remote-json`
- **Multi-Backend Support**: Fetch configuration from disk, S3-compatible storage, or MongoDB
- **Environment Variable Resolution**: Mode, router, and key can be specified via `ENV.VARIABLE` patterns
- **Unified Pipeline**: Remote configs are processed through the standard `initConfig` pipeline
- **Type-Safe**: Full TypeScript support with generic type parameters
- **Integration**: Seamlessly integrates with existing validation, schema, and documentation features

#### Enhanced Array Type
- **JSON Array Support**: `:array` type now supports both JSON arrays (`[1,2,3]`) and comma-separated strings (`"a,b,c"`)
- **Type Preservation**: JSON arrays preserve element types (numbers, strings, objects)
- **Backward Compatible**: Comma-separated strings continue to work as before

#### Lenient JSON Parsing
- **New Option**: `lenientJson` option for `:json` type to return original string on parse failure instead of throwing
- **Default Behavior**: Strict parsing (throws on error) remains the default

### Configuration Options Added
- `heuristicJsonParsing`: Enable/disable automatic JSON detection (default: `true`)
- `lenientJson`: Return original string on JSON parse failure instead of throwing (default: `false`)

## [2.0.0] - 2024-10-25

### Changed
- **Package Name**: Renamed from `nxconfig-js` to `nx-config2` for better clarity and availability
- **Repository**: Moved to `nx-morpheus` organization on GitHub
- **Documentation**: Updated all references to use the new package name

### Breaking Changes
- **Import Path**: All imports now use `nx-config2` instead of `nxconfig-js`
- **Installation**: Package must be installed as `nx-config2`

## [1.1.1] - 2024-01-XX

### Added

#### Environment Variable Inspection (NEW!)
- **getEnvVariables()**: New function to inspect and analyze environment variables in .env files
- **Variable Information**: Returns detailed information including variable name, length, and synthesized value
- **Sensitive Data Masking**: Automatically masks passwords, keys, and tokens for safe inspection
- **Multi-Environment Support**: Works with any environment file path (test, prod, prod.database, etc.)
- **TypeScript Support**: Full type definitions with EnvVariableInfo interface

#### Enhanced Documentation
- **Detailed Feature Descriptions**: Comprehensive explanations for each capability
- **Feature-Specific Details**: Technical details, use cases, and benefits for each feature
- **Improved Readability**: Better organization and developer experience
- **Removed Comparison Matrix**: Replaced with detailed feature descriptions

### Technical Details

#### New Function: getEnvVariables()
```typescript
interface EnvVariableInfo {
  varName: string;        // Variable name
  length: number;         // Original value length
  synthesizedValue: string; // Masked or original value
}

// Usage examples
const defaultVars = getEnvVariables();           // .env file
const testVars = getEnvVariables('test');        // .env.test file
const prodVars = getEnvVariables('prod.database'); // .env.prod.database file
```

#### Sensitive Field Detection
- Automatically detects sensitive fields by name patterns (password, key, token, secret, etc.)
- Applies smart masking: preserves first/last characters for readability
- Safe for logging and debugging purposes

## [1.1.0] - 2024-01-XX

### Added

#### Multi-Environment File Support (NEW!)
- **Dynamic Environment File Loading**: Load variables from different `.env` files based on token pattern
- **Multi-Segment Token Pattern**: `ENV.TEST.VARIABLE` loads from `.env.test`, `ENV.PROD.DATABASE.HOST` loads from `.env.prod.database`
- **Environment File Cache**: Automatic caching of loaded environment files to avoid reloading
- **Backward Compatibility**: Simple `ENV.VARIABLE` pattern continues to work as before
- **Flexible File Naming**: Support for any number of segments in environment file paths

#### Pattern Examples
- `ENV.VARIABLE` → loads from `.env` file or `process.env`
- `ENV.TEST.VARIABLE` → loads from `.env.test` file
- `ENV.PROD.DATABASE.HOST` → loads from `.env.prod.database` file
- `ENV.STAGING.API.KEY` → loads from `.env.staging.api` file

#### Enhanced Examples
- Updated all example files with multi-environment file demonstrations
- Added comprehensive multi-environment file usage patterns
- Demonstrated integration with multi-config features

### Technical Details

#### New Regex Patterns
- `ENV_TOKEN_PATTERN`: Matches multi-segment patterns like `ENV.TEST.VARIABLE||default`
- `ENV_TOKEN_WITH_TYPE`: Matches multi-segment patterns with type coercion like `ENV.TEST.VARIABLE:port||3000`
- `ENV_SIMPLE_TOKEN_PATTERN`: Matches simple patterns like `ENV.VARIABLE||default` (backward compatibility)
- `ENV_SIMPLE_TOKEN_WITH_TYPE`: Matches simple patterns with type coercion like `ENV.VARIABLE:port||3000`

#### New Functions
- `loadEnvFileByToken()`: Dynamically loads environment files based on token pattern
- `loadSimpleEnvVariable()`: Loads variables from default environment (process.env or .env)
- Enhanced `extractEnvTokens()`: Now extracts tokens from both simple and multi-segment patterns

#### Environment File Loading
- Automatic file path construction: `ENV.TEST.API.KEY` → `.env.test.api`
- Case-insensitive file path conversion
- Comprehensive error handling for missing files
- Silent failure for non-existent files (returns undefined)

### Breaking Changes

None - This is a backward-compatible enhancement.

### Migration Guide

No migration needed! All existing `ENV.VARIABLE` patterns continue to work exactly as before.

#### New Usage Examples

```typescript
// Before (v1.0.0) - only simple patterns
const config = {
  apiKey: 'ENV.API_KEY',
  dbHost: 'ENV.DB_HOST'
};

// After (v1.1.0) - now supports multi-environment files
const config = {
  // Simple pattern (still works)
  apiKey: 'ENV.API_KEY',
  
  // New multi-environment patterns
  test: {
    apiKey: 'ENV.TEST.API_KEY',        // loads from .env.test
    dbHost: 'ENV.TEST.DB_HOST'         // loads from .env.test
  },
  production: {
    database: {
      host: 'ENV.PROD.DATABASE.HOST',  // loads from .env.prod.database
      port: 'ENV.PROD.DATABASE.PORT:port'  // loads from .env.prod.database
    }
  }
};
```

## [1.0.0] - 2024-01-XX

### Added

#### Core Features (Original Specification)
- **ENV Token Resolution**: `ENV.VARIABLE` and `ENV.VARIABLE:type||default` patterns
- **parseConfig()**: Recursive configuration parsing with ENV token resolution
- **verify()**: Environment variable validation with detailed results
- **initConfig()**: One-call configuration initialization with validation
- **Type Coercion**: 15+ built-in types (string, number, int, nat, port, boolean, bigint, json, array, url, email, ipaddress, duration, timestamp, regex)
- **Default Values**: Support for `||default` syntax in ENV tokens
- **Strict/Non-strict Modes**: Configurable error handling
- **Secret Detection**: Automatic detection and masking of sensitive fields
- **Error Classes**: Comprehensive error handling with detailed context

#### Multi-Config Features (NEW!)
- **parseMultiConfig()**: Smart multi-configuration handling
- **Merge Strategies**: deep, shallow, override, append
- **Named Config Access**: Keep configs separate but accessible by name
- **Configuration Inheritance**: Base config extension with `extends` option
- **Priority Control**: first/last priority for source ordering
- **Source Types**: Support for files, objects, and file paths

#### Advanced Features
- **Schema Validation**: Convict-style schema validation with format validators
- **Prefix/Suffix Support**: Environment-specific variable resolution (AWS_*, *_PROD)
- **Auto Documentation**: Generate markdown documentation tables
- **Directory Creation**: Auto-create directories for path configurations
- **Command-line Arguments**: Parse --flag=value from process.argv
- **Hot Reload**: watchConfig with onChange callbacks
- **Hierarchical Stores**: ConfigurationHierarchy class (nconf-style)
- **Path Validation**: File and directory existence validation
- **Custom Transformers**: Transform values during parsing
- **Dotenv Integration**: .env file loading and processing
- **Verbose Logging**: Detailed logging of resolved variables
- **Nested Key Support**: Colon-separated nested key access

#### Developer Experience
- **TypeScript-First**: Full TypeScript support with generics
- **Zero Dependencies**: Pure Node.js implementation
- **Comprehensive Types**: Detailed type definitions for all interfaces
- **Error Context**: Detailed error messages with variable names and paths
- **Source Maps**: Full source map support for debugging
- **Tree-shakeable**: ESM and CommonJS support

### Technical Details

#### Type Coercion Support
- `string` - String values (default)
- `number` - Numeric values with NaN validation
- `int` - Integer values only
- `nat` - Natural numbers (>= 0)
- `port` - Port numbers (0-65535)
- `boolean` - Boolean values (true/false, 1/0)
- `bigint` - BigInt support
- `json` - JSON parsing
- `array` - Comma-separated arrays
- `url` - URL validation
- `email` - Email validation
- `ipaddress` - IPv4/IPv6 validation
- `duration` - Duration parsing (5s, 10m, 2h, 1d, 1w)
- `timestamp` - Date/timestamp parsing
- `regex` - Regular expression parsing

#### Merge Strategies
- **deep**: Recursively merge objects, concatenate arrays
- **shallow**: Object.assign style merging
- **override**: Last source wins completely
- **append**: Array-focused merging

#### Schema Validation Features
- Type validation with custom error messages
- Required field validation
- Default value support
- Enum constraint validation
- Min/max numeric validation
- Pattern matching with RegExp
- Custom format validators
- Nullable field support
- Sensitive field marking
- File/directory path validation
- Command-line argument mapping
- Environment variable mapping

#### Error Classes
- `ConfigParseError` - ENV token parsing errors
- `ConfigValidationError` - Validation failures
- `ConfigFileError` - File reading/parsing errors
- `ConfigSchemaError` - Schema validation errors
- `ConfigMergeError` - Multi-config merge errors

### Inspired By

This project combines the best features from 9+ MIT-licensed packages:

1. **[dotenv](https://github.com/motdotla/dotenv)** - .env file loading and environment variable substitution
2. **[env-var](https://github.com/evanshortiss/env-var)** - Type coercion, validation, and TypeScript support
3. **[node-config](https://github.com/node-config/node-config)** - Multi-environment support and configuration hierarchies
4. **[convict](https://github.com/mozilla/node-convict)** - Schema-based validation and documentation generation
5. **[nconf](https://github.com/flatiron/nconf)** - Hierarchical configuration stores and priority-based merging
6. **[xconfig](https://github.com/kaelzhang/node-xconfig)** - Flexible configuration architecture
7. **[envconfig](https://github.com/jsonxr/envconfig)** - Directory validation and auto-creation
8. **[@jondotsoy/envconfig](https://github.com/jondotsoy/envconfig)** - Prefix/suffix support and BigInt type
9. **[@logdna/env-config](https://github.com/logdna/env-config)** - Auto-documentation generation and regex support

### Breaking Changes

None - This is the initial release.

### Migration Guide

This is the initial release, so no migration is needed. However, if you're coming from other configuration libraries:

#### From dotenv
```typescript
// Before (dotenv)
require('dotenv').config();
const port = process.env.PORT;

// After (nxconfig)
const { initConfig } = require('nxconfig');
const { config } = initConfig({ port: 'ENV.PORT:port||3000' });
```

#### From env-var
```typescript
// Before (env-var)
const env = require('env-var');
const port = env.get('PORT').required().asPortNumber();

// After (nxconfig)
const { initConfig } = require('nxconfig');
const { config } = initConfig({ port: 'ENV.PORT:port' }, { requiredVars: ['PORT'] });
```

#### From convict
```typescript
// Before (convict)
const convict = require('convict');
const config = convict({
  port: {
    doc: 'The port to bind',
    format: 'port',
    default: 8080,
    env: 'PORT'
  }
});

// After (nxconfig)
const { createConfig } = require('nxconfig');
const config = createConfig({
  port: {
    type: 'port',
    doc: 'The port to bind',
    default: 8080,
    env: 'PORT'
  }
});
```

### Roadmap

#### v1.1.0 (Planned)
- [ ] YAML configuration file support
- [ ] TOML configuration file support
- [ ] Environment-specific .env file loading
- [ ] Configuration validation schemas from JSON Schema
- [ ] Plugin system for custom type coercions
- [ ] Configuration diffing and change detection

#### v1.2.0 (Planned)
- [ ] Remote configuration loading (HTTP/HTTPS)
- [ ] Configuration encryption/decryption
- [ ] Configuration versioning
- [ ] Configuration rollback support
- [ ] Performance optimizations for large configs

#### v2.0.0 (Future)
- [ ] Configuration streaming for large files
- [ ] Real-time configuration updates via WebSocket
- [ ] Configuration conflict resolution
- [ ] Advanced merge strategies
- [ ] Configuration templates and inheritance chains

---


## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.

## License

MIT License - see [LICENSE](LICENSE) for details.