mcard-js/README.md

MCard - Content-addressable storage with cryptographic hashing, handle resolution, and vector search for Node.js and browsers

# MCard JavaScript / TypeScript

Full-featured TypeScript implementation of MCard content-addressable storage, supporting both browser and Node.js environments.

[![Tests](https://img.shields.io/badge/tests-155%20passed-brightgreen)](./tests)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue)](https://www.typescriptlang.org/)
[![Node.js](https://img.shields.io/badge/Node.js-18%2B-green)](https://nodejs.org/)

## Features

- **Content Addressing**: SHA-256 via Web Crypto API (browser) or Node.js crypto
- **Dual Storage**: IndexedDB (browser) + SQLite via better-sqlite3 (Node.js)
- **UTF-8 Handles**: International support (文檔, مستند, ドキュメント, документ)
- **Monadic API**: Maybe, Either, IO monads for functional composition
- **PTR Runtime**: Polynomial Type Runtime with polyglot execution (JS, Python, Rust, C, WASM, Lean)
- **Bridgelet**: Universal Vehicle for cross-runtime execution (JS ↔ Python)
- **Introspection**: Recursive analysis of CLM composition and dependencies
- **LLM Integration**: Ollama, WebLLM (browser), and MLC-LLM providers
- **Vector Search**: sqlite-vec extension for semantic similarity search

## Environments

MCard runs in two primary environments:

1.  **Node.js**: Full filesystem access, native SQLite, spawning subprocesses (Python, etc.).
2.  **Browser (Service Worker)**: Uses `ServiceWorkerPTR` to run the kernel in a background thread, using `sql.js` + `IndexedDB` for storage and `fetch` interception for networking.

## Quick Start

### Browser (IndexedDB)
```typescript
import { MCard, IndexedDBEngine, CardCollection } from 'mcard-js';

const db = new IndexedDBEngine();
await db.init();

const collection = new CardCollection(db);
const card = await MCard.create('Hello, 世界!');
await collection.addWithHandle(card, 'greeting');

// Monadic retrieval
const result = await collection.getByHandleM('greeting');
if (result.isJust) {
  console.log(result.value.getContentAsText());
}
```

### Node.js (SQLite)
```typescript
import { MCard } from './model/MCard';
import { SqliteNodeEngine } from './storage/SqliteNodeEngine';

// File-based or in-memory database
const engine = new SqliteNodeEngine('./data/mcard.db');
// or: const engine = new SqliteNodeEngine(':memory:');

// Store a card
const card = await MCard.create('Hello from Node.js!');
await engine.save(card);

// Retrieve by hash
const retrieved = await engine.get(card.hash);
console.log(retrieved?.getContentAsText());

// Search
const results = engine.searchByString('Hello');
console.log(`Found ${results.totalItems} cards`);

// Clean up
engine.close();
```

## Storage Backends

### IndexedDB (Browser)
```typescript
const db = new IndexedDBEngine();
await db.init();
```

### SQLite WASM (Browser)
```typescript
const db = new SqliteWasmEngine();
await db.init();
const data = db.export(); // Persist to IndexedDB
```

### SQLite Node.js (Server)
```typescript
import { SqliteNodeEngine } from './storage/SqliteNodeEngine';

const db = new SqliteNodeEngine('./mcard.db');

// Synchronous API available for performance
db.saveSync(card);
const card = db.getSync(hash);
const count = db.countSync();

db.close();
```

## Database Schema

The SQLite backend uses schemas matching the Python implementation for full interoperability.
Schemas are defined in `src/storage/schema.ts` and mirror `mcard/model/schema.py` and `mcard/rag/*/schema.py`.

### Core Storage (matches `mcard/model/schema.py`)

```sql
-- Content-addressable card storage
CREATE TABLE card (
  hash TEXT PRIMARY KEY,
  content BLOB NOT NULL,
  g_time TEXT NOT NULL
);

-- Handle registry (UTF-8 string → hash mapping)
CREATE TABLE handle_registry (
  handle TEXT PRIMARY KEY,
  current_hash TEXT NOT NULL,
  created_at TEXT NOT NULL,
  updated_at TEXT NOT NULL,
  FOREIGN KEY (current_hash) REFERENCES card(hash)
);

-- Handle version history
CREATE TABLE handle_history (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  handle TEXT NOT NULL,
  previous_hash TEXT NOT NULL,
  changed_at TEXT NOT NULL,
  FOREIGN KEY (handle) REFERENCES handle_registry(handle),
  FOREIGN KEY (previous_hash) REFERENCES card(hash)
);
```

### Vector Storage (matches `mcard/rag/vector/schema.py`)

```sql
-- Vector metadata
CREATE TABLE mcard_vector_metadata (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  hash TEXT NOT NULL,
  model_name TEXT NOT NULL,
  dimensions INTEGER NOT NULL,
  chunk_index INTEGER DEFAULT 0,
  chunk_total INTEGER DEFAULT 1,
  chunk_text TEXT,
  created_at TEXT NOT NULL,
  UNIQUE(hash, chunk_index)
);

-- Fallback embeddings (when sqlite-vec unavailable)
CREATE TABLE mcard_embeddings (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  metadata_id INTEGER NOT NULL REFERENCES mcard_vector_metadata(id),
  embedding BLOB NOT NULL,
  UNIQUE(metadata_id)
);

-- FTS5 for hybrid search
CREATE VIRTUAL TABLE mcard_fts USING fts5(
  hash, content, tokenize='porter unicode61'
);

-- sqlite-vec virtual table (when extension available)
CREATE VIRTUAL TABLE mcard_vec USING vec0(
  metadata_id INTEGER PRIMARY KEY,
  embedding float[768]
);
```

### Graph Storage (matches `mcard/rag/graph/schema.py`)

```sql
-- Knowledge graph entities
CREATE TABLE graph_entities (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  name TEXT NOT NULL,
  type TEXT NOT NULL,
  description TEXT,
  source_hash TEXT NOT NULL,
  embedding BLOB,
  created_at TEXT NOT NULL,
  UNIQUE(name, type, source_hash)
);

-- Knowledge graph relationships
CREATE TABLE graph_relationships (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  source_entity_id INTEGER NOT NULL,
  target_entity_id INTEGER NOT NULL,
  relationship TEXT NOT NULL,
  description TEXT,
  weight REAL DEFAULT 1.0,
  source_hash TEXT NOT NULL,
  created_at TEXT NOT NULL
);
```

## Monads

```typescript
import { Maybe, Either, IO } from 'mcard-js';

// Maybe - optional values
Maybe.just(5).bind(x => Maybe.just(x * 2)); // Just(10)
Maybe.nothing().map(x => x * 2);            // Nothing

// Either - error handling
Either.right(5).map(x => x * 2);            // Right(10)
Either.left('error').map(x => x * 2);       // Left('error')

// IO - deferred side effects
const io = IO.of(() => fetch('/api'));
await io.run();
```

## PTR (Polynomial Type Runtime)

### CLM Execution
```typescript
import { CLMRunner } from './ptr/node/CLMRunner';
import { CLMLoader } from './ptr/node/CLMLoader';

const runner = new CLMRunner();
const loader = new CLMLoader();

// Load and execute a CLM specification
const clm = loader.load('chapters/chapter_01_arithmetic/addition.yaml');
const result = await runner.executeCLM(clm, './chapters/chapter_01_arithmetic', { a: 10, b: 20 });

console.log(result.result); // 30
```

### Recursive CLM (Meta-Execution)
```typescript
// CLM files can reference other CLM files as their runtime
// runtime: meta.clm  → loads and executes meta.clm with original context

const clm = {
  chapter: { title: 'Orchestrator' },
  clm: {
    concrete: { runtime: 'meta.clm' }  // Recursive execution
  }
};

const result = await runner.executeCLM(clm, chapterDir, input);
```

### Polyglot Runtimes
```typescript
import { RuntimeFactory } from './ptr/node/Runtimes';

// Available runtimes
const jsRuntime = RuntimeFactory.getRuntime('javascript');
const pyRuntime = RuntimeFactory.getRuntime('python');
const rustRuntime = RuntimeFactory.getRuntime('rust');
const cRuntime = RuntimeFactory.getRuntime('c');
const wasmRuntime = RuntimeFactory.getRuntime('wasm');
const leanRuntime = RuntimeFactory.getRuntime('lean');
const llmRuntime = RuntimeFactory.getRuntime('llm');
const loaderRuntime = RuntimeFactory.getRuntime('loader');

// Check availability
const available = RuntimeFactory.getAvailableRuntimes();
```

### File Loader Runtime

The `LoaderRuntime` provides high-performance directory loading using native Node.js APIs:

```typescript
// CLM files can use operation: loader for built-in file loading
// Example CLM:
// concrete:
//   runtime: python
//   operation: loader
// balanced:
//   input_arguments:
//     source_dir: "path/to/files"
//     recursive: true

// Features:
// - Recursive/flat directory traversal
// - SHA-256 content hashing
// - MIME type detection (extension + magic bytes)
// - Problematic file filtering (>50MB, binary garbage)
// - ~5000 files/sec performance
```

### LLM Runtime (Ollama)
```typescript
import { LLMRuntime } from './ptr/node/llm/LLMRuntime';
import { OllamaProvider } from './ptr/node/llm/providers/OllamaProvider';

const provider = new OllamaProvider({ model: 'gemma3:latest' });
const runtime = new LLMRuntime(provider);

const result = await runtime.execute('prompt', { question: 'What is 2+2?' }, config, dir);
```

### Node.js CLI
```bash
# Run a CLM file
npx tsx src/ptr/node/cli.ts run chapters/chapter_01_arithmetic/addition.yaml

# Check runtime status
npx tsx src/ptr/node/cli.ts status

# List available CLM files
npx tsx src/ptr/node/cli.ts list
```

### Lambda Calculus Runtime

The `LambdaRuntime` implements α-β-η conversions on MCard-stored Lambda terms:

```typescript
import { Lambda, LambdaRuntime, CardCollection, SqliteNodeEngine } from 'mcard-js';

// Initialize
const engine = new SqliteNodeEngine(':memory:');
const collection = new CardCollection(engine);
const runtime = new LambdaRuntime(collection);

// Parse a Lambda expression - terms are stored as MCards!
const { termHash } = await runtime.execute('', { expression: '(\\x.x) y' }, { operation: 'parse' }, '');

// Normalize (apply β-reductions until normal form)
const result = await runtime.execute(termHash, {}, { operation: 'normalize' }, '');
console.log(result.prettyPrint);  // "y"

// Or use the API directly:
const { parseLambdaExpression, normalize, alphaRename, betaReduce, etaReduce } = Lambda;

// Parse: (λx.λy.x) a b
const termHash2 = await parseLambdaExpression(collection, '(\\x.\\y.x) a b');

// Normalize to "a"
const normResult = await normalize(collection, termHash2, 'normal', 100).run();
console.log(normResult.right.normalForm);  // Hash of normalized term
```

**Key Features:**
- **α-conversion**: Rename bound variables (`alphaRename`, `alphaEquivalent`)
- **β-reduction**: Function application (`betaReduce`, `normalize`)
- **η-conversion**: Extensional equivalence (`etaReduce`, `etaExpand`)
- **Parser**: Simple Lambda syntax with `\x.M` or `λx.M`
- **MCard Storage**: Terms are hashes, enabling structural sharing and memoization

### Multi-Runtime CLM Execution

The PTR supports executing CLMs across multiple runtimes with **consensus verification**:

```typescript
import { CLMRunner, CLMLoader } from 'mcard-js/ptr/node';

const loader = new CLMLoader('.');
const runner = new CLMRunner('.');

// Load a multi-runtime CLM
const clm = loader.load('polyglot_comparison.yaml');

// Execute across all configured runtimes (Python, JS, Rust, C, WASM, Lean)
const result = await runner.executeMultiRuntime(clm, './chapters', { a: 5, b: 3 });

console.log(result.consensus);       // true - all runtimes agree
console.log(result.consensusValue);  // 8 - the agreed result
console.log(result.results);         // Per-runtime results
```

**CLI Usage:**
```bash
# Run all CLMs across all chapters
npm run clm:all

# Run specific chapter
npm run clm:chapter chapter_01_arithmetic

# Run Lambda CLMs (specialized runner)
npm run clm:lambda
```

**Features:**
- **Polyglot Consensus**: Verifies all runtimes produce identical results
- **Floating-Point Tolerance**: Handles precision differences with configurable tolerance (1e-9)
- **Graceful Fallback**: Unknown runtimes (R, Julia) are skipped, consensus among available runtimes
- **Legacy Format Support**: Handles both top-level `examples` and `balanced.examples`

## P2P Session Management

MCard now supports decentralized, append-only session recording with summarization capabilities, ideal for multi-agent systems and audit logs.

### Feature Highlights
- **Immutable Chain**: Each message is a discrete MCard linked to the previous one (blockchain-like structure).
- **Session Summarization**: Compress a long conversation chain into a single Summary MCard for efficient retrieval.
- **Persistence & Resume**: Sessions can be paused, serialized to disk, and resumed seamlessly.
- **Orchestration**: Nested CLM execution allowing "Root" agents to coordinate multiple "Child" agents.

### Example: P2P Chat Recording
```typescript
import { P2PChatSession, CardCollection } from 'mcard-js';

// Initialize session
const session = new P2PChatSession(collection, 'session_123', 5); // buffer size 5

// Add messages
await session.addMessage('Alice', 'Hello World');
await session.addMessage('Bob', 'Ack');

// Auto-checkpointing happens when buffer fills, or manually:
const headHash = await session.checkpoint();

// Summarize and cleanup old segments
const summaryHash = await session.summarize(false); // pass true to keep originals
```

### Multi-Agent Orchestration (Recursive CLMs)

The CLM Runner now supports "Lambda-style" execution, where a script can dynamically invoke other CLMs. This enables complex multi-agent workflows.

**Orchestrator CLM (`orchestrator.yaml`)**:
```yaml
concrete:
  manifestation: Multi-Agent Logic
  runtime: javascript
  code_file: orchestrator_logic.js
```

**Orchestrator Logic (`orchestrator_logic.js`)**:
```javascript
// Function runCLM is injected into the context!
await context.runCLM('send_message.yaml', { sender: 'Agent A', content: 'Ping' });
await context.runCLM('send_message.yaml', { sender: 'Agent B', content: 'Pong' });

// Summarize
const result = await context.runCLM('summarize_session.yaml', { sessionId: ... });
```

## Content Detection

The `ContentTypeInterpreter` module provides intelligent content type detection with parity to the Python implementation.

### Features
-   **Binary Signature Detection**: Identifies images (PNG, JPEG, GIF, WEBP), ZIP archives, PDFs, etc.
-   **Extension-Based Fallback**: robust detection for Video (MP4, MKV, WebM) and Audio types when binary signatures are unavailable or content is streamed.
-   **Programming Language Detection**: Identifies Python, C/C++, JavaScript, TypeScript, etc.
-   **Structured Data Detection**: Validates JSON, YAML, CSV, SQL, XML.
-   **Parity with Python**: Validated against the same test dataset as the core Python library.

### Usage

```typescript
import { ContentTypeInterpreter } from './src/model/ContentTypeInterpreter';

// Detect from string
const pythonCode = "import os\nprint('Hello')";
const result = ContentTypeInterpreter.detectContentType(pythonCode);
console.log(result.mimeType); // 'text/x-python'
console.log(result.extension); // '.py'

// Detect from binary buffer
const pngBuffer = new Uint8Array([0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A]);
const binResult = ContentTypeInterpreter.detectContentType(pngBuffer);
console.log(binResult.mimeType); // 'image/png'
```

## Vector Search (sqlite-vec)

The `MCardVectorStore` provides semantic search using the sqlite-vec extension:

```typescript
import { MCardVectorStore } from './storage/VectorStore';

// Initialize vector store
const store = new MCardVectorStore('./vectors.db', {
    embeddingModel: 'nomic-embed-text',
    dimensions: 768,
    ollamaBaseUrl: 'http://localhost:11434'
});

// Index content
await store.index('abc123...', 'MCard provides content-addressable storage...');

// Search similar content
const results = await store.search('What is MCard?', 5);
for (const result of results) {
    console.log(`${result.hash}: ${result.score}`);
}

// Hybrid search (vector + FTS)
const hybridResults = await store.hybridSearch('content storage', 5, 0.7);

// Check if sqlite-vec is available
console.log('Has vec extension:', store.hasVectorExtension());

store.close();
```

### Features
- **sqlite-vec extension**: Native KNN search with vec0 virtual tables
- **Fallback mode**: Brute-force cosine similarity when extension unavailable
- **Chunking**: Automatic text splitting with overlap
- **Hybrid search**: Combined vector + FTS5 full-text search
- **Ollama integration**: Embedding generation via local LLM

## RAG (Retrieval-Augmented Generation)

MCard includes a complete RAG pipeline compatible with the Python implementation, powered by Ollama.

### Components

-   **PersistentIndexer**: Automates indexing of MCards into the vector store.
-   **GraphExtractor**: Extracts entities and relationships from text using LLMs (e.g., `gemma3`).
-   **VisionEmbeddingProvider**: Generates embeddings for images by first describing them with a vision model (e.g., `moondream`).
-   **CommunityDetection**: Detects communities within the knowledge graph using Label Propagation.

### Example: Graph Extraction

```typescript
import { GraphExtractor } from './rag/graph/extractor';

const extractor = new GraphExtractor({
    model: 'gemma3:latest',
    temperature: 0.0
});

const result = await extractor.extract("MCard uses content-addressable storage.");
console.log(result.entities); 
// [{ name: "MCard", type: "TECHNOLOGY", ... }]
```

### Example: Vision Embedding

```typescript
import { VisionEmbeddingProvider } from './rag/embeddings/VisionEmbeddingProvider';

const provider = new VisionEmbeddingProvider();
const embedding = await provider.embedImage(base64Image);
```

### Graph RAG Engine

Combines vector search with knowledge graphs and LLMs for enhanced retrieval:

```typescript
import { GraphRAGEngine } from 'mcard-js/rag/GraphRAGEngine';

const engine = new GraphRAGEngine('./rag.db', null, 'gemma3:latest');

// Index with graph extraction
await engine.index(mcard);

// Query (Hybrid Search + Graph Context + LLM Synthesis)
const response = await engine.query("How does MCard relate to PTR?", 5, true);
console.log(response.answer);
console.log(response.graphContext);
```

## Bulk Loading

Efficiently load directories into CardCollection with safety checks (skips binary/large files):

```typescript
import { Loader } from 'mcard-js';

// Load directory recursively
await Loader.loadFileToCollection('./docs', collection, { 
    recursive: true,
    includeProblematic: false
});
```

## Frontend Observability (Grafana Faro)

MCard integrates [Grafana Faro](https://grafana.com/oss/faro/) as a **frontend observability sidecar**, complementing backend observability (e.g., Grafana Beyla). This enables full-stack visibility into your CLM verifications when running in browser environments.

```typescript
import { FaroSidecar } from 'mcard-js';

// Initialize the sidecar (Browser only)
const faro = FaroSidecar.getInstance().initialize({
    url: 'https://faro-collector-us-central1.grafana.net/collect/your-app-id',
    apiKey: 'your-api-key', // Optional if using public collector
    appName: 'mcard-studio-runtime',
    appVersion: '1.0.0',
    enableTracing: true, // Enables OpenTelemetry tracing
    namespace: 'ptr_verification' // Custom namespace for logs
});

if (faro) {
    faro.api.pushLog(['PTR verification started']);
}
```

The Sidecar implementation:
1.  **Safe Initialization**: Automatically detects non-browser environments (Node.js) and no-ops gracefully.
2.  **Auto-Instrumentation**: Captures console logs, uncaught exceptions, Web Vitals, and network traces.
3.  **OpenTelemetry Tracing**: Correlates frontend traces with backend spans for end-to-end distributed tracing.

## Integration Testing

### Live Integration Tests

To run integration tests against a live Ollama instance:

1.  Ensure Ollama is running (`ollama serve`).
2.  Pull required models:
    ```bash
    ollama pull gemma3:latest
    ollama pull nomic-embed-text
    ollama pull moondream
    ```
3.  Run the integration suite:
    ```bash
    npx vitest tests/rag/Integration.test.ts
    ```

## Version 0.1.47 / 2.1.28 Release Notes (January 17, 2026)
> Directory Structure Alignment, Strict Schema, and Store Refactoring

### 📂 Directory Structure Alignment
*   **Python Parity**: Reorganized `mcard-js` to strictly match the Python `mcard` library structure.
    *   Moved `src/hash` → `src/model/hash`.
    *   Moved `src/util/FileIO.ts` → `src/FileIO.ts`.
    *   Moved `src/util/Loader.ts` → `src/Loader.ts`.
    *   Moved Validators → `src/model/validators`.
    *   Moved Detectors → `src/model/detectors`.
*   **Cleanup**: Removed `src/util` directory.
*   **Duplicate Removal**: Removed duplicate `ContentTypeInterpreter` in `detectors/`.

### 💾 MCardStore Refactoring
*   **Strict SQL Schema**: Browser `IndexedDB` schema now mirrors the canonical SQL schema exactly.
    *   Tables: `card`, `handle_registry`, `handle_history`.
    *   Indexes: `by_g_time`, `by_updated_at`, `by_previous_hash`.
*   **API Updates**:
    *   `putCard(hash, content, g_time?)`: Replaces generic storage methods.
    *   `setHandle(handle, hash)`: Transactional handle updates with history tracking.
*   **Verification**: Verified against the "Mapping SQL to IndexedDB" specification.

### 🔒 Standardized Hashing
*   **SHA-256**: Implemented standard SHA-256 hashing using Web Crypto API (Browser) and Node `crypto` (Server).
*   **Location**: `src/model/hash/algorithms/LocalSHA256.ts`.
*   **Verification**: Validated identical hash output for identical content across Python and JS.

### 🐛 Bug Fixes
*   **CSV Detection**: Fixed regression where short text strings were misclassified as CSV.
*   **CLMLoader**: Fixed import path error in `ptr/node/runtimes/loader.ts`.

---

## Version 0.1.44 / 2.1.24 Release Notes (January 13, 2026)
> Single Source of Truth Schema Synchronization

### 🔄 Schema Synchronization
*   **Unified Schema Source**: `mcard-js` now strictly uses the canonical SQL schema files (`mcard_schema.sql`, `mcard_vector_schema.sql`) from the project root.
*   **Distribution Integrity**: The `npm` package now includes the raw SQL schema files, ensuring guaranteed parity with the Python implementation.
*   **Path Resolution**: Robust logic to resolve schema files in both development (monorepo) and production (dependency) contexts.

---

## Version 0.1.43 / 2.1.23 Release Notes (January 8, 2026)
> CLM REPL Architecture, Bridgelet, and Introspection

### 🌉 Bridgelet Universal Vehicle
*   **Cross-Language Execution**: New `Bridgelet` class allows moving execution between runtimes (e.g., JS ↔ Python) via content-addressable MCards.
*   **Unified Interface**: `invoke(pcardHash, inputVCardHash)` works consistently across adapters.
*   **Architecture**: Implements the "Universal Vehicle" pattern from the CLM REPL Specification.

### 🔍 Recursive CLM Introspection
*   **Hierarchy Analysis**: `CLMIntrospector` builds a tree of CLM dependencies, enabling visualization of complex compositions.
*   **Cycle Detection**: Automatically detects and warns about infinite recursion loops in CLM references.
*   **Composition Analysis**: Distinguishes between Sequential (Coend) and Parallel (Tensor) compositions.

### 🔭 OpenTelemetry Observability
*   **REPL Tracer**: `OpenTelemetrySidecar` traces `prep`, `exec`, `post`, and `await` phases of the CLM execution loop.
*   **Distributed Tracing**: Correlates spans across polyglot boundaries (e.g., JS calling Python).

### 🧪 Test Coverage
*   **New Tests**: Added 32 new unit tests covering Bridgelet, Introspection, and Observability.
*   **Stability**: All 500+ existing tests passing.

---

## Version 2.1.16 Release Notes (December 16, 2025)
> Feature Parity, Refactoring, and UPTV Alignment

### 🏗️ Code Refactoring & Cleanup
*   **Binary Detector Parity**: Refactored `BinaryDetector.ts` to expose `detectFromBytes` and match Python's logic structure, ensuring consistent file signatures detection.
*   **FileSystemUtils Optimization**: `detectContentType` in `FileSystemUtils` now delegates to the central `ContentTypeInterpreter`, removing code duplication and ensuring consistent MIME type detection across the loader.
*   **Handle Validation**: Relaxed validation to allow colons (`:`), supporting URI-like handles (e.g., `vcard://auth/1`) used in the UPTV architecture.

### 🎭 Feature Parity
*   **Petri Net Metadata**: The CLM Runner now attaches `petriNet` metadata (pcardHash, vcardHash, handle) to execution results, matching the Python implementation.
*   **VCard Standardization**:
    -   Aligned field names (`success` instead of `verified`, `token` types).
    -   Handles now default to `vcard://` namespace.
*   **Global Time**: `GTime` now consistently uses UTC with `Z` suffix, ensuring hash consistency between runtimes.

### 📜 Unifying Protocol of Truth Verification (UPTV)
*   **Documentation Overhaul**: Updated PTR, CLM, and PCard documentation to align with the **Categorical Petri Net** architecture of UPTV.
*   **CLM Triad**: Explicitly defined the Abstract ($A$) × Concrete ($C$) × Balanced ($B$) structure as the core verification unit.
*   **Petri Net Mapping**: Clarified roles: Handle (Place), PCard (Transport/Function), VCard (Token/Evidence).
*   **Loader Persistence**: Fixed database corruption issues during recursive loading.

---

## Version 2.1.14 Release Notes (December 13, 2025)
> VCard Application Vocabulary — Data-Driven Resource Factory

### 🏗️ Modular VCard Vocabulary

Refactored `vcard_vocabulary.ts` to be **fully data-driven** following the Empty Schema principle. All 31 resource types are now defined as pure data in modular extension files (`vcard_ext/`).

**New Structure** (`src/model/vcard_ext/`):
```
vcard_ext/
├── index.ts           # Auto-loader and exports
├── core.ts            # env, file, directory (3 types)
├── storage.ts         # sqlite, postgres, s3, litefs, turso (5 types)
├── network.ts         # api, webhook (2 types)
├── observability.ts   # grafana, prometheus, loki, tempo, faro, otlp (6 types)
└── vendors.ts         # google, github, meta, whatsapp, telegram, line, wechat, slack, trello, miro, figma, linkedin, aws, azure, gcp (15 types)
```

### 📦 31 Resource Types

| Category | Types |
|----------|-------|
| **Core** | `env`, `file`, `directory` |
| **Storage** | `sqlite`, `postgres`, `s3`, `litefs`, `turso` |
| **Network** | `api`, `webhook` |
| **Observability** | `grafana`, `prometheus`, `loki`, `tempo`, `faro`, `otlp` |
| **Vendors** | `google`, `github`, `meta`, `whatsapp`, `telegram`, `line`, `wechat`, `slack`, `trello`, `miro`, `figma`, `linkedin`, `aws`, `azure`, `gcp` |

### 🚀 Unified Factory API

```typescript
import { Resource } from 'mcard-js/model/vcard_vocabulary';

// Any resource type via unified factory
const ref = await Resource.create('github', 'xlp0', 'MCard_TDD');
const ref = await Resource.create('slack', 'my-workspace', 'general');
const ref = await Resource.create('turso', 'my-database', { group: 'us-east' });

// Check available types
console.log(Resource.types()); // ['env', 'file', 'sqlite', 'github', ...]
```

### 📉 Code Reduction
*   `vcard_vocabulary.ts`: **700 → 221 lines** (68% reduction)
*   Resource types now defined as data (no code changes needed to add new types)

---

## Version 2.1.13 Release Notes (December 11, 2025)
> Lean 4.25.2 Support & CLM Test Case Improvements

### 🔧 Lean Runtime Improvements
*   **Lean 4.25.2 Support**: `LeanRuntime` now uses `elan run leanprover/lean4:v4.25.2` to ensure correct toolchain version regardless of system PATH configuration.
*   **Modern Syntax**: Updated `lean_gcd.lean` to use Lean 4.8+ syntax (`termination_by b`, `Nat.pos_of_ne_zero`).
*   **Elan Integration**: Automatic detection of `~/.elan/bin/elan` for reliable version selection.

### 📝 CLM Test Case Additions
*   **`loader_orchestrator.clm`**: Added `test_cases` section with proper `given`/`then` assertions.
*   **`verify_loaders.clm`**: Added missing `then` clause with `success: true` and `match: true` assertions.

---

## Version 2.1.12 Release Notes (December 11, 2025)
> Major Modularization Refactoring

### 🏗️ Runtime Module Refactoring

**New Structure** (`src/ptr/node/runtimes/`):
```
runtimes/
├── index.ts           # Re-exports all components
├── base.ts            # Runtime interface, types, utilities
├── javascript.ts      # JavaScriptRuntime (VM or subprocess)
├── python.ts          # PythonRuntime
├── binary.ts          # BinaryRuntime (Rust, C)
├── wasm.ts            # WasmRuntime
├── lean.ts            # LeanRuntime + caching
├── loader.ts          # LoaderRuntime, CollectionLoaderRuntime
└── factory.ts         # RuntimeFactory with lazy imports
```

*   **Reduced Complexity**: `Runtimes.ts` from 583 lines → 62 lines (89% reduction).
*   **Lazy Loading**: LLM, Lambda, and Network runtimes loaded on-demand.
*   **Backward Compatible**: Existing imports from `Runtimes.ts` still work.

### 🏃 CLM Module Refactoring

**New Structure** (`src/ptr/node/clm/`):
```
clm/
├── index.ts           # Re-exports all components
├── types.ts           # All TypeScript interfaces
├── utils.ts           # Helper functions (resultsEqual, asObject, etc.)
├── loader.ts          # CLMLoader (YAML parsing)
├── runner.ts          # CLMRunner (execution engine)
├── multiruntime.ts    # Multi-runtime consensus logic
└── builtins/
    ├── index.ts       # Builtin detection (isNetworkBuiltin, etc.)
    └── handle.ts      # Handle version/prune operations
```

*   **Reduced Complexity**: `CLMRunner.ts` from 863 lines → 55 lines (94% reduction).
*   **Extracted Builtins**: Handle operations isolated in dedicated module.
*   **Type Safety**: Centralized interface definitions in `types.ts`.

### 🐛 JavaScript Runtime Fix
*   **IIFE Pattern**: Fixed variable declaration conflicts (`var`, `let`, `const`) by wrapping user code in IIFE.
*   **All Tests Pass**: 501 tests passed, 91 CLM files verified.

## Version 2.1.10 Release Notes (December 10, 2025)
> Handle Validation, Loader, and Python Runtime Improvements

### 🔧 Handle Validation Improvements
*   **Relaxed Validation Rules**: Handles now support periods (`.`), spaces (` `), and forward slashes (`/`) to accommodate file paths and filenames.
*   **Extended Maximum Length**: Increased from 63 to 255 characters for better file path compatibility.
*   **Cross-Runtime Parity**: Synchronized validation logic with Python implementation (`mcard/model/handle.py`).
*   **Updated Tests**: All handle validation tests (`Handle.test.ts`, `HandleParity.test.ts`) updated to reflect new rules.

### 📦 Loader Return Type Standardization
*   **Structured Response**: `loadFileToCollection` now returns `{metrics, results}` instead of a plain array.
*   **Metrics Object**: Includes `filesCount`, `directoriesCount`, and `directoryLevels` for better observability.
*   **Test Updates**: Updated `Loader.test.ts` to use `response.metrics.filesCount` and `response.results` array.

### 🐍 Python Runtime Wrapper Enhancements
*   **Smart Function Invocation**: Python wrapper now tries calling with `context` dict first, then `target`, then no args.
*   **Error Discrimination**: Added `_is_arg_error()` helper to distinguish TypeError about function arguments from other TypeErrors (e.g., sorting errors).
*   **Proper Scope Resolution**: Fixed entry point lookup using `dir()` and `globals()` instead of `locals()`.
*   **Builtin Loader Support**: CLMRunner now recognizes `builtin: loader` and `builtin: load_files` for Python CLMs.

### 🐛 Bug Fixes
*   **CLM Loader Detection**: Added support for `builtin: load_files` in addition to existing loader detection methods.
*   **Reflection Logic**: Fixed sorting bugs in Python reflection scripts to handle mixed ID types (int, float, string).

## Version 2.1.7 Release Notes (December 2025)
> CLM Test Infrastructure & Runtime Fixes

### 🚀 New Features
*   **Multi-Runtime Consensus**: Execute CLMs across Python, JS, Rust, C, WASM, Lean with automatic consensus verification.
*   **Lambda Calculus Runtime**: Full α-β-η conversion engine in `src/ptr/lambda/` with parser and MCard-based term storage.
*   **GraphRAGEngine**: Full RAG orchestration in `src/rag/GraphRAGEngine.ts` matching Python's capabilities.
*   **Bulk Loader**: New `Loader` module for robust file ingestion with safety checks.
*   **LLM Monads**: `promptMonad` and `chatMonad` added to `LLMRuntime` for functional composition.
*   **Collection Search**: Enhanced `CardCollection` with search capabilities (content, hash, string).
*   **CLM Test Runner**: New CLI (`npm run clm:all`) for running all CLMs with summary reporting.

### 🌐 Network IO Enhancements
*   **Retry with Backoff**: Configurable retry logic with `exponential`, `linear`, or `constant` backoff strategies.
    ```typescript
    config: {
        url: 'https://api.example.com/data',
        retry: {
            max_attempts: 3,
            backoff: 'exponential',
            base_delay: 1000,
            max_delay: 30000,
            retry_on: [503, 429, 500]  // Optional: custom status codes
        }
    }
    ```
*   **Response Caching**: Memory or MCard-persistent caching with TTL support.
    ```typescript
    config: {
        url: 'https://api.example.com/cacheable',
        cache: {
            enabled: true,
            ttl: 300,  // seconds
            storage: 'memory'  // or 'mcard' for persistent
        }
    }
    ```
*   **Rate Limiting**: Token-bucket rate limiting per domain (10 req/sec default with burst of 20).
*   **Bidirectional Sync**: New `both` and `bidirectional` modes for `mcard_sync` to push and pull in one operation.
    ```typescript
    config: {
        url: 'http://remote:3000/sync',
        mode: 'both'  // pushes local cards, then pulls remote cards
    }
    ```


### 🛠️ CLM Execution Fixes (December 2025)
*   **JavaScript Runtime Variable**: Changed all CLMs from `runtime: node` to `runtime: javascript` and updated inline code to use `target` instead of `input`.
*   **Input Context Preservation**: `CLMLoader` now passes `__input_content__` to preserve original `given` values in merged `when` blocks.
*   **Lambda Parser Fix**: Fixed beta reduction parser to correctly handle parenthesized expressions before splitting for application.
*   **Orchestrator Filter Fix**: `run_clm_background` now strips file extensions from filter for proper CLM matching.
*   **Floating-Point Tolerance**: Added 1e-6 tolerance for numeric comparisons in test result verification.
*   **Execution Timing**: Added millisecond timing logs to `run-all-clms.ts` for performance profiling.

### 🛠️ Polyglot Runtime Fixes
*   **ESM Compatibility**: Fixed `ReferenceError: require is not defined` in `SqliteNodeEngine` and `schema.ts`.
*   **Double-Encoding Fix**: Resolved input context double-JSON-encoding.
*   **Loader Context Injection**: Fixed `CLMRunner` context passing.
*   **CLM Example Iteration**: Updated `CLMLoader` for better example handling.
*   **Floating-Point Tolerance**: Added configurable tolerance for numeric consensus (1e-9).

### Previous Fixes (2.0.0)
### SQLite Foreign Key Constraint Fix

**Problem**: `clearSync()` was deleting tables in wrong order, causing `SQLITE_CONSTRAINT_FOREIGNKEY` errors.

**Solution**: Delete child tables before parent tables:
```typescript
clearSync(): void {
    // FK-safe order: children first, then parents
    this.db.exec('DELETE FROM handle_history');   // ✓ Child
    this.db.exec('DELETE FROM handle_registry');  // ✓ Child
    this.db.exec('DELETE FROM card');             // ✓ Parent
}
```

**Impact**: All 155 tests pass ✅

### CLMRunner Recursive Runtime Fix

**Problem**: Test mocks for recursive runtime execution weren't wiring correctly due to Vitest module path resolution.

**Solution**: Verified `executeRecursive()` implementation correctly calls `this.loader.load()` for meta CLM files.

**Impact**: Both CLMRunner tests pass ✅

## Development

```bash
# Install dependencies
npm install

# Run tests (Vitest)
npm test

# Run tests in watch mode
npm run test:watch

# Type checking
npx tsc --noEmit

# Build
npm run build
```

## Project Structure

```
mcard-js/
├── src/
│   ├── model/
│   │   ├── MCard.ts              # Core MCard implementation
│   │   ├── Handle.ts             # Handle validation & normalization
│   │   └── GTime.ts              # Global time ordering
│   ├── storage/
│   │   ├── StorageAdapter.ts     # Storage interface
│   │   ├── SqliteNodeEngine.ts   # Node.js SQLite (better-sqlite3)
│   │   ├── IndexedDBEngine.ts    # Browser IndexedDB
│   │   └── SqliteWasmEngine.ts   # Browser SQLite (sql.js)
│   ├── ptr/
│   │   ├── node/
│   │   │   ├── CLMLoader.ts      # Re-export (→ clm/loader.ts)
│   │   │   ├── CLMRunner.ts      # Re-export (→ clm/runner.ts)
│   │   │   ├── Runtimes.ts       # Re-export (→ runtimes/index.ts)
│   │   │   ├── cli.ts            # Node.js CLI
│   │   │   ├── runtimes/         # Modular runtime executors
│   │   │   │   ├── base.ts       # Interface, types, utilities
│   │   │   │   ├── javascript.ts # JavaScriptRuntime
│   │   │   │   ├── python.ts     # PythonRuntime
│   │   │   │   ├── binary.ts     # BinaryRuntime (Rust, C)
│   │   │   │   ├── wasm.ts       # WasmRuntime
│   │   │   │   ├── lean.ts       # LeanRuntime
│   │   │   │   ├── loader.ts     # LoaderRuntime
│   │   │   │   └── factory.ts    # RuntimeFactory
│   │   │   ├── clm/              # Modular CLM execution
│   │   │   │   ├── types.ts      # All interfaces
│   │   │   │   ├── utils.ts      # Helper functions
│   │   │   │   ├── loader.ts     # CLMLoader
│   │   │   │   ├── runner.ts     # CLMRunner
│   │   │   │   ├── multiruntime.ts
│   │   │   │   └── builtins/     # Builtin handlers
│   │   │   └── llm/              # LLM runtime (Ollama)
│   │   └── lambda/               # Lambda calculus runtime
│   ├── hash/
│   │   └── HashValidator.ts      # SHA-256 validation
│   └── monads/
│       └── Monads.ts             # Maybe, Either, IO
├── tests/
│   ├── storage/
│   │   └── SqliteNodeEngine.test.ts
│   ├── ptr/
│   │   ├── CLMLoader.test.ts
│   │   └── CLMRunner.test.ts
│   └── ...
└── package.json
```

## License

MIT