UNPKG

miniml

Version:

A minimal, embeddable semantic data modeling language for generating SQL queries from YAML model definitions. Inspired by LookML.

42 lines (24 loc) 7.04 kB
# WAVE 1 — THE TARGET USER (simulation) > Reconstructed brief. The dispatched subagent got through the Snowflake happy-path and began the realistic raw-TIMESTAMP case before a session-limit API error terminated it. The Orchestrator completed the adoption walk against the repo (README as the contract, code/CLI as the reality), per the protocol's sequential-fallback clause. All [MEASURED] tags reflect commands actually run. ## Persona **Maya Chen, senior full-stack engineer at a 40-person B2B SaaS.** Her PM wants a "chat with your data" panel shipped this sprint (10 working days) on their Snowflake warehouse: users ask "revenue by plan tier last month," an LLM turns it into a query, results render. Maya is TypeScript-native, does not want to run another server, and found MiniML by searching "safe semantic layer for LLM text-to-SQL node." Her headline metric: time-to-first-correct-SQL from a cold start, and whether she'd trust it in prod. ## Position (2 sentences) The library's happy path genuinely works and the value is legible within minutes — a terse YAML model compiles to clean, correct-looking Snowflake SQL, which is exactly what Maya wants. But she hits a documentation-vs-reality failure inside the first five minutes (the README's own Quick Start command is wrong), and when she probes the realistic cases her feature actually needs, she finds bugs the tool documents against itself — enough that she'd prototype with it but not ship it unsupervised into a customer-facing agent. ## Evidence (adoption log: step → expected → actual → friction) 1. **Discover / install.** `npm install miniml` — package exists on npm (v1.0.17), MIT, small dep tree. Expected clean install; no reason to believe otherwise. Friction: **searching "miniml" is polluted** — an ML teaching language and a deep-learning tool (delvendahl/miniML) outrank it; Maya only found it because she had the exact name. [MEASURED — npm/README; SOURCED — name collision noted across briefs] 2. **First model.** Copy the README `sales.yaml`. The terse YAML is genuinely pleasant — `total_amount: [description, SUM(...)]`, dimensions with `[desc, sql, join]`. Expected: intuitive. Actual: intuitive. **This is the moment she likes it.** [MEASURED — README + load.ts expansion logic] 3. **First CLI run.** `npx miniml sales.yaml --dimensions=date --measures=total_amount` → emits correct Snowflake SQL with the default 7-day range. ✅ Works exactly as documented. Time-to-first-SQL: ~4–5 minutes. [MEASURED — ran `node command.js test/models/snowflake/sales.yaml --dimensions=date --measures=total_amount`] 4. **Second CLI run — FIRST FAILURE.** She copies the README's fuller example verbatim: `... --date-from=2025-01-01 --date-to=2025-01-31 --order-by=-date --limit=10`. Expected: the documented SQL. Actual: **`Invalid argument(s): --date-from, --date-to`**. The CLI accepts only underscore flags (`--date_from`, `--date_to`, `--order_by`, `--date_granularity`); the README documents them with hyphens (lines 128–135). A copy-paste of the docs fails in the first five minutes. This is the single worst finding for adoption: the canonical example is broken. [MEASURED — ran it, got the error] 5. **Programmatic path.** `renderQuery(model, {...})` with underscore keys works and returns the documented SQL. Once past the CLI flag trap, the API is clean and TypeScript types are present. ✅ [MEASURED] 6. **Realistic case A — raw TIMESTAMP column.** Maya's `created_at` is a raw `TIMESTAMP`, not `DATE(...)`-wrapped like the demo. The default-range path emits `created_at >= CURRENT_TIMESTAMP - INTERVAL '7 DAY'`; fine for TIMESTAMP. But TASKS.md itself warns non-TIMESTAMP date fields generate failing SQL, and there's no `date_type` control — so her correctness depends on matching the demo's exact typing. Friction: she has to read the source to know which shape is safe. [MEASURED — dialect.ts + TASKS.md] 7. **Realistic case B — single-day query.** A user asks "sales on Jan 1." `date_from == date_to``BETWEEN '2025-01-01' AND '2025-01-01'`. Works for her DATE-typed demo, silently returns nothing on a TIMESTAMP column (midnight only). This is a *common* end-user question and it's an open TASKS.md bug. [MEASURED — ran renderQuery, saw the BETWEEN] 8. **Realistic case C — the tool lies to her LLM.** MiniML auto-generates model `info` text (meant to be fed to the LLM) that says: *"Specify `date_to` as `null` to bypass applying a default date range."* She wires that into her agent's prompt. It **doesn't work**`date_to:null` still applies the default range (confirmed: generated SQL keeps `CURRENT_TIMESTAMP`; root cause is a duplicated condition at query.ts:119). Her agent will confidently do the documented thing and get wrong results. [MEASURED — ran renderQuery(model,{date_to:null})] 9. **What's missing for her actual feature.** No MCP server (despite the npm "mcp" keyword) — she has to hand-roll the tool wiring. No row-level security enforcement (multi-tenant SaaS needs it). No fan-out protection — if she joins a one-to-many table her revenue silently doubles, and the README honestly warns her it won't protect against this. Two dialects only (she's fine, she's on Snowflake). [MEASURED — repo contents, README fan-out warning] ## Strongest point FOR the project Time-to-*understanding* is excellent: within five minutes Maya has a working model and reads clean, auditable SQL she could paste into Snowflake herself — the deterministic "LLM picks from a vocabulary, library writes the SQL" story is immediately tangible and the YAML is the nicest part. For a prototype or an internal tool where a developer reviews the generated SQL, this is genuinely usable *today*. ## Strongest point AGAINST The three things Maya's real feature depends on — copy-pasteable docs, correct single-day/date-bypass behavior, and instructions to her LLM that are actually true — all fail, and the last one is worst: the tool feeds her agent guidance that produces wrong answers, which is the exact failure mode ("confident wrong numbers") that a safety layer exists to prevent. She cannot ship this to customers unsupervised this sprint. ## Viability score: 30/100 Great first impression, real value for a reviewed/internal use case, but too many docs-vs-reality and correctness gaps for the customer-facing agent it's marketed for. Every one of these is individually cheap to fix, which is why the score isn't lower — this is a polish-and-correctness problem, not a concept problem. ## The one question that would change my mind If I `npm install miniml` again in 90 days, do the README commands run verbatim, does a single-day query return rows, and does the LLM-facing `info` describe behavior that actually happens — and is there an MCP server so I don't hand-wire the agent? If yes, Maya ships it and tells two friends; if the canonical Quick Start command is still wrong, no engineer past the first five minutes will trust anything else the docs claim.