miniml/.eval/wave1_ai_futurist.md

A minimal, embeddable semantic data modeling language for generating SQL queries from YAML model definitions. Inspired by LookML.

# Wave 1 — AI Futurist: Disruption Analysis (MiniML)

## Position (2 sentences)
MiniML's core thesis — constrain the LLM to a model-defined vocabulary instead of letting it write raw SQL — is not at risk of absorption by frontier models; it is the thesis every major platform (Snowflake, Databricks, dbt, Cube) independently converged on, and the failure data says the layer gets MORE valuable as deployments scale. But the agentic wave MiniML claims to ride in its npm keywords has already been ridden by others: the MCP semantic-layer niche is occupied by shipped, registry-listed servers (dbt-mcp, Cube, Boring Semantic Layer) while MiniML has 1 GitHub star, ~57 npm downloads/month, and zero MCP code — it is marketing a race it has not entered.

## Evidence

### Obsolescence analysis: does model improvement kill the layer?

**The benchmark data says no — raw text-to-SQL still fails on enterprise schemas.**
- [SOURCED] Spider 2.0 (632 real enterprise workflows, 3,000+ column schemas, Snowflake/BigQuery): o1-preview scored ~21.3%; the current leader ReFoRCE scores ~35.8 (Snow) / 36.6 (Lite). Compare classic Spider at ~86% — the enterprise gap is the whole story. https://spider2-sql.github.io/ , https://github.com/xlang-ai/Spider2 , https://arxiv.org/pdf/2502.00675
- [SOURCED] BIRD: best systems ~80–82% execution accuracy vs. human 92.96% — a 12+ point gap persisting into mid-2026 despite dedicated models (Gemini-SQL2, June 2026). https://llm-stats.com/benchmarks/bird-sql-(dev) , https://aiweekly.co/alerts/googles-gemini-sql2-tops-bird-text-to-sql-at-8004
- [SOURCED] Snowflake engineering measured GPT-4o at 51% on 150 real internal BI questions; adding a semantic model produced ~+20% accuracy, and Cortex Analyst's whole architecture is "agentic system + semantic model" to reach 90%+. https://www.snowflake.com/en/blog/engineering/cortex-analyst-text-to-sql-accuracy-bi/ , https://colrows.com/blogs/text-to-sql-accuracy-cliff/
- [SOURCED] Databricks built Metric Views precisely so Genie "is no longer hallucinating metrics; it's resolving them from a single source of truth," with documented accuracy gains. https://www.databricks.com/blog/redefining-semantics-data-layer-future-bi-and-ai , https://docs.databricks.com/aws/en/genie/best-practices
- [SOURCED] dbt's 2026 paired benchmark: raw text-to-SQL improved from ~32.7% to ~64.5% as models improved, but queries covered by a semantic layer approach ~100% — and they conclude reliability requires "a compiler that enforces the patterns, not a model that may or may not follow them." https://docs.getdbt.com/blog/semantic-layer-vs-text-to-sql-2026
- [SOURCED] The dominant production failure mode is silent-wrong answers (fan-out over-counting, schema-linking errors — 80%+ of execution failures trace to schema linking), which better models mitigate but do not eliminate, and which auditors/regulators specifically flag. https://tianpan.co/blog/2026-04-10-text-to-sql-failure-modes-production , https://omni.co/blog/why-text-to-sql-fails

**Verdict on direction:** the guardrail layer becomes MORE valuable as models improve, because the residual failures are exactly the expensive kind (confident, silent, ungoverned). Every hyperscaler chose to BUILD a semantic model rather than bet on model improvement. MiniML is not a thin wrapper over a model capability; it is deterministic SQL compilation — the thing the industry decided it needs.

**The real obsolescence vector is not models — it is platforms and standards.**
- [SOURCED] The Open Semantic Interchange (OSI) v1.0 spec (finalized Jan 2026; Snowflake, dbt Labs, Databricks-adjacent consortium, Salesforce, AtScale, Qlik) standardizes semantic-model interchange. MiniML's bespoke YAML dialect has no OSI story; a nonstandard format becomes a liability the moment agents expect OSI. https://www.snowflake.com/en/blog/open-semantic-interchanges-specs-finalized/ , https://github.com/open-semantic-interchange/OSI
- [MEASURED] MiniML is ~1,040 LOC of source. The engineering moat is roughly a week of frontier-model-assisted work; the durable asset in this category is model *content* and distribution, and MiniML owns neither outside Wayvia.
- [MEASURED] Two dialects only (BigQuery/Snowflake, inferred from file paths in lib/load.ts inferModelDialect), and TASKS.md openly lists date-handling bugs in the exact code paths ("last N days", granularity) that the README sells as the sophisticated part.

**Obsolescence risk score: 38/100.** Low risk from frontier models (the data is unambiguous); moderate-to-high risk from warehouse vendors bundling semantic models for free (Cortex Analyst, Genie ship with the platform) and from OSI commoditizing the format. What survives is the embeddable, warehouse-neutral niche — which is real but narrow.

### Amplification analysis: can it ride the agentic wave?

**The architecture is genuinely agent-legible — this is measured, not marketing.**
- [MEASURED] lib/load.ts `expandModelInfo()` auto-generates `model.info` as literal LLM prompt material: a markdown data dictionary of dimensions/measures plus injected imperatives ("**IMPORTANT** ... Use SNOWFLAKE syntax for generating SQL filter expressions", default-date-range instructions). lib/jinja.ts renders user `info` sections via Nunjucks. This is a prompt-generation pipeline, designed before it had a consumer.
- [MEASURED] The API surface (`loadModel`, `renderQuery` with a flat MinimlQueryOptions of dimensions/measures/dates/where/having/order/limit) maps one-to-one onto the canonical semantic-layer MCP tool set. An MCP server is ~200 LOC with @modelcontextprotocol/sdk: `list_models` (scan YAML dir), `get_model_info` (return model.info), `render_query`/`query_model` (renderQuery + optional warehouse execution). [SPECULATION on LOC, MEASURED on API fit]
- [MEASURED] A real differentiator: lib/validation.ts AST-validates *LLM-supplied* `where`/`having` SQL fragments (node-sql-parser, allowlist, model-aware column checks, subquery/DDL blocking). dbt-mcp and BSL constrain agents to structured parameters; MiniML uniquely tries to safely accept freeform filter SQL from an untrusted model. That is a defensible design point for agent use — and it's where 68 of its tests live.

**But the niche it names is already occupied, and it hasn't shipped.**
- [SOURCED] dbt-mcp (official dbt Labs): list_metrics, get_dimensions, get_entities, query_metrics, get_metrics_compiled_sql — the exact toolset MiniML would offer, backed by MetricFlow and dbt's install base. https://github.com/dbt-labs/dbt-mcp , https://docs.getdbt.com/docs/dbt-ai/about-mcp
- [SOURCED] Boring Semantic Layer (Python/Ibis) is MiniML's positioning executed: "lightweight, MCP-friendly semantic layer," shipped MCP tools (list_models, get_model, get_time_range, query_model), community traction (TLDR Data feature, dltHub demos, MotherDuck coverage). https://github.com/boringdata/boring-semantic-layer , https://juhache.substack.com/p/boring-semantic-layer-mcp
- [SOURCED] Cube exposes its semantic layer over MCP as a headline feature; Drizzle Cube ships a built-in MCP server in the TypeScript ecosystem — MiniML's home turf. https://cube.dev/articles/semantic-layer-for-ai-agents-2026 , https://www.drizzle-cube.dev/ai/mcp-endpoints/
- [MEASURED] MiniML's package.json claims keywords "mcp" and "model context protocol"; grep confirms zero MCP code anywhere in the repo. In an ecosystem where agents discover tools via MCP registries (PulseMCP, mcp.so), an unshipped keyword is invisible.
- [MEASURED] Adoption ground truth: GitHub 1 star / 0 forks / 0 issues / no releases; npm ~57 downloads in the last month (2026-06-03→07-02), with historical spikes correlating with the author's own publish days — i.e., mirrors and CI, not users. Discoverability is further damaged by the name collision with MiniML-the-teaching-language: a search for the package surfaces semantic-release noise before the project. [MEASURED via api.npmjs.org and github.com/dtempx/miniml]

**Amplification potential score: 42/100.** The raw material is unusually good — the `info` pipeline and validated filter passthrough are exactly what an agent-facing tool needs, and the MCP server is a weekend of work. But potential unexercised for 12+ months in a land-grab market decays fast; every month, another registry-listed competitor becomes the default answer to "semantic layer MCP server," and agents amplify whatever is already legible to them.

## Strongest point FOR the project
Its central bet is empirically vindicated by every major platform's own engineering data — Snowflake (+20% accuracy from a semantic model, GPT-4o at 51% without), Databricks (Metric Views to stop Genie hallucinating), dbt (~100% with layer vs 64.5% raw) — and within that validated category MiniML holds a real unoccupied micro-niche: a zero-infrastructure, embeddable, TypeScript-native compiler whose `info` output is purpose-built LLM prompt material and whose AST validation safely accepts LLM-written filter SQL, something neither dbt-mcp nor BSL attempts.

## Strongest point AGAINST
It put "mcp" in its npm keywords and then shipped nothing, while dbt Labs, Cube, Drizzle Cube, and Boring Semantic Layer shipped actual MCP servers with actual users — so the project's own stated growth vector is a race it watched from the stands, and the scoreboard reads 1 star, 0 forks, ~57 downloads/month, single author, and an open TASKS.md of date-logic bugs in its flagship feature.

## Viability score: 31/100

## The one question that would change my mind
If the author ships a first-class `miniml-mcp` server (list_models / get_model_info / query_model, with the validated-filter passthrough as its headline) and lists it on PulseMCP/mcp.so this quarter, do strangers install it within 90 days — or does Boring Semantic Layer's JS-shaped shadow prove the niche was already spoken for?