UNPKG

@dudousxd/nestjs-filter-clickhouse

Version:

ClickHouse adapter for @dudousxd/nestjs-filter: compiles structured filter input to parameterized ClickHouse SQL.

123 lines (98 loc) • 5.33 kB
# @dudousxd/nestjs-filter-clickhouse ClickHouse adapter for [`@dudousxd/nestjs-filter`](../core/README.md): compiles the structured input — `filter` / `where` operators, `sort`, `paginate`, `search`, `distinct`, `select`, cursor pages, `groupByCount`, `extent` — to **one parameterized ClickHouse statement**. Every client value is bound as a typed `{pN:Type}` query parameter; client-supplied field names only select among declared expressions. Nothing a client sends is ever part of the SQL text. ```bash pnpm add @dudousxd/nestjs-filter @dudousxd/nestjs-filter-clickhouse @clickhouse/client ``` ## Declare a table The table is the adapter's "entity". Its `fields` are the allowlist: a trusted SQL expression per field plus its ClickHouse type (which types every bound parameter). ```ts import { defineClickHouseTable } from '@dudousxd/nestjs-filter-clickhouse'; export const events = defineClickHouseTable({ table: 'events', primaryKey: 'id', where: 'at >= now64(3) - toIntervalDay(90)', // optional always-on guard (trusted SQL) fields: { id: 'UUID', at: "DateTime64(3, 'UTC')", day: { type: 'Date', expr: 'toDate(at)' }, event: { type: 'LowCardinality(String)', expr: 'name' }, provider: 'Nullable(String)', durationMs: 'UInt64', tags: 'Array(String)', }, }); ``` ### Aggregated tables (measures) Fields marked `measure` are aggregate expressions. A table with measures answers with **groups**: rows are grouped by the requested dimensions (`select`, else the table's `groupBy`), a filter that touches a measure goes to `HAVING`, sorting by a measure orders groups, and the total counts groups. ```ts export const chatDaily = defineClickHouseTable({ table: 'chat_daily', groupBy: ['provider', 'model'], fields: { day: 'Date', provider: 'LowCardinality(String)', model: 'LowCardinality(String)', turns: { type: 'UInt64', expr: 'sum(turns)', measure: true }, errorRate: { type: 'Float64', expr: 'sum(errors) / nullIf(sum(turns), 0)', measure: true }, }, }); // GET /stats/chat?select=day&filter[provider]=openai&filter[where][0][field]=turns&filter[where][0][operator]=>=&filter[where][0][value]=100&sort=-turns ``` ## Register and query ```ts import { createClient } from '@clickhouse/client'; import { clickHouseAdapter, ClickHouseFilter, type ClickHouseQuery } from '@dudousxd/nestjs-filter-clickhouse'; FilterModule.forRoot({ adapter: clickHouseAdapter({ client: createClient({ url }) }) }); // or { connection: CLICKHOUSE } — the token of a provider holding the client @Injectable() @Filterable({ entity: events, defaultSort: '-at' }) export class EventFilter extends ClickHouseFilter { static readonly search = ['event', 'provider']; @FilterFor('days') lastDays(value: string) { this.$query.where(`at >= now64(3) - toIntervalDay(${this.$query.bind('UInt32', Number(value))})`); } } @Get() list(@ApplyFilter(EventFilter) q: ClickHouseQuery) { return q.executeAndCount(); // { rows, total } } ``` `q.toSQL()` / `q.toCountSQL()` return `{ query, params }` to run with any client. Next to a database adapter, provide `ClickHouseAdapter` under `CLICKHOUSE_FILTER_ADAPTER` and name it per filter: `@Filterable({ entity: events, adapter: CLICKHOUSE_FILTER_ADAPTER })`. ## Semantics The same answers as the SQL and memory adapters (Postgres as the reference): - **NULL logic** — negated operators do not match NULL values. ClickHouse answers `NULL NOT IN (…)` with `1` (`transform_null_in = 0`), so `notIn` on a `Nullable` field adds `isNotNull(…)`. - **Case & wildcards** — `contains`/`startsWith`/`endsWith` are case-sensitive, `iContains` and `search` case-insensitive (UTF-8). Matching uses `position()`/`startsWith()`, not `LIKE`: `%` and `_` in values are literal. Non-string fields are matched via `toString()`. - **Coercion** — values are encoded for the field type (`'30'` → UInt, `'true'` → Bool, ISO strings → `DateTime64` text in UTC); a value that does not fit is a 400 (`ClickHouseValueError`), not a failed query. - **`isEmpty`** — NULL-or-`''` on strings, NULL-or-`[]` on arrays, NULL elsewhere. - **Array fields** — positive operators hold when any element matches (`arrayExists`), negated ones when none does. - **Ordering** — `NULLS LAST` ascending, `NULLS FIRST` descending. - **Output columns** are aliased `__<field>` in the SQL (ClickHouse resolves aliases inside WHERE and aggregates, so `sum(errors) AS errors` would recurse) and un-prefixed on execution. - **Computed fields** — SQL expressions (strings, or functions returning a string). - Not supported: relations / includes / to-many aggregates (ClickHouse has no correlated subqueries to build them on), `tsvector` search. Result values are whatever `JSONEachRow` returns: 64-bit integers arrive as strings unless you set `output_format_json_quote_64bit_integers: 0` on the client, and dates as text. ## Testing `pnpm test` runs the SQL-snapshot suite. With a server, the real-server suite runs the cross-adapter contract's operator expectations against ClickHouse: ```bash docker run -d --name ch -p 8123:8123 -e CLICKHOUSE_USER=test -e CLICKHOUSE_PASSWORD=test clickhouse/clickhouse-server:25.8 CLICKHOUSE_URL=http://test:test@localhost:8123 pnpm --filter @dudousxd/nestjs-filter-clickhouse test ```