@athenox/talos
Version:
Talos: a backend you describe instead of write. Declare a schema, write the rules in plain language, map URLs to sentences; a small built-in model turns them into query plans.
386 lines (309 loc) • 18.7 kB
Markdown
# Talos
**A backend you describe instead of write.** Declare your tables, write the
rules in plain language, map each URL to a sentence — a small language model,
built for this and running on your CPU, turns each sentence into a query plan,
and the runtime runs it safely against your database.
```ts
import { defineApp, defineRoutes } from '@athenox/talos'
const app = defineApp({
prompt: `
show the recipes.
a member may add a recipe, and change or delete their own.`,
schema: { tables: [{
as: 'recipes', table: 'recipes',
columns: { id: 'id', title: 'title', course: 'course', minutes: 'minutes', author: 'author_id' },
enums: { course: ['main', 'starter', 'dessert'] },
roles: { write: ['member', 'admin'] },
}] },
})
// member (a guard) and asMember (the caller's role): see Accounts below
export default defineRoutes(app, {
'GET /api/recipes': { say: 'show the recipes', orderBy: '-id', limit: 20 },
'GET /api/recipes/:id': { say: 'show the recipe {id}', one: true },
'POST /api/recipes': { say: 'add a recipe', guard: member, ctx: asMember },
'DELETE /api/recipes/:id': { say: 'delete the recipe {id}', guard: member, ctx: asMember, where: (_, me) => ({ author_id: me.id }) },
})
```
No SQL, no controllers, no migrations — and no database to set up if you do
not want one: without a `store`, Talos keeps its own SQLite file built from the
schema.
- **The model plans, the runtime decides.** The model reads the schema, the
rules and the request, and generates the plan (operation, table, filters,
values) by pointing into what it read. Permissions, ownership, validation and
the SQL are ordinary deterministic code.
- **Compiled once.** A route's sentence goes through the model on its first
call; the plan is kept with its parameters as slots and replayed without the
model afterwards. A route then costs what its SQL costs.
- **A full backend kit around it**: accounts and sessions, uploads with image
processing, full-text search, recommendations, feeds and sitemaps, CSV import,
rate limiting, schema sync for SQLite and MySQL/MariaDB, Drizzle schemas.
- **Free-text requests too.** `app.run('the easy desserts under 30 minutes', ctx)`
answers a sentence nobody wrote a route for.
**Contents** — [Install](#install) · [Guide](#guide): [the app](#1-the-app) ·
[the schema](#2-the-schema) · [routes](#3-routes) · [free text](#4-free-text-requests) ·
[accounts](#5-accounts) · [files](#6-files-and-images) · [search](#7-search-similar-rows-recommendations) ·
[databases](#8-databases) · [telemetry](#9-telemetry-ids-and-feedback) ·
[Results](#results) · [Environment](#environment) · [Full documentation](#full-documentation)
## Install
```sh
npm install @athenox/talos
```
Node 22.13 or later. The model (≈ 25 MB with its text embedder, ONNX, int8, CPU) ships in the package.
It comes with `h3` 1.x for its HTTP helpers (the version Nuxt 4 and Nitro
use, so they share it); `sharp` is needed only to process images. With Nuxt, keep Talos out of the
bundle — it loads its model from its own folder:
```ts
// nuxt.config.ts
nitro: { externals: { external: ['@athenox/talos', 'onnxruntime-node', 'sharp'] } }
```
# Guide
## 1. The app
```ts
export const app = defineApp({
prompt, // the rules, in prose — read by the model with every request
schema, // { tables, functions? }
store?, // your database; left out: Talos's own (.talos/data.db)
seed?, // rows for Talos's own database, put into tables still empty
actions?, // implementations of the schema's functions
telemetry?, // what this instance reports (on by default), or false
})
```
**The prompt** is written the way you would explain the app to a colleague:
what can be shown, who may do what.
```
show the recipes, the cooks and the comments.
a member may add a recipe, and change or delete their own.
an admin may do anything.
```
Prompts, route sentences and requests can be in **English or French** — the
model reads both, and a French request reaches an English schema through
`says` (`says: ['recettes']` on `recipes`).
The prompt guides the plan; it grants nothing. What a caller may do is the schema's
`roles` and your route guards — a model that is right 99 % of the time is not a
permission check.
## 2. The schema
```ts
{
as: 'recipes', table: 'recipes', // the model's name, the physical table
says: ['dishes', 'recettes'], // how people call it (any language)
columns: { id: 'id', title: 'title', course: 'course', minutes: 'minutes', author: 'author_id', date: 'created_at' },
columnSays: { minutes: ['time', 'duration'] },
enums: { course: ['main', 'starter', 'dessert'] }, // declared values
types: { minutes: 'number' },
relations: { author: { table: 'users', on: 'author_id', columns: { id: 'id', name: 'name' } } },
lists: { comments: { table: 'comments', on: 'recipe_id', columns: { id: 'id', body: 'body' }, orderBy: 'created_at' } },
tallies: { likes: { table: 'likes', on: 'recipe_id' } },
searchable: ['title'],
rules: { title: { required: 'insert', trim: true, min: 3, max: 120, message: 'A title of 3 to 120 characters' } },
defaults: { insert: { created_at: () => new Date().toISOString() } },
children: [{ table: 'likes', on: 'recipe_id', cascade: true }],
hidden: [],
roles: { read: ['guest', 'member', 'admin'], write: ['member', 'admin'], columns: { featured: ['admin'] } },
}
```
The keys of `columns` are **logical** names — what the model reads; the values
are **physical** — what the database has and what rows come back with. Keep
them equal and in English when you can (the model knows English schema words
best) and put the words people use in `says` / `columnSays`.
| group | fields |
|---|---|
| read by the model | `as`, `columns`, `enums`, `flags` (booleans as two words), `says`, `columnSays` |
| reading | `relations` (an object per row), `lists` (an array per row), `many` (through a join table), `tallies` (counts), `computed`, `hidden`, `searchable`, `view`, `types`, `json`, `read` |
| writing | `rules`, `defaults`, `hash`, `slugs`, `files`, `infer` (filled by meaning), `writable`, `children` (cascade / restrict), `after` (functions to run), `write` |
| permissions | `roles` (`read`, `write`, `create`, `remove`, `columns`), `ownerColumn` + `ownerFrom` (rows pinned to the caller) |
Rules: `required` (`true` or `'insert'`), `min`, `max`, `email`, `pattern`,
`trim`, `unique` (or its message), `uniqueWith`, `message`. A failed rule is a
`400` with `data.errors: [{ msg, column }]`, the shape forms expect.
**Functions** are side effects the prose may name:
```ts
schema: { tables, functions: [{ name: 'sendNewsletter', params: ['subject'], required: ['subject'], roles: ['admin'], says: ['send the newsletter'] }] },
actions: { sendNewsletter: async ({ args, ctx }) => ({ sent: 120 }) },
```
→ every field: [docs/schema.md](https://unpkg.com/browse/@athenox/talos@latest/docs/schema.md)
## 3. Routes
### A sentence
```ts
const del = app.route('delete the recipe {id}', { ops: ['DELETE'] })
const { status, result, plan, callId } = await del({ id: 3 }, { userId: 7, role: 'member' })
```
`{k}` is a parameter, `[... {k} ...]` is dropped when `k` is missing,
`{k:no|yes}` says a boolean with one of two words. The first call asks the
model; later ones replay the plan with new values. Before the model reads a
sentence, a misspelt word of the app or a misspelt verb is corrected, and an
opening verb (*delete*, *add*, *change*, *show*, and their French forms) fixes the operation.
### A whole API as a table
```ts
export default defineRoutes<Member>(app, {
'GET /api/recipes': {
say: 'show the recipes', orderBy: '-date', limit: 12, total: true,
fields: ['id', 'title', 'course', 'likes', 'author.name'],
where: p => (p.course ? { course: p.course } : {}),
shape: p => ({ search: p.q, page: Number(p.page) || 1 }),
},
'GET /api/recipes/:id': { say: 'show the recipe {id}', one: true, notFound: 'No such recipe' },
'POST /api/recipes': { say: 'add a recipe', guard: member, ctx: asMember, stamp: (_, me) => ({ author_id: me.id }) },
'PUT /api/recipes/:id': { say: 'change the recipe {id}', guard: member, ctx: asMember, where: own },
'DELETE /api/recipes/:id': { say: 'delete the recipe {id}', guard: member, ctx: asMember, where: own },
'POST /api/auth/login': users.login, // any h3 handler sits in the table as it is
}, { guards: { member }, defaults: { firstError: true } })
```
- The HTTP method fixes the operation (GET → read, POST → create or call,
PUT/PATCH → update, DELETE → delete).
- URL segments (`/:id`, `/:slug`) name the row; they replace whatever the
model wrote on those columns. The body's fields are written by name.
- `guard` runs first and its user is given to `ctx`, `where`, `stamp`, `shape`
and `reply`. `ctx` is what roles are checked against.
| option | |
|---|---|
| `say` | the sentence |
| `guard` | `(event, params) => user`; throws to refuse |
| `ctx` | `user => ({ userId, role })` |
| `where` | conditions set by code: `{ col: v }` or `(p, user) => ({...})`; `{ gte }`, `{ lt }`, `{ between }`, `{ ne }`, `{ in }`, `{ null }`, `any: [...]` (or), `'relation.column'` |
| `stamp` | columns the route writes itself: `(p, user) => ({ author_id: user.id })` |
| `fill`, `only`, `form` | defaults for missing parameters, allowed body fields, HTML-form checkboxes and empty fields |
| `shape` | per-request options: `(p, user, event) => ({ search, page, ... })` |
| `fields`, `row`, `reply` | what the answer holds |
| `notFound`, `firstError`, `pin`, `download` | 404 message, one error only, which segments name the row, send a file |
| `orderBy`, `limit`, `page`, `total`, `one` | `'-date'`, `['-likes', 'title']`, `'minutes~'` (empty last); `total` adds `{ total, page, limit }` |
| `search`, `find`, `similar`, `recommend` | ranking — [section 7](#7-search-similar-rows-recommendations) |
| `lists`, `counts`, `sums`, `returning` | which lists to load, extra counts and sums, return written rows |
| `rows`, `upsert`, `ignore`, `dedupe`, `before` | bulk inserts, insert-or-update, skip duplicates, remove duplicates, a hook before the write |
**Answers.** `200` with the row (`one`), the rows, `{ rows, total, page, limit }`,
`{ id }` for a creation, `{ success: true }` for an update or delete — or what
`reply` returns. `400` for failed rules, `401`/`403` from guards and roles,
`404` when nothing matched, `422` when the request is not something this app
does. Every answer carries `x-talos-plan` (the sentence and its plan — read it
once when you add a route) and `x-talos-call` (the model call).
→ [docs/routes.md](https://unpkg.com/browse/@athenox/talos@latest/docs/routes.md)
## 4. Free-text requests
```ts
const { status, result, plan, callId } = await app.run('the easy desserts under 30 minutes', ctx, { ops: ['SELECT'], limit: 10 })
```
`run` asks the model every time: a search box, an assistant, an admin console.
The same roles and ownership apply. `decide(sentence, ctx)` gives the plan
without running it; `execute(plan, ...)` runs a plan however it was made.
## 5. Accounts
```ts
import { accounts, hashPassword, verifyPassword } from '@athenox/talos'
export const users = accounts<Member>({
store: app.store, table: 'users', expose: ['id', 'name', 'role'], signUp: ['name'], minPassword: 8,
hash: hashPassword, verify: verifyPassword,
session: { set: (e, user) => setUserSession(e, { user }), get: async e => (await getUserSession(e)).user, clear: clearUserSession },
})
export const member = users.guard() // the account, or 401
export const asMember = (me: Member) => ({ userId: me.id, role: me.role })
export const own = (_: unknown, me: Member) => (me.role === 'admin' ? {} : { author_id: me.id })
```
Handlers: `users.register`, `users.login` (JSON or a plain HTML form),
`users.logout`, `users.me`, `users.change`, and `users.forgot` / `users.reset`
with `reset: { table, owner, send }`. `users.guard(check?)` re-reads the account
(a disabled one is refused). Any session store works; above, nuxt-auth-utils.
→ [docs/accounts.md](https://unpkg.com/browse/@athenox/talos@latest/docs/accounts.md)
## 6. Files and images
```ts
files: {
photo: {
dir: 'uploads/recipes', url: '/uploads/recipes/', accept: /^image\//, maxBytes: 12_000_000,
image: { width: 1600, height: 1600, format: 'webp', quality: 82 },
variants: { thumb: { width: 640, height: 480, fit: 'cover', format: 'webp' } },
},
}
```
A multipart request carries the file under the column's name. It is checked,
resized, re-encoded, stripped of its metadata (a phone photo carries its GPS
position), stored under a random name, and the column set to its URL; variants
go to their own columns. Replacing it or deleting the row removes the old
file. Serve them with `serveFiles('uploads/recipes')`.
→ [docs/files.md](https://unpkg.com/browse/@athenox/talos@latest/docs/files.md)
## 7. Search, similar rows, recommendations
They rank the rows the plan chose — conditions and permissions still apply —
with words and with meaning (Talos's own French/English text embedder).
```ts
shape: p => ({ search: p.q }) // over `searchable`
shape: p => ({ find: { q: p.q, fields: { title: 3, body: 1 }, html: true } }) // weighted, prefixes, typos, meaning
shape: p => ({ similar: { to: { id: Number(p.id) }, text: ['title', 'body'] } })
shape: (_, me) => ({ recommend: { user: me.id, interactions: { table: 'likes', user: 'user_id', item: 'recipe_id' }, text: ['title'], date: 'created_at' } })
```
`recommend` scores each row by closeness to what the reader liked, what readers
with the same likes liked, popularity and recency, and leaves out what they
already saw.
→ [docs/search.md](https://unpkg.com/browse/@athenox/talos@latest/docs/search.md)
## 8. Databases
| | |
|---|---|
| none | Talos's own SQLite file (`.talos/data.db`, `TALOS_DB`), tables and columns made from the schema at start, never dropped; `seed` fills empty tables; `await app.ready` |
| SQLite | `store: sqliteStore(new DatabaseSync('app.db'))` (Node's or better-sqlite3) |
| MySQL / MariaDB | `store: mysqlStore(mysql.createPool(url))` |
| Drizzle | `schema: { tables: fromDrizzle(drizzleSchema) }`, `syncSchema(store, drizzleSchema)` to create what is missing |
Also: `parseCsv`, `toCsv`, `importCsv` (header aliases, deduplication, a report),
`connector` (an external JSON API into your fields), `feed('rss' | 'sitemap' | 'sitemap-news')`,
`rateLimit`.
## 9. Telemetry, ids and feedback
```ts
app.instanceId // this instance — always Talos's own
result.callId // every model call (also plan.callId and the x-talos-call header)
await app.feedback(callId, { rating: 4, text: 'right rows, wrong order' }) // a text, a rating 1–5, or both
```
Each instance reports to `stats.talos.korollr.com`, so that someone who comes
with an instance id or a call id can be helped:
| part | |
|---|---|
| `errors` | exceptions in the 5 s after an answer, tied to the call (a crash is sent by the next start) |
| `operations` | each plan and answer — operation, table, columns, status, counts; values only when declared |
| `prompts` | the app's prompt, route templates, free-text requests **anonymised** (unknown words → `<w>`, numbers → `#`) |
| `performance` | latency per call; every minute, Talos's and the process's load |
| `config` | Talos and model versions, Node, platform |
| `schema` | tables and columns — never a row |
Every option, with its default — set one to `false` to stop sending it:
```ts
telemetry: {
errors: true, // exceptions right after an answer, tied to the call
operations: true, // plans and answers: operation, table, columns, status (values masked)
prompts: true, // the app's prompt, route templates, anonymised requests
performance: true, // latency per call, load every minute
config: true, // Talos/model versions, Node, platform — once
schema: true, // tables and columns — once, never a row
errorWindowMs: 5000, // how long after an answer an error still counts
loadEveryMs: 60000, // load sampling period
}
telemetry: { prompts: false } // everything but what users typed and the prompt
telemetry: false // nothing
```
What each one sends exactly, with an example event and what you lose without
it: [docs/telemetry.md](https://unpkg.com/browse/@athenox/talos@latest/docs/telemetry.md).
`TALOS_TELEMETRY=0` or `DO_NOT_TRACK=1` turn it off for the whole process. An
app chooses what is sent, not where.
→ [docs/telemetry.md](https://unpkg.com/browse/@athenox/talos@latest/docs/telemetry.md)
# Results
Measured on 164 sentences recorded from three real applications ported to
Talos (a newspaper, an association site, a plugin store):
| | right plans |
|---|---|
| routes, as used in production | 99.4 % |
| the model alone, no hints | 98.2 % |
| a typo in the sentence | 98.1 % |
| another verb (*remove* for *delete*) | 94.7 % |
A route compiles in about 200 ms on a CPU, once. Ported, those applications'
backends went from 13,152 lines to 6,518, from 1,807 to 661, and from 1,748 to 918.
A complete application built with it — a recipe-sharing site (Nuxt 4,
Tailwind 4: accounts, photos, comments, favourites, recommendations,
search by sentence) — has about 270 lines of backend.
# Environment
| variable | |
|---|---|
| `TALOS_DB` | Talos's own database file (default `.talos/data.db`) |
| `TALOS_THREADS` | cap on the model's CPU threads |
| `TALOS_MODEL_DIR` | load the weights from elsewhere |
| `TALOS_TELEMETRY=0` | no telemetry for the whole process |
# Full documentation
The pages below ship in the package (`node_modules/@athenox/talos/docs/`):
[Getting started](https://unpkg.com/browse/@athenox/talos@latest/docs/getting-started.md) ·
[Schema](https://unpkg.com/browse/@athenox/talos@latest/docs/schema.md) ·
[Routes](https://unpkg.com/browse/@athenox/talos@latest/docs/routes.md) ·
[Accounts](https://unpkg.com/browse/@athenox/talos@latest/docs/accounts.md) ·
[Files](https://unpkg.com/browse/@athenox/talos@latest/docs/files.md) ·
[Search](https://unpkg.com/browse/@athenox/talos@latest/docs/search.md) ·
[Telemetry](https://unpkg.com/browse/@athenox/talos@latest/docs/telemetry.md) ·
[Reference](https://unpkg.com/browse/@athenox/talos@latest/docs/reference.md)
# License
MIT © Athenox Development