harper-fabric-embeddings
Version:
Minimal llama.cpp embedding wrapper for Harper Fabric. Talks directly to the native addon — no build tools, no CLI, no chat wrappers.
212 lines (163 loc) • 8.6 kB
Markdown
# harper-fabric-embeddings
Minimal llama.cpp embedding wrapper for Harper Fabric. Talks directly to the `@node-llama-cpp` native N-API addon — no build tools, no CLI, no chat wrappers, no model downloaders beyond a simple HuggingFace fetch.
~19 MB installed (native binary only) vs ~250 MB+ for `node-llama-cpp`.
## Installation
```sh
npm install harper-fabric-embeddings
```
The package uses `@node-llama-cpp` platform-specific binaries. The `linux-x64` binary is included as an optional dependency. For other platforms, install the appropriate package:
```sh
npm install @node-llama-cpp/mac-arm64-metal # macOS Apple Silicon
npm install @node-llama-cpp/mac-x64 # macOS Intel
npm install @node-llama-cpp/linux-arm64 # Linux ARM64
```
## Use as a Harper models backend
Harper's models bootstrap can load this package directly as an embedding
backend. Install it in the Harper instance root and name it in
`harperdb-config.yaml`:
```yaml
models:
embedding:
default:
backend: harper-fabric-embeddings
modelName: nomic-embed-text
modelsDir: ./models
```
Everything that consumes Harper's models API then routes through the local
GGUF engine: `models.embed()`, `@embed` table directives, and model-call
analytics (`hdb_model_calls` gets `embeddingTokens` + `latencyMs` per call).
- `modelsDir` or `modelPath` is required. Relative paths resolve against
Harper's working directory.
- `model` is accepted as an alias for `modelName` (the field Harper's
built-in backends use). `contextSize`, `batchSize`, `threads`, `gpuLayers`,
and `addonPath` pass through as with `init()`.
- Boot is not blocked on the model: registration kicks off the load/download
in the background and the first embed call awaits it. Misconfiguration
(wrong kind, missing model source, unknown model name) fails at boot, where
Harper logs and skips the entry.
- `inputType` is honored: each model's **prompt templates** (see below) shape
document vs query encodings — nomic models get their `search_document: ` /
`search_query: ` task prefixes. Harper's `@embed` directive passes
`inputType: 'document'`; when `inputType` is omitted (the default through
`models.embed()`), **no template is applied, ever** — input handling
identical to the raw API and to 0.2.x, so pre-existing vectors stay
comparable. Corpora embedded template-less need a one-time re-embed to
benefit from templated queries.
- Vector dimensionality: Harper's `models` facade has no model-metadata
accessor yet, so read it from the first embed result
(`(await models.embed('x'))[0].length` — 768 for both built-in nomic
models), or use `dimensions()` on the raw API.
- Multiple entries work — each gets its own engine (own model + context),
sharing one native addon binding:
```yaml
models:
embedding:
default:
backend: harper-fabric-embeddings
modelName: nomic-embed-text
modelsDir: ./models
fallback: [remote]
moe:
backend: harper-fabric-embeddings
modelName: nomic-embed-text-v2-moe
modelsDir: ./models
remote:
backend: openai
model: text-embedding-3-small
apiKey: ${OPENAI_API_KEY}
```
## Usage
```typescript
import { init, embed, dimensions, dispose } from 'harper-fabric-embeddings';
// Initialize with a models directory (finds or downloads the model)
await init({ modelsDir: '/path/to/models' });
// Generate an embedding (L2-normalized)
const vector = await embed('Hello world');
// Get vector dimensionality
const dims = dimensions();
// Clean up native resources
await dispose();
```
## API
### `init(options)`
Initialize the embedding engine. Call once before using `embed()`.
| Option | Type | Default | Description |
| ------------- | ------ | -------------------- | ----------------------------------------------------------------------- |
| `modelPath` | string | — | Absolute path to a .gguf model file |
| `modelsDir` | string | — | Directory to search/download model files |
| `modelName` | string | `"nomic-embed-text"` | Model name from the built-in registry |
| `contextSize` | number | `2048` | Token context window size |
| `batchSize` | number | `contextSize` | Batch size (`n_batch`/`n_ubatch`); longer inputs are truncated |
| `threads` | number | `6` | CPU threads for inference |
| `gpuLayers` | number | `0` | Layers to offload to GPU (0 = CPU only) |
| `addonPath` | string | — | Override path to `llama-addon.node` |
| `templates` | object | registry entry | Per-inputType prompt templates (`document`/`query`/`defaults`) |
| `pooling` | string | — | Expected pooling declared by the model file — verified at init, not set |
`pooling` (`'none' | 'mean' | 'cls' | 'last' | 'rank'`) is verification, not
override: the native addon exposes no pooling option, so llama.cpp always uses
the model's own `<arch>.pooling_type` metadata. Declaring the expectation makes
init fail loudly when a GGUF omits or contradicts it — instead of a
metadata-less conversion silently mean-pooling a last-token model.
Either `modelPath` or `modelsDir` is required.
### `embed(text)`
Generate an L2-normalized embedding vector for the given text. Returns `number[]`.
### `dimensions()`
Returns the embedding vector dimensionality.
### `dispose()`
Clean up native resources (model, context, binding).
### `downloadModel(dir, modelName?)`
Download a model from HuggingFace. Called automatically by `init()` when using `modelsDir` and no local model is found.
## Prompt templates
Embedding models disagree about how document and query inputs should be
affixed — nomic wants fixed prefixes, instruct-style embedders (Qwen3-Embedding
and friends) want a free-text task instruction on the query side only. That
convention is **data on the model entry**, not code:
```yaml
models:
embedding:
default:
backend: harper-fabric-embeddings
modelPath: ./models/my-model.Q8_0.gguf
templates:
document: '{text}'
query: "Instruct: {task}\nQuery: {text}"
defaults:
task: 'Given a search query, retrieve relevant passages that answer the query'
```
- `{text}` is the input text. `{task}` comes from the embed call's `task`
option (`models.embed(q, { inputType: 'query', task: '...' })`), falling
back to `defaults.task`. Any other placeholder must be covered by
`defaults`. Escape literal braces as `{{` / `}}`.
- Interpolation is single-pass; invalid templates (unknown placeholders,
unescaped braces) fail at registration — Harper logs and skips the entry at
boot rather than surfacing at first embed.
- Omitted `inputType` is always passthrough, templates or not. A missing side
(e.g. only `query` declared) falls back to the legacy nomic name-prefix
heuristic, then passthrough.
- The built-in nomic entries declare their prefixes as templates; explicit
`templates` in config override an entry's own.
- Note for typed Harper consumers: `task` passes through `models.embed()` at
runtime, but isn't on core's `EmbedOpts` type yet — cast until core widens it.
## Models
Two models are built in:
| Name | Source | Quantization |
| ------------------------- | -------------------------------- | ------------ |
| `nomic-embed-text` | nomic-ai/nomic-embed-text-v1.5 | Q4_K_M |
| `nomic-embed-text-v2-moe` | nomic-ai/nomic-embed-text-v2-moe | Q4_K_M |
Models are resolved in order: HuggingFace-prefixed filename, bare filename, stem match scan, then download from HuggingFace.
HuggingFace may reject anonymous large-file downloads (HTTP 403). Set
`HF_TOKEN` (or `HUGGING_FACE_HUB_TOKEN`) to a free account token and the
download sends it as a bearer; alternatively pre-seed `modelsDir` with the
model file and no download happens at all.
## Testing
```sh
# Unit tests (no model file needed)
npm test
# Integration tests (requires a model file)
MODEL_PATH=/path/to/model.gguf npm test
```
## Requirements
- Node.js 22+
- A `@node-llama-cpp` platform package for your architecture
## License
MIT