UNPKG

@claude-flow/embeddings

Version:

V3 Embedding Service - OpenAI, Transformers.js, Agentic-Flow (ONNX), Mock providers with hyperbolic embeddings, normalization, and chunking

120 lines • 4.66 kB
/** * ADR-121 Phase 5b — `@ruvector/diskann` snapshot wrapper. * * `RabitqSnapshot` (Phase 3) gives 32× memory reduction via 1-bit * quantization, batch-build-only, in-memory. That covers the * common case (≤100k cached vectors, fits in RAM, ANN search needed). * * Once the corpus grows past what fits in RAM — agent fleets where * each agent owns a slice of a shared memory namespace, multi-million * cached embeddings across long-running sessions — the index needs * to live on disk. `@ruvector/diskann` implements DiskANN/Vamana * with optional product quantization for SSD-friendly billion-scale * ANN search. This module is the thin adapter. * * Lifecycle differs from RabitqSnapshot: * - DiskANN supports **streaming insert** (then a build pass to * finalize the Vamana graph), so callers can grow the index * incrementally before building * - **Persistence** via `save(dir)` / `static load(dir)` — * surviving restarts is what makes the on-SSD story useful * * Optional peer dep. When `@ruvector/diskann` isn't installed, * `DiskannSnapshot.create` throws a clear named error pointing at * the install command + the `RabitqSnapshot` in-memory alternative. */ export interface DiskannSnapshotOptions { /** Embedding dimension. Required. */ readonly dimension: number; /** * Vamana max out-degree (default 64). Higher = better recall at * the cost of memory + build time. */ readonly maxDegree?: number; /** * Product-quantization subspaces (default 0 = no PQ). Set to e.g. * `dimension / 4` for 4× compression with a small recall hit. */ readonly pqSubspaces?: number; /** * Optional directory for on-disk persistence. When set, the * underlying DiskANN will spill its index to disk so it survives * process restarts; pair with `save()` + `DiskannSnapshot.load()`. */ readonly storagePath?: string; } export interface DiskannSnapshotHit { readonly id: string; /** L2² distance to the query. */ readonly distance: number; } /** * Disk-friendly ANN snapshot over a corpus of vectors via DiskANN/Vamana. * * Workflow: * const snap = await DiskannSnapshot.create({ dimension: 384 }); * snap.add([{ id: 'a', vector: vec1 }, { id: 'b', vector: vec2 }, ...]); * await snap.build(); * const hits = await snap.search(query, 10); * snap.save('./cache/diskann'); * * Reload after restart: * const snap = await DiskannSnapshot.load('./cache/diskann', { dimension: 384 }); * const hits = await snap.search(query, 10); */ export declare class DiskannSnapshot { private index; private readonly dimension; private built; private constructor(); /** * Create a new empty snapshot. Add vectors with `add()`, then call * `build()` before the first `search()`. * * @throws when `@ruvector/diskann` isn't installed — error message * names the package + recommends `RabitqSnapshot` as the * in-memory alternative. */ static create(options: DiskannSnapshotOptions): Promise<DiskannSnapshot>; /** * Load a previously-saved snapshot from disk. The directory must * contain a snapshot produced by `save()`. */ static load(dir: string, options: DiskannSnapshotOptions): Promise<DiskannSnapshot>; /** * Add vectors to the snapshot. Streaming — can be called multiple * times before `build()`. Each vector's dimension must match the * snapshot's configured dimension. */ add(entries: ReadonlyArray<{ id: string; vector: Float32Array; }>): void; /** * Build (or rebuild) the Vamana graph. Async — yields to the * event loop during construction so callers can show progress. * Required before `search()`. */ build(): Promise<void>; /** * Query for the `k` nearest neighbors. `build()` must have been * called at least once after the most recent `add()`. */ search(query: Float32Array, k: number): Promise<DiskannSnapshotHit[]>; /** Remove a vector by id. */ delete(id: string): boolean; /** Current vector count. */ count(): number; /** * Persist the snapshot to a directory. Reload with * `DiskannSnapshot.load(dir, options)`. */ save(dir: string): void; } /** * Detect whether `@ruvector/diskann` is reachable. Useful for * `ruflo doctor` / observability reporting alongside the existing * `hyperbolicAttentionAvailable` and `probeRuvectorSidecar` probes. */ export declare function diskannAvailable(): Promise<boolean>; //# sourceMappingURL=diskann-snapshot.d.ts.map