agentfootprint
Version:
The explainable agent framework — backtrack a wrong answer to the exact context that caused it (evidence, not guesses). Built on footprintjs.
244 lines (243 loc) • 11.7 kB
TypeScript
/**
* adapters/memory/pgVector — the target our own port has named since 2.x.
*
* `MemoryStore`'s docstring has listed the backends it was designed for from
* the beginning — *"Every storage backend (InMemory, Redis, DynamoDB,
* **Postgres**, Bedrock AgentCore) implements this interface"* — and named the
* query, twice, in the places an implementer would look: *"**Postgres**:
* multi-row INSERT … ON CONFLICT DO UPDATE"* for `putMany`, *"**pgvector**:
* `ORDER BY embedding <=> query LIMIT k`"* for `search`. Every one of those
* sentences was true about the design and false about the shipped package.
* This is the adapter that makes them the same sentence.
*
* It matters more than one row in a table of backends. Postgres is the database
* most teams already run, `pgvector` is an extension away, and a corpus that
* lives beside the application's own data inherits its backups, its failover,
* its access control and its migrations. `sqliteVectorStore` is one machine;
* `s3VectorsStore` is serverless and eventually consistent; this is the one for
* a fleet that already has a database.
*
* ── The table, and why this does not create it ──────────────────────────────
* A `vector(N)` column fixes N at creation, and N is a fact about your
* embedder. Creating the table implicitly would pick that number — and the
* index type, and the operator class — on your behalf, in a migration you never
* reviewed, in a database whose DDL is usually somebody's job. So the schema is
* yours to run, and this store REFUSES a table that is missing rather than
* silently answering "no matches" against nothing:
*
* ```sql
* CREATE EXTENSION IF NOT EXISTS vector;
*
* -- 1024 = your embedder's dimensions. bedrockEmbedder() default: 1024.
* -- openaiEmbedder() default: 1536. staticEmbedder(): 256.
* CREATE TABLE af_vectors (
* namespace TEXT NOT NULL,
* id TEXT NOT NULL,
* value JSONB NOT NULL,
* metadata JSONB,
* embedding vector(1024),
* embedder_fp TEXT,
* version INTEGER NOT NULL,
* created_at BIGINT NOT NULL,
* updated_at BIGINT NOT NULL,
* last_accessed_at BIGINT NOT NULL,
* access_count INTEGER NOT NULL,
* ttl BIGINT,
* tier TEXT,
* source JSONB,
* embedding_model TEXT,
* PRIMARY KEY (namespace, id)
* );
*
* -- Cosine, because that is the score this port reports and every threshold
* -- in this library is calibrated on. Match the operator class to the metric.
* CREATE INDEX af_vectors_hnsw ON af_vectors
* USING hnsw (embedding vector_cosine_ops);
* CREATE INDEX af_vectors_ns ON af_vectors (namespace);
*
* -- Recognition (`seen`/`recordSignature`), usefulness feedback, and the
* -- per-namespace embedder fingerprint. Small, and each one is a port method
* -- that would otherwise have to be refused.
* CREATE TABLE af_signatures (
* namespace TEXT NOT NULL, signature TEXT NOT NULL,
* PRIMARY KEY (namespace, signature)
* );
* CREATE TABLE af_feedback (
* namespace TEXT NOT NULL, id TEXT NOT NULL,
* total DOUBLE PRECISION NOT NULL, count INTEGER NOT NULL,
* PRIMARY KEY (namespace, id)
* );
* CREATE TABLE af_index_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
* ```
*
* Every table and column name above is an OPTION with that value as its
* default, so this drops into a schema that already has naming conventions —
* see {@link PgVectorStoreOptions}. Identifiers are validated and quoted; a
* name that is not a plain SQL identifier is refused rather than interpolated.
*
* ── Cosine, and only cosine ─────────────────────────────────────────────────
* `search` is `1 - (embedding <=> $query::vector)` — pgvector's cosine
* DISTANCE, converted to the cosine SIMILARITY the port reports and
* `defineRAG`'s 0.7 default is calibrated on. `<->` (L2) and `<#>` (inner
* product) are deliberately not options: their ranges are not that range, and a
* number that reads like a cosine and is not one is the failure mode the whole
* fingerprint machinery exists to prevent. Build the HNSW index with
* `vector_cosine_ops` so the operator and the index agree — with the wrong
* operator class the query still returns the right answer, slowly, by scanning.
*
* ── One statement at a time, on purpose ─────────────────────────────────────
* The client here is anything with `query()` — a `pg.Pool` is the expected one,
* and a Pool hands each `query()` its own connection. `BEGIN` on one and the
* next statement on another is a transaction that silently is not one, so this
* adapter never writes multi-statement transactions. Everything that must be
* atomic is ONE statement: `putMany` is one multi-row upsert, `putIfVersion` is
* one conditional upsert, `forget` is one statement with CTEs across all four
* tables. That is a constraint that made the code better.
*
* ── Lazy peer dependency ────────────────────────────────────────────────────
* `pg` is an OPTIONAL peer dependency, required at construction time. Importing
* `agentfootprint/memory` costs nothing for consumers who never build one of
* these. Pass `client` to reuse the pool your app already has — which is the
* recommended shape, because a second pool to the same database is a second set
* of connections nobody counted.
*/
import type { MemoryIdentity } from '../../memory/identity/index.js';
import type { MemoryStore } from '../../memory/store/types.js';
/** One result set, as this adapter reads it. */
export interface PgQueryResult {
readonly rows: readonly Record<string, unknown>[];
}
/**
* The slice of a `pg` client this adapter calls.
*
* Structural, so a `Pool`, a `Client`, a pgBouncer-fronted wrapper or a test
* double all satisfy it without this package taking a hard type dependency on
* the optional peer.
*/
export interface PgLikeClient {
query(text: string, params?: readonly unknown[]): Promise<PgQueryResult>;
/** Optional — awaited by {@link PgVectorStore.close} when this store built the pool. */
end?(): Promise<void>;
}
/** The one constructor this adapter needs out of `pg`. */
export interface PgSdkModule {
readonly Pool?: new (config: {
connectionString?: string;
}) => PgLikeClient;
}
/**
* Column names, so this store fits a schema that already has conventions.
* Every one defaults to the name in the `CREATE TABLE` above.
*/
export interface PgVectorColumns {
readonly namespace?: string;
readonly id?: string;
readonly value?: string;
readonly metadata?: string;
readonly embedding?: string;
readonly embedderFp?: string;
readonly version?: string;
readonly createdAt?: string;
readonly updatedAt?: string;
readonly lastAccessedAt?: string;
readonly accessCount?: string;
readonly ttl?: string;
readonly tier?: string;
readonly source?: string;
readonly embeddingModel?: string;
}
export interface PgVectorStoreOptions {
/**
* Postgres connection string, used only when this store builds its own pool.
* Prefer `client` — a second pool to one database is a second set of
* connections nobody counted.
*/
readonly connectionString?: string;
/** A pre-built `pg.Pool` (or anything with `query`). Recommended. */
readonly client?: PgLikeClient;
/** Schema the tables live in. Default `'public'`. */
readonly schema?: string;
/** The vectors table. Default `'af_vectors'`. */
readonly table?: string;
/** The recognition-set table (`seen`/`recordSignature`). Default `'af_signatures'`. */
readonly signaturesTable?: string;
/** The usefulness-aggregate table. Default `'af_feedback'`. */
readonly feedbackTable?: string;
/** The key/value table holding one embedder fingerprint per namespace. Default `'af_index_meta'`. */
readonly metaTable?: string;
/** Column names inside {@link table}. Each defaults to the documented one. */
readonly columns?: PgVectorColumns;
/**
* Rows per multi-row upsert. Default 500.
*
* Postgres caps a statement at 65,535 bound parameters and this store binds
* 15 per row, so 500 leaves an order of magnitude of headroom. Raising it
* trades round-trips for a statement that fails all-or-nothing on a bigger
* unit.
*/
readonly batchSize?: number;
/** @internal Test injection — skips the `pg` require entirely. */
readonly _client?: PgLikeClient;
/** @internal Test injection — the `pg` module (exercises the real shim with a mock module). */
readonly _pg?: PgSdkModule;
}
/** A Postgres-backed vector store, plus the two things it owns beyond the port. */
export interface PgVectorStore extends MemoryStore {
/**
* The embedder fingerprint (`'<id>@<dims>'`) a namespace was built with, or
* `undefined` when nothing with a vector has been written to it yet.
*
* Read this before an embedder swap: it is the fact `EmbedderMismatchError`
* refuses on, available up front instead of at the first failed write.
*/
fingerprintOf(identity: MemoryIdentity): Promise<string | undefined>;
/**
* Release the pool, if this store built one. Idempotent. A client you passed
* in is yours and is left alone.
*/
close(): Promise<void>;
}
/**
* Raised when the database is reachable but its schema is not this store's.
*
* The law `sqliteVectorStore` states for a file, one backend over: **an
* unreadable index and an empty one are different facts, and only one of them
* is safe to answer with "no matches".** A store that treated a missing table
* as an empty corpus would answer every question from the model's own weights
* and log nothing.
*/
export declare class PgVectorSchemaError extends Error {
readonly code: "ERR_PGVECTOR_SCHEMA";
/** The schema-qualified table that could not be used. */
readonly table: string;
/** Columns this store needs and did not find. Empty when the table is absent entirely. */
readonly missingColumns: readonly string[];
constructor(table: string, missingColumns: readonly string[], detail: string);
}
/**
* Open a `MemoryStore` over an existing Postgres + pgvector table.
*
* @throws when `pg` is absent and no `client` was passed.
* @throws PgVectorSchemaError on the first call, when the table or a column it
* needs is not there.
* @throws EmbedderMismatchError from `put`/`putMany`/`search` when a vector
* meets a namespace built by a different embedder.
*
* @example A corpus beside the application's own data
* ```ts
* import { Pool } from 'pg';
* import { defineRAG, indexDocuments } from 'agentfootprint';
* import { pgVectorStore } from 'agentfootprint/memory';
* import { openaiEmbedder } from 'agentfootprint/providers';
*
* const store = pgVectorStore({ client: new Pool({ connectionString: process.env.DATABASE_URL }) });
* const embedder = openaiEmbedder();
*
* await indexDocuments(store, embedder, docs, { embedderId: embedder.id });
*
* const agent = Agent.create({ provider })
* .rag(defineRAG({ id: 'docs', store, embedder, embedderId: embedder.id }))
* .build();
* ```
*/
export declare function pgVectorStore(options?: PgVectorStoreOptions): PgVectorStore;