@mastra/core
Version:
Mastra is a framework for building AI-powered applications and agents with a modern TypeScript stack.
245 lines (177 loc) • 9.7 kB
Markdown
# Composite storage
`MastraCompositeStore` can compose storage domains from different providers. Use it when you need different databases for different purposes. For example, use LibSQL for memory and PostgreSQL for workflows.
## Installation
`MastraCompositeStore` is included in `/core`:
**npm**:
```bash
npm install /core
```
**pnpm**:
```bash
pnpm add /core
```
**Yarn**:
```bash
yarn add /core
```
**Bun**:
```bash
bun add /core
```
You'll also need to install the storage providers you want to compose:
**npm**:
```bash
npm install /pg@latest @mastra/libsql
```
**pnpm**:
```bash
pnpm add /pg@latest @mastra/libsql
```
**Yarn**:
```bash
yarn add /pg@latest @mastra/libsql
```
**Bun**:
```bash
bun add /pg@latest @mastra/libsql
```
## Storage domains
Mastra organizes storage into domains, each handling a specific type of data. Each domain can be backed by a different storage adapter, and domain classes are exported from each storage package.
| Domain | Description |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `memory` | Conversation persistence for agents. Stores threads (conversation sessions), messages, resources (user identities), and working memory (persistent context across conversations). |
| `workflows` | Workflow execution state. When workflows suspend for human input, external events, or scheduled resumption, their state is persisted here to enable resumption after server restarts. |
| `scores` | Evaluation results from Mastra's evals system. Scores and metrics are persisted here for analysis and comparison over time. |
| `observability` | Telemetry data including traces and spans. Agent interactions, tool calls, and LLM requests generate spans collected into traces for debugging and performance analysis. |
| `agents` | Agent configurations for stored agents. Enables agents to be defined and updated at runtime without code deployments. |
| `datasets` | Evaluation datasets used for experiment runs. Stores dataset definitions, schemas, and versioned items. |
| `experiments` | Experiment runs and per-item experiment results linked to datasets and targets. |
> **Note:** `MastraCompositeStore` accepts all of the domain keys above, but storage adapter support varies by package. You can mix adapters per domain, but only for domains implemented and exported by those adapters. For example, `memory: new MemoryLibSQL(...)` and `workflows: new WorkflowsPG(...)` is valid because both packages export those domain classes.
## Usage
### Basic composition
Import domain classes directly from each store package and compose them:
```typescript
import { MastraCompositeStore } from '@mastra/core/storage'
import { WorkflowsPG, ScoresPG } from '@mastra/pg'
import { MemoryLibSQL } from '@mastra/libsql'
import { Mastra } from '@mastra/core'
export const mastra = new Mastra({
storage: new MastraCompositeStore({
id: 'composite',
domains: {
memory: new MemoryLibSQL({ url: 'file:./local.db' }),
workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
},
}),
})
```
### With a default storage
Use `default` to specify a fallback storage, then override specific domains:
```typescript
import { MastraCompositeStore } from '@mastra/core/storage'
import { PostgresStore } from '@mastra/pg'
import { MemoryLibSQL } from '@mastra/libsql'
import { Mastra } from '@mastra/core'
const pgStore = new PostgresStore({
id: 'pg',
connectionString: process.env.DATABASE_URL,
})
export const mastra = new Mastra({
storage: new MastraCompositeStore({
id: 'composite',
default: pgStore,
domains: {
memory: new MemoryLibSQL({ url: 'file:./local.db' }),
},
}),
})
```
## Options
**id** (`string`): Unique identifier for this storage instance.
**default** (`MastraCompositeStore`): Default storage adapter. Domains not explicitly specified in \`domains\` will use this storage's domains as fallbacks.
**disableInit** (`boolean`): When true, automatic initialization is disabled. You must call init() explicitly.
**domains** (`object`): Individual domain overrides. Each domain can come from a different storage adapter. These take precedence over both \`editor\` and \`default\` storage.
**domains.memory** (`MemoryStorage`): Storage for threads, messages, and resources.
**domains.workflows** (`WorkflowsStorage`): Storage for workflow snapshots.
**domains.scores** (`ScoresStorage`): Storage for evaluation scores.
**domains.observability** (`ObservabilityStorage`): Storage for traces and spans.
**domains.agents** (`AgentsStorage`): Storage for stored agent configurations.
**domains.datasets** (`DatasetsStorage`): Storage for dataset metadata, dataset items, and dataset versions.
**domains.experiments** (`ExperimentsStorage`): Storage for experiment runs and per-item experiment results.
## Initialization
`MastraCompositeStore` initializes each configured domain independently. When passed to the Mastra class, `init()` is called automatically:
```typescript
import { MastraCompositeStore } from '@mastra/core/storage'
import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
import { Mastra } from '@mastra/core'
const storage = new MastraCompositeStore({
id: 'composite',
domains: {
memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
},
})
export const mastra = new Mastra({
storage, // init() called automatically
})
```
If using storage directly, call `init()` explicitly:
```typescript
import { MastraCompositeStore } from '@mastra/core/storage'
import { MemoryPG } from '@mastra/pg'
const storage = new MastraCompositeStore({
id: 'composite',
domains: {
memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
},
})
await storage.init()
// Access domain-specific stores via getStore()
const memoryStore = await storage.getStore('memory')
const thread = await memoryStore?.getThreadById({ threadId: '...' })
```
## Use cases
### Separate databases for different workloads
Use a local database for development while keeping production data in a managed service:
```typescript
import { MastraCompositeStore } from '@mastra/core/storage'
import { MemoryPG, WorkflowsPG, ScoresPG } from '@mastra/pg'
import { MemoryLibSQL } from '@mastra/libsql'
const storage = new MastraCompositeStore({
id: 'composite',
domains: {
// Use local SQLite for development, PostgreSQL for production
memory:
process.env.NODE_ENV === 'development'
? new MemoryLibSQL({ url: 'file:./dev.db' })
: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
},
})
```
### Specialized storage for observability
Observability data can quickly overwhelm general-purpose databases in production. A single agent interaction can generate hundreds of spans, and high-traffic applications can produce thousands of traces per day.
**[ClickHouse](https://mastra.ai/reference/storage/clickhouse)** is recommended for production observability because it's optimized for high-volume, write-heavy analytics workloads. Use composite storage to route observability to ClickHouse while keeping other data in your primary database:
```typescript
import { MastraCompositeStore } from '/core/storage'
import { MemoryPG, WorkflowsPG, ScoresPG } from '/pg'
import { ObservabilityStorageClickhouseVNext } from '/clickhouse'
const storage = new MastraCompositeStore({
id: 'composite',
domains: {
memory: new MemoryPG({ connectionString: process.env.DATABASE_URL }),
workflows: new WorkflowsPG({ connectionString: process.env.DATABASE_URL }),
scores: new ScoresPG({ connectionString: process.env.DATABASE_URL }),
observability: new ObservabilityStorageClickhouseVNext({
url: process.env.CLICKHOUSE_URL,
username: process.env.CLICKHOUSE_USERNAME,
password: process.env.CLICKHOUSE_PASSWORD,
}),
},
})
```
> **Note:** `ObservabilityStorageClickhouseVNext` is the current observability domain implementation. The legacy `ObservabilityStorageClickhouse` class is also exported and remains supported for projects that have not migrated. See the [ClickHouse storage reference](https://mastra.ai/reference/storage/clickhouse) for details.
> **Info:** This approach is also required when using storage providers that don't support observability (like Convex, DynamoDB, or Cloudflare). See the [MastraStorageExporter documentation](https://mastra.ai/docs/observability/tracing/exporters/mastra-storage) for the full list of supported providers.