@mastra/core
Version:
129 lines (86 loc) • 7.7 kB
Markdown
> Discover all available pages from the documentation index: https://mastra.ai/llms.txt
# PubSub
Mastra uses a publish/subscribe (pub/sub) system as its internal event bus. Components publish events to topics, and other components subscribe to those topics to react. The backend you configure decides how far those events travel: within one process or across processes on one host, or alternatively across separate instances.
You set the backend once on the `Mastra` instance, and the rest of the system uses it without changes. By default, Mastra uses an in-process backend that needs no setup.
## How Mastra uses PubSub
Several built-in systems publish and subscribe to events through the same pub/sub bus:
- **Workflow execution**: The scheduler publishes a `workflow.start` event, and a long-lived worker consumes it to run the workflow. Step and lifecycle events flow over pub/sub during execution.
- **Scheduled workflows**: The scheduler dispatches due runs by publishing to the workflow topic. See [Scheduled workflows](https://mastra.ai/docs/workflows/scheduled-workflows).
- **Background tasks**: A task manager dispatches work to a worker group and fans task lifecycle updates out to subscribers, which is how background task streams stay live. See [Background task streaming](https://mastra.ai/docs/long-running-agents/background-tasks).
- **Agent signals**: Sending a signal to an active agent run publishes an event on a thread topic, so a run executing in another process receives the signal.
- **Resumable streams**: Stream chunks are published per run, so a client that reconnects can replay what it missed.
Because these systems run on one bus, the backend you choose applies to all of them at once.
## Delivery modes
Backends deliver events in one of two modes, defined by the [`PubSub`](https://mastra.ai/reference/pubsub/base) contract:
- **Pull**: Consumers read from the backend on their own, which Mastra does with a long-lived worker loop. Distributed backends such as [`RedisStreamsPubSub`](https://mastra.ai/reference/pubsub/redis-streams) use this mode.
- **Push**: Events arrive without the consumer asking, either in process or over HTTP. The default [`EventEmitterPubSub`](https://mastra.ai/reference/pubsub/event-emitter) delivers this way in process.
Subscribers can also distribute work through consumer groups. Members of the same group split the events so each event is handled once. A subscriber without a group receives every event, which fans the stream out to all ungrouped subscribers.
## Default backend
When you don't set the `pubsub` option, Mastra uses [`EventEmitterPubSub`](https://mastra.ai/reference/pubsub/event-emitter). It delivers events in process using a Node.js [`EventEmitter`](https://nodejs.org/api/events.html#class-eventemitter), so it works without any external service.
```typescript
import { Mastra } from '@mastra/core'
// No pubsub option: Mastra uses EventEmitterPubSub
export const mastra = new Mastra({})
```
Because it runs in process, events aren't persisted and don't reach other processes. The default suits a single instance, which covers most applications.
## Choosing a backend
Set the `pubsub` option on the `Mastra` instance to choose a backend. Each backend implements the same [`PubSub`](https://mastra.ai/reference/pubsub/base) contract, so the rest of your application doesn't change.
The deciding question is where the subscriber runs.
| Backend | Scope | Mode | Package |
| ----------------------------------------------------------------------------- | ---------------------------- | ------------- | ----------------------------- |
| [`EventEmitterPubSub`](https://mastra.ai/reference/pubsub/event-emitter) | Single process | Pull and push | `@mastra/core` |
| [`UnixSocketPubSub`](https://mastra.ai/reference/pubsub/unix-socket-pubsub) | Multiple processes, one host | Push | `@mastra/core` |
| [`RedisStreamsPubSub`](https://mastra.ai/reference/pubsub/redis-streams) | Distributed, multiple hosts | Pull | `@mastra/redis-streams` |
| [`GoogleCloudPubSub`](https://mastra.ai/reference/pubsub/google-cloud-pubsub) | Distributed, multiple hosts | Pull | `@mastra/google-cloud-pubsub` |
### Multiple processes on one host
Use [`UnixSocketPubSub`](https://mastra.ai/reference/pubsub/unix-socket-pubsub) when several processes on the same machine need to share a stream. It delivers events over a Unix domain socket and elects one process as a broker. If the broker exits, the remaining processes elect a new one.
```typescript
import { Mastra } from '@mastra/core'
import { UnixSocketPubSub } from '@mastra/core/events'
export const mastra = new Mastra({
pubsub: new UnixSocketPubSub('/tmp/mastra/events.sock'),
})
```
### Distributed deployments
Use a distributed backend when you run more than one instance or host, so every instance receives the same events. This matters whenever a request handled by one instance must reach work running on another.
For example, sending a signal to an agent run requires the signal event to cross the process boundary to the instance that owns the run. With the in-process default, that instance never receives the event.
Both backends below deliver across processes and hosts and persist events for redelivery.
The following example uses [Redis Streams](https://redis.io/docs/latest/develop/data-types/streams/):
```typescript
import { Mastra } from '@mastra/core'
import { RedisStreamsPubSub } from '@mastra/redis-streams'
export const mastra = new Mastra({
pubsub: new RedisStreamsPubSub({
url: 'redis://localhost:6379',
}),
})
```
The following example uses [Google Cloud Pub/Sub](https://cloud.google.com/pubsub/docs):
```typescript
import { Mastra } from '@mastra/core'
import { GoogleCloudPubSub } from '@mastra/google-cloud-pubsub'
export const mastra = new Mastra({
pubsub: new GoogleCloudPubSub({
projectId: 'my-project',
}),
})
```
## Resumable streams
Resumable streams let a client reconnect and replay events it missed, which requires the backend to keep recent history. Distributed backends such as [`RedisStreamsPubSub`](https://mastra.ai/reference/pubsub/redis-streams) persist events, so they support replay on their own.
In-process delivery doesn't keep history. To add replay on top of [`EventEmitterPubSub`](https://mastra.ai/reference/pubsub/event-emitter), wrap it in [`CachingPubSub`](https://mastra.ai/reference/pubsub/caching-pubsub), which records published events per topic so a late or reconnecting subscriber can catch up before continuing with live events.
```typescript
import { Mastra } from '@mastra/core'
import { CachingPubSub, EventEmitterPubSub } from '@mastra/core/events'
import { InMemoryServerCache } from '@mastra/core/cache'
const cache = new InMemoryServerCache()
export const mastra = new Mastra({
pubsub: new CachingPubSub(new EventEmitterPubSub(), cache),
})
```
Visit the [PubSub reference](https://mastra.ai/reference/pubsub/base) for the full delivery contract and the configuration options for each backend.
## Related
- [PubSub reference](https://mastra.ai/reference/pubsub/base)
- [Mastra class](https://mastra.ai/reference/core/mastra-class)
- [Workers](https://mastra.ai/docs/deployment/workers): Run workflow orchestration and background tasks in dedicated processes using PubSub
- [Background task streaming](https://mastra.ai/docs/long-running-agents/background-tasks)
- [Scheduled workflows](https://mastra.ai/docs/workflows/scheduled-workflows)