@mastra/core
Version:
Mastra is a framework for building AI-powered applications and agents with a modern TypeScript stack.
96 lines (74 loc) • 3.4 kB
Markdown
# voice.connect()
The `connect()` method establishes a WebSocket or WebRTC connection for real-time speech-to-speech communication. This method must be called before using other real-time features like `send()` or `answer()`.
## Usage example
```typescript
import { OpenAIRealtimeVoice } from '@mastra/voice-openai-realtime'
import Speaker from '@mastra/node-speaker'
const speaker = new Speaker({
sampleRate: 24100, // Audio sample rate in Hz - standard for high-quality audio on MacBook Pro
channels: 1, // Mono audio output (as opposed to stereo which would be 2)
bitDepth: 16, // Bit depth for audio quality - CD quality standard (16-bit resolution)
})
// Initialize a real-time voice provider
const voice = new OpenAIRealtimeVoice({
realtimeConfig: {
model: 'gpt-5.1-realtime',
apiKey: process.env.OPENAI_API_KEY,
options: {
sessionConfig: {
turn_detection: {
type: 'server_vad',
threshold: 0.6,
silence_duration_ms: 1200,
},
},
},
},
speaker: 'alloy', // Default voice
})
// Connect to the real-time service
await voice.connect()
// Now you can use real-time features
voice.on('speaker', stream => {
stream.pipe(speaker)
})
// With connection options
await voice.connect({
timeout: 10000, // 10 seconds timeout
reconnect: true,
})
```
## Parameters
**options** (`Record<string, unknown>`): Provider-specific connection options
## Return value
Returns a `Promise<void>` that resolves when the connection is successfully established.
## Provider-specific options
Each real-time voice provider may support different options for the `connect()` method:
### OpenAI Realtime
**options** (`Options`): Configuration options.
**options.timeout** (`number`): Connection timeout in milliseconds
**options.reconnect** (`boolean`): Whether to automatically reconnect on connection loss
## Using with `CompositeVoice`
When using `CompositeVoice`, the `connect()` method delegates to the configured real-time provider:
```typescript
import { CompositeVoice } from '@mastra/core/voice'
import { OpenAIRealtimeVoice } from '@mastra/voice-openai-realtime'
const realtimeVoice = new OpenAIRealtimeVoice()
const voice = new CompositeVoice({
realtime: realtimeVoice,
})
// This will use the OpenAIRealtimeVoice provider
await voice.connect()
```
## Notes
- This method is only implemented by real-time voice providers that support speech-to-speech capabilities
- If called on a voice provider that doesn't support this functionality, it will log a warning and resolve immediately
- The connection must be established before using other real-time methods like `send()` or `answer()`
- When you're done with the voice instance, call `close()` to properly clean up resources
- Some providers may automatically reconnect on connection loss, depending on their implementation
- Connection errors will typically be thrown as exceptions that should be caught and handled
## Related methods
- [voice.send()](https://mastra.ai/reference/voice/voice.send): Sends audio data to the voice provider
- [voice.answer()](https://mastra.ai/reference/voice/voice.answer): Triggers the voice provider to respond
- [voice.close()](https://mastra.ai/reference/voice/voice.close): Disconnects from the real-time service
- [voice.on()](https://mastra.ai/reference/voice/voice.on): Registers an event listener for voice events