@cloudkinetix/bmad-enhanced
Version:
Cloud-Kinetix enhanced fork of BMAD-METHOD - Breakthrough Method of Agile AI-driven Development with robust versioning and unified validation.
521 lines (442 loc) • 12 kB
Markdown
# {{Agent Name}} Voice Agent Configuration
[[LLM: Initial Setup
1. Replace {{Agent Name}} with the actual agent name
2. Choose voice platform (Gemini Live, OpenAI Realtime, ElevenLabs, etc.)
3. Define voice interaction patterns and personas
4. Configure audio processing pipeline
5. Set up multimodal capabilities
Output file location: `config/voice-agents/{{agent-name}}-voice-config.md`]]
## Voice Agent Overview
[[LLM: Provide a comprehensive overview of the voice agent's purpose, capabilities, and user experience goals.]]
- **Agent Name**: {{agent-name}}
- **Voice Platform**: {{Gemini-Live/OpenAI-Realtime/Custom}}
- **Interaction Mode**: {{Voice-only/Voice+Text/Multimodal}}
- **Primary Use Case**: {{education/customer-service/companion}}
- **Target Audience**: {{user-demographics}}
## Voice Configuration
### Voice Profile
[[LLM: Define the agent's voice characteristics and personality.]]
#### Voice Selection
- **Voice Model**: {{voice-id-or-name}}
- **Gender**: {{male/female/neutral}}
- **Age Range**: {{young/middle-aged/mature}}
- **Accent/Language**: {{en-US/en-GB/other}}
- **Speaking Style**: {{formal/casual/friendly/professional}}
#### Prosody Settings
````yaml
voice_parameters:
speaking_rate:
'[object Object]': null
pitch:
'[object Object]': null
volume:
'[object Object]': null
emphasis_level:
'[object Object]': null
emotion_settings:
default_emotion:
'[object Object]': null
emotion_range:
'[object Object]': null
context_awareness:
'[object Object]': null
```text
### Audio Processing
#### Input Configuration
```yaml
audio_input:
sample_rate:
'[object Object]': null
encoding:
'[object Object]': null
channels:
'[object Object]': null
preprocessing:
noise_reduction:
'[object Object]': null
echo_cancellation:
'[object Object]': null
automatic_gain_control:
'[object Object]': null
voice_activity_detection:
enabled: true
sensitivity:
'[object Object]': null
silence_threshold:
'[object Object]': null
````
#### Output Configuration
````yaml
audio_output:
format:
'[object Object]': null
sample_rate:
'[object Object]': null
bitrate:
'[object Object]': null
enhancements:
dynamic_range_compression:
'[object Object]': null
equalization_preset:
'[object Object]': null
spatial_audio:
'[object Object]': null
```text
## Conversation Design
### Turn-Taking Management
[[LLM: Define how the agent handles conversation flow in voice interactions.]]
#### Interruption Handling
```yaml
interruption_policy:
allow_interruptions:
'[object Object]': null
interruption_threshold:
'[object Object]': null
on_interruption:
- pause_immediately:
'[object Object]': null
- save_context: true
- acknowledgment: '{{I heard you, go ahead}}'
- resume_strategy:
'[object Object]': null
````
#### Silence Management
````yaml
silence_handling:
end_of_turn_silence:
'[object Object]': null
thinking_silence_filler:
'[object Object]': null
filler_phrases:
- Let me think about that...
- Hmm, that's interesting...
- Give me a moment...
max_silence_before_prompt:
'[object Object]': null
silence_prompt: '{{Are you still there?}}'
```text
### Voice-Specific Prompting
#### System Prompt for Voice
````
You are {{agent-name}}, a voice-based LLM assistant. Your responses should be:
1. **Conversational**: Use natural speech patterns, contractions, and casual language
2. **Concise**: Keep responses brief and to the point for voice interaction
3. **Clear**: Avoid complex sentences or jargon that's hard to understand when heard
4. **Interactive**: Ask clarifying questions and confirm understanding
5. **Contextual**: Remember this is a voice conversation, reference sounds or verbal cues
Special instructions:
- {{voice-specific-behavior-guidelines}}
- {{personality-traits-for-voice}}
- {{domain-specific-knowledge}}
Remember: Users can't see text, so spell out acronyms, provide verbal descriptions, and use voice-appropriate formatting.
````text
#### Response Formatting
```yaml
voice_formatting:
numbers: spell_out_below_10
urls: provide_verbal_description
lists: use_verbal_markers
emphasis: use_prosody_not_caps
code_handling:
strategy:
'[object Object]': null
explanation_style:
'[object Object]': null
````
## Multimodal Capabilities
### Voice + Visual Integration
[[LLM: For agents that combine voice with visual elements.]]
````yaml
multimodal_config:
screen_sharing:
enabled:
'[object Object]': null
annotations:
'[object Object]': null
pointer_control:
'[object Object]': null
visual_references:
description_style:
'[object Object]': null
spatial_awareness:
'[object Object]': null
gesture_recognition:
'[object Object]': null
document_interaction:
read_aloud:
'[object Object]': null
summarization:
'[object Object]': null
navigation_commands:
- next
- previous
- go to
```text
### Context Switching
```yaml
modality_switching:
voice_to_text:
trigger:
'[object Object]': null
preservation: maintain_conversation_history
notification: '{{Switching to text mode}}'
text_to_voice:
trigger:
'[object Object]': null
warm_start: true
greeting: '{{Let''s continue our conversation}}'
````
## Educational Features (Example: Tutor Agent)
### Learning Interaction Patterns
[[LLM: Define specialized features for educational voice agents.]]
````yaml
educational_config:
questioning_strategies:
socratic_method: true
wait_time_after_question:
'[object Object]': null
encouragement_phrases:
- Take your time thinking about it
- That's a great start, what else?
- You're on the right track
feedback_delivery:
positive_reinforcement: immediate
correction_style: gentle_guidance
explanation_depth: adaptive
pacing:
adaptive_speed: true
comprehension_checks: true
break_reminders: every_20_minutes
```text
### Progress Tracking
```yaml
learning_metrics:
track_understanding: true
vocabulary_growth: true
concept_mastery: true
verbal_assessments:
quiz_format: conversational
self_assessment_prompts: true
progress_summaries: daily
````
## Platform-Specific Configuration
### Gemini Live Integration
[[LLM: Specific configuration for Gemini Live API.]]
````python
# Gemini Live configuration example
gemini_config = {
"model": "gemini-live",
"voice_settings": {
"voice_name": "{{voice-selection}}",
"streaming": True,
"audio_config": {
"encoding": "LINEAR16",
"sample_rate_hertz": 16000
}
},
"interaction_config": {
"enable_interruptions": True,
"live_transcription": True,
"emotion_detection": True
}
}
```text
### OpenAI Realtime API
```python
# OpenAI Realtime configuration
openai_config = {
"model": "gpt-4-realtime",
"voice": "{{alloy/echo/fable/onyx/nova/shimmer}}",
"response_format": {
"type": "audio",
"voice_settings": {
"speed": {{0.25-4.0}},
"temperature": {{0-1}}
}
}
}
````
## Safety and Moderation
### Voice-Specific Safety
[[LLM: Address unique safety concerns for voice interactions.]]
````yaml
voice_safety:
content_filtering:
profanity_filter:
'[object Object]': null
inappropriate_content: block_and_redirect
impersonation_prevention:
verify_ai_identity: true
disclosure_frequency: every_session
cannot_pretend_to_be:
- human
- specific_person
- authority
emotional_safety:
detect_distress: true
emergency_protocols: true
supportive_responses: true
```text
### Privacy Protection
```yaml
privacy_config:
audio_retention:
store_audio:
'[object Object]': null
retention_period:
'[object Object]': null
transcription_handling:
store_transcripts:
'[object Object]': null
anonymization: true
pii_detection: true
voice_biometrics:
collect_voiceprint: false
speaker_identification: disabled
````
## Performance Optimization
### Latency Optimization
[[LLM: Configure for optimal voice interaction latency.]]
````yaml
performance_tuning:
streaming_config:
chunk_size:
'[object Object]': null
buffer_size:
'[object Object]': null
prefetching:
common_responses: true
contextual_predictions: true
caching:
tts_cache: true
response_cache: true
cache_ttl:
'[object Object]': null
```text
### Resource Management
```yaml
resource_limits:
concurrent_sessions:
'[object Object]': null
session_timeout:
'[object Object]': null
compute_allocation:
cpu_cores:
'[object Object]': null
memory:
'[object Object]': null
gpu:
'[object Object]': null
bandwidth_management:
audio_codec:
'[object Object]': null
adaptive_bitrate: true
minimum_quality:
'[object Object]': null
````
## Integration Examples
### Web Application Integration
````javascript
// Browser-based voice agent integration
class VoiceAgent {
constructor(config) {
this.config = config;
this.mediaStream = null;
this.recognition = new webkitSpeechRecognition();
this.synthesis = window.speechSynthesis;
}
async startConversation() {
// Initialize audio context
const audioContext = new AudioContext();
// Configure speech recognition
this.recognition.continuous = true;
this.recognition.interimResults = true;
// Start listening
this.recognition.start();
}
}
```text
### Mobile App Integration
```swift
// iOS voice agent example
class VoiceAgentManager {
let speechRecognizer = SFSpeechRecognizer()
let audioEngine = AVAudioEngine()
func configureVoiceInteraction() {
// Configure audio session
let audioSession = AVAudioSession.sharedInstance()
try audioSession.setCategory(.playAndRecord)
// Set up voice processing
// ... implementation details
}
}
````
## Testing and Validation
### Voice Quality Testing
````yaml
test_scenarios:
acoustic_conditions:
- quiet_room
- background_noise
- echo_environment
- multiple_speakers
speech_variations:
- different_accents
- speech_impediments
- children_voices
- elderly_speakers
interaction_patterns:
- rapid_exchanges
- long_pauses
- interruptions
- parallel_talking
```text
### Performance Benchmarks
- **First Token Latency**: < {{200ms}}
- **End-to-End Latency**: < {{500ms}}
- **Speech Recognition Accuracy**: > {{95%}}
- **Natural Conversation Flow**: {{subjective-rating}}
## Monitoring and Analytics
### Voice-Specific Metrics
```yaml
voice_analytics:
conversation_metrics:
- average_turn_duration
- interruption_rate
- silence_ratio
- conversation_completion_rate
quality_metrics:
- audio_quality_score
- recognition_confidence
- tts_naturalness_rating
- user_satisfaction_score
technical_metrics:
- audio_packet_loss
- jitter_measurements
- bandwidth_usage
- processing_latency
````
### User Feedback Collection
```yaml
feedback_system:
implicit_signals:
- conversation_length
- return_rate
- task_completion
explicit_feedback:
- voice_quality_rating
- understanding_rating
- helpfulness_score
- would_recommend
```
## Deployment Checklist
- [ ] Voice model selected and tested
- [ ] Audio pipeline configured
- [ ] Latency targets met
- [ ] Safety measures implemented
- [ ] Privacy compliance verified
- [ ] Integration tested across platforms
- [ ] Monitoring dashboard configured
- [ ] Feedback system active
- [ ] Documentation complete
- [ ] User training materials ready