@airsurfer09/web-handsfree

# @convai/web-sdk The convai/web-sdk helps power characters in websites (brain) and the communication UI with Convai characters. This SDK facilitates the capture of user audio streams and provides appropriate responses in the form of audio, actions, and facial expressions. ## Installation ```bash npm install @convai/web-sdk ``` ## Quick Start ```tsx import { useConvaiClient, RTCWidget } from "@convai/web-sdk"; function App() { const convaiClient = useConvaiClient(); return ( <RTCWidget convaiClient={convaiClient} apiKey="your-convai-api-key" characterId="your-character-id" enableVideo={true} startWithVideoOn={false} enableAudio={true} /> ); } ``` ## Get Convai Credentials 1. Visit [convai.com](https://convai.com) and create an account 2. Navigate to your dashboard 3. Create a new character or use an existing one 4. Copy your **API Key** from the dashboard 5. Copy your **Character ID** from the character details ## API Reference ### useConvaiClient() Main hook for managing Convai connections. ```tsx const convaiClient = useConvaiClient(); ``` **Returns:** ```tsx { state: ConvaiClientState; // Connection and activity state connect: (config) => Promise<void>; // Connect to Convai disconnect: () => Promise<void>; // Disconnect from Convai resetSession: () => void; // Reset the session ID room: Room; // Internal room instance sendUserTextMessage: (text) => void; // Send text message sendTriggerMessage: (name?, msg?) => void; // Send trigger message updateTemplateKeys: (keys) => void; // Update template keys updateDynamicInfo: (info) => void; // Update dynamic info chatMessages: ChatMessage[]; // Message history audioControls: AudioControls; // Audio control methods videoControls: VideoControls; // Video control methods screenShareControls: ScreenShareControls; // Screen share controls toggleTts: (enabled) => void; // Toggle text-to-speech } ``` **State Object:** ```tsx interface ConvaiClientState { isConnected: boolean; // Connected to Convai isConnecting: boolean; // Connection in progress isListening: boolean; // Listening to user isThinking: boolean; // Processing response isSpeaking: boolean; // Speaking to user agentState: | "disconnected" | "connected" | "listening" | "thinking" | "speaking"; } ``` **Audio Controls:** ```tsx audioControls: { isAudioMuted: boolean; toggleAudio: () => Promise<void>; muteAudio: () => Promise<void>; unmuteAudio: () => Promise<void>; } ``` **Video Controls:** ```tsx videoControls: { isVideoEnabled: boolean; enableVideo: () => Promise<void>; disableVideo: () => Promise<void>; } ``` **Screen Share Controls:** ```tsx screenShareControls: { isScreenShareActive: boolean; toggleScreenShare: () => Promise<void>; } ``` ### RTCWidget Component Complete all-in-one chat widget with automatic connection, voice mode, and video support. **Features:** - Collapsed circular button that expands on click - Auto-connects on first interaction - Text and voice mode support - Video controls with floating video window - Screen sharing capability ```tsx <RTCWidget convaiClient={convaiClient} apiKey="your-api-key" characterId="your-character-id" enableVideo={true} startWithVideoOn={false} enableAudio={true} url="https://your-custom-url.com" /> ``` **Props:** | Prop | Type | Required | Default | Description | | ------------------ | -------------- | -------- | ------- | --------------------------------------------------- | | `convaiClient` | `ConvaiClient` | Yes | - | ConvaiClient instance from useConvaiClient() | | `apiKey` | `string` | Yes | - | Your Convai API key | | `characterId` | `string` | No | - | Character ID to connect to | | `enableVideo` | `boolean` | No | `false` | Enable video capability (shows video toggle button) | | `startWithVideoOn` | `boolean` | No | `false` | Start with camera on when connecting | | `enableAudio` | `boolean` | No | `true` | Enable audio | | `url` | `string` | No | - | Custom Convai API URL | ## Configuration **Config Interface:** ```tsx interface ConvaiConfig { apiKey: string; // Required: Your Convai API key characterId: string; // Required: Character ID url?: string; // Optional: Custom API URL enableVideo?: boolean; // Optional: Enable video (default: false) enableAudio?: boolean; // Optional: Enable audio (default: true) actionConfig?: ActionConfig; // Optional: Action configuration } ``` ## Advanced Usage ### Sending Messages ```tsx // Send text message convaiClient.sendUserTextMessage("Hello!"); // Send trigger message convaiClient.sendTriggerMessage("greet", "Custom greeting"); // Update template keys convaiClient.updateTemplateKeys({ user_name: "John", location: "New York", }); // Update dynamic info convaiClient.updateDynamicInfo({ text: "Additional context for the character", }); ``` ### Control Audio/Video ```tsx // Toggle microphone await convaiClient.audioControls.toggleAudio(); // Mute/unmute await convaiClient.audioControls.muteAudio(); await convaiClient.audioControls.unmuteAudio(); // Enable/disable video await convaiClient.videoControls.enableVideo(); await convaiClient.videoControls.disableVideo(); // Toggle screen share await convaiClient.screenShareControls.toggleScreenShare(); ``` ### Session Management ```tsx // Reset session (clears history) convaiClient.resetSession(); // Disconnect await convaiClient.disconnect(); // Reconnect with new session await convaiClient.connect(config); ``` ## TypeScript Support The SDK is written in TypeScript and provides full type definitions. Import types as needed: ```tsx import { ConvaiClient, ConvaiConfig, ConvaiClientState, ChatMessage, } from "@convai/web-sdk"; ``` ## License MIT