electron-window-rtc
Version:
Exchange media streams between Electron windows with WebRTC.
364 lines (252 loc) • 15 kB
Markdown
# electron-window-rtc
<a href='https://en.wikipedia.org/wiki/Marseille'>
<img src="assets/badges/made_with_♥_in-Marseille-cc0040.svg">
</a>
Inspired by [electron-peer-connection](https://github.com/han-gyeol/electron-peer-connection), `electron-window-rtc` is a zero-dependency package that allows sharing medias between [Electron](https://www.electronjs.org/) windows through [WebRTC](https://webrtc.org/) with (almost) zero-latency, depending on application configuration.
It works by creating a **main process** events hub that acts as a signaling server for the windows through Electron's IPC. Once windows are registered, each **renderer process** creates a `WindowRTCPeerConnection` to another window and begins to send/receive media streams.
This package was primarily released to handle video/canvas manipulation before sending it to a [window rendered offscreen](https://www.electronjs.org/docs/latest/tutorial/offscreen-rendering) passed to the [Syphon framework](https://github.com/Syphon/Syphon-Framework) with same author's [node-syphon](https://github.com/benoitlahoz/node-syphon) package.
### Donate
If you find this package useful, contribute to the author's open source work by donating here! Thank you!
[](https://www.paypal.com/donate/?hosted_button_id=C2ABZ3KBUXF92)
## Table of Contents
- [electron-window-rtc](#electron-window-rtc)
- [Donate](#donate)
- [Table of Contents](#table-of-contents)
- [Install](#install)
- [Usage](#usage)
- [Main process](#main-process)
- [Renderer process](#renderer-process)
- [Sender](#sender)
- [Receiver](#receiver)
- [API](#api)
- [Main process](#main-process-1)
- [`WindowRTCMain.register`](#windowrtcmainregister)
- [`WindowRTCMain.unregister`](#windowrtcmainunregister)
- [`WindowRTCMain.dispose`](#windowrtcmaindispose)
- [Renderer process](#renderer-process-1)
- [`defineIpc`](#defineipc)
- [`WindowRTCPeerConnection.with`](#windowrtcpeerconnectionwith)
- [`windowPeerConnectionInstance.addStream`](#windowpeerconnectioninstanceaddstream)
- [`windowPeerConnectionInstance.requestOffer`](#windowpeerconnectioninstancerequestoffer)
- [`windowPeerConnectionInstance.on`](#windowpeerconnectioninstanceon)
- [`windowPeerConnectionInstance.off`](#windowpeerconnectioninstanceoff)
- [`windowPeerConnectionInstance.dispose`](#windowpeerconnectioninstancedispose)
- [Events](#events)
- [Types](#types)
- [`IpcObject`](#ipcobject)
- [`WindowRTCEvent`](#windowrtcevent)
- [Using `canvas`](#using-canvas)
- [Setting `frameRequestRate`](#setting-framerequestrate)
- [Without setting `frameRequestRate`](#without-setting-framerequestrate)
- [Using WebAudio API](#using-webaudio-api)
- [Known Issues](#known-issues)
- [License](#license)
## Install
```sh
npm i -s electron-window-rtc
```
```sh
yarn add electron-window-rtc
```
## Usage
See [Electron+Vue example](https://github.com/benoitlahoz/electron-window-rtc/tree/main/example) for a complete integration example.
### Main process
```typescript
// WindowRTCMain is a singleton.
import { WindowRTCMain } from 'electron-window-rtc';
const senderWindow: BrowserWindow = createWindow(); // Left to application.
const receiverWindow: BrowserWindow = createWindow();
// Note that windows can both send and receive by creating two connection (see below in 'renderer process' section).
WindowRTCMain.register('sender', senderWindow);
WindowRTCMain.register('receiver', receiverWindow);
```
### Renderer process
#### Sender
```typescript
import {
WindowRTCPeerConnection,
defineIpc,
} from 'electron-window-rtc/renderer';
// Important: define early how to access to IPC object, according to application 'preload' script.
// The IPC object must at least expose `on`, `removeListener`, `send` and `invoke` methods (see IPCObject below).
defineIpc(window.electron.ipcRenderer);
document.addEventListener('DOMContentLoaded', (event) => {
const windowConnection = new WindowRTCPeerConnection('receiver');
// Canvas for example...
const canvas: HTMLCanvasElement = document.getElementById('canvas');
// Add track from canvas: this will create an 'offer' for 'receiver' window.
// Note the '240' fps framerate: leaving it empty creates latency in the receiver.
windowConnection.addStream(canvas.captureStream(240));
});
```
#### Receiver
```typescript
import {
WindowRTCPeerConnection,
defineIpc,
} from 'electron-window-rtc/renderer';
// Important: define early how to access to IPC object, according to application 'preload' script.
// The IPC object must at least expose `on`, `removeListener`, `send` and `invoke` methods (see IPCObject below).
defineIpc(window.electron.ipcRenderer);
document.addEventListener('DOMContentLoaded', (event) => {
const windowConnection = new WindowRTCPeerConnection('sender');
// Listen to 'track' added by 'sender' window.
windowConnection.on('track', (event: EventManagerDTO) => {
const video: HTMLVideoElement = document.getElementById('video');
const trackEvent: RTCTrackEvent = event.payload;
const streams = trackEvent.streams;
for (const stream of streams) {
// For the sake of this example, keep only one stream, we could also get `streams[0]`.
video.srcObject = null;
video.srcObject = stream;
}
});
});
```
## API
### Main process
Import the singleton.
`import { WindowRTCMain } from 'electron-window-rtc';`
#### `WindowRTCMain.register`
Registers a `BrowserWindow` for message passing with a unique name.
**Parameters**
| Name | Type | Default | Optional |
| -------- | --------------- | ----------- | -------- |
| `name` | `string` | `undefined` | false |
| `window` | `BrowserWindow` | `undefined` | false |
**Returns** `void`
**Throws** `Error` if a window with this name or this window has already been registered.
#### `WindowRTCMain.unregister`
Unregister a `BrowserWindow` from message passing. Fails silently if window can not be found.
**Parameters**
| Name | Type | Default | Optional |
| ------ | -------- | ----------- | -------- |
| `name` | `string` | `undefined` | false |
**Returns** `void`
#### `WindowRTCMain.dispose`
Dispose the `WindowRTCMain` singleton by unregistering windows and removing `IpcMain` listeners.
**Returns** `void`
### Renderer process
`import { WindowRTCPeerConnection, defineIpc } from 'electron-window-rtc/renderer';`
#### `defineIpc`
Define the global `IpcObject` to use for thiw window for communicating with main process.
**Parameters**
| Name | Type | Default | Optional |
| ----- | ----------- | ----------- | -------- |
| `ipc` | `IpcObject` | `undefined` | false |
**Returns** `void`
#### `WindowRTCPeerConnection.with`
Create a `WindowRTCPeerConnection` with window of given name.
**Parameters**
| Name | Type | Default | Optional |
| ------ | -------- | ----------- | -------- |
| `name` | `string` | `undefined` | false |
**Returns** `Promise<WindowRTCPeerConnection>` An instance of `WindowRTCPeerConnection`.
**Throws** `Error` if `IpcObject` was not defined with `defineIpc`, or if the window with `name` was not registered, or if this window was not registered.
#### `windowPeerConnectionInstance.addStream`
Add a stream to send to connected window and create an `offer` sent to receiving window.
**Parameters**
| Name | Type | Default | Optional |
| ------------ | ----------------------------------- | ---------------------- | -------- |
| `stream` | `MediaStream` | `undefined` | false |
| `maxBitrate` | `{ audio: number; video: number; }` | `5000` (Kbps) for both | true |
**Returns** `Promise<void>`
#### `windowPeerConnectionInstance.requestOffer`
Request the peer window to send an offer.
**Returns** `Promise<void>`
#### `windowPeerConnectionInstance.on`
Register a listener to `WindowRTCPeerConnection` instance's events.
**Parameters**
| Name | Type | Default | Optional |
| ---------- | --------------------------------- | ----------- | -------- |
| `channel` | `WindowRTCEventChannel` | `undefined` | false |
| `listener` | `(event: WindowRTCEvent) => void` | `undefined` | false |
**Returns** `void`
#### `windowPeerConnectionInstance.off`
Unregister a listener or a whole channel from `WindowRTCPeerConnection` instance's events. If `listener` is left undefined, unregisters all listeners for this channel.
**Parameters**
| Name | Type | Default | Optional |
| ----------- | --------------------------------- | ----------- | -------- |
| `channel` | `WindowRTCEventChannel` | `undefined` | false |
| `listener?` | `(event: WindowRTCEvent) => void` | `undefined` | true |
**Returns** `void`
#### `windowPeerConnectionInstance.dispose`
Close the connection with the other window and remove all listeners.
**Returns** `void`
### Events
`WindowRTCEvent` (see below) sent by `WindowRTCPeerConnection` with different payloads according to the event emitted.
| Channel | Payload Type | Emitted |
| -------------------------- | --------------------------------------------------------- | ----------------------------------------------------------- |
| `*` | `any` | For each event. |
| `icecandidate` | `RTCIceCandidate` | On `icecandidate` RTCPeerConnection event. |
| `iceconnectionstatechange` | `Event` | On `iceconnectionstatechange` RTCPeerConnection event. |
| `icecandidateerror` | `RTCPeerConnectionIceErrorEvent` | On `icecandidateerror` RTCPeerConnection event. |
| `icegatheringstatechange` | `Event` | On `icegatheringstatechange` RTCPeerConnection event. |
| `negotiationneeded` | `Event` | On `negotiationneeded` RTCPeerConnection event. |
| `signalingstatechange` | `Event` | On `signalingstatechange` RTCPeerConnection event. |
| `track` | `RTCTrackEvent` | On `track` RTCPeerConnection event. |
| `leave` | `undefined` \| `Error` | When `local` window leaves. |
| `peer-left` | `undefined` \| `Error` | When `remote` window leaves. |
| `request-offer` | `undefined` | When a window requests an offer from its peer window. |
| `sent-offer` | `RTCSessionDescriptionInit` | When a window has sent an offer. |
| `received-offer` | `{ offer: SdpObject, answer: RTCSessionDescriptionInit }` | When a window has received an offer and **sent an answer**. |
| `received-answer` | `SdpObject` | When a window has received an answer. |
| `received-candidate` | `RTCIceCandidate` | When a window has received an ICE candidate. |
| `error` | `Error` | When an error occurred in IPC communication. |
### Types
#### `IpcObject`
Describes the IPC object used by `electron-window-rtc`
```typescript
interface IpcObject {
on: (
channel: string,
callback: (event: IpcRendererEvent, ...args: any[]) => void
) => void;
removeListener: (
channel: string,
callback: (event: IpcRendererEvent, ...args: any[]) => void
) => void;
send: (channel: string, ...args: any[]) => void;
invoke: (channel: string, ...args: any[]) => Promise<any>;
}
```
#### `WindowRTCEvent`
Describes the generic event data sent and received by `WindowRTCPeerConnection`.
```typescript
interface WindowRTCEvent {
/**
* This window name the event was emitted from.
*/
local: string;
/**
* Peer window name.
*/
remote: string;
payload: any;
}
// Example.
windowConnection.on('track', (event: WindowRTCEvent) => {
const local: string = event.local;
const receiveremoter: string = event.remote;
const trackEvent: RTCTrackEvent = event.payload;
});
```
## Using `canvas`
When using `canvas` to get an image to send to other windows, application should set the `frameRequestRate` parameter of `canvas.captureStream` to a high framerate to avoid latency on the receiver side.
### Setting `frameRequestRate`
**Fig. 1:** _`const stream = canvas.captureStream(240);` in `Sender` window doesn't induce latency. Look at the result of `performance.now()` sent by the `Sender`._
### Without setting `frameRequestRate`
**Fig. 2:** _Without setting `frameRequestRate`: a latency of ~30ms is induced._
## Using WebAudio API
See example's [`SenderWindow`](https://github.com/benoitlahoz/electron-window-rtc/blob/main/example/src/renderer/src/components/SenderWindow.vue) and [`ReceiverWindow`](https://github.com/benoitlahoz/electron-window-rtc/blob/main/example/src/renderer/src/components/ReceiverWindow.vue) for the setup of the final stream.
**Fig. 3:** _Sending audio for visualization with or without playing it in the `Sender` window._
## Known Issues
- Electron example doesn't handle very well reloading windows and introduces latency. **We may explore recreating stream on reload.**
- Reloading `Sender` window takes a lot of time for `Receiver` window to reconnect, whereas the `requestOffer` method allows reconnecting quickly on `Receiver` window's reload.
- Closing and opening again windows has not been tested: it may involve some logic in the `main process` to be integrated in `WindowRTCMain`.
- Latency differs when using the example in `development` mode or `production` mode.
## License
[MIT](https://github.com/benoitlahoz/electron-window-rtc/blob/main/LICENSE)