@speechmatics/browser-audio-input-react/README.md

React hooks for managing audio inputs and permissions across browsers

# Browser audio input (React)

React bindings for the `@speechmatics/browser-audio-input` package, letting you manage audio input devices and permissions across browsers.

## Installation

```
npm i @speechmatics/browser-audio-input-react
```

## Usage

### Microphone selection

Below is an example of a Microphone selection component.

```TSX
import { useAudioDevices } from "@speechmatics/browser-audio-input-react";

function MicrophoneSelect({
  setDeviceId,
}: { setDeviceId: (deviceId: string) => void }) {
  const devices = useAudioDevices();

  switch (devices.permissionState) {
    case 'prompt':
      return (
        <label>
          Enable mic permissions
          <select
            onClick={devices.promptPermissions}
            onKeyDown={devices.promptPermissions}
          />
        </label>
      );
    case 'prompting':
      return (
        <label>
          Enable mic permissions
          <select aria-busy="true" />
        </label>
      );
    case 'granted': {
      const onChange = (e: ChangeEvent<HTMLSelectElement>) => {
        setDeviceId(e.target.value);
      };
      return (
        <label>
          Select audio device
          <select onChange={onChange}>
            {devices.deviceList.map((d) => (
              <option key={d.deviceId} value={d.deviceId}>
                {d.label}
              </option>
            ))}
          </select>
        </label>
      );
    }
    case 'denied':
      return (
        <label>
          Microphone permission disabled
          <select disabled />
        </label>
      );
    default:
      devices satisfies never;
      return null;
  }
}

```

### PCM recording

This package exposes a context provider that can be used to share a **single PCM recorder across the app**. This is quite handy, as you can control the recorder from any component in your app!

```TSX
import { PCMAudioRecorderProvider } from '@speechmatics/browser-audio-input-react';

function App() {
  return (
    // See note in the next section about the AudioWorklet script
    <PCMAudioRecorderProvider workletScriptURL="/path/to/pcm-audio-worklet.min.js">
      <Component>
    </PCMAudioRecorderProvider>
  );
}

```
Now all child components can use the provided hooks:

### Start/stop recording

The only required argument to `startRecording` is an [`AudioContext`](https://developer.mozilla.org/en-US/docs/Web/API/AudioContext). Note that 

`stopRecording` stops the active `MediaStream` source, but leaves the `AudioContext` open, so it can be re-used.

```TSX
import { usePCMAudioRecorderContext } from "@speechmatics/browser-audio-input-react";

function RecordingButton() {
  const { startRecording, stopRecording, isRecording } = usePCMAudioRecorderContext();

  const onClick = () => {
    if (isRecording) {
      stopRecording();
    } else {
      const audioContext = new AudioContext();
      startRecording({ audioContext });
    }
  }

  return <button onClick={onClick}>
    {isRecording ? "Stop recording" : "Start recording" }
  </button>
}
```
You can specify the device for recording by passing the `deviceId`option to `startRecording`.


#### Recording options

You can pass whatever ['MediaTrackSettings'](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackSettings) you want through the `recordingOptions` property:

```typescript
    pcmRecorder.startRecording({
      audioContext,
      deviceId,
      recordingOptions: {
        noiseSuppression: false,
      },
    });
```

By default we enable the following to optimize for speech:

```javascript
{
  noiseSuppression: true,
  echoCancellation: true,
  autoGainControl: true,
}
```

Note that the last two [may not be supported in Safari](https://developer.mozilla.org/en-US/docs/Web/API/MediaTrackConstraints/autoGainControl#browser_compatibility)

### Read recorded data

Recorded data can be read from any child component of the context provider with the `usePCMAudioListener` hook:

```TSX

function Component() {
  usePCMAudioListener((audio: Float32Array) => {
    // Handle Float32Array of audio however you like
  });
}

```

### Note about `AudioWorklet` script URL

When recording audio in the browser, there are generally three approaches:

- ❌ [`createScriptProcessor()`](https://developer.mozilla.org/en-US/docs/Web/API/BaseAudioContext/createScriptProcessor): Can capture PCM data on the main thread, but is deprecated and suffers from poor performance easily.
- ❌ [`MediaRecorder`](https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder): Provides a simple API, but cannot capture PCM data (only MPEG/OGG)
- ✅ [`AudioWorklet`](https://developer.mozilla.org/en-US/docs/Web/API/AudioWorklet): Captures/processes PCM on dedicated thread.

This library leverages `AudioWorklet` to capture PCM audio (specifically 32-bit Float PCM, which is the underlying representation in the browser).

Since `AudioWorklets` run outside the main thread, their code must be run from an external source (i.e. a URL).

### Getting the AudioWorklet script

First make sure the base package (the one this package wraps) is installed:

```
npm i @speechmatics/browser-audio-input
```

The code for this PCM audio processor is provided by that library at `/dist/pcm-audio-worklet.min.js`. However, **how this script is loaded depends on your bundler setup**.

Modern bundlers can emit the worklet as an asset and give you its URL — that's the approach we recommend. Each example below produces a URL string, which you then pass to `<PCMAudioRecorderProvider workletScriptURL={workletScriptURL}>`.

### Turbopack (e.g. Next.js 16+)

Turbopack can emit the worklet as an [asset](https://nextjs.org/docs/app/api-reference/config/next-config-js/turbopack) and resolve the import to its URL. Add a rule to your `next.config.ts`:

```typescript
import type { NextConfig } from 'next';

const nextConfig: NextConfig = {
  turbopack: {
    rules: {
      // Emit the worklet as an asset; the import below resolves to its URL.
      '**/pcm-audio-worklet.min.js': {
        type: 'asset',
      },
    },
  },
};

export default nextConfig;
```

Then import the worklet URL directly:

```TSX
import { PCMAudioRecorderProvider } from '@speechmatics/browser-audio-input-react';
import workletScriptURL from '@speechmatics/browser-audio-input/pcm-audio-worklet.min.js';

function App() {
  return (
    <PCMAudioRecorderProvider workletScriptURL={workletScriptURL}>
      <Component>
    </PCMAudioRecorderProvider>
  );
}
```

The base package ships a type declaration for the `pcm-audio-worklet.min.js` subpath, so the import is typed as a `string` with no extra setup.

### Webpack (v5+)

Webpack 5 has built-in [asset modules](https://webpack.js.org/guides/asset-modules/), so no plugins are required. Configure the worklet to be emitted as a resource:

```javascript
module.exports = {
  // ... rest of your Webpack config
  module: {
    rules: [
      {
        test: /pcm-audio-worklet\.min\.js$/,
        type: 'asset/resource',
      },
    ],
  },
};
```

The import then resolves to the emitted file's URL:

```TSX
import { PCMAudioRecorderProvider } from '@speechmatics/browser-audio-input-react';
import workletScriptURL from '@speechmatics/browser-audio-input/pcm-audio-worklet.min.js';

function App() {
  return (
    <PCMAudioRecorderProvider workletScriptURL={workletScriptURL}>
      <Component>
    </PCMAudioRecorderProvider>
  );
}
```

<details>
<summary>Older Webpack (last resort)</summary>

If you're on Webpack 4, which predates asset modules, you can copy the worklet into your `public/` folder with [`copy-webpack-plugin`](https://webpack.js.org/plugins/copy-webpack-plugin):

```javascript
const CopyWebpackPlugin = require("copy-webpack-plugin");

module.exports = {
  // ... rest of your Webpack config
  plugins: [
    new CopyWebpackPlugin({
      patterns: [
        {
          from: require.resolve(
            '@speechmatics/browser-audio-input/pcm-audio-worklet.min.js',
          ),
          to: path.resolve(__dirname, 'public/js/[name][ext]'),
        },
      ],
    }),
  ],
};
```

Then pass its public path, e.g. `<PCMAudioRecorderProvider workletScriptURL="/js/pcm-audio-worklet.min.js">`.

</details>

### Vite

Vite supports referencing bundled code by URL with the [`?url` suffix](https://vite.dev/guide/assets#explicit-url-imports):

```TSX
import { PCMAudioRecorderProvider } from '@speechmatics/browser-audio-input-react';
import workletScriptURL from '@speechmatics/browser-audio-input/pcm-audio-worklet.min.js?url';

function App() {
  return (
    <PCMAudioRecorderProvider workletScriptURL={workletScriptURL}>
      <Component>
    </PCMAudioRecorderProvider>
  );
}
```

### Creating audio visualizers

The hook `usePCMAudioRecorderContext` provides an `analyser` object, which is an instance of [`AnalyserNode`](https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode).

```typescript
const { analyser } = usePCMAudioRecorderContext();
```

MDN has a [great guide on audio visualizers for the WebAudio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API). The basic idea though is that you can use `requestAnimationFrame` to repeatedly read the [`analyser.getFloatFrequencyData`](https://developer.mozilla.org/en-US/docs/Web/API/AnalyserNode/getFloatFrequencyData) method to animate whatever DOM elements you like.

See the [`AudioVisualizer`](../../examples/nextjs/src/lib/components/AudioVisualizer.tsx) in the NextJS demo app for a complete example.