UNPKG

gpt-token-utils

Version:

Isomorphic utilities for GPT-3 tokenization and prompt building.

github.com/sister-software/gpt-token-utils

sister-software/gpt-token-utils

25 lines (24 loc) • 600 B
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
/** * @copyright Sister Software. All rights reserved. * @author Teffen Ellis, et al. * @license * See LICENSE file in the project root for full license information. */ /** * Serialized vocabulary. Used with the bundled vocabulary. * @internal * @ignore */ export type BundledVocab = readonly string[]; /** * Serialized encoder keys. Used with the bundled encoder. * @internal * @ignore */ export type BundledEncoderKeys = readonly string[]; /** * Serialized encoder keys. Used with the bundled encoder. * @internal * @ignore */ export type BundledEncoderValues = readonly number[];