UNPKG

gpt-tokenizer

Version:

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

github.com/niieani/gpt-tokenizer

niieani/gpt-tokenizer

4 lines • 224 B
1
2
3
4
// eslint-disable-next-line no-restricted-exports, import/no-default-export export { default } from '../encoding/cl100k_base.js'; export * from '../encoding/cl100k_base.js'; //# sourceMappingURL=text-embedding-3-small.js.map