@agnai/web-tokenizers
Version:
| [NPM Package](https://www.npmjs.com/package/@mlc-ai/web-tokenizers) | [WebLLM](https://github.com/mlc-ai/web-llm) |
1 lines • 576 B
Source Map (JSON)
{"version":3,"file":"tokenizers.d.ts","sourceRoot":"","sources":["../src/tokenizers.ts"],"names":[],"mappings":"AAUA;;;GAGG;AACH,qBAAa,SAAS;IACpB,OAAO,CAAC,MAAM,CAAM;IAEpB,OAAO;IAIP;;;;OAIG;IACH,OAAO;IAIP;;;;;OAKG;IACH,MAAM,CAAC,IAAI,EAAE,MAAM,GAAG,UAAU;IAOhC;;;;;OAKG;IACH,MAAM,CAAC,GAAG,EAAE,UAAU,GAAG,MAAM;IAO/B;;;;;OAKG;WACU,QAAQ,CAAC,IAAI,EAAE,WAAW,GAAG,OAAO,CAAC,SAAS,CAAC;IAK5D;;;;;;;OAOG;WACU,gBAAgB,CACzB,KAAK,EAAE,WAAW,EAClB,MAAM,EAAE,WAAW,EACnB,WAAW,GAAE,MAAU,GACvB,OAAO,CAAC,SAAS,CAAC;IAMtB;;;;;OAKG;WACW,iBAAiB,CAAC,KAAK,EAAE,WAAW,GAAI,OAAO,CAAC,SAAS,CAAC;CAKzE"}