UNPKG

@anush008/tokenizers

Version:

Multi-arch builds of HuggingFace tokenizers

github.com/Anush008/tokenizers

Anush008/tokenizers

3 files
../
mod.rs
35 B
models.rs
2 kB
tokenizer.rs
2.9 kB