happynodetokenizer
Version:
A simple, Twitter-aware tokenizer.
1 lines • 244 B
Source Map (JSON)
{"version":3,"file":"tokenizer.d.ts","sourceRoot":"","sources":["../src/tokenizer.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAyB,SAAS,EAAiB,gBAAgB,EAAE,MAAM,YAAY,CAAC;AAmBpG;;;;;;;GAOG;AACH,wBAAgB,SAAS,CAAC,IAAI,GAAE,gBAA+B,GAAG,SAAS,CAsB1E"}