UNPKG

nlp-tokenizer

Version:

tokenizer

github.com/josherich/nlp-tokenizer

josherich/nlp-tokenizer

15 files

	Filename	Content Type	Size
	../
	NNTokenizer.d.ts	text/typescript	1.63 kB
	NNTokenizer.js	text/javascript	6.19 kB
	NNTokenizer.js.map	application/json	6.31 kB
	Tokenizer.d.ts	text/typescript	312 B
	Tokenizer.js	text/javascript	2.35 kB
	Tokenizer.js.map	application/json	2.38 kB
	WordPieceTokenizer.d.ts	text/typescript	331 B
	WordPieceTokenizer.js	text/javascript	1.83 kB
	WordPieceTokenizer.js.map	application/json	1.6 kB
	index.d.ts	text/typescript	214 B
	index.js	text/javascript	737 B
	index.js.map	application/json	477 B
	utils.d.ts	text/typescript	300 B
	utils.js	text/javascript	1.89 kB
	utils.js.map	application/json	1.83 kB