pos-tokenizer
Version:
POS tokenization of words int meaningful components usable in POS-Bayes & Elastic Search Indexes
9 lines (6 loc) • 379 B
Markdown
This is an implementation of the Porter2 stemming algorithm in JavaScript, currently for English, French and German.
var tokenizer = require('./../lib/tokenizer.js'),
words = tokenizer('The quick brown fox jumps over the lazy dog'),
frWords = tokenizer('Servez à ce monsieur une bière et des kiwis', 'fr');