wink-tokenizer

Multilingual tokenizer that automatically tags each token with its type

v5.3.0 URL: https://unpkg.com/wink-tokenizer@5.3.0/src/wink-tokenizer.js

Open Browse Files

Tokenizer Tokenize Tags Tagging NLP email twitter URL Emoji Emoticon Multilingual French German Spanish Icelandic wink

@coder-seb/tokenizer

Lexical tokenizer with grammar classes ready to go.

v1.0.10 URL: https://unpkg.com/@coder-seb/tokenizer@1.0.10/src/app.js

Open Browse Files

Tokenizer coder-seb

lucene-tokenizers

Port from Apache Lucene

v5.3.3 URL: https://unpkg.com/lucene-tokenizers@5.3.3/lucene-tokenizers.babel.js

Open Browse Files

Lucene Tokenizer JFlex

bleuscore-js

A fast bleu score calculator

v0.1.5 URL: https://unpkg.com/bleuscore-js@0.1.5/bleuscore_js.js

Open Browse Files

NLP Tokenizer BLEU DeepLearning

kingchop

A text tokenizing library that handles strings, by tokenizing them into arrays, depending on intended format, like sentences, sub sentences, paragraphs, words...

v2.0.2 URL: https://unpkg.com/kingchop@2.0.2/index.js

Open Browse Files

NLP tokenizer Tokenizer Sentence tokenizer Text tokenizer Node.js tokenizer Flexible text tokenizer CommonJS text tokenizer Paragraph tokenizer Sub sentence tokenizer Stable tokenizer Easy tokenizer Light text tokenizer