wink-tokenizer

Multilingual tokenizer that automatically tags each token with its type

v5.3.0 URL: https://unpkg.com/wink-tokenizer@5.3.0/src/wink-tokenizer.js

Open Browse Files

Tokenizer Tokenize Tags Tagging NLP email twitter URL Emoji Emoticon Multilingual French German Spanish Icelandic wink

lucene-tokenizers

Port from Apache Lucene

v5.3.3 URL: https://unpkg.com/lucene-tokenizers@5.3.3/lucene-tokenizers.babel.js

Open Browse Files

Lucene Tokenizer JFlex

@dmitrysoshnikov/lex-js

Lexer generator from RegExp spec

v0.1.10 URL: https://unpkg.com/@dmitrysoshnikov/lex-js@0.1.10/index.js

Open Browse Files

Lexer Tokenizer RegExp Parser Token Scanner JavaScript

@enigmaoffline/node-exp-solver

Mathematical expression solver / Reverse Polish Notation calculator for NodeJS

v1.0.4 URL: https://unpkg.com/@enigmaoffline/node-exp-solver@1.0.4/lib/index.js

Open Browse Files

RPN Solver Equation Tokenizer Mathematical Expression

libcore-tokenizer

NFA Tokenizer

v0.1.8 URL: https://unpkg.com/libcore-tokenizer@0.1.8/dist/libcore-tokenizer.js

Open Browse Files

libcore Tokenizer Automata Automaton Finite Non-Deterministic ES5