happynodetokenizer

A simple, Twitter-aware tokenizer.

v7.1.0 URL: https://unpkg.com/happynodetokenizer@7.1.0/dist/index.js

Open Browse Files

tokenise tokenize tokenising tokenizing tokeniser tokenizer token twitter NLP language text strings stanford dlatk wwbp

complexion-time-map

Time a JavaScript tokenizer

v1.1.0 URL: https://unpkg.com/complexion-time-map@1.1.0/lib/complexion-time-map.js

Open Browse Files

complexion tokenizer tokeniser tokenize tokenise profile

complexion

A fast tokenizer/lexer core that is well tested.

v0.2.2 URL: https://unpkg.com/complexion@0.2.2/lib/complexion.js

Open Browse Files

tokenizer tokeniser tokenize tokenise lexer token lex parser fast

complexion-js

Tokenize JavaScript

v0.2.2 URL: https://unpkg.com/complexion-js@0.2.2/lib/complexion-js.js

Open Browse Files

complexion javascript tokenizer tokeniser tokenize tokenise lexer token lex parser fast

xml-zero-lexer

Friendly and forgiving HTML5/XML5/JSX lexer/parser with lots of tests. Memory-efficient and Web Worker compatible.

v4.0.1 URL: https://unpkg.com/xml-zero-lexer@4.0.1/build/index.js

Open Browse Files

xml html jsx react parser lexer string stringto stringtoreact tokenise token markup typescript