mixed-english-and-chinese-tokenizer

Tokenizes utterances that contain a mix of English and Chinese words.
v1.1.0 URL: https://unpkg.com/mixed-english-and-chinese-tokenizer@1.1.0/index.js
OpenBrowse Files
tokenizerchineseenglishlanguagetokenizenltknlpnatural-language-processingchinese-tokenizercontains-chinese