bpe-tokenizer-encoding

A simple JavaScript implementation of Byte Pair Encoding (BPE)

v2.0.0 URL: https://unpkg.com/bpe-tokenizer-encoding@2.0.0/index.js

Open Browse Files

bpe tokenizer byte pair encoding nlp

bpe-merge-visualizer

CLI tool to visualize Byte Pair Encoding (BPE) merge steps

v3.1.0 URL: https://unpkg.com/bpe-merge-visualizer@3.1.0/index.js

Open Browse Files

bpe tokenizer visualizer cli

n8n-nodes-gpt-tokenizer

n8n node for working with BPE Tokens with OpenAI's GPT models in mind.

v0.1.1 URL: https://unpkg.com/n8n-nodes-gpt-tokenizer@0.1.1/index.js

Open Browse Files

n8n-community-node-package n8n nodemation n8n-node gpt bpe tokenizer

kitoken

Fast and versatile tokenizer for language models, supporting BPE, Unigram and WordPiece tokenization

v0.10.1 URL: https://unpkg.com/kitoken@0.10.1/dist/index.js

Open Browse Files

tokenizer nlp bpe unigram wordpiece

gpt-turbo-plugin-stats

GPT Turbo plugin that calculates your conversation size and cost

v5.0.1 URL: https://unpkg.com/gpt-turbo-plugin-stats@5.0.1/dist/index.js

Open Browse Files

openai chatgpt chat gpt gpt3 gpt-3 gpt3.5 gpt-3.5 gpt4 gpt-4 completion chatcompletion conversation conversation ai ai ml bot chatbot gpt-turbo gpt-turbo-plugin tokenizer bpe