@lenml/tokenizers

a lightweight no-dependency fork of transformers.js (only tokenizers)

v3.4.0 URL: https://unpkg.com/@lenml/tokenizers@3.4.0/dist/main.global.js

Open Browse Files

llama llama2 llama3 chatgpt mistral tokenizer transformers transformers.js gpt4 gpt4o gpt3.5 vicuna chatglm baichuan qwen

@lenml/tokenizer-llama3

llama3 tokenizer for NodeJS/Browser

v3.4.2 URL: https://unpkg.com/@lenml/tokenizer-llama3@3.4.2/dist/main.global.js

Open Browse Files

llama llama2 llama3 chatgpt mistral tokenizer llama3

@lenml/tokenizer-gemini

gemini tokenizer for NodeJS/Browser

v3.4.2 URL: https://unpkg.com/@lenml/tokenizer-gemini@3.4.2/dist/main.global.js

Open Browse Files

llama llama2 llama3 chatgpt mistral tokenizer gemini

@lenml/tokenizer-claude

claude tokenizer for NodeJS/Browser

v3.4.2 URL: https://unpkg.com/@lenml/tokenizer-claude@3.4.2/dist/main.global.js

Open Browse Files

llama llama2 llama3 chatgpt mistral tokenizer claude

aicommit2

A Reactive CLI that generates git commit messages with various AI

v2.3.6 URL: https://unpkg.com/aicommit2@2.3.6

Open Browse Files