@lenml/tokenizers

a lightweight no-dependency fork of transformers.js (only tokenizers)

v3.4.0 URL: https://unpkg.com/@lenml/tokenizers@3.4.0/dist/main.global.js

Open Browse Files

llama llama2 llama3 chatgpt mistral tokenizer transformers transformers.js gpt4 gpt4o gpt3.5 vicuna chatglm baichuan qwen

@lenml/llama2-tokenizer

Our library `@lenml/llama2-tokenizer` has been deprecated. We are excited to introduce our new library `@lenml/tokenizers` as its replacement, offering a broader set of features and an enhanced experience.

v1.1.4 URL: https://unpkg.com/@lenml/llama2-tokenizer@1.1.4/dist/main.global.js

Open Browse Files

llama llama2 chatgpt mistral

@lenml/llama2-tokenizer-vocab-llama2

v1.1.1 URL: https://unpkg.com/@lenml/llama2-tokenizer-vocab-llama2@1.1.1/dist/main.global.js

Open Browse Files

llama llama2 chatgpt mistral

@lenml/tokenizer-llama3

llama3 tokenizer for NodeJS/Browser

v3.4.2 URL: https://unpkg.com/@lenml/tokenizer-llama3@3.4.2/dist/main.global.js

Open Browse Files

llama llama2 llama3 chatgpt mistral tokenizer llama3

@axflow/models

Zero-dependency, modular SDK for building robust natural language applications

v0.0.25 URL: https://unpkg.com/@axflow/models@0.0.25

Open Browse Files

ai llm stream streaming streams openai azure anthropic cohere ollama palm llama2 embeddings generative ai gpt4 models useChat nextjs react hugging face huggingface