mlc-ai/web-llm
★ 19,357 stars
Covered in week 2026-W39 · the 2026-09 report
High-performance In-browser LLM Inference Engine
What web-llm is
WebLLM is a high-performance in-browser LLM inference engine that brings language model inference directly onto web browsers with hardware acceleration. Everything runs inside the browser with no server support and is accelerated with WebGPU.
From the repository README · read it on GitHub →
Transcript
AI features usually mean sending user prompts to remote servers. web-llm runs the whole inference inside your browser, accelerated by WebGPU. It stays OpenAI API compatible, streams completions, generates structured JSON, loads Llama 3, Phi 3, and Mistral, though function calling remains work in progress. Custom models drop in via MLC format, and every prompt stays on the user's machine.