sqliteai/waste
★ 2,511 stars
Covered in week 2026-W35 · the 2026-08 report
Run the full 2.78-trillion-parameter Kimi K3 model beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.
What waste is
WARP is an embeddable inference engine written in C, with no third-party runtime dependencies. It keeps the model trunk in memory, streams selected experts directly from disk, and uses the remaining RAM as a bounded expert cache.
From the repository README · read it on GitHub →
Transcript
WARP streams 2.78-trillion-parameter Kimi K3 from NVMe beyond your RAM by keeping the trunk in memory and reading selected experts directly from disk. The full model runs on a 64 GB MacBook Pro at 0.6 tokens per second, with weights compressed from 1.42 TB to 982 GB. Sixty-four gigabytes is the practical minimum. You can now run frontier models locally.