Janas Tool Runs LLMs Larger Than RAM by Streaming Experts from SSD
A Hacker News post introduces Janas, a tool that enables running large language models (LLMs) larger than available RAM by streaming expert modules from SSD storage. This approach allows users to execute models that exceed memory capacity, potentially expanding the range of models deployable on standard hardware.
Coverage timeline
Hacker Newskhalidkhair