Back to News

Janas Tool Runs LLMs Larger Than RAM by Streaming Experts from SSD

219 points · 93 comments#llm#ssd#streaming#memory

A Hacker News post introduces Janas, a tool that enables running large language models (LLMs) larger than available RAM by streaming expert modules from SSD storage. This approach allows users to execute models that exceed memory capacity, potentially expanding the range of models deployable on standard hardware.

Coverage timeline

  1. Hacker Newskhalidkhair