Swift+Metal runtimes stream MoE experts from SSD to run large models in low RAM
Two open-source projects demonstrate running large MoE models on Apple Silicon with minimal RAM by streaming expert weights from SSD. TurboFieldfare runs Gemma 4 26B-A4B in about 2 GB of RAM on any M-series Mac, while another tool runs Qwen 3.6 35B in 4.3 GB and Qwen3-Next 80B in 4.3 GB on a Mac, with a 35B version on iPhone. Both use custom Swift and Metal runtimes rather than wrappers around MLX or llama.cpp.
Coverage timeline
Hacker Newsgitpusher42
## TurboFieldfare **Gemma 4 26B-A4B inference in about 2 GB of RAM** A custom Swift + Metal runtime for any Apple Silicon Mac, even the 8 GB ones. Quick start · Local server · Benchmarks · Contribute results · How it works · Experiments · References Memory got expensive. So I gave a 26-billion-parameter model a ~2 GB budget. TurboFieldfare runs the instruction-tuned **Gemma 4 26B-A4B** without loading the entire 14.3 GB model into memory. It keeps the shared 1.35 GB core and FP16 KV cache in memory, then streams only the experts needed for each token from SSD. This is what lets the model run on Macs with 8 GB of RAM. The runtime, streaming installer, CLI, and native Mac app are written in Swift and Metal. TurboFieldfare is model-specific rather than a wrapper around MLX or llama.cpp. The curated experiment record summarizes 103 measured results across kernels, caching, I/O, prefill, and decode. ## Try it git clone https://github.com/drumih/turbo-fieldfare.git cd turbo-fieldfare swift b
# Download the 35B container from the R2 mirror (fast and consistent; # Hugging Face throttles anonymous downloads harder the longer they run): .build/release/swiftlet-repack \ --from-url https://pub-c0cfece2dbc340dbb2cd9d94310a7d68.r2.dev/qwen3.6-35b-qpack \ --output ~/models/qwen3.6-35b.qpack # Or the 80B (42 GB on disk, still only ~4.3 GB of RAM): .build/release/swiftlet-repack \ --from-url https://pub-c0cfece2dbc340dbb2cd9d94310a7d68.r2.dev/qwen3-next-80b-qpack \ --output ~/models/qwen3-next-80b.qpack # Or the 8-bit 35B (best writing quality, 34 GB disk, ~7.6 GB RAM): .build/release/swiftlet-repack \ --from-url https://pub-c0cfece2dbc340dbb2cd9d94310a7d68.r2.dev/qwen3.6-35b-8bit-qpack \ --output ~/models/qwen3.6-35b-8bit.qpack # The same containers are on Hugging Face (anonymous downloads are # rate-limited there, so the mirror is usually much faster): # --from-hf Leonickson/Qwen3.6-35B-A3B-qpack # --from-hf Leonickson/Qwen3.6-35B-A3B-8bit-qpack # --from-hf Leonickson/Qwen3-Next-80