Hugging Face Runs 180B-Parameter MoE Model in 32GB RAM
Hugging Face engineers detailed how they optimized inference of a 180B-parameter mixture-of-experts (MoE) model to fit within 32GB of RAM using the Transformers library. The approach leverages MoE sparsity and quantization to reduce memory footprint while maintaining performance. This demonstrates the feasibility of running large MoE models on consumer-grade hardware.
Coverage timeline
Hugging Face Blog
Related stories
Diffusion Intelligence Unveils 350B-Parameter MoE Diffusion Language Model Clover-35B-A3B-Preview
Diffusion Intelligence released Clover-35B-A3B-Preview, a 350B-parameter MoE diffusion language model with about 3B active parameters per token, supporting thinking mode. In the WindTunnel web agent benchmark across 49 tasks with 3 attempts each, Clover achieved a median execution time of 1.52 seconds and a success rate of 98.6%, compared to 3.21 seconds and 95.9% for the Jev + Mercury 2.5 combination. The model can handle coding, web operations, tool calls, and HTML generation, and is positioned as faster than autoregressive models at similar scale.
1 reports · 1 sources

MiniCPM-V 4.7 open-sourced: 35B MoE multimodal model runs on iPhone at 30+ tokens/s
面壁智能 and OpenBMB have open-sourced MiniCPM-V 4.7, including a 35B-A3B MoE multimodal model that runs fully on-device, occupying about 3GB of memory and exceeding 30 tokens per second on the iPhone 18 Pro Max. The team calls this a milestone toward edge intelligence, with co-founder 刘知远 stating that GPT-6-level intelligence on phones within two years is feasible.
1 reports · 1 sources

NES 6502 CPU Emulator Runs Inside Qwen3.5-4B Model Weights
Drew Breunig built a working NES 6502 CPU emulator inside the weights of Qwen3.5-4B, inspired by qwenputer by Divyansh Agrawal. The emulator can run NES ROMs such as Super Mario Bros, albeit very slowly. Breunig shared the Opus 5.5 prompts used to create the system, which involved exploring chip and RAM emulation techniques and using NES CPU conformance tests.
2 reports · 2 sources293 points · 151 comments
Baidu open-sources ERNIE 5.5 Flash Base, 280B MoE model with 14B active parameters
On November 5, Baidu open-sourced ERNIE 5.5 Flash Base, a sparse MoE language model with 280B total parameters and 14B active per token, pretrained on 33.5T tokens with up to 128K training context and 1M-token context for fine-tuning and inference. The company claims SOTA results among same-size models on benchmarks covering knowledge, reasoning, math, code, multilingual, and long-context tasks, surpassing DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base. The release includes weights on Hugging Face and ecosystem support with vLLM and SGLang.
3 reports · 2 sources

News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe