Tsinghua and Qujing Introduce KTransformers-FineTune for CPU-GPU MoE Fine-Tuning
Tsinghua University's MADSys lab and Qujing Technology, with LlamaFactory, introduced KTransformers-FineTune (KT-FT), a CPU-GPU co-computation method for fine-tuning mixture-of-experts (MoE) models. It claims 2.5-21.2x throughput gains over traditional offloading by having the CPU compute expert weights directly instead of transferring them to the GPU. The approach includes a semantic-aware proactive layout engine and dynamic thread scheduling for routing skew, and is fully open-sourced.
Coverage timeline
机器之心机器之心
从 DeepSeek-V4 到 Qwen-3.8、Kimi-K3、GLM-5.3，每⼀次超⼤模型的开源都带来性能上的巨⼤跃升。然⽽，随着模型规模越来越大，多数研究者与开发者只能尝试调用 API，无法在资源受限条件下进行微调，这进一步限制了大家探索前沿模型的特性，也难以把他们适配到自己的业务任务中。为降低 GPU 显存需求，之前的工作把 CPU 和 GPU 搭配起来，用容量更大的 CPU 内存存储专家权重，再通过 PCIe 搬到 GPU 上计算。这种办法能够让模型装得下了，但是微调速度很慢，效率难以达到大家的使用预期。 这次， 在清华大学 MADSys 实验室、趋境科技主导下，我们提出 KTransformers-FineTune（KT-FT），并联合知名开源项目 LlamaFactory 团队一起，打造用户友好、易用的开源生态 。KT-FT 通过 CPU 与 GPU 协作，吞吐达到传统卸载范式的 2.5～21.2 倍。同时，我们也坚持全面开源，为社区、学界和业界提供了低成本微调 - 推理方案闭环，实现 LoRA 微调和推理可以在同等硬件条件下一键执行。 之前微调中的 GPU/CPU 协作的工作主要都在围绕搬运权重的范式来思考怎么进一步优化，希望用 GPU 计算时间更多的掩盖 PCIe 搬运。可 MoE 的稀疏计算耗时很短，根本盖不住大规模权重的传输。 在单机有限显存的场景下，即使传输与计算完美重叠，权重搬运仍然决定了微调的吞吐上限，传统范式无论如何优化权重的搬运方式都是徒劳 。 要真正提速，就必须改变专家的执行位置，让 CPU 用留在主存中的权重直接完成专家计算，与 GPU 协同工作。 KT-FT 发现，在当前前沿 MoE 模型的微调负载下，把这些权重搬到 GPU 的耗时，甚至长于在 CPU 上直接计算。并因此提出 CPU/GPU 协同计算（Co-Computation）范式，直接利用 CPU 的计算能力，把 CPU 从一个 “存储库” 转为协同执行专家计算，GPU 则继续负责计算密集的注意力模块。然而，仅仅通过 Co-Computation 范式让 CPU 参与计算，还不足以带来足够有效的微调方案。KT-FT 提出语义感知主动布局引擎提前转换布局，并优化路由倾斜的动态线程调度。这些设计使 KT-FT 的 Co-Computation 范式吞吐达到传统卸载范式（如
Related stories
HKUST and Huawei propose BalanceRoute to cut MoE LLM serving load-balancing bottleneck
Researchers from HKUST and Huawei's 2012 Lab propose BalanceRoute, an online routing method that models synchronous waiting in MoE LLM serving and converts short-term load prediction into routing decisions. In a 96-GPU test, the predictor-based variant BR-H improves decode throughput by about 15% over the strongest baseline. The work is accepted by NeurIPS 2026 and deployed in the jointly built MaaS system MathBot-Pie.
1 reports · 1 sources
Tsinghua, Infinigence-AI, SJTU Open-Source On-Device Inference Engine APXInf for Embodied Models
On September 15, Tsinghua University, Infinigence-AI, and Shanghai Jiao Tong University open-sourced APXInf, an on-device inference engine for embodied models. The engine reportedly achieves a 10.7x end-to-end speedup on robot hardware, reducing inference latency from 278ms to 26ms on Thor chips with FP8 configuration, enabling real-time control at 38.46Hz.
Microsoft unveils MAI-Base-2 MoE model with 2.8T total parameters
Microsoft AI announced MAI-Base-2, a mixture-of-experts model with 2.8 trillion total parameters and 80 billion active parameters. The company claims it achieves 4x-12x greater pretraining FLOPs-efficiency compared to DeepSeek-V4-Pro-Base. This follows the June announcement of MAI-Thinking-1, a reasoning model post-trained on MAI-Base-1.
1 reports · 1 sources
Zhizhi Innovation open-sources IQuest-Q1, 320B-param sparse MoE coding model
Chinese research institute Zhizhi Innovation released IQuest-Q1, a sparse MoE model with ~320B total parameters and ~15B active parameters, focused on coding, software engineering, and complex agentic tasks. The model shows balanced performance across benchmarks including NL2Repo, CyberGym, Terminal-Bench 2.1, DeepSWE v1.1, and JobBench, with weights and blog published on GitHub and Hugging Face.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe