Tsinghua, Infinigence-AI, SJTU Open-Source On-Device Inference Engine APXInf for Embodied Models
On September 15, Tsinghua University, Infinigence-AI, and Shanghai Jiao Tong University open-sourced APXInf, an on-device inference engine for embodied models. The engine reportedly achieves a 10.7x end-to-end speedup on robot hardware, reducing inference latency from 278ms to 26ms on Thor chips with FP8 configuration, enabling real-time control at 38.46Hz.
Coverage timeline
机器之心机器之心
编辑｜Panda 9 月的第二周，具身圈很热闹。 8 日，松延动力发布 HERON-World Model；9 日，智元一口气放出 AGILE 2.0 与 GE-Act 2.0；10 日，宇树开源 UnifoLM-WLA-1.0，6B 参数，约 2500 小时真机数据，一个模型统筹 64 项任务；到 15 日，松延又补上了 HERON-CRA。八天，三家本体厂商，五个模型。 与此同时，机器人正在加速迈入物理世界。9 月 20 日，启元机器人举办新品发布会，两款个人机器人正式发售，Q1 定价 19999 元起；此前 9 月 10 日，优必选宣布斩获超 5000 万元海外订单，涉及 Walker C1、优世界 U1 等产品。资本市场的评价标准也随之变化，开始用脱水复购率、经营性净现金流，以及真实履约成本（即交付之后的部署、调试与维护投入）来重新审视具身企业。 两条线索交汇，一个核心痛点浮现： 如何将复杂强大的具身大模型，高效、稳定地部署到算力极其有限的机器人本体硬件上？ 这正是 端侧推理引擎 的核心价值所在。 9 月 15 日，清华大学联合无问芯穹与上海交通大学开源了 APXInf，一款面向具身模型的端侧推理引擎 。它同时回答了两个问题： 在算力、内存和功耗受限的端侧环境中，如何让具身模型在机器人本体上达到可用的推理速度？ 当模型不断迭代进化，如何构建持续适配、永不掉队的端侧优化能力？ 第一个问题，APXInf 交出的答案是一组数字： 无需改变 π0.5 模型本身，通过端到端的全栈优化，在 Thor 芯片 FP8 配置下把推理延迟从 278ms 降至 26ms，端到端提速约 10.7 倍，38.46Hz 的频率让机器人控制进入实时区间。 第二个问题的答案则写在 APXInf 仓库的构建方式中。它把原本依赖少数专家的模型适配、优化与验证，沉淀成了一套可被持续复用并且可被 Agent 使用的工作流程。 项目地址：https://github.com/RLinf/APXinf-robo 为什么具身需要专门的端侧推理优化？ 要回答这个问题，先要看清推理引擎在一台机器人中的位置。 一台具身本体大致由主控、算力盒子和外设构成。一次控制循环是：主控采集观测，算力盒子推理出动作 chunk，推送给手臂或底盘执行，然后进入下一帧。推理引擎就嵌在这个循环的中间，它决定一次推理花多少毫
Related stories
Open-source projects unlock hidden Tensor Core throughput on GeForce GPUs, doubling RTX 4090 performance
Open-source developers released projects on GitHub that unlock higher Tensor Core throughput on NVIDIA GeForce GPUs by modifying boot-time configuration, without driver swaps or BIOS flashes. Tests reportedly doubled the FP16/FP32 and BF16/FP32 throughput of an RTX 4090 from about 165 TFLOPS to over 330 TFLOPS, with gains also observed on RTX 3090 Ti, RTX 4070 Ti SUPER, and RTX 2080 Ti. The projects target RTX 20, 30, and 40 series, with some experiments done on Windows 11.
1 reports · 1 sources
RAX: Harvard, Berkeley, NVIDIA Introduce Online Exploration Scaling for Robot Learning, Outperforming SOTA by 37.4 Points on Average
Researchers from Harvard, NVIDIA, UC Berkeley, MIT, and Stanford proposed RAX (Robot Agent eXploration), which introduces scaling online exploration to robot learning. RAX uses a general coding agent to connect directly to a robot's perception and control interfaces, enabling the robot to gain experience through exploration and write successful decisions as executable programs, accumulating reusable skills. With frozen model parameters, RAX outperforms previous state-of-the-art models by an average of 37.4 percentage points across five simulation benchmarks: RoboDojo, BEHAVIOR-1K, HumanoidArena, DexJoCo, and LIBERO-Pro.
Alibaba Qwen and OSU Propose TRACE to Speed Up FP4 RL Rollout by 5.4x
Alibaba Qwen and Ohio State University researchers proposed TRACE, a rollout-guided quantization-aware training method for FP4 reinforcement learning of mixture-of-experts (MoE) language models. By using the actual FP4 quantization results from the rollout side to guide rounding on the training side, TRACE reduces discrepancies between training and rollout paths, recovering performance close to BF16 while achieving up to 5.4x decoding throughput improvement in rollout with FP4 weights/activations and FP4 KV cache at 128K output length. The paper is available on arXiv and Hugging Face.
DeepSeek open-sources DeepInfer inference stack, reports 82% gross margin
DeepSeek released its internal inference framework DeepInfer under the Apache 2.0 license, along with the token codec component deepseek-recipe. The framework, which supports DeepSeek-V4.1-Flash and targets Nvidia SM100 GPUs and Huawei Ascend 950 NPUs, reportedly achieves a throughput of 155 trillion tokens per day per cluster, with API revenue of $38 per GPU-hour at off-peak pricing, implying a gross margin of 82% after GPU rental costs. The release sparked discussions about the profitability of AI inference services.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe