UniSteer: Human Corrections as Noise Supervision for VLA Reinforcement Learning
Researchers from Microsoft Research Asia, University of Technology Sydney, and Tsinghua University introduced UniSteer, a method that converts human corrections into noise supervision for vision-language-action (VLA) reinforcement learning via approximate action-to-noise inversion. It enables supervised and reinforcement learning to jointly update a lightweight noise actor while keeping the pretrained VLA action decoder frozen. Demonstrated on a bead art assembly task, the robot placed 16 beads to form a Microsoft logo using only two full demonstration trajectories as human data budget.
Coverage timeline
机器之心机器之心
视觉 - 语言 - 动作模型（Vision-Language-Action Model，VLA）正在成为机器人操作的重要基础模型。经过少量示范微调后，VLA 已经能够完成抓取、放置、堆叠等多种任务。然而，一旦任务涉及精密接触、狭小容差或分布外位置，模型的成功率仍会明显下降。 这也是为什么真机强化学习如此重要：预训练和模仿学习赋予机器人一个不错的起点，而强化学习让它能够在真实环境中继续试错，适应具体物体、视角和接触动力学。但真机试错的成本远高于仿真。每一次失败都会消耗设备时间、实验人员精力，甚至破坏任务现场。 因此，真机强化学习面对的核心问题不是能否学习，而是： 如何用尽可能少的真实交互，让策略尽快学会成功？ 一个自然的答案是让人类参与。模型探索错误方向时，人类可以立即接管并完成纠正。然而，对于采用流匹配（flow matching）的 VLA，新的矛盾随之出现：人类给出的是机器人动作，轻量强化学习优化的却是初始噪声。人类知道机械臂应该怎样移动，却无法直接告诉模型应该选择哪一个噪声。 人类动作与噪声空间之间，缺少一个可靠的接口 。 来自 微软亚洲研究院、悉尼科技大学和清华大学的研究者提出了 UniSteer 。它通过动作到噪声的近似反演，将人类纠正转换成噪声监督，使监督学习和强化学习能够共同更新同一个轻量 noise actor，而预训练 VLA 的动作解码器始终保持冻结。 为了展示这种方法在高精度任务上的潜力，研究者让机械臂挑战了一项颇具难度的手工任务： 拼豆 。 机械臂需要逐颗夹取细小的聚乙烯中空颗粒，再将其准确放到带有柱状凸起的拼豆板上。力度过大，拼豆可能弹飞；姿态稍有偏差，拼豆就无法顺利套入凸起；释放时稍有碰撞，还可能推倒已经放好的颗粒。 在 模型基础上，UniSteer 训练了额外的 noise actor，逐颗放置 16 颗拼豆，最终拼出了微软 Logo。整个任务的在线学习阶段人类数据预算仅为两条完整演示轨迹。 下面是 UniSteer 微调后的 VLA 完成拼豆任务的视频。 作者已在 arXiv 发布论文预印本，并开源训练代码。 论文标题：UniSteer: Unified Noise Steering for Efficient Human-Guided VLA Adaptation 论文链接：https://arxiv.org/abs/2605.10
Related stories
Anthropic reports automated AI researchers mitigate alignment failures across all 10 benchmarks
Anthropic released a report on its Automated Alignment Researcher (AAR) project, where Claude agents autonomously search papers, propose methods, generate data, and train models to fix alignment failures. The system improved performance on all 10 misalignment benchmarks without degrading overall performance, and outperformed human researchers on some tasks at a fraction of the cost.
4 reports · 4 sources
TensorCast: Unified Tensor Management Layer Cuts LLM TTFT by up to 93.2%
Researchers from Peking University, StepFun, and Beijing University of Posts and Telecommunications propose TensorCast, a unified programmable tensor lifecycle management layer for LLM infrastructure. It decouples tensor states like model weights and KV Cache from specific systems, enabling reuse and combination of management capabilities, and reduces time-to-first-token by up to 93.2% in high-concurrency multi-turn agent scenarios.
1 reports · 1 sources
RL-100 framework combines imitation and reinforcement learning for robust robot manipulation
Researchers introduced RL-100, a framework published in Science Robotics on July 22, 2026, that unifies imitation learning and reinforcement learning to optimize diffusion-based robot control policies. It uses human demonstrations for a stable base, offline RL for policy improvement, and minimal online interaction for adjustment, achieving 100% success across eight real-world tasks and 1000 evaluation trials. The framework also employs consistency distillation to compress multi-step diffusion into a one-step controller for high-frequency control, matching or exceeding human teleoperation efficiency.
Tsinghua Researchers Propose XDEM, a Unified Deep Learning Method for Fracture Mechanics
Researchers at Tsinghua University and collaborators published a paper in Nature Communications on August 15, 2026, introducing the Extended Deep Energy Method (XDEM), a unified deep learning framework for fracture mechanics. XDEM combines discrete and continuous fracture models, building on the Deep Energy Method (DEM) and incorporating crack and extended functions to handle discontinuous cracks while leveraging variational potential energy minimization for high accuracy with sparse collocation points.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe