Samsung, Oxford, Peking University propose TrOPD for on-policy LLM distillation
Researchers from Samsung, Oxford, and Peking University propose TrOPD (Trust Region On-Policy Distillation), a method to compress frontier LLM capabilities into edge devices. The work first systematically benchmarks existing OPD methods under a unified setup, identifying that reliable token-level supervision from the teacher, rather than divergence formula choice, determines training success. TrOPD addresses supervision distortion when teacher-student gaps are large, offering a path for lower-cost edge deployment.
Coverage timeline
机器之心机器之心
标题：Trust Region On-Policy Distillation 文章地址：https://arxiv.org/abs/2606.01249 项目地址：https://github.com/Xingrun-Xing2/TrOPD GPT-6 等前沿大模型的能力仍在快速 Scaling，但随之攀升的推理成本，正在成为 AI 进一步规模化普及的关键障碍。当 AI 从云端走向手机、平板、智能家居等终端，模型不仅要 “足够聪明”，还必须塞进有限的内存和算力预算，并满足功耗、时延等严苛约束。 对于拥有数亿级终端设备的三星而言，这一矛盾更加直接：如何用更小的模型、更低的推理成本承载更多智能，让前沿 AI 能力进入更多终端，已经成为端侧智能继续 Scaling 的关键问题。端侧模型能否有效继承大模型的复杂推理能力，不仅决定 AI 能覆盖多少设备和应用场景，也直接影响智能服务的成本与用户体验。 On-Policy Distillation (OPD) 正是解决这一问题的重要技术路径。相比通过结果奖励驱动模型自主探索的 GRPO，OPD 利用教师大模型提供细粒度监督，让端侧模型直接学习更强模型的推理能力。然而，现有 OPD 面临一个关键瓶颈：当教师与学生模型能力差距较大时，监督信号可能失真，甚至导致训练不稳定。 针对这一问题， 三星大模型团队联合牛津大学、北京大学提出基于信任域 OPD 方法（Trust Region On-Policy Distillation） ，通过识别教师 “可信” 的监督区域，让端侧模型更稳定、更有效地继承大模型能力，为前沿智能以更低成本走向数亿终端提供了一条新路径。 核心结论 当前关于 On-Policy Distillation（OPD）的研究，大多缺少统一 benchmark 下的公平对比：不同工作的训练步数、KL 散度估计方式各不相同，很难判断某一种改进是不是真的有效。这篇先把主流 OPD 方法拉到统一设定下系统测了一遍，由此发现了真正决定 OPD 训练成败的关键： 不是该用哪种散度公式（FKL、RKL 还是 JSD），而是 教师能否对学生生成的每一个 token 给出可靠监督 。一旦学生和教师的输出分布差得太远，监督信号本身就会失真，再精巧的散度公式也救不回来。 沿着这个发现，提出 TrOPD （Trust Region On-Poli
Related stories
LAMA Framework Auctions Ads Token-by-Token During LLM Generation
Researchers from Renmin University and Stanford propose LAMA, a large-model-native advertising framework that auctions ads per generated token, integrating ad allocation into the generation process. This shifts from traditional pre-defined ad slots to dynamic allocation where each token influences both content continuation and ad exposure.
1 reports · 1 sources
On-Policy Distillation With One Training Example Recovers Over 70% of Full-Data Gains
Researchers from Tsinghua and collaborators show that on-policy distillation (OPD) with a single math problem can sustain performance gains over hundreds of training steps, recovering over 70% of the improvement achieved with the full 17,000-problem dataset. The second paper in the Rethinking On-Policy Distillation series argues that OPD is 'data-saturated but algorithm-starved,' shifting focus from data scale to algorithmic efficiency.
UCL team unveils Large Discovery Models for experiment design
Researchers at University College London led by Wang Jun released Large Discovery Models (LDMs), a recursive foundation-model architecture that combines generative foundation models with Gaussian process reward models to design next experiments in scientific discovery. Validated on neural-network training, antibody design, and molecular optimization, LDM v0.1 reportedly outperforms pure LLM reflection baselines, with antibody binding energy reduced by 18.2% on average and molecular multi-objective hypervolume improved by over 60%.
Anthropic CEO proposes 'pacing the frontier' plan; OpenAI, Musk, Hugging Face respond
Anthropic CEO Dario Amodei published an essay on September 12, 2026, urging the AI industry to slow the pace of frontier model development to address safety risks, proposing a three-part plan: embedded third-party evaluators, coordination among democratic countries, and global coordination including with authoritarian governments. Anthropic unilaterally committed to giving third-party evaluators permanent employee-level access. OpenAI CEO Sam Altman agreed and said OpenAI would do the same, while Elon Musk endorsed the call and Hugging Face launched an Open Alignment Initiative seeking to join the embedded evaluators program.
25 reports · 6 sources330 points · 260 comments
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe