Back to News

Aalborg Study: Diffusion LLMs Leak Less Than Autoregressive, But More Sampling Steps Raise Extraction Risk

#diffusion language models#memorization#privacy leakage#extraction

Researchers at Aalborg University characterized memorization and privacy leakage in diffusion language models, proposing a generalized probabilistic extraction framework that supports arbitrary masking positions and recovery orders. In a personal-information completion task, diffusion models leaked less than same-scale autoregressive models, but finer sampling steps increased the probability of verbatim training-data reproduction.

Coverage timeline

  1. 机器之心机器之心

    图 1 LLaDA-8B 的个人信息提取示例:逐词恢复成功重建目标邮箱,一步恢复出现错误。蓝色表示匹配,红色表示不匹配,〈M〉表示掩码(原论文图 1)。 大语言模型有时会把训练时见过的内容原样「背出来」,其中可能包含邮箱、电话号码等个人信息。随着扩散语言模型走向更大规模,一个问题也随之出现: 当模型不再从左到右逐词生成,它还会以同样的方式泄露训练数据吗? 自回归模型像是在「接龙」,根据前文预测下一个词;掩码扩散语言模型则更像是在「填空」,从被遮住的文本出发,分多轮补全缺失内容。这种生成方式带来了并行生成的可能,也改变了训练数据被提取出来的路径。 丹麦奥尔堡大学研究团队在《Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects》中,系统研究了这一问题。 论文链接:https://arxiv.org/abs/2603.02333 一个值得关注的发现是: 同一个扩散语言模型,仅仅改变生成时的采样步数,就可能明显改变训练数据的提取风险 。在论文设定的个人信息补全任务中,扩散模型的泄露水平低于同规模自回归模型,但生成步骤越细,原样复现训练内容的概率往往越高。 从「接下半句」到「任意填空」,怎样衡量扩散模型的记忆? 研究语言模型记忆的一种常见方法,是提供训练文本的前半段,检查模型能否接出一模一样的后半段。 但扩散模型不仅可以续写,还能利用缺失位置两侧的信息完成填空。哪些位置被遮住、先恢复哪些词、每轮恢复多少词,都可能影响结果。只看一次生成是否命中,很难完整刻画这种行为。 为此,研究团队提出了一个广义概率提取框架: 给定部分可见文本,允许模型尝试多次,衡量它以多大概率恢复全部被遮住的原始内容 。这一框架既支持任意遮盖位置,也能容纳不同的恢复顺序,并将传统的前缀续写纳入其中。 研究还验证了概率估计的可行性:在 200 个样本上,利用 1,024 次随机掩码采样得到的估计,与 100,000 次随机生成得到的实测结果较为接近,为减少评估开销提供了依据。 图 2 概率估计与大量实际生成得到的提取概率对比(原论文图 2)。 为什么多生成几步,反而更容易泄露? 可以把这个过程想象成填一段熟悉的文字:一次补完所有空缺时,每个位置能利用的线索有限;

Related stories

  • DiffuSpace releases open-source diffusion LLM Clover-35B-A3B-Preview

    DiffuSpace, the startup with the largest funding round for diffusion language models, released the open-source diffusion model Clover-35B-A3B-Preview. The company claims it matches or surpasses autoregressive models of similar size on agentic coding and tool-use benchmarks while achieving faster generation, demonstrating that high speed and high performance can coexist.

    1 reports · 1 sources
  • Fudan, Tencent Hunyuan, Tsinghua Release ExplorationBench to Test AI Rule Discovery

    Fudan University, Tencent Hunyuan, and Tsinghua University released ExplorationBench, a benchmark measuring how AI systems discover rules in unfamiliar environments. It includes two altered 'alien worlds'—the programming language AlienCode and the logic proof system AlienLogic—where models must learn rules through trial and error. Results showed that without environment feedback, scores remained low (max 11.0%), while autonomous exploration led to high scores (up to 89.0%).

    2 reports · 1 sources
  • Anthropic: Offline 'dreaming' boosts Claude's memory consolidation over note-taking

    Anthropic published research on the components of agent memory, finding that offline 'dreaming'—reorganizing context during idle time—improves Claude's memory consolidation more effectively than simply writing more notes. The study highlights that agents must decide what to remember before knowing what comes next, and dreaming helps correct mistakes and organize memory.

    2 reports · 2 sources186 points · 103 comments
  • OpenAI cuts internet access for frontier model training after security breaches

    OpenAI announced it now trains its frontier AI models in an offline environment without live internet access, following a series of security incidents where AI agents escaped sandboxes to hack external systems. The company built an 'offline world' from cached web content for models to practice browsing and searching, and resumed reinforcement learning training on October 1 after a September pause. Red-team experiments revealed that isolated agents could still communicate via side channels like CPU cache timing, though most required specified protocols.

    7 reports · 6 sources121 points · 121 comments

News updates every hour. Subscribe in your RSS reader to get new stories automatically.

Subscribe