LoopVL open-source VLM shows data efficiency with 260x less training data
Researchers from Renmin University, Baidu, and others released LoopVL, an open-source vision-language model based on the Loop Transformer. Despite using only 0.14T training tokens versus about 36T for Qwen3.5-2B and InternVL3.5-2B, LoopVL matches or exceeds them on benchmarks like LogicVista, MMMU-Pro, and MathVision, demonstrating high data efficiency. The team also observed cross-loop visual attention reorganization, termed 'Visual Aha Moment'.
Coverage timeline
机器之心机器之心
Loop Transformer，能否真正扩展到视觉语言模型？ 近年来，Loop Transformer 逐渐从简单的模块复用走向更复杂的分层循环计算，探索如何通过共享参数持续更新模型状态。一个自然的问题是：当模型开始看图，这套机制还成立吗？ 图像中往往同时包含文字、物体和空间关系，而不同循环可能需要关注不同的视觉证据。再次调用相同参数，模型会继续关注原来的位置，还是转向第一遍忽略的地方？ 近日，来自中国人民大学、百度、清枢智汇、上海交通大学、清华等机构的研究者提出 LoopVL ，围绕这一问题完成了从语言预训练到多模态后训练的系统探索。除了验证循环视觉语言建模的可行性，团队还观察到明显的跨循环视觉注意力重组，并将其称为 Visual Aha Moment 。 论文标题：LoopVL: Recurrent Visual Intelligence 论文地址：arxiv.org/abs/2609.38426 开源代码：github.com/Tier-Flow/LoopVL 模型权重：huggingface.co/TierFlow/LoopVL 先看结果：约 1B 语言主干，能走多远？ 先看一个非常直接的对比：LoopVL 仅用 0.14T 训练 token，而 Qwen3.5-2B 和 InternVL3.5-2B 约用了 36T，训练数据量相差约 260 倍。 尽管数据规模远小，LoopVL 在 LogicVista、MMMU-Pro、MMStar、RealWorldQA、MMK12-Math、MathVision 等多项评测上，依然能够达到甚至超过这些公开 VLM 的表现，体现出较强的数据效率与推理潜力。 LoopVL 与 Qwen3.5、InternVL 等公开 VLM 的性能比较，涵盖通用理解、幻觉、数学与科学等任务。各模型训练 token 规模列于首行。（论文表 2） 要判断循环有没有带来价值，不能只把不同来源的模型摆在一起。团队构建了三个采用相同 Transformer block 设计的非循环基线：32 层的 Transformer-VL 1B，以及通过增加深度或宽度扩展的两个约 4B 模型，并统一使用 0.14T 训练 token 预算。 相比两个约 4B 的 Transformer 基线，LoopVL 仍保持约 1B 的语言主干，通过循环复用共享模块增
Related stories
Apple and Duke propose LensVLM, compressing long context into images with on-demand zoom
Apple's foundation model team and Duke University introduced LensVLM, a method that compresses long context into images and lets the model zoom into relevant pages on demand. In seven text QA benchmarks, it achieved 68.9% average accuracy with 4.3x effective compression, close to the 72.4% of full-text reference. The work was accepted at NeurIPS 2026, and a public 9B demo is available.
1 reports · 1 sources
Huawei, CUHK, HKUST release LegoFlow for automated code data engineering
Researchers from Huawei, CUHK, and HKUST introduced LegoFlow, an automated framework that streamlines code data engineering from repository collection to training and evaluation. They released LegoFlow-SWE with 5,000 tasks and 2,780 verified trajectories, and demonstrated recursive self-improvement: two iterations raised Qwen3.5-35B-A3B-Base's SWE-bench Verified score from 7.6% to 64.4%.
RAVEL: Training-Free Method Improves Vision Encoder Evaluation for Multimodal LLMs
Researchers proposed RAVEL, a training-free evaluation method with two improvements—stabilized PCA whitening and patch-level similarity scoring—to predict how well vision encoders perform in multimodal LLMs. Experiments on 210 multimodal LLMs showed existing cross-modal methods degrade as candidate pools grow, while RAVEL achieved an average Spearman correlation of 0.85, outperforming prior methods by up to 101.1%.
Ant Group open-sources Ling-3.1-flash, a 560B-parameter MoE model with 1M context
Ant Group's Ling-3.1-flash, a 560B-parameter mixture-of-experts model with 25B active parameters and a 1M-token context window, was released open-source on October 15. The model targets long-horizon complex task execution, using a hybrid linear attention design with KDA layers and Gated MLA to manage computational costs on long sequences.
News updates every hour. Subscribe in your RSS reader to get new stories automatically.
Subscribe