Related papers
- Closing the Loop: Practical Training Recipes for Looped Language Models
Looped language models beat parameter-matched dense transformers on all 12 benchmarks using 25x less training data, via a two-stage recipe and minimal dense-to-loop conversion.
- Adaptive-Depth Is a Finishing Stage
Fixed-first training cuts adaptive-depth transformer training compute by 30-45% with negligible quality loss by reserving full-depth updates for a brief final phase.
- Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
Looped transformers retain a global workspace, but recurrence reshapes its interfaces: reads and writes follow iteration boundaries, not absolute depth, yet neither model preserves injected content beyond roughly two recurrences.