Looping Is Not Free: Scaling Boundaries of Post-Training Loops and Positive RLVR Results
Highlights of This Issue
The most prominent signal this issue is the debate over "whether looping is worth it" being brought to the table—two papers provide evidence from opposite ends. Dory1.0-4B is the first large-scale fully post-trained looped language model (4B, 6T tokens), combining RLVR with a difficulty-aware looping curriculum: routing harder prompts to deeper loops based on prompt pass rates, yielding the largest gains on agentic benchmarks (Terminal-Bench v2.1 from 14.0% to 21.4%), and providing last-loop KV caching (decoupling KV memory from loop count), self-speculative decoding with different loop counts from the same checkpoint, and a token efficiency analysis showing "deeper loops generate fewer output tokens"—looping becomes a routable compute axis in RLVR. Meanwhile, There Is No Free Loop proposes a dual-test protocol (isoparametric twins + equal-FLOP best non-looped usage), systematically proving on 0.8B–9B that only 0.8B looping passes both tests simultaneously, with 4B/9B showing no gains even ignoring compute, and noting that part of the 0.8B gain comes from the decoding artifact of "completing the answer." The differences between the two—pretraining-built-in looping vs. post-training-added looping, RLVR vs. fine-tuning, difficulty routing vs. uniform depth—are precisely the boundaries most worth questioning this issue.
The second theme is the mechanism lens continuing to answer "what looping actually computes." Not All Thinking is Created Equal provides the first mechanistic comparison of five thinking interfaces under a unified backbone and training budget, finding that implicit latent deliberation (especially Bottleneck-latent) can spontaneously discover sparse looped search circuits and achieve deep generalization, while explicit CoT performs worst—this is an empirical mechanistic complement to the constructive theory of Internalizing CoT from Issue 4. Inside the Loop uses probes and activation patching to confirm that looped models execute approximately one hop per iteration and compress multi-hop computations into a single iteration when the iteration budget is insufficient, while untied models with equal effective depth fail to learn even with 8x parameters—providing direct evidence for the algorithmic execution properties of looping.
The third theme is new axes in residual design and training objectives. Loopback Transformer unifies looped residual connections with linear attention and looped memory into a "depth-memory" perspective, decomposing update coefficients into three independently controllable quantities: input strength, total update weight, and cross-update allocation, providing matched token and depth comparisons at 191M–3B. ARLoop adapts attention residuals to looped transformers, finding that sliding-window depth attention (retaining only the embedding plus the most recent 3 residual blocks) matches or even exceeds full-history attention at deeper loops, with large-scale validation at 0.6B/1.2B. RELOAD proves for the first time that damped looping of middle blocks in pretrained decision models with joint training significantly improves long-chain reasoning, and this gain cannot be replaced by matched training compute, more data, or non-looped depth scaling with nearly double the parameters. Best-of-Depth treats looped trajectories as a candidate set rather than a single solution path, taking the minimum loss along depth. Theoretically, Iterated Narrow Networks proves Turing completeness of ReLU networks with iterated width 4 on convex input regions, providing a verification complexity spectrum of NP/NEXP/RE-complete.
Community and Updates
- The debate over "whether looping is worth it" is fermenting in the community: There Is No Free Loop: Post-Training Loops Only Pay for Small Models disseminates this negative result as a community interpretation, while Three preprints ask whether extra inference compute buys accuracy places the negative result on loop depth into the broader skeptical framework of "whether test-time compute buys accuracy"—both are secondhand accounts of this issue's core debate, and specific conclusions should still be based on the original papers.
- Scaling evidence continues to fill in: The Benefit of Looping Twice reports a scaling study of approximately 600 training runs, quantifying the Pareto frontier of loop count versus parameters/compute in dense and MoE models, identifying two fully trained loops as a practical inflection point—echoing the truncation debate in Issue 3's Truncate or Not.
Open Questions
- Dory achieves large gains at 4B through RLVR with a difficulty-aware looping curriculum, while There Is No Free Loop proves no gains from post-training loops at 4B/9B—among the differences (pretraining-built-in looping vs. post-training-added looping, RLVR vs. fine-tuning, difficulty routing vs. uniform depth), which is the key variable determining "whether looping is worth it"?
- Not All Thinking is Created Equal localizes the sparse looped search circuit in Bottleneck-latent (L4H1 as soft tracker, L4MLP as filter)—how does this circuit correspond to the "computation state" direction in Issue 4's Beyond the Training Horizon and the carry/re-read mechanisms in Issue 2's Monitoring? Is the sparse circuit a sufficient condition for successful depth extrapolation?
- Loopback Transformer's robustness to extra accesses directly addresses the overthinking problem—can a unified residual perspective provide a more reliable criterion for adaptive depth stopping than "stop when the answer is readable"? How does this reconcile with the "readable but not frozen for reuse" finding in Issue 3's Learning to Compute?
- Best-of-Depth treats trajectories as a candidate set and takes the minimum loss along depth—coverage of the candidate set improves significantly on synthetic tasks, but does this come at the cost of single-depth accuracy? Can the candidate-set idea transfer to test-time scaling in language model reasoning?
Papers in this issue
Dory1.0-4B, a looped 4B model, outperforms models up to 9x larger by scaling compute per token in latent space rather than generating longer outputs.
Editor's noteFirst large-scale fully post-trained looped language model (4B, 6T tokens), combining RLVR with a difficulty-aware looping curriculum—routing harder prompts to deeper loops based on prompt pass rates, yielding the largest gains on agentic benchmarks (Terminal-Bench v2.1 from 14.0% to 21.4%). Also provides last-loop KV caching (decoupling KV memory from loop count), self-speculative decoding with different loop counts from the same checkpoint, and a token efficiency analysis showing "deeper loops generate fewer output tokens." Relative to the recipe improvements in Issue 4's Closing the Loop, the increment is treating looping as a routable compute axis in RLVR with a complete production pipeline.
Looping transformer layers only improves accuracy in small 0.8B models, not at 4B or 9B, and is only worthwhile under a fixed parameter budget.
Editor's noteA direct cold shower on "post-training-added looping." Proposes a dual-test protocol (isoparametric twins + equal-FLOP best non-looped usage), systematically proving on 0.8B–9B that only 0.8B looping passes both tests, with 4B/9B showing no gains even ignoring compute, and using the benchmark gap between 4B and 9B to rule out ceiling effects. Relative to the positive recipes in Issue 4's Closing the Loop and How to Loop, this is the first scale-dependent negative result with a rigorous control protocol, noting that part of the 0.8B gain comes from the decoding artifact of "completing the answer," requiring budget-forced matching.
RELOAD retrofits damped loops into pretrained decision models, boosting accuracy up to 23 points on long reasoning chains with zero added parameters via joint training.
Editor's noteFirst to prove in typed decision models that damped looping of middle blocks in pretrained models with joint training significantly improves long-chain reasoning (up to +23 points on encoder tasks beyond training chain lengths), and this gain cannot be replaced by matched training compute, more data, or non-looped depth scaling with nearly double the parameters. Relative to Issue 1's SanSi (looped decision models but relying on looped pretraining backbones) and Issue 4's Accordion (fixed-prefix elastic depth), the increment is a retrofit looping recipe for non-looped pretrained models (damped updates + full-budget supervision + request-level controller), proving that the update rule (damping vs. gating) is key to retrofit success.
Latent reasoning models discover a generalizable recurrent forward-search algorithm, while token-based methods like CoT rely on local shortcuts that fail out-of-distribution.
Editor's noteFirst mechanistic comparison of five thinking interfaces under a unified backbone and training budget, finding that implicit latent deliberation (especially Bottleneck-latent) can spontaneously discover sparse looped search circuits and achieve deep generalization, while explicit CoT performs worst. Relative to Issue 4's Internalizing CoT constructive theory interpolation, this provides empirical mechanistic evidence; relative to Issue 2's Monitoring latent reasoning cross-family comparison, the increment is isolating the thinking interface itself and localizing specific circuit components (L4H1 as soft tracker, L4MLP as filter). Limitation: synthetic graph reasoning tasks; generalization to large language models remains to be verified.
The Loopback Transformer stores block updates separately with decay-bounded attention, beating standard Transformers with 40-75% of their parameters.
Editor's noteUnifies looped transformer residual connections with linear attention and looped memory into a "depth-memory" perspective, decomposing update coefficients into three independently controllable quantities: input strength, total update weight, and cross-update allocation. Key innovation: storing a single block update rather than full block output, constraining total weight with geometric decay, softmax attention reallocating by content, and input kept on an independent path with coefficient one. A clear increment over residual designs like AttnRes and Hyperloop, providing matched token and depth comparisons at 191M–3B, with robustness to extra accesses directly addressing the overthinking problem.
Best-of-Depth training, which minimizes loss across all recurrent depths, maximizes solution coverage in looped transformers, outperforming standard objectives on combinatorial tasks and code generation.
Editor's noteChanges the looped transformer training objective from "final-depth supervision" to "minimum loss along depth" candidate-set training, allowing different depths to match different valid solutions, with theoretical links between exact match loss and solution coverage. Relative to Issue 4's Internalizing CoT constructive bounds and Issue 3's PAR progress-aligned objectives, this is a new axis at the training-objective level—treating looped trajectories as a candidate set rather than a single solution path, requiring no architectural changes or depth labels. Provides comparisons with Last-Depth and Mean-Depth on N-Queens, graph coloring, and code generation, along with depth extrapolation curves.
ARLOOP uses sliding-window depth attention over residuals to outperform prior looped transformer designs, matching larger models with far less training data.
Editor's noteAdapts attention residuals (Kimi team's design on standard transformers) to looped transformers, finding that sliding-window depth attention (retaining only the embedding plus the most recent 3 residual blocks) matches or even exceeds full-history attention at deeper loops—because representations across loops are highly redundant, contrasting with the finding in standard transformers that sliding windows hurt gains. A clear increment over Issue 3's Towards Looped Models Done Right residual designs and Sandwich Norm, Parcae, etc., with large-scale validation on math and code at 0.6B/1.2B, 180B/500B tokens, showing gains beyond perplexity. Limitation: fixed loops, no adaptive depth.
A single width-4 iterated ReLU network can simulate any Turing machine on convex inputs, making verification NP-complete, NEXP-complete, or undecidable depending on the time bound.
Editor's noteProves Turing completeness of ReLU networks with iterated width 4 on convex input regions, using the fourth coordinate as an online validity certificate to resolve the continuity obstacle, and provides a verification complexity spectrum of NP/NEXP/RE-complete along time bounds. Relative to prior width-3 simulations, the increment is making the reduction work on convex sets, thereby obtaining certificate mechanisms for NEXP and undecidability results. Direct value for understanding expressiveness and verification cost of looped/iterated models; purely theoretical, no experiments.
