Weekly Gallery

Aug 8 – 15, 2026

Directions: 7 · Picks: 63

AI for Formal Math · Issue 1

Issue 1: Neural Theorem Proving Enters Open Research Problems, Kernel Soundness and Benchmark Reliability Take Center Stage

The most central signal this issue is that neural theorem proving has officially moved from competition problems to open research problems. In AlphaProof Nexus,…

  1. Advancing Mathematics Research with AI-Driven Formal Proof SearchThe first large-scale evaluation of LLM-driven formal proof search on open research-level problems: autonomously solving 9 open Erdős problems (including two open for 56 years) and 44 OEIS conjectures in Lean 4. The key increment is that ablation experiments show the basic agent of "LLM generation + Lean compiler feedback" performs comparably to the full agent on most problems, and all proofs and cost analyses are publicly available, making it essential reading for understanding current agent design trade-offs.
  2. Formal Conjectures: An Open and Evolving Benchmark for Verified Discovery in MathematicsProvides 2,615 Lean 4 formalized propositions (including 1,029 zero-contamination open conjectures and 836 solvable propositions), along with frozen evaluation subsets and a misformalization taxonomy, directly addressing the data leakage and saturation issues of MiniF2F/PutnamBench. As the vehicle for systems like AlphaProof Nexus to solve open problems, it has become the de facto standard for research-grade formal benchmarks, worth reading for anyone working on neural theorem proving or automated formalization.
  3. MechGeo: Autoformalizing and Proving Euclidean Geometry in Lean 4Proposes the GeoIR deterministic intermediate representation and the selective algebraization prover GeoProver, unifying automated formalization and proving of Euclidean geometry into Mathlib-native Lean 4. It jumps from the previous best 2/14 to 12/14 on Lean-IMO-Bench, and completes the remaining propositions through counterexample-guided repair of missing non-degeneracy conditions, with all code and proofs publicly available, marking a reproducible milestone in geometry formalization.
All picks: 9

Pretraining Data · Issue 1

Issue 1: The Bitter Fruit of Filtering — Mixing, Duplication, and Extraction Quality Under Data Constraints

This week's leads pull "data quality" back to the first-principles level, away from just cleaning pipelines. Document extraction quality is being systematically…

  1. A Bitter Lesson for Data FilteringProvides systematic scaling evidence in the high-compute, data-scarce regime, arguing that 'the best filter is no filter': with sufficiently large models and training steps, training directly on raw Common Crawl can surpass heavily filtered DCLM-Baseline/RefinedWeb, and even shuffled word documents provide benefits. It advances the filtering question from 'threshold tightness' to 'whether any filter is superior' and provides a crossover prediction at ~1e30 FLOPs. Limitation: only evaluates base loss and small benchmarks, not post-training ceiling; however, it's enough to challenge the default assumption that 'more filtering is better'.
  2. InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and RepetitionProposes an 'information accumulation' scaling law that explicitly models quality-weighted mixing and duplication jointly. It extrapolates loss to unseen recipes and larger scales (252M–7B, up to 425B tokens) and releases the recipe search. Compared to Data Mixing Laws/RegMix/CMR, the increment is treating duplication as a term with diminishing returns with scale, rather than handling it separately from quality mixing. Limitation: evaluation metric remains validation loss, not post-training ceiling, but it is one of the few empirical frameworks that can guide both mixing and overtraining decisions.
  3. Internal Data Repetition Destroys Language ModelsModernizes duplication harm research using compute-equivalent units (CEG/CEL), finding that at a fixed 10% duplicated token ratio, harm peaks at intermediate repetition counts, and the peak location shifts with model scale according to a power law (R_peak ∝ N^-0.96). It provides a closed-form statistical analogy (misspecified linear regression with block-diagonal noise covariance) showing this effect is not unique to transformers. Limitation: small scale (max 344M) and single architecture/corpus, but the controlled iso-FLOP design and theoretical analogy make the conclusions robust enough to influence deduplication and mixing decisions.
All picks: 8

MoE & Sparse Experts · Issue 1

Issue 1: Verifiability of Sparse Expert Structures—Load Balancing, Causal Auditing, and System Measurements

This issue focuses on the "verifiability of sparse expert structures": from load balancing to expert specialization, multiple works raise the bar of evidence to…

  1. Kimi K3: Open Frontier IntelligenceAt the extreme sparse frontier scale of 2.8T parameters and 896 experts (16 active), proposes Quantile Balancing (QB)—setting expert biases directly from router-score quantiles to exactly match target load, replacing fixed-step sign-based bias updates, paired with RMSNorm and SiTU-GLU for stable optimization. This is the first scalable scheme for auxiliary-loss-free load balancing under extreme sparsity, reporting full routing statistics and a 2.5× scaling efficiency improvement, making it a must-read for load balancing and sparse stability directions.
  2. TEMPO: Makespan-Aware Expert-Parallel Load Balancing Across Memory- and Compute-Bound RegimesFirst to model expert costs in EP distribution as a max-affine cost function with memory-bound (flat) and compute-bound (linear) regimes, formalized as a fixed-charge makespan problem (NP-hard with additive approximation guarantees). On real batches, 92–100% contain both regimes, and token-count proxies systematically fail at regime mixtures; TEMPO yields 4–6% throughput and ~15.6% p99 latency improvements on Qwen3-235B, and uses phase diagrams to predict when adaptive distribution is worth the investment. A significant methodological contribution to expert-parallel serving and load balancing.
  3. How Modular Is a Frontier Mixture-of-Experts? A Pre-registered Causal Test in Which Apparent Expert Modularity Mostly DissolvesOn Command A+, performs causal ablations of six pre-registered expert families, treating corpus, metric, and statistical criteria as variables. The conclusion is cautionary: only the Arabic family is a robust module among the six, while the apparent modularity of the others flips with corpus/metric/statistical criteria. This is a benchmark work for the causal evidence threshold of expert specialization, and its "measurement-as-variable" protocol should become standard for subsequent modularity claims.
All picks: 9

Efficient Sequence Modeling · Issue 1

Issue 1: The Compound Effects of Architectural Priors and Cross-Layer Reuse of Indexers

This week's research on efficient sequence modeling presents three clear threads. The first is the "compound effects of architectural priors": Cracks in the Fou…

  1. Cracks in the Foundation: Seemingly Minor Architectural Choices Impact Long Context ExtensionConducts a four-factor ablation on 26 7B models with fully aligned data and optimization, demonstrating that combinations of "small architectural choices" such as QK norm, GQA, sliding window, and pretraining length suppress long-context scalability by up to 47% in an additive manner, and short-context metrics cannot predict long-context performance. It provides real degradation curves for HELMET/RULER/LongPPL rather than NIAH scores, and releases the OlmPool model suite, serving as an essential comparison benchmark for long-context scaling and efficient attention research.
  2. Information Abundance Paradox: Long-Context Training Undermines Parametric KnowledgeProposes the "information abundance paradox": the richer the training context, the less incentive the model has to parameterize knowledge, instead relying on context, leading to performance degradation when context is missing or misleading at inference. It provides compute-matched comparisons across context lengths with degradation curves, and offers mechanistic evidence from gradient allocation, module-constrained fine-tuning, and attention allocation, directly relevant to the design of long-context pretraining curricula.
  3. You Only Index Once: Cross-Layer Sparse Attention with Shared RoutingExtends the reuse of KV-sharing architectures (YOCO-like) from memory to routing—a single indexer computes token-level top-k once, and the results are shared across multiple cross-decoder layers, amortizing the expensive per-token routing cost. It provides compute-matched comparisons with dense baselines and degradation curves for RULER/validation loss, representing a key work on the new axis of cross-layer index reuse.
All picks: 10

Scaling Laws & Training Methods · Issue 1

Issue 1: Extrapolability of Scaling Laws, MoE Parameterization, and Large-Batch Optimizer Stability

This week's focus is on two main threads: the extrapolability and experimental design of scaling laws themselves, and the parameterization and hyperparameter tr…

  1. Skaling: Chinchilla’s Exponents Meet Kaplan’s CouplingUnifies Chinchilla's additive form and Kaplan's coupled form with a single coupling exponent k, recovering the interaction between N and D, and reducing MAPE by 1.5–3x in interpolation and extrapolation regimes. Mixed derivative analysis and boundary residual diagnostics directly address the failure of the additive independence assumption. L-shaped sparse sampling reduces fitting computation by ~10x. Recommended reading for anyone fitting scaling laws.
  2. How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable ParameterizationPoints out that although muP formally satisfies maximal-update conditions, it fails to transfer learning rate reliably on MoE or improve monotonically with scale, and proposes the 'Maximal Scale Stability' (MSSP) principle with an architectural fix in three scale regimes. Each regime has independent DMFT analysis support, marking a key advance in the muP-for-MoE line.
  3. Tokens-per-Parameter Coverage Is Critical for Robust LLM Scaling Law ExtrapolationTheoretically proves that fixed tokens-per-parameter grids cause ill-conditioned scaling law parameter estimation, with condition numbers growing inversely with exponent differences, and provides closed-form thresholds for TPP diversity to ensure identifiability. Validated on ~1900 trained models with four scaling law forms, offering actionable prior diagnostics for experimental design.
All picks: 9

Coding Agents · Issue 1

Issue 1: Benchmark Validity and Harness Attribution—When Scores Are No Longer Trustworthy

The strongest signal this issue is the rare convergence of industry and academia on "benchmark validity." OpenAI's audit of SWE-Bench Pro estimates that about 3…

  1. Don’t Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent QualitySkeptical evidence explicitly required for this direction: fixes the base model, varies only the harness across 35 sequential release versions, and is the first longitudinal controlled study to cleanly separate harness contributions from model contributions. The conclusion is that harness evolution does not improve resolve rate but significantly increases tokens and tool calls, attributing these changes to specific development patterns and architectural components. Includes a reproduction package and is the most direct empirical support for 'harness update ≠ harness benefit.'
  2. Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM AgentsDecomposes harness self-evolution into two orthogonal capabilities—'update capability' on the evolver side and 'benefit capability' on the agent side—and independently varies both in controlled analyses. Finds that update capability is flat with base capability (Qwen3.5-9B's update gains are comparable to Claude Opus 4.6), while benefit capability is non-monotonic (weak models fail to activate/follow, mid-tier benefits most, strong tiers see diminishing returns). Provides actionable guidance for capability budget allocation, with public code and failure mode analysis.
  3. One Recipe, Many Harnesses: What Self-Evolution Encodes Across Languages and ModelsFirst to fix the evolution recipe on a language×model grid and mechanistically decompose what self-evolving harnesses encode: gains are compensation for 'recoverable execution defects,' harnesses share abstract concepts but diverge on ecosystem-specific instances, and portability is constrained by ecosystem slack. On Python/GPT-5-mini, gains are zero with explanation, providing practical criteria for when to reuse, distill, or re-evolve. Code is reproducible, evaluated on held-out splits of the real codebase Multi-SWE-Bench.
All picks: 9

Recursive Self-Improvement · Issue 1

Issue 1: Making the 'Self-Improvement Rate' a Measurable, Falsifiable Object

This issue is the inaugural issue of the 'Recursive Self-Improvement' direction. Centered on the main theme of 'treating the self-improvement rate as a measurem…

  1. DarwinX: Evolving Agent Harnesses Through Natural SelectionReframes self-evolution as 'frozen model + population selection harness': the preserve-and-extend contract only accepts non-regressive extensions, the archive retains complementary lineages for recombination, and three types of signals (failure/teacher/self-derived) share the same editing interface. The four-level evaluation ladder (in-domain, held-out, synthetic-to-real, cross-benchmark) gradually separates evolution signals from testing, and explicitly reports the determination of regression variants and invalid trajectories. Read it to see how 'the cost of one improvement' is explicitly contractualized.
  2. Memory Reward Inflation in Self-Improving LLM AgentsFormalizes the 'Echo Gap' in memory circuits—frequency-dependent reward inflation—and proves that the Error-Independence Assumption is necessary for de-inflation: stronger re-scorers and ensembles cannot debias, only answer-free execution signals (LUCID) correlated with self-evaluation bias can. Provides end-to-end gains and failure rate reports on BIRD. Read it to understand why 'reward grounding' is not just about being more accurate, but must be correlated with bias solutions.
  3. Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-TrainingSystematically compares three intervention levels (inter-campaign memory, intra-campaign early stopping, algorithm-level GRPO) on the same testbed, providing an empirical decomposition of the 'rise-then-collapse' collapse pattern: campaign-level memory is only effective in fragile small-model regimes, and early stopping and GRPO target different failure modes. Multi-seed, bootstrap CI, explicit reporting of failure rates and mixed results. Read it as an empirical map for 'at which level to intervene in the self-improvement loop.'
All picks: 9