Grokking Can Be Steered: Selecting Desirable Solutions by Choice of Learning Rule
Summary (Overview)
-
Key finding: The learning rule (combination of credit assignment algorithm and optimizer) controls the size of the neural circuit that implements a grokked solution, independently of the learned algorithm itself.
-
Mechanism identified: A neuron stays in use when its credit (gradient magnitude) is large relative to the non-gradient term of its update (coupled weight decay, Adam's , or decayed per-neuron gain). This single condition unifies three previously separate routes to unused neurons.
-
Controlled intervention: "Credit equalization" — rescaling each neuron's gradient magnitude to the layer mean while preserving direction — increases effective circuit size from ~154 to ~254 of 256 neurons on modular addition, with test accuracy dropping only from 0.998 to 0.976.
-
Group-size threshold: Normalizing neurons individually distributes the circuit (participation 0.99), but groups of 8+ neurons sharing one normalization factor remain concentrated (participation ~0.60), matching the reference.
-
Trade-off discovered: Distributed circuits tolerate neuron deletion better (tolerance 0.770 vs 0.674), while concentrated circuits can be pruned further (62% vs 20% removable) and adapt ~2.4× faster to subtraction by recruiting their nearly unused neurons.
Introduction and Theoretical Foundation
Background
Grokking refers to the phenomenon where a neural network reaches near-perfect training accuracy while test accuracy remains near zero for many epochs before improving abruptly. The paper distinguishes two aspects of a grokked solution:
- Algorithm: The computational procedure used to compute the function (e.g., Fourier-based computation for modular addition)
- Circuit: How that procedure is implemented across neurons
Key Questions
The authors ask whether circuit concentration is:
- Required by the task itself, or
- Produced by the training procedure (learning rule)
Previous work (Nanda et al., 2023; Chughtai et al., 2023) described Fourier circuits for modular addition but left open whether concentration is inherent to the task. Optimization research (Gunasekar et al., 2018; Woodworth et al., 2020) suggests learning rules select among many parameter settings that fit training data, but circuit studies all used similar backpropagation-based rules.
Central Hypothesis
Circuit concentration depends on how the learning rule distributes credit across neurons, where credit denotes the learning signal assigned to a neuron.
Theoretical Foundation
The paper builds on:
- Implicit bias literature: How optimization selects solutions among many fitting parameter settings
- Grokking mechanism: Weight decay rewards efficient circuits (larger logits at given parameter norm)
- Adaptive optimizer behavior: Adam with L2 penalty drives nodes to zero (Yaguchi et al., 2018; Mehta et al., 2019); Adam's can kill units with tiny second-moment estimates (Dufort-Labbe et al., 2025); decayed per-unit gains act as sparsity-inducing gates (Kolb et al., 2025)
Methodology
Task and Network
- Task: Modular arithmetic: , target with
- Primary task: Addition modulo prime
- Network: Fully connected ReLU network, two hidden layers of width 256, linear readout
- Training: Backpropagation with Adam, cross-entropy loss, coupled weight decay (L2 regularization added to gradient before Adam's moment estimates)
- Data split: 40% of operand pairs held out, full-batch training
Key Definitions
Circuit participation ratio (from outgoing weight norms ):
Effective neuron count = .
Credit equalization intervention — rescales each neuron's gradient magnitude toward the layer mean with strength :
where is the mean credit magnitude.
Learning Rules Tested
| Learning Rule | Transformation of Learning Signal |
|---|---|
| Backpropagation with Adam (reference) | per-parameter adaptive scaling |
| Credit equalization, then Adam | one positive factor per neuron, applied before Adam |
| LARS | one factor per layer |
| LAMB | per-parameter scaling + layerwise trust ratio |
| RMSProp | per-parameter second-moment scaling |
| Lion | elementwise sign of momentum signal |
| Adafactor | factored second-moment scaling |
| Muon | matrix orthogonalization |
| Shampoo | matrix preconditioning |
| Nero | one factor per neuron + weight constraints |
| DFC | controller-based credit, no gradient rescaling |
Group Normalization Sweep
Dividing neurons into groups of size and rescaling all gradients in a group by one common factor:
Empirical Validation / Results
1. Credit Equalization Distributes the Circuit
- Increasing from 0 to 1 raises circuit participation from 0.60 to 0.99 (effective neurons: ~154 → 254 of 256)
- Training loss, margin, and parameter norm stay within a few percent of reference
- Test accuracy decreases from 0.998 to 0.976
- Post-grokking application: Switching to equalization after grokking redistributes the circuit within 300 epochs at unchanged test accuracy
- At width 2048: reference uses 332 effective neurons, equalized uses 2046
2. The Per-Neuron Condition Unifies Three Routes to Unused Neurons
| Non-gradient term | Network | Reference | Equalized | Term rescaled | Ratio held |
|---|---|---|---|---|---|
| Coupled decay | MLP, Adam with L2 | 0.606 | 0.993 | 0.993 | 0.574 |
| Adam's | Transformer, AdamW | 0.609 | 0.994 | 0.981 | 0.818 |
| Decay of the gain | MLP, layer normalization | 0.366 | 0.996 | 0.997 | 0.343 |
Key insight: Rescaling only the non-gradient term reproduces equalization; holding the credit-to-term ratio fixed keeps the circuit concentrated.
3. Group Size Determines Whether Normalization Distributes
- Groups of 8+ neurons: participation stays ~0.60 (concentrated)
- Groups of 4: 0.70
- Pairs: 0.75
- Single neurons: 0.99 (distributed)
4. Optimizer Comparison
- Distributed circuits: Nero (participation 0.986, normalizes per neuron), DFC (credit participation 0.94)
- Concentrated circuits: LARS, RMSProp, LAMB, Adafactor, Lion, Muon, Shampoo (participation 0.470–0.613)
- Lion gives equal update norms but keeps circuit concentrated because it normalizes the coupled decay together with the gradient
5. Which Learning Rules Grok
- Grok: Backpropagation with tested optimizers, credit equalization, DFC, rules with fresh or fading orthogonal components
- Memorize only: Feedback alignment, direct feedback alignment, sign-SGD, rule with fixed orthogonal component
- The null-space weight norm separates outcomes without overlap (means 5.3–8.5 for grokking, 26–135 for memorization)
6. Trade-off: Robustness vs. Compressibility vs. Adaptation
| Metric | Concentrated () | Distributed () | From scratch |
|---|---|---|---|
| Random deletion tolerance | 0.674 | 0.770 | — |
| Norm-ranked deletion tolerance | 0.332 | 0.581 | — |
| Greedy deletion, neurons removed | 23 | 61 | — |
| Magnitude pruning, fraction removed | 0.62 | 0.20 | — |
| Greedy pruning, neurons removed | 191 | 160 | — |
| Adaptation to subtraction, epochs | 640 (128) | 1540 (180) | 1410 (137) |
| Adaptation to multiplication, epochs | 1800 (141) | 1940 (136) | 1200 (63) |
Key findings on adaptation:
- Concentrated circuit adapts ~2.4× faster to subtraction (uses same Fourier frequencies differently)
- Nearly unused neurons account for ~2/3 of the advantage (removing them: 1204 vs 652 epochs)
- Multiplication (different frequencies) shows no advantage from addition pretraining
Theoretical and Practical Implications
Theoretical Implications
-
Circuit size is not task-determined: The same Fourier algorithm can be implemented in circuits ranging from 60% to 99% participation, depending on the learning rule.
-
Unified mechanism for neuron death: Three previously separate routes to unused neurons (coupled weight decay, Adam's , decayed per-neuron gain) follow one condition: credit relative to non-gradient update term.
-
Credit allocation as implicit bias: The learning rule's normalization scheme acts as an implicit bias that selects circuit size, complementing existing implicit-bias literature.
-
Mechanistic interpretability caveat: Circuit size found in trained networks reflects the learning rule, not just the task. Comparing learning rules separates task-required from optimizer-selected properties.
Practical Implications
-
Robustness engineering: Distributed circuits (via Nero, DFC, or credit equalization) tolerate neuron deletion better — relevant for hardware faults.
-
Compressibility: Concentrated circuits can be pruned more aggressively (62% vs 20% removable), enabling smaller deployed models.
-
Transfer learning: Concentrated circuits adapt faster to related tasks by recruiting unused neurons — a form of built-in spare capacity.
-
Architecture design: The choice of optimizer normalization scheme (per-neuron vs. per-layer vs. per-parameter) directly controls the robustness-compressibility trade-off.
-
Scaling to language models: The condition governs MLP units in a 30M-parameter language model trained with Adam and L2 regularization — balancing each unit's decay leaves 1.3% instead of 24% nearly unused at lower validation loss.
Conclusion
Main takeaways:
- Circuit concentration is a consequence of training dynamics, not the task alone
- A single per-neuron condition (credit relative to non-gradient update term) unifies multiple routes to unused neurons
- Per-neuron normalization (credit equalization, Nero, DFC) distributes circuits; group normalization retains concentration
- The learning rule trades robustness against compressibility and transfer speed without changing the learned algorithm
Future directions and limitations:
- The theory isolates the distribution mechanism in a simplified linear model without adaptive normalization; nonlinear results are empirical
- Under AdamW without decay on normalization parameters, the remaining effect is small (participation 0.91–0.96) and not explained by the condition
- Algorithm checks cover modular addition only
- Adaptation advantage appears for targets needing new hidden combinations of operand frequencies, not for readout-only changes
- Robustness advantage concerns neuron deletion specifically
Core message: The learning rule steers which of many possible circuit implementations of the same algorithm a network finds, enabling deliberate selection of desirable properties (robustness vs. compressibility vs. transfer speed) through the choice of credit assignment and optimizer.
Related papers
- Done, But Still Working: Anytime Evaluation of When Coding Agents Should Stop
Anytime evaluation of coding agents shows they waste 42-66% of run cost after passing self-tests, and stopping rules can save up to 57% without losing quality.
- From Repositories to Training Environments: Scaling Long-Horizon Coding Agents with Environment Evolution Loop
RepoFactory's proxy-driven environment evolution loop yields RepoHorizen-35B-A3B, a model improving coding-agent benchmarks by 22.6 points on NL2Repo and 8.5 points on average.
- Rethinking Fixed-Horizon Latent Reasoning
Adaptive latent inference that stops reasoning when progress stalls improves accuracy by up to 7.2 points while cutting computation by 29%, as completing fixed latent horizons can degrade performance.