Grokking Can Be Steered: Selecting Desirable Solutions by Choice of Learning Rule

Summary (Overview)

  • Key finding: The learning rule (combination of credit assignment algorithm and optimizer) controls the size of the neural circuit that implements a grokked solution, independently of the learned algorithm itself.

  • Mechanism identified: A neuron stays in use when its credit (gradient magnitude) is large relative to the non-gradient term of its update (coupled weight decay, Adam's ϵ\epsilon, or decayed per-neuron gain). This single condition unifies three previously separate routes to unused neurons.

  • Controlled intervention: "Credit equalization" — rescaling each neuron's gradient magnitude to the layer mean while preserving direction — increases effective circuit size from ~154 to ~254 of 256 neurons on modular addition, with test accuracy dropping only from 0.998 to 0.976.

  • Group-size threshold: Normalizing neurons individually distributes the circuit (participation 0.99), but groups of 8+ neurons sharing one normalization factor remain concentrated (participation ~0.60), matching the reference.

  • Trade-off discovered: Distributed circuits tolerate neuron deletion better (tolerance 0.770 vs 0.674), while concentrated circuits can be pruned further (62% vs 20% removable) and adapt ~2.4× faster to subtraction by recruiting their nearly unused neurons.

Introduction and Theoretical Foundation

Background

Grokking refers to the phenomenon where a neural network reaches near-perfect training accuracy while test accuracy remains near zero for many epochs before improving abruptly. The paper distinguishes two aspects of a grokked solution:

  1. Algorithm: The computational procedure used to compute the function (e.g., Fourier-based computation for modular addition)
  2. Circuit: How that procedure is implemented across neurons

Key Questions

The authors ask whether circuit concentration is:

  • Required by the task itself, or
  • Produced by the training procedure (learning rule)

Previous work (Nanda et al., 2023; Chughtai et al., 2023) described Fourier circuits for modular addition but left open whether concentration is inherent to the task. Optimization research (Gunasekar et al., 2018; Woodworth et al., 2020) suggests learning rules select among many parameter settings that fit training data, but circuit studies all used similar backpropagation-based rules.

Central Hypothesis

Circuit concentration depends on how the learning rule distributes credit across neurons, where credit denotes the learning signal assigned to a neuron.

Theoretical Foundation

The paper builds on:

  • Implicit bias literature: How optimization selects solutions among many fitting parameter settings
  • Grokking mechanism: Weight decay rewards efficient circuits (larger logits at given parameter norm)
  • Adaptive optimizer behavior: Adam with L2 penalty drives nodes to zero (Yaguchi et al., 2018; Mehta et al., 2019); Adam's ϵ\epsilon can kill units with tiny second-moment estimates (Dufort-Labbe et al., 2025); decayed per-unit gains act as sparsity-inducing gates (Kolb et al., 2025)

Methodology

Task and Network

  • Task: Modular arithmetic: a,b∈Zp={0,1,…,p−1}a, b \in \mathbb{Z}_p = \{0, 1, \dots, p-1\}, target c=(a∘b) mod pc = (a \circ b) \bmod p with ∘∈{+,−,×}\circ \in \{+, -, \times\}
  • Primary task: Addition modulo prime p=31p = 31
  • Network: Fully connected ReLU network, two hidden layers of width 256, linear readout
  • Training: Backpropagation with Adam, cross-entropy loss, coupled weight decay 10−310^{-3} (L2 regularization added to gradient before Adam's moment estimates)
  • Data split: 40% of operand pairs held out, full-batch training

Key Definitions

Circuit participation ratio (from outgoing weight norms aj=∥W:,jout∥2a_j = \|\mathbf{W}^{\text{out}}_{:,j}\|_2):

PR=(∑jaj)2n∑jaj2∈[1/n,1]\mathrm{PR} = \frac{\left(\sum_{j} a_j\right)^2}{n \sum_{j} a_j^2} \in [1/n, 1]

Effective neuron count = n⋅PRn \cdot \text{PR}.

Credit equalization intervention — rescales each neuron's gradient magnitude toward the layer mean with strength β∈[0,1]\beta \in [0, 1]:

g~j=gj∥gj∥2((1−β)∥gj∥2+βgˉ)for ∥gj∥2>0\widetilde{\mathbf{g}}_j = \frac{\mathbf{g}_j}{\|\mathbf{g}_j\|_2}\left((1 - \beta)\|\mathbf{g}_j\|_2 + \beta \bar{g}\right) \quad \text{for } \|\mathbf{g}_j\|_2 > 0

where gˉ=1n∑k=1n∥gk∥2\bar{g} = \frac{1}{n}\sum_{k=1}^n \|\mathbf{g}_k\|_2 is the mean credit magnitude.

Learning Rules Tested

Learning RuleTransformation of Learning Signal
Backpropagation with Adam (reference)per-parameter adaptive scaling
Credit equalization, then Adamone positive factor per neuron, applied before Adam
LARSone factor per layer
LAMBper-parameter scaling + layerwise trust ratio
RMSPropper-parameter second-moment scaling
Lionelementwise sign of momentum signal
Adafactorfactored second-moment scaling
Muonmatrix orthogonalization
Shampoomatrix preconditioning
Neroone factor per neuron + weight constraints
DFCcontroller-based credit, no gradient rescaling

Group Normalization Sweep

Dividing neurons into groups of size mm and rescaling all gradients in a group by one common factor:

g~j=gjgˉgˉB(j),gˉB(j)=1m∑k∈B(j)∥gk∥2\widetilde{\mathbf{g}}_j = \mathbf{g}_j \frac{\bar{g}}{\bar{g}_{B(j)}}, \quad \bar{g}_{B(j)} = \frac{1}{m}\sum_{k \in B(j)} \|\mathbf{g}_k\|_2

Empirical Validation / Results

1. Credit Equalization Distributes the Circuit

  • Increasing β\beta from 0 to 1 raises circuit participation from 0.60 to 0.99 (effective neurons: ~154 → 254 of 256)
  • Training loss, margin, and parameter norm stay within a few percent of reference
  • Test accuracy decreases from 0.998 to 0.976
  • Post-grokking application: Switching to equalization after grokking redistributes the circuit within 300 epochs at unchanged test accuracy
  • At width 2048: reference uses 332 effective neurons, equalized uses 2046

2. The Per-Neuron Condition Unifies Three Routes to Unused Neurons

Non-gradient termNetworkReferenceEqualizedTerm rescaledRatio held
Coupled decayMLP, Adam with L20.6060.9930.9930.574
Adam's ϵ\epsilonTransformer, AdamW0.6090.9940.9810.818
Decay of the gainMLP, layer normalization0.3660.9960.9970.343

Key insight: Rescaling only the non-gradient term reproduces equalization; holding the credit-to-term ratio fixed keeps the circuit concentrated.

3. Group Size Determines Whether Normalization Distributes

  • Groups of 8+ neurons: participation stays ~0.60 (concentrated)
  • Groups of 4: 0.70
  • Pairs: 0.75
  • Single neurons: 0.99 (distributed)

4. Optimizer Comparison

  • Distributed circuits: Nero (participation 0.986, normalizes per neuron), DFC (credit participation 0.94)
  • Concentrated circuits: LARS, RMSProp, LAMB, Adafactor, Lion, Muon, Shampoo (participation 0.470–0.613)
  • Lion gives equal update norms but keeps circuit concentrated because it normalizes the coupled decay together with the gradient

5. Which Learning Rules Grok

  • Grok: Backpropagation with tested optimizers, credit equalization, DFC, rules with fresh or fading orthogonal components
  • Memorize only: Feedback alignment, direct feedback alignment, sign-SGD, rule with fixed orthogonal component
  • The null-space weight norm qq separates outcomes without overlap (means 5.3–8.5 for grokking, 26–135 for memorization)

6. Trade-off: Robustness vs. Compressibility vs. Adaptation

MetricConcentrated (β=0\beta = 0)Distributed (β=1\beta = 1)From scratch
Random deletion tolerance0.6740.770—
Norm-ranked deletion tolerance0.3320.581—
Greedy deletion, neurons removed2361—
Magnitude pruning, fraction removed0.620.20—
Greedy pruning, neurons removed191160—
Adaptation to subtraction, epochs640 (128)1540 (180)1410 (137)
Adaptation to multiplication, epochs1800 (141)1940 (136)1200 (63)

Key findings on adaptation:

  • Concentrated circuit adapts ~2.4× faster to subtraction (uses same Fourier frequencies differently)
  • Nearly unused neurons account for ~2/3 of the advantage (removing them: 1204 vs 652 epochs)
  • Multiplication (different frequencies) shows no advantage from addition pretraining

Theoretical and Practical Implications

Theoretical Implications

  1. Circuit size is not task-determined: The same Fourier algorithm can be implemented in circuits ranging from 60% to 99% participation, depending on the learning rule.

  2. Unified mechanism for neuron death: Three previously separate routes to unused neurons (coupled weight decay, Adam's ϵ\epsilon, decayed per-neuron gain) follow one condition: credit relative to non-gradient update term.

  3. Credit allocation as implicit bias: The learning rule's normalization scheme acts as an implicit bias that selects circuit size, complementing existing implicit-bias literature.

  4. Mechanistic interpretability caveat: Circuit size found in trained networks reflects the learning rule, not just the task. Comparing learning rules separates task-required from optimizer-selected properties.

Practical Implications

  1. Robustness engineering: Distributed circuits (via Nero, DFC, or credit equalization) tolerate neuron deletion better — relevant for hardware faults.

  2. Compressibility: Concentrated circuits can be pruned more aggressively (62% vs 20% removable), enabling smaller deployed models.

  3. Transfer learning: Concentrated circuits adapt faster to related tasks by recruiting unused neurons — a form of built-in spare capacity.

  4. Architecture design: The choice of optimizer normalization scheme (per-neuron vs. per-layer vs. per-parameter) directly controls the robustness-compressibility trade-off.

  5. Scaling to language models: The condition governs MLP units in a 30M-parameter language model trained with Adam and L2 regularization — balancing each unit's decay leaves 1.3% instead of 24% nearly unused at lower validation loss.

Conclusion

Main takeaways:

  1. Circuit concentration is a consequence of training dynamics, not the task alone
  2. A single per-neuron condition (credit relative to non-gradient update term) unifies multiple routes to unused neurons
  3. Per-neuron normalization (credit equalization, Nero, DFC) distributes circuits; group normalization retains concentration
  4. The learning rule trades robustness against compressibility and transfer speed without changing the learned algorithm

Future directions and limitations:

  • The theory isolates the distribution mechanism in a simplified linear model without adaptive normalization; nonlinear results are empirical
  • Under AdamW without decay on normalization parameters, the remaining effect is small (participation 0.91–0.96) and not explained by the condition
  • Algorithm checks cover modular addition only
  • Adaptation advantage appears for targets needing new hidden combinations of operand frequencies, not for readout-only changes
  • Robustness advantage concerns neuron deletion specifically

Core message: The learning rule steers which of many possible circuit implementations of the same algorithm a network finds, enabling deliberate selection of desirable properties (robustness vs. compressibility vs. transfer speed) through the choice of credit assignment and optimizer.

Related papers