# Grokking Can Be Steered: Selecting Desirable Solutions By Choice Of Learning Rule

> The learning rule, not the task, determines neural circuit size in grokked networks, enabling deliberate trade-offs between robustness, compressibility, and transfer speed.

- **Source:** [arXiv](https://arxiv.org/abs/2611.19477)
- **Published:** 2026-11-21
- **Permalink:** https://picx.dev/p/BBW1ug
- **Whiteboard:** https://picx.dev/p/BBW1ug/image

## Summary

# Grokking Can Be Steered: Selecting Desirable Solutions by Choice of Learning Rule

## Summary (Overview)

- **Key finding**: The learning rule (combination of credit assignment algorithm and optimizer) controls the size of the neural circuit that implements a grokked solution, independently of the learned algorithm itself.

- **Mechanism identified**: A neuron stays in use when its credit (gradient magnitude) is large relative to the non-gradient term of its update (coupled weight decay, Adam's $\epsilon$, or decayed per-neuron gain). This single condition unifies three previously separate routes to unused neurons.

- **Controlled intervention**: "Credit equalization" — rescaling each neuron's gradient magnitude to the layer mean while preserving direction — increases effective circuit size from ~154 to ~254 of 256 neurons on modular addition, with test accuracy dropping only from 0.998 to 0.976.

- **Group-size threshold**: Normalizing neurons individually distributes the circuit (participation 0.99), but groups of 8+ neurons sharing one normalization factor remain concentrated (participation ~0.60), matching the reference.

- **Trade-off discovered**: Distributed circuits tolerate neuron deletion better (tolerance 0.770 vs 0.674), while concentrated circuits can be pruned further (62% vs 20% removable) and adapt ~2.4× faster to subtraction by recruiting their nearly unused neurons.

## Introduction and Theoretical Foundation

### Background

Grokking refers to the phenomenon where a neural network reaches near-perfect training accuracy while test accuracy remains near zero for many epochs before improving abruptly. The paper distinguishes two aspects of a grokked solution:

1. **Algorithm**: The computational procedure used to compute the function (e.g., Fourier-based computation for modular addition)
2. **Circuit**: How that procedure is implemented across neurons

### Key Questions

The authors ask whether circuit concentration is:
- Required by the task itself, or
- Produced by the training procedure (learning rule)

Previous work (Nanda et al., 2023; Chughtai et al., 2023) described Fourier circuits for modular addition but left open whether concentration is inherent to the task. Optimization research (Gunasekar et al., 2018; Woodworth et al., 2020) suggests learning rules select among many parameter settings that fit training data, but circuit studies all used similar backpropagation-based rules.

### Central Hypothesis

> Circuit concentration depends on how the learning rule distributes credit across neurons, where **credit** denotes the learning signal assigned to a neuron.

### Theoretical Foundation

The paper builds on:
- **Implicit bias literature**: How optimization selects solutions among many fitting parameter settings
- **Grokking mechanism**: Weight decay rewards efficient circuits (larger logits at given parameter norm)
- **Adaptive optimizer behavior**: Adam with L2 penalty drives nodes to zero (Yaguchi et al., 2018; Mehta et al., 2019); Adam's $\epsilon$ can kill units with tiny second-moment estimates (Dufort-Labbe et al., 2025); decayed per-unit gains act as sparsity-inducing gates (Kolb et al., 2025)

## Methodology

### Task and Network

- **Task**: Modular arithmetic: $a, b \in \mathbb{Z}_p = \{0, 1, \dots, p-1\}$, target $c = (a \circ b) \bmod p$ with $\circ \in \{+, -, \times\}$
- **Primary task**: Addition modulo prime $p = 31$
- **Network**: Fully connected ReLU network, two hidden layers of width 256, linear readout
- **Training**: Backpropagation with Adam, cross-entropy loss, coupled weight decay $10^{-3}$ (L2 regularization added to gradient before Adam's moment estimates)
- **Data split**: 40% of operand pairs held out, full-batch training

### Key Definitions

**Circuit participation ratio** (from outgoing weight norms $a_j = \|\mathbf{W}^{\text{out}}_{:,j}\|_2$):

$$
\mathrm{PR} = \frac{\left(\sum_{j} a_j\right)^2}{n \sum_{j} a_j^2} \in [1/n, 1]
$$

Effective neuron count = $n \cdot \text{PR}$.

**Credit equalization intervention** — rescales each neuron's gradient magnitude toward the layer mean with strength $\beta \in [0, 1]$:

$$
\widetilde{\mathbf{g}}_j = \frac{\mathbf{g}_j}{\|\mathbf{g}_j\|_2}\left((1 - \beta)\|\mathbf{g}_j\|_2 + \beta \bar{g}\right) \quad \text{for } \|\mathbf{g}_j\|_2 > 0
$$

where $\bar{g} = \frac{1}{n}\sum_{k=1}^n \|\mathbf{g}_k\|_2$ is the mean credit magnitude.

### Learning Rules Tested

| Learning Rule | Transformation of Learning Signal |
|---|---|
| Backpropagation with Adam (reference) | per-parameter adaptive scaling |
| Credit equalization, then Adam | one positive factor per neuron, applied before Adam |
| LARS | one factor per layer |
| LAMB | per-parameter scaling + layerwise trust ratio |
| RMSProp | per-parameter second-moment scaling |
| Lion | elementwise sign of momentum signal |
| Adafactor | factored second-moment scaling |
| Muon | matrix orthogonalization |
| Shampoo | matrix preconditioning |
| Nero | one factor per neuron + weight constraints |
| DFC | controller-based credit, no gradient rescaling |

### Group Normalization Sweep

Dividing neurons into groups of size $m$ and rescaling all gradients in a group by one common factor:

$$
\widetilde{\mathbf{g}}_j = \mathbf{g}_j \frac{\bar{g}}{\bar{g}_{B(j)}}, \quad \bar{g}_{B(j)} = \frac{1}{m}\sum_{k \in B(j)} \|\mathbf{g}_k\|_2
$$

## Empirical Validation / Results

### 1. Credit Equalization Distributes the Circuit

- Increasing $\beta$ from 0 to 1 raises circuit participation from **0.60 to 0.99** (effective neurons: ~154 → 254 of 256)
- Training loss, margin, and parameter norm stay within a few percent of reference
- Test accuracy decreases from 0.998 to 0.976
- **Post-grokking application**: Switching to equalization after grokking redistributes the circuit within 300 epochs at unchanged test accuracy
- At width 2048: reference uses 332 effective neurons, equalized uses 2046

### 2. The Per-Neuron Condition Unifies Three Routes to Unused Neurons

| Non-gradient term | Network | Reference | Equalized | Term rescaled | Ratio held |
|---|---|---|---|---|---|
| Coupled decay | MLP, Adam with L2 | 0.606 | 0.993 | 0.993 | 0.574 |
| Adam's $\epsilon$ | Transformer, AdamW | 0.609 | 0.994 | 0.981 | 0.818 |
| Decay of the gain | MLP, layer normalization | 0.366 | 0.996 | 0.997 | 0.343 |

**Key insight**: Rescaling only the non-gradient term reproduces equalization; holding the credit-to-term ratio fixed keeps the circuit concentrated.

### 3. Group Size Determines Whether Normalization Distributes

- Groups of 8+ neurons: participation stays ~0.60 (concentrated)
- Groups of 4: 0.70
- Pairs: 0.75
- Single neurons: 0.99 (distributed)

### 4. Optimizer Comparison

- **Distributed circuits**: Nero (participation 0.986, normalizes per neuron), DFC (credit participation 0.94)
- **Concentrated circuits**: LARS, RMSProp, LAMB, Adafactor, Lion, Muon, Shampoo (participation 0.470–0.613)
- Lion gives equal update norms but keeps circuit concentrated because it normalizes the coupled decay together with the gradient

### 5. Which Learning Rules Grok

- **Grok**: Backpropagation with tested optimizers, credit equalization, DFC, rules with fresh or fading orthogonal components
- **Memorize only**: Feedback alignment, direct feedback alignment, sign-SGD, rule with fixed orthogonal component
- The null-space weight norm $q$ separates outcomes without overlap (means 5.3–8.5 for grokking, 26–135 for memorization)

### 6. Trade-off: Robustness vs. Compressibility vs. Adaptation

| Metric | Concentrated ($\beta = 0$) | Distributed ($\beta = 1$) | From scratch |
|---|---|---|---|
| Random deletion tolerance | 0.674 | 0.770 | — |
| Norm-ranked deletion tolerance | 0.332 | 0.581 | — |
| Greedy deletion, neurons removed | 23 | 61 | — |
| Magnitude pruning, fraction removed | 0.62 | 0.20 | — |
| Greedy pruning, neurons removed | 191 | 160 | — |
| Adaptation to subtraction, epochs | 640 (128) | 1540 (180) | 1410 (137) |
| Adaptation to multiplication, epochs | 1800 (141) | 1940 (136) | 1200 (63) |

**Key findings on adaptation**:
- Concentrated circuit adapts ~2.4× faster to subtraction (uses same Fourier frequencies differently)
- Nearly unused neurons account for ~2/3 of the advantage (removing them: 1204 vs 652 epochs)
- Multiplication (different frequencies) shows no advantage from addition pretraining

## Theoretical and Practical Implications

### Theoretical Implications

1. **Circuit size is not task-determined**: The same Fourier algorithm can be implemented in circuits ranging from 60% to 99% participation, depending on the learning rule.

2. **Unified mechanism for neuron death**: Three previously separate routes to unused neurons (coupled weight decay, Adam's $\epsilon$, decayed per-neuron gain) follow one condition: credit relative to non-gradient update term.

3. **Credit allocation as implicit bias**: The learning rule's normalization scheme acts as an implicit bias that selects circuit size, complementing existing implicit-bias literature.

4. **Mechanistic interpretability caveat**: Circuit size found in trained networks reflects the learning rule, not just the task. Comparing learning rules separates task-required from optimizer-selected properties.

### Practical Implications

1. **Robustness engineering**: Distributed circuits (via Nero, DFC, or credit equalization) tolerate neuron deletion better — relevant for hardware faults.

2. **Compressibility**: Concentrated circuits can be pruned more aggressively (62% vs 20% removable), enabling smaller deployed models.

3. **Transfer learning**: Concentrated circuits adapt faster to related tasks by recruiting unused neurons — a form of built-in spare capacity.

4. **Architecture design**: The choice of optimizer normalization scheme (per-neuron vs. per-layer vs. per-parameter) directly controls the robustness-compressibility trade-off.

5. **Scaling to language models**: The condition governs MLP units in a 30M-parameter language model trained with Adam and L2 regularization — balancing each unit's decay leaves 1.3% instead of 24% nearly unused at lower validation loss.

## Conclusion

**Main takeaways**:

1. Circuit concentration is a consequence of training dynamics, not the task alone
2. A single per-neuron condition (credit relative to non-gradient update term) unifies multiple routes to unused neurons
3. Per-neuron normalization (credit equalization, Nero, DFC) distributes circuits; group normalization retains concentration
4. The learning rule trades robustness against compressibility and transfer speed without changing the learned algorithm

**Future directions and limitations**:

- The theory isolates the distribution mechanism in a simplified linear model without adaptive normalization; nonlinear results are empirical
- Under AdamW without decay on normalization parameters, the remaining effect is small (participation 0.91–0.96) and not explained by the condition
- Algorithm checks cover modular addition only
- Adaptation advantage appears for targets needing new hidden combinations of operand frequencies, not for readout-only changes
- Robustness advantage concerns neuron deletion specifically

**Core message**: The learning rule steers which of many possible circuit implementations of the same algorithm a network finds, enabling deliberate selection of desirable properties (robustness vs. compressibility vs. transfer speed) through the choice of credit assignment and optimizer.

---

_Markdown view of https://picx.dev/p/BBW1ug, served by PicX — AI-generated visual whiteboard summaries of research papers._
