DeGMatch: Target-Aware Reasoning Data Selection via Deconfounded Gradient Matching
Summary (Overview)
-
Problem: Reasoning post-training on long trajectories is computationally expensive, and existing target-aware data selection methods exhibit limited target discrimination when applied to long reasoning trajectories due to dominant population-level gradient structure.
-
Key Finding: Candidate gradients in reasoning datasets are dominated by a small number of population-level directions—the top 50 principal components explain 86.7–89.4% of centered-gradient variance across five mathematical target-conditioned populations—which obscures finer target-dependent variation during gradient matching.
-
Proposed Method: DeGMatch combines three stages: (1) a target-relevance gate using semantic embedding similarity, (2) deconfounded gradient geometry via population centering and dominant-component removal, and (3) target-conditioned gradient matching combining directional alignment with target-subspace magnitude.
-
Results: DeGMatch achieves the strongest aggregate performance among selection methods on mathematical reasoning (60.73 avg. across 8 benchmarks), outperforms all budget-matched baselines, and generalizes to science and code domains without domain-specific retuning.
-
Robustness: DeGMatch remains effective across selection budgets (1K–15K examples), model families (Qwen, Phi), and scales (1.7B–7B parameters), with its largest advantage at 10K examples where it surpasses full-data training.
Introduction and Theoretical Foundation
Background and Motivation
Large language models have shown substantial improvements in complex reasoning capabilities, with reasoning post-training emerging as a standard approach for enhancing these capabilities through training on long reasoning trajectories. However, the growing scale of reasoning datasets and the length of individual trajectories make full-data training increasingly expensive.
While data-efficient selection seeks compact subsets broadly useful for post-training, many real-world applications require specialization to a particular downstream task or capability. This motivates target-aware data selection, which identifies training examples that most effectively support a specified downstream objective.
Key Theoretical Insight
The authors identify a critical limitation in applying gradient-based matching to long reasoning trajectories:
"Candidate gradients are dominated by a small number of population-level directions: across five mathematical target-conditioned candidate populations, the top 50 principal components explain 86.7–89.4% of the centered-gradient variance."
This dominant population-level structure obscures finer target-dependent variation during gradient matching, as illustrated in Figure 1: target-identification accuracy increases from 56% to 71% when removing the population mean and top 50 principal components.
Theoretical Foundation
The work builds on several theoretical pillars:
-
Common-component removal (Mu et al., 2018): Removing dominant shared directions in representation spaces can expose more discriminative variation.
-
Gradient-based influence estimation (Koh & Liang, 2017): Gradient information can estimate training utility for specified targets.
-
Gradient geometry: Prior work shows gradient representations exhibit substantial covariance, low-rank, and spectral structure, and that normalization or whitening can substantially change their similarity geometry.
Methodology
Problem Formulation
Let denote a large pool of reasoning trajectories, where consists of a problem and its reasoning trajectory . Let be a small reference set specifying a target capability. Given a model with parameters and a selection budget , the goal is to select a subset , , that maximizes post-training performance on the target capability.
Ideally:
where denotes downstream performance on the target capability. Since evaluating this utility requires repeated post-training, DeGMatch optimizes a tractable surrogate.
Stage 1: Target-Relevance Gate
Using a frozen embedding model , encode problem statements and rank candidates by cosine similarity between normalized embeddings. Construct a relevance gate of fixed size by cycling through target references and adding each reference's highest-ranked unseen candidate until .
Stage 2: Deconfounded Gradient Geometry
Step 1: Gradient sketching. Compute the gradient of the token-averaged completion loss with respect to the initial student model and sketch it:
using CountSketch (Charikar et al., 2002).
Step 2: Population centering. Subtract the mean gradient of the candidate population:
Step 3: Dominant-component removal. Let contain the top orthonormal principal directions of the centered candidate gradients. Remove this dominant subspace:
transforming both candidate and target gradients:
Both and are estimated exclusively from candidate gradients, without directly incorporating target gradients into population statistics.
Stage 3: Target-Conditioned Gradient Matching
Directional alignment with target references:
where denotes the target gradients with highest cosine similarity.
Target-subspace magnitude: Let and :
Final score combining both terms multiplicatively:
Empirical Validation / Results
Mathematical Reasoning Results
The main results on mathematical reasoning (Table 1) show DeGMatch achieves the highest aggregate performance:
| Method | # Ex. | Tokens | AIME24 | AIME25 | AIME26 | AMC23 | HMMT | MATH500 | GK24 | GK25 | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Base Model | - | - | 1.2 | 1.9 | 0.2 | 12 | 0.2 | 17.3 | 9.4 | 7.2 | 6.18 |
| Random | 10K | 60.3M | 39.2 | 30.8 | 34.4 | 77.7 | 18.3 | 89.6 | 70.2 | 80.0 | 55.03 |
| Length | 10K | 142.3M | 45.0 | 33.3 | 40.4 | 82.3 | 21.7 | 91.8 | 74.3 | 81.0 | 58.73 |
| LESS | 10K | 70.6M | 43.1 | 31.7 | 38.5 | 79.7 | 20.2 | 90.2 | 72.8 | 79.6 | 56.98 |
| RDS+ | 10K | 85.2M | 45.0 | 31.9 | 38.8 | 82.3 | 21.5 | 91.2 | 74.5 | 80.3 | 58.19 |
| TAROT | 10K | 74.7M | 39.4 | 34.0 | 40.3 | 78.6 | 21.0 | 91.7 | 70.2 | 81.7 | 57.11 |
| DeGMatch | 10K | 86.9M | 47.7 | 33.8 | 42.6 | 85.6 | 23.7 | 92.8 | 76.9 | 82.7 | 60.73 |
Key findings:
- DeGMatch outperforms all budget-matched baselines and the released Select2Reason checkpoint (58.99)
- Compares favorably with full-data OpenR1-Qwen-7B reference (58.80) despite using only 10K of 93K examples
- Existing target-aware methods (LESS, RDS+, GIST, TAROT) do not consistently outperform the strong Length baseline
Cross-Domain Generalization
DeGMatch generalizes to science and code:
| Domain | Benchmark | Random | Length | RDS+ | TEMP | DeGMatch | Full FT |
|---|---|---|---|---|---|---|---|
| Science | GPQA-D | 46.8 | 48.2 | 46.4 | 47.5 | 49.0 | 48.2 |
| Science | JEE | 45.8 | 46.9 | 51.6 | 46.6 | 51.8 | 52.3 |
| Code | LCB v6 | 19.4 | 16.3 | 18.7 | 21.4 | 21.4 | 20.5 |
| Code | CF | 15.5 | 13.7 | 15.2 | 16.2 | 17.4 | 21.1 |
Component Validation
| Variant | A24 | A25 | A26 | AMC | Avg. |
|---|---|---|---|---|---|
| Random | 30.4 | 27.3 | 32.3 | 75.5 | 41.38 |
| Relevance only | 32.7 | 30.2 | 32.3 | 79.1 | 43.58 |
| Raw-gradient dir. | 36.9 | 30.8 | 34.0 | 78.0 | 44.93 |
| Centered dir. | 33.3 | 33.1 | 35.6 | 79.1 | 45.28 |
| Deconfounded dir. | 34.6 | 31.9 | 36.7 | 80.3 | 45.88 |
| Deconfounded mag. | 32.5 | 32.5 | 33.3 | 80.5 | 44.70 |
| DeGMatch | 37.9 | 31.0 | 35.6 | 81.6 | 46.53 |
Cross-Model Robustness
| Model | Random | Length | RDS+ | DeGMatch |
|---|---|---|---|---|
| Qwen2.5-3B-Base | 14.55 | 15.80 | 15.75 | 16.05 |
| Qwen3-1.7B-Base | 14.20 | 12.83 | 14.95 | 14.85 |
| Phi-3.5-mini-Instruct | 8.05 | 9.58 | 10.98 | 12.58 |
Theoretical and Practical Implications
Theoretical Implications
-
Gradient geometry in reasoning: The paper provides empirical evidence that reasoning-trajectory gradients exhibit strong low-rank structure dominated by population-level directions, extending prior findings on gradient covariance and spectral structure to the reasoning domain.
-
Deconfounding principle: The work demonstrates that suppressing dominant shared structure before target matching improves target discrimination, analogous to common-component removal in representation learning but applied to gradient spaces.
-
Complementary signals: Directional alignment and target-subspace magnitude capture complementary aspects of candidate utility—direction captures compatibility with target gradients while magnitude measures update strength within the target subspace.
Practical Implications
-
Data efficiency: DeGMatch achieves competitive or superior performance with substantially less training data (10K vs. 93K examples), reducing computational costs for reasoning post-training.
-
Transferability: The method transfers across reasoning domains (math, science, code) and model families/scales without domain-specific retuning, suggesting broad applicability.
-
Limitations of existing methods: Prior target-aware methods developed for instruction tuning do not transfer uniformly to long reasoning trajectories, motivating reasoning-specific adaptations.
Conclusion
DeGMatch introduces a three-stage approach for target-aware reasoning data selection: target-relevant retrieval, deconfounded gradient geometry, and target-conditioned gradient matching. By centering gradients and suppressing dominant candidate-population principal directions, DeGMatch reduces the influence of population-level structure that obscures target-dependent variation.
Key takeaways:
- Dominant population-level gradient structure is a key obstacle to effective target-aware selection on reasoning trajectories
- Deconfounding improves target discrimination in gradient matching
- Combining directional alignment with target-subspace magnitude provides complementary selection signals
- DeGMatch achieves strong performance across domains, budgets, and model families
Future directions:
- Scaling student-gradient-based selection to substantially larger models
- Jointly optimizing data generation, evolution, and selection
- Understanding what removed gradient directions represent in terms of reasoning skills and behaviors
Related papers
- Shared Weights, Selected Computations: How Looped Transformers Route What Each Loop Does
Looped Transformers perform state-dependent computation, not fixed one-loop-one-step algorithms, and a learned linear layer can steer frozen shared weights via attention routing.
- The Critic Can Hurt: On the Safety of Critics in LLM-Guided AutoML Search
A naive LLM critic can drive AutoML search below the no-critic random-search floor on 7 of 8 tabular datasets, with losses up to 0.245 AUC.
- RIPPLE: Recovery Internalization through Process-Level Policy Learning and Effect-Aware Credit for Long-Horizon Agentic Coding
RIPPLE internalizes prefix-only recovery guidance into a single policy via stage-specific credit, improving coding agents by 4.8 points across four benchmarks without deployment-time critics.