DeGMatch: Target-Aware Reasoning Data Selection via Deconfounded Gradient Matching

Summary (Overview)

  • Problem: Reasoning post-training on long trajectories is computationally expensive, and existing target-aware data selection methods exhibit limited target discrimination when applied to long reasoning trajectories due to dominant population-level gradient structure.

  • Key Finding: Candidate gradients in reasoning datasets are dominated by a small number of population-level directions—the top 50 principal components explain 86.7–89.4% of centered-gradient variance across five mathematical target-conditioned populations—which obscures finer target-dependent variation during gradient matching.

  • Proposed Method: DeGMatch combines three stages: (1) a target-relevance gate using semantic embedding similarity, (2) deconfounded gradient geometry via population centering and dominant-component removal, and (3) target-conditioned gradient matching combining directional alignment with target-subspace magnitude.

  • Results: DeGMatch achieves the strongest aggregate performance among selection methods on mathematical reasoning (60.73 avg. across 8 benchmarks), outperforms all budget-matched baselines, and generalizes to science and code domains without domain-specific retuning.

  • Robustness: DeGMatch remains effective across selection budgets (1K–15K examples), model families (Qwen, Phi), and scales (1.7B–7B parameters), with its largest advantage at 10K examples where it surpasses full-data training.

Introduction and Theoretical Foundation

Background and Motivation

Large language models have shown substantial improvements in complex reasoning capabilities, with reasoning post-training emerging as a standard approach for enhancing these capabilities through training on long reasoning trajectories. However, the growing scale of reasoning datasets and the length of individual trajectories make full-data training increasingly expensive.

While data-efficient selection seeks compact subsets broadly useful for post-training, many real-world applications require specialization to a particular downstream task or capability. This motivates target-aware data selection, which identifies training examples that most effectively support a specified downstream objective.

Key Theoretical Insight

The authors identify a critical limitation in applying gradient-based matching to long reasoning trajectories:

"Candidate gradients are dominated by a small number of population-level directions: across five mathematical target-conditioned candidate populations, the top 50 principal components explain 86.7–89.4% of the centered-gradient variance."

This dominant population-level structure obscures finer target-dependent variation during gradient matching, as illustrated in Figure 1: target-identification accuracy increases from 56% to 71% when removing the population mean and top 50 principal components.

Theoretical Foundation

The work builds on several theoretical pillars:

  1. Common-component removal (Mu et al., 2018): Removing dominant shared directions in representation spaces can expose more discriminative variation.

  2. Gradient-based influence estimation (Koh & Liang, 2017): Gradient information can estimate training utility for specified targets.

  3. Gradient geometry: Prior work shows gradient representations exhibit substantial covariance, low-rank, and spectral structure, and that normalization or whitening can substantially change their similarity geometry.

Methodology

Problem Formulation

Let Dpool={zi}i=1N\mathcal{D}_{\text{pool}} = \{z_i\}_{i=1}^N denote a large pool of reasoning trajectories, where zi=(xi,yi)z_i = (x_i, y_i) consists of a problem xix_i and its reasoning trajectory yiy_i. Let Dtarget={aj}j=1M\mathcal{D}_{\text{target}} = \{a_j\}_{j=1}^M be a small reference set specifying a target capability. Given a model with parameters θ0\theta_0 and a selection budget BB, the goal is to select a subset S⊂Dpool\mathcal{S} \subset \mathcal{D}_{\text{pool}}, ∣S∣=B|\mathcal{S}| = B, that maximizes post-training performance on the target capability.

Ideally:

S∗=arg⁡max⁡S⊂Dpool∣S∣=BUtarget(Train(θ0,S)),(1)\mathcal{S}^{*} = \arg \max_{\substack{\mathcal{S}\subset \mathcal{D}_{\text{pool}}\\ |\mathcal{S}| = B}}\mathcal{U}_{\text{target}}\left(\text{Train}(\theta_{0},\mathcal{S})\right),\tag{1}

where Utarget\mathcal{U}_{\text{target}} denotes downstream performance on the target capability. Since evaluating this utility requires repeated post-training, DeGMatch optimizes a tractable surrogate.

Stage 1: Target-Relevance Gate

Using a frozen embedding model ϕ\phi, encode problem statements and rank candidates by cosine similarity between normalized embeddings. Construct a relevance gate G\mathcal{G} of fixed size G>BG > B by cycling through target references and adding each reference's highest-ranked unseen candidate until ∣G∣=G|\mathcal{G}| = G.

Stage 2: Deconfounded Gradient Geometry

Step 1: Gradient sketching. Compute the gradient of the token-averaged completion loss with respect to the initial student model and sketch it:

gi=Sketch(∇θ0L(zi;θ0)),(2)g_i = \text{Sketch}\left(\nabla_{\theta_0}\mathcal{L}(z_i; \theta_0)\right),\tag{2}

using CountSketch (Charikar et al., 2002).

Step 2: Population centering. Subtract the mean gradient of the candidate population:

μ=1∣G∣∑zi∈Ggi,gˉi=gi−μ,gˉjt=gjt−μ.(3)\mu = \frac{1}{|\mathcal{G}|} \sum_{z_i \in \mathcal{G}} g_i, \quad \bar{g}_i = g_i - \mu, \quad \bar{g}_j^t = g_j^t - \mu.\tag{3}

Step 3: Dominant-component removal. Let UkU_k contain the top kk orthonormal principal directions of the centered candidate gradients. Remove this dominant subspace:

P⊥=I−UkUk⊤,(4)P_{\perp} = I - U_k U_k^{\top},\tag{4}

transforming both candidate and target gradients:

g~i=P⊥gˉi,g~jt=P⊥gˉjt.(5)\tilde{g}_i = P_{\perp} \bar{g}_i, \qquad \tilde{g}_j^t = P_{\perp} \bar{g}_j^t.\tag{5}

Both μ\mu and UkU_k are estimated exclusively from candidate gradients, without directly incorporating target gradients into population statistics.

Stage 3: Target-Conditioned Gradient Matching

Directional alignment with target references:

di=1q∑j∈Nq(i)cos⁡(g~i,g~jt),(6)d_i = \frac{1}{q} \sum_{j \in \mathcal{N}_q(i)} \cos(\tilde{g}_i, \tilde{g}_j^t),\tag{6}

where Nq(i)\mathcal{N}_q(i) denotes the qq target gradients with highest cosine similarity.

Target-subspace magnitude: Let T=[g~1t,…,g~Mt]T = [\tilde{g}_1^t, \dots, \tilde{g}_M^t] and Q=orth(T)Q = \text{orth}(T):

mi=∥Q⊤g~i∥2.(7)m_i = \left\| Q^{\top} \tilde{g}_i \right\|_2.\tag{7}

Final score combining both terms multiplicatively:

si=di⋅mi,S=TopBzi∈Gsi.(8)s_i = d_i \cdot m_i, \qquad \mathcal{S} = \text{TopB}_{z_i \in \mathcal{G}} s_i.\tag{8}

Empirical Validation / Results

Mathematical Reasoning Results

The main results on mathematical reasoning (Table 1) show DeGMatch achieves the highest aggregate performance:

Method# Ex.TokensAIME24AIME25AIME26AMC23HMMTMATH500GK24GK25Avg.
Base Model--1.21.90.2120.217.39.47.26.18
Random10K60.3M39.230.834.477.718.389.670.280.055.03
Length10K142.3M45.033.340.482.321.791.874.381.058.73
LESS10K70.6M43.131.738.579.720.290.272.879.656.98
RDS+10K85.2M45.031.938.882.321.591.274.580.358.19
TAROT10K74.7M39.434.040.378.621.091.770.281.757.11
DeGMatch10K86.9M47.733.842.685.623.792.876.982.760.73

Key findings:

  • DeGMatch outperforms all budget-matched baselines and the released Select2Reason checkpoint (58.99)
  • Compares favorably with full-data OpenR1-Qwen-7B reference (58.80) despite using only 10K of 93K examples
  • Existing target-aware methods (LESS, RDS+, GIST, TAROT) do not consistently outperform the strong Length baseline

Cross-Domain Generalization

DeGMatch generalizes to science and code:

DomainBenchmarkRandomLengthRDS+TEMPDeGMatchFull FT
ScienceGPQA-D46.848.246.447.549.048.2
ScienceJEE45.846.951.646.651.852.3
CodeLCB v619.416.318.721.421.420.5
CodeCF15.513.715.216.217.421.1

Component Validation

VariantA24A25A26AMCAvg.
Random30.427.332.375.541.38
Relevance only32.730.232.379.143.58
Raw-gradient dir.36.930.834.078.044.93
Centered dir.33.333.135.679.145.28
Deconfounded dir.34.631.936.780.345.88
Deconfounded mag.32.532.533.380.544.70
DeGMatch37.931.035.681.646.53

Cross-Model Robustness

ModelRandomLengthRDS+DeGMatch
Qwen2.5-3B-Base14.5515.8015.7516.05
Qwen3-1.7B-Base14.2012.8314.9514.85
Phi-3.5-mini-Instruct8.059.5810.9812.58

Theoretical and Practical Implications

Theoretical Implications

  1. Gradient geometry in reasoning: The paper provides empirical evidence that reasoning-trajectory gradients exhibit strong low-rank structure dominated by population-level directions, extending prior findings on gradient covariance and spectral structure to the reasoning domain.

  2. Deconfounding principle: The work demonstrates that suppressing dominant shared structure before target matching improves target discrimination, analogous to common-component removal in representation learning but applied to gradient spaces.

  3. Complementary signals: Directional alignment and target-subspace magnitude capture complementary aspects of candidate utility—direction captures compatibility with target gradients while magnitude measures update strength within the target subspace.

Practical Implications

  1. Data efficiency: DeGMatch achieves competitive or superior performance with substantially less training data (10K vs. 93K examples), reducing computational costs for reasoning post-training.

  2. Transferability: The method transfers across reasoning domains (math, science, code) and model families/scales without domain-specific retuning, suggesting broad applicability.

  3. Limitations of existing methods: Prior target-aware methods developed for instruction tuning do not transfer uniformly to long reasoning trajectories, motivating reasoning-specific adaptations.

Conclusion

DeGMatch introduces a three-stage approach for target-aware reasoning data selection: target-relevant retrieval, deconfounded gradient geometry, and target-conditioned gradient matching. By centering gradients and suppressing dominant candidate-population principal directions, DeGMatch reduces the influence of population-level structure that obscures target-dependent variation.

Key takeaways:

  • Dominant population-level gradient structure is a key obstacle to effective target-aware selection on reasoning trajectories
  • Deconfounding improves target discrimination in gradient matching
  • Combining directional alignment with target-subspace magnitude provides complementary selection signals
  • DeGMatch achieves strong performance across domains, budgets, and model families

Future directions:

  • Scaling student-gradient-based selection to substantially larger models
  • Jointly optimizing data generation, evolution, and selection
  • Understanding what removed gradient directions represent in terms of reasoning skills and behaviors

Related papers