# Learning from the Gap Between Pass@K and Pass@1 > GapFT selects verified responses from problems the source policy fails at first but solves via search, outperforming uniform rejection sampling on logic benchmarks with far fewer training examples. - **Source:** [arXiv](https://arxiv.org/abs/2609.35793) - **Published:** 2026-10-24 - **Permalink:** https://picx.dev/p/csjr0h - **Whiteboard:** https://picx.dev/p/csjr0h/image ## Summary ## Summary (Overview) - **GapFT (Gap Fine-Tuning)** is a novel data-selection method that identifies which verified responses should enter post-training when the training budget is fixed. Rather than selecting responses by correctness alone, GapFT conditions selection on whether the *source policy* fails in a single attempt but succeeds within K search samples. - The paper introduces a formal partition of problems into three evidence states: **$G_K$** (single-sample success), **$S_K$** (failure recovered by search), and **$U_K$** (failure missed by search), with GapFT selecting exclusively from $S_K$. - An **exact decomposition identity** attributes any Pass@1 change to corrections on $S_K$ and $U_K$ problems minus regressions on $G_K$ problems: $A_{dep}(\pi) - A_{dep}(\pi_{src}) = p_S \kappa_S + p_U \kappa_U - p_G \rho_G$. - On LogiQA 2.0 and ReClor across Llama-3.1-8B, Qwen2.5-7B, and Mistral-7B, GapFT outperforms budget-matched uniform RFT in every setting, recovering **two-thirds to four-fifths** of the full verified pool's gain with only 11–34% of its problems. - Key controls reveal that gains come from covering **reachable** failures, that failures found only by deep search trade correction for regression, and that gold-label selection of unreachable failures *lowers* accuracy. --- ## Introduction and Theoretical Foundation ### Background and Motivation Large language models exhibit a well-documented gap between Pass@K and Pass@1: a single response often fails on problems the model can still solve when multiple responses are sampled and verified (Chen et al., 2021; Li et al., 2022). This has made test-time search a standard approach (Wang et al., 2023; Snell et al., 2025), but search must be paid again for every query. Post-training (e.g., STaR, ReST, rejection-sampling fine-tuning) can amortize this cost by searching a training pool once and using verified responses to update the policy (Zelikman et al., 2022; Gulcehre et al., 2023). However, existing recipes judge verified responses by properties of the response itself (correctness, reward, diversity), ignoring a critical question: **whether the response can change single-attempt behavior depends on the source model, not the response alone.** ### The Data-Selection Gap On problems the source model already answers correctly, a verified response repeats existing behavior—it is redundant. On problems missed in one attempt but solved by search, the verified response supplies the exact target that deployment lacks. Because problems already solved typically outnumber those recovered by search, allocation without this distinction wastes a fixed budget on redundant examples. The paper calls this the **data-selection gap**: not how to obtain correct responses, but how to allocate them relative to what the deployed model already does. ### Theoretical Foundation The paper formalizes the setting with an exact verifier $V(x,y) \in \{0,1\}$ and defines deployment accuracy and search success: $$A_{dep}(\pi) = \mathbb{E}_{x \sim \mathcal{D}}[V(x, d_\pi(x))], \qquad A_{search}(\pi; K) = \mathbb{E}_{x \sim \mathcal{D}, y_{1:K} \sim \pi(\cdot|x)} \left[ \max_{k \leq K} V(x, y_k) \right] \tag{1}$$ The offline RFT objective is maximum likelihood: $$\mathcal{L}(\theta) = -\sum_{(x,y) \in \mathcal{T}} \sum_{t=1}^{|y|} \log \pi_\theta(y_t \mid x, y_{