# Post-Training Leaves Behavioral Shadows on Unrelated Decisions > Post-training updates leave behavioral shadows on unrelated inputs that can be extracted via single-word queries to transfer capabilities, yielding +5.34 points on HumanEval+. - **Source:** [arXiv](https://arxiv.org/abs/2609.29233) - **Published:** 2026-09-30 - **Permalink:** https://picx.dev/p/TI7Ijc - **Whiteboard:** https://picx.dev/p/TI7Ijc/image ## Summary # Post-Training Leaves Behavioral Shadows on Unrelated Decisions ## Summary (Overview) - **Core finding**: Post-training updates on language models create "behavioral shadows"—changes in decisions on inputs *unrelated* to the training task. These shadows carry capability-relevant information that can be extracted and transferred to another model. - **Key contribution**: The authors introduce **Active Taskless Distillation (ATD)**, a method that achieves capability transfer using only a *single word* from the teacher per prompt, without target-task examples, teacher logits, or teacher parameters. - **Primary result**: On HumanEval+ (code generation), a student trained on 5,664 single-word teacher responses gains **+5.34 percentage points** over an exact nuisance-matched control, with the effect persisting across independently constructed acquisitions. - **Generality**: Transfer is demonstrated across seven target tasks (code, scientific knowledge, commonsense reasoning, reading comprehension) and across model generations, sizes, and families (Qwen2.5, Qwen3, Llama). - **Boundary condition**: Large teacher gains do *not* guarantee transfer—the transferred capability must be one the student can be elicited to express (e.g., memorized answers and substitution ciphers do not transfer). --- ## Introduction and Theoretical Foundation ### Background and Motivation Post-training (e.g., fine-tuning, DPO) is typically understood through changes in target-task behavior—a coding update makes a model write better code. However, the influence of an update may extend beyond the trained task: it may change which *ordinary word* a model prefers in a short story, even when neither the story nor the answer contains code or mathematics. These changes on task-unrelated inputs constitute the **behavioral shadow** of post-training. Prior work on *subliminal learning* showed that behavioral traits can pass through semantically unrelated training data, but largely focused on traits/preferences using extensive teacher outputs. ATD asks two sharper questions: 1. **Observability**: Can a private post-training update be observed through individual decisions on unrelated inputs? 2. **Capability transfer**: Can learning from these decisions transfer part of the teacher's target-task improvement? ### Theoretical Foundation The theoretical motivation rests on a first-order expansion around the public ancestor. For a carrier prompt $x_i$ with two candidate words $a_i$ and $b_i$, define the logit difference: $$ m_i(\theta) = z_\theta(a_i \mid x_i) - z_\theta(b_i \mid x_i) \tag{2} $$ Expanding around the ancestor parameters $\theta_0$: $$ m_i(\theta_0 + \delta) \approx m_i(\theta_0) + g_i^\top \delta, \qquad g_i = \nabla_\theta m_i(\theta_0) \tag{3} $$ where $\delta$ is the unknown post-training update and $g_i$ is the gradient of the margin with respect to parameters. The observed binary choice is modeled as: $$ y_i \approx \mathrm{sign}\big(m_i(\theta_0) + g_i^\top \delta\big) \tag{4} $$ **Key insight**: Near a tie (where $|m_i(\theta_0)|$ is small), even a small preference change induced by the update can *reverse* the selected word. Each observed choice thus provides a **one-bit observation** of the update's behavioral shadow. The teacher parameters are: $$ \theta_T = \theta_0 + \delta \tag{1} $$ --- ## Methodology ### Active Taskless Distillation (ATD) ATD consists of three stages: **1. Selecting near-tie prompts.** Using only the public ancestor $M_0$, compute the probability of candidate $b_i$ after normalizing over the pair: $$ q_i = \frac{\exp z_0(b_i \mid x_i)}{\exp z_0(a_i \mid x_i) + \exp z_0(b_i \mid x_i)} \tag{5} $$ Retain prompts satisfying $|q_i - 0.5| \le \epsilon$ with $\epsilon = 0.02$. Also require that the public model's highest-probability token over the full vocabulary is one of the two candidates. **2. Querying the teacher.** The query set is fixed before observing teacher responses. Each prompt is queried once; the teacher returns its highest-probability next token via greedy decoding over the full vocabulary. Responses are retained only if they belong to $\{a_i, b_i\}$; otherwise discarded without resampling. **3. Training the student.** The student is initialized from the same public ancestor $M_0$ and trained via full-vocabulary cross-entropy on the $n$ retained prompt–word pairs: $$ \mathcal{L}_{\mathrm{CE}}(\theta) = -\frac{1}{n} \sum_{i=1}^{n} \log p_\theta(w_i^T \mid x_i) \tag{6} $$ ### Experimental Design - **Primary lineage**: Qwen2.5-1.5B-Instruct as public ancestor and student initialization; teacher is a frozen rank-16 code-DPO LoRA; student is a rank-16 LoRA. - **Primary acquisition**: 5,664 (prompt, word) training rows over 128 words—audited to be free of code, mathematics, benchmark, and task terms. - **Controls**: The key control is the **exact nuisance-matched permutation** (same carriers, completion multiset, and per-bin difficulty, but broken prompt–token pairing). Other controls include teacher-label shuffle, public-label (teacher-free), and random-marginal labels. - **Evaluation**: HumanEval+ scored by greedy pass@1 through execution; GSM8K by exact match; multiple-choice benchmarks by held-out accuracy. ### Instrument Extensions **Reference-relative pair-margin (RPM) objective**: $$ \mathcal{L}_{\mathrm{RPM}} = -\mathbb{E}_i \log \sigma \left(\beta \left[ \log \frac{p_\theta(w_i^T \mid x_i)}{p_\theta(\bar{w}_i \mid x_i)} - \log \frac{p_0(w_i^T \mid x_i)}{p_0(\bar{w}_i \mid x_i)} \right] \right) \tag{7} $$ where $\bar{w}_i$ is the non-selected word. This subtracts the frozen public margin to focus on the update-induced change. **Multi-token observations** (K-word answers): $$ \mathcal{L}_{\mathrm{multi}} = -\frac{1}{n} \sum_i \frac{1}{K} \sum_{k=1}^{K} \log p_\theta(y_{i,k} \mid x_i, y_{i,