Summary (Overview)

• DecoEvo introduces a score-decoupled co-evolution framework for adapting large language models (LLMs) in text space, jointly optimizing a solver skill (task strategy) and a rubric-generator skill (principles for constructing question-specific rubrics) without using gold rubrics during optimization.
• The key technical contribution is separating generator updates from aggregate solver score: generator revisions are guided by two complementary, frozen audits—a task-conditioned structural audit (identifying omitted criteria) and a near-tie contrastive audit (identifying missed response distinctions)—rather than by the solver’s training proxy.
• Across five benchmarks and three LLM backbones (GPT-4o, Qwen3-4B/8B), DecoEvo achieves the highest mean score in all 15 backbone–benchmark combinations, with average relative gains of 2.8–5.0% over SkillOpt (the best solver-only baseline).
• Ablation studies confirm that both audits, rubric-blind preference formation, held-out verification, and cross-example distillation are essential; removing verification causes the largest performance drop (2.4–2.9 points on HealthBench).
• Analyses of rubric alignment show that DecoEvo improves criterion-level overlap with gold rubrics (F1 increase of ~12 points), while a score-coupled baseline (SC-CoEvo) degrades alignment, confirming that decoupling prevents rubric drift.

Introduction and Theoretical Foundation

Conventional LLM adaptation methods (SFT, RLHF, DPO) modify model weights. Text-space optimization instead edits external natural-language artifacts (e.g., prompts, skills), keeping the backbone frozen and the optimized state inspectable. However, most text-space methods use a fixed evaluation objective (e.g., a static rubric). On open-ended tasks (medical consultation, creative writing, research QA), a fixed rubric can become a bottleneck: once the solver improves on measured criteria, omitted dimensions remain invisible to the optimization signal.

Simply evolving the rubric alongside the solver introduces a score-coupled co-adaptation risk: if generator updates are rewarded for increasing the current solver’s aggregate score, apparent progress can come from making the rubric easier to satisfy rather than improving true task quality. This is related to reward overoptimization (Gao et al., 2023) and classical co-evolutionary instability (Popovici et al., 2012).

DecoEvo addresses this by decoupling generator updates from aggregate solver score. The generator is updated through two frozen audits that assess requirement coverage (structural audit) and response discrimination (contrastive audit) independently of solver performance. This design allows the evaluation signal to adapt without rewarding rubric changes that favor the current solver.

Methodology

Problem Setup

• Let $Q$ be a set of open-ended questions. Given $q \in Q$, the solver produces a response $r_q(s) = \text{Solve}(q; s)$, and the generator produces a question-specific rubric: $$R_q(g) = \text{Gen}(q; g) = \{(c_i, w_i)\}_{i=1}^{m_q}, \quad m_q \geq 1$$ where $c_i$ is a natural-language criterion and $w_i > 0$ its point value.
• A fixed criterion-level judge $E$ returns $e(q, r, c_i) \in [0, 1]$. The rubric score is: $$S(q, r, R_q) = \frac{\sum_i w_i \, e(q, r, c_i)}{\sum_i w_i}$$
• The training-time proxy on a question set $D \subseteq Q$ is: $$J_D(s, g) = \frac{1}{|D|} \sum_{q \in D} S\!\left(q, r_q(s), R_q(g)\right)$$
• The target quantity is the solver’s test utility $U(s)$, computed by the benchmark’s official grader using held-out gold rubrics (never exposed during optimization).

Score-Decoupled Generator Update

Definition 1 formalizes that a generator update is score-decoupled if neither $J_D(s_t, g_t)$, $J_D(s_t, g')$, nor their difference is used as the generator objective. Response-level scores are retained only for locating near-ties; aggregate solver score does not indicate generator improvement.

Solver Skill Evolution

At inner-loop step $t$, the solver answers a batch $B_t \subset D_{\text{tr}}$ with skill $s_t$, the generator produces a rubric $R_q(g_t)$, and the judge returns criterion-level scores and failure rationales. Selected failures are distilled into a candidate solver skill $s'$. It is accepted when:

$$J_{V_s}(s', g_t) > J_{V_s}(s_t, g_t) + \varepsilon$$

otherwise a stall counter is incremented, triggering an outer-loop audit of the generator.

Generator Skill Evolution via Dual-View Auditing

The outer loop runs every $K$ inner steps or after $M$ consecutive solver rejections. It involves three phases: audit, distill, and verify.

1. Task-Conditioned Structural Audit: For each task family $\tau$, a fixed structural-audit specification $\pi^{\text{str}}_\tau$ (derived from public task description and evaluation instructions) is used to evaluate the generated rubric $R_q(g)$ without seeing any solver response or score. It returns an adequacy score $u_q(g) \in [0, 1]$, a rationale $\xi_q$, and a repair suggestion $\kappa_q$. The structural adequacy on a set $D$ is:

   $$\phi_{\text{str}}(g; D) = \frac{1}{|D|} \sum_{q \in D} u_q(g)$$

2. Near-Tie Contrastive Audit: The current solver samples $L$ stochastic rollouts per probe question $q$, and they are scored under the current rubric. Pairs with score gap $\leq \tau_{\text{tie}}$ are selected (prioritizing small gaps). For each selected pair, the auditor first sees only the question and two responses (no rubric, no scores) and returns a rubric-blind preference $y_{qij} \in \{i \succ j, j \succ i, \text{equivalent}\}$ with a rationale. Only after fixing this preference does it receive the rubric and explains why the rubric misses the difference. For a retained record $p = (q, r^+, r^-, \xi, \kappa)$, the rubric-induced margin is:

   $$m_g(p) = S(q, r^+, R_q(g)) - S(q, r^-, R_q(g))$$

   The verification uses the resolution rate:

   $$\phi_{\text{ctr}}(g; \Gamma) = \frac{1}{|\mathcal{Q}_\Gamma|} \sum_{q \in \mathcal{Q}_\Gamma} \phi_q(g; \Gamma_q), \quad \phi_q(g; \Gamma_q) = \frac{1}{|\Gamma_q|} \sum_{p \in \Gamma_q} \mathbb{1}\{m_g(p) \geq \gamma\}$$

   where $0 < \gamma \leq 1$ is the required resolution margin.

3. Audit Distillation and Pareto Verification: The generator-skill rewriter consolidates recurring findings into reusable principles for constructing rubrics. The candidate $g'$ is accepted when (using disjoint validation set $V_g$):

   $$\exists k \in \mathcal{K}_t : \Delta_k > \delta, \quad \forall j \in \mathcal{K}_t : \Delta_j \geq -\eta$$

   where $\Delta_k = \Phi_k(g') - \Phi_k(g_t)$, $\Phi_{\text{str}}(g) = \phi_{\text{str}}(g; V_g)$, $\Phi_{\text{ctr}}(g) = \phi_{\text{ctr}}(g; \Gamma^{t, V_g}_{\text{ctr}})$. At least one audit objective must improve by $\delta > 0$, while none may regress beyond tolerance $\eta \geq 0$.

Empirical Validation / Results

Experimental Setup

• Tasks: HealthBench, WritingBench, ResearchQA (source); LLMEval-Med and EQ-Bench Creative Writing v3 (cross-benchmark transfer, never used for optimization).
• Backbones: GPT-4o, Qwen3-4B, Qwen3-8B; same frozen backbone fills all optimization roles.
• Baselines: Zero-shot (no optimization), SkillOpt (fixed rubric generator), Score-Coupled Co-Evolution (SC-CoEvo, same audits but generator accepted by aggregate solver score).
• Reporting: Five matched runs per setting; final scores computed by official graders with gold rubrics.

Main Results (Table 1)

Table 1: Main results under gold rubrics. † denotes direct within-domain cross-benchmark transfer. Mean ± sample standard deviation over five runs.

• Key findings: DecoEvo leads all baselines on all 15 backbone–benchmark pairs. SkillOpt exceeds Zero-shot in 14/15 cases, but DecoEvo further improves by 2.8–5.0%. SC-CoEvo falls below SkillOpt in 13/15 cases and below Zero-shot in 7, showing that score-coupled selection can erase solver-only gains.
• Paired analysis: Average gains over SkillOpt are 3.10 (GPT-4o, 95% CI [2.88, 3.32]), 1.74 (Qwen3-4B, [1.42, 2.06]), 1.78 (Qwen3-8B, [1.69, 1.87]); all remain significant after Holm correction.

Ablation Study (Table 2, HealthBench)

Table 2: Ablation on HealthBench. Each variant removes one mechanism; all share the same splits, schedules, hyperparameters, and matched seeds.

• Key findings: Both audits are critical; removing contrastive auditing causes slightly larger losses (1.7–2.1 pts) than removing structural auditing (1.3–1.7). Removing verification causes the largest loss (2.4–2.9 pts). Rubric blindness, near-tie sampling, and distillation each contribute 1.0–1.8 pts.

Alternative-Explanation Controls (Table 3, HealthBench)

*Table 3: Controls testing richer priors, more compute, and direct routing. SkillOpt with task-prior specifications; Budget-Matched with additional solver updates to match DecoEvo’s token budget; Direct Audit routes audit records directly to the solver re