Evolution Fine-Tuning: Learning to Discover Across 371 Optimization Tasks

Summary (Overview)

• Core contribution: Introduces Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across optimization tasks by converting evolutionary search trajectories into supervised training signals, internalizing the discovery capability directly into model weights.

• Dataset: Constructs ℱinch Collection, a large-scale dataset of ~156K evolutionary trajectories spanning 10 domains and 371 optimization tasks, collected using OpenEvolve scaffold with Qwen3.5-397B-A17B as the teacher mutation operator.

• Model family: Produces ℱinch models (2B–9B parameters) that demonstrate cross-task generalization, outperforming base counterparts by 10.22% on average across 22 held-out tasks.

• Synergy with test-time RL: When paired with test-time reinforcement learning, ℱinch achieves state-of-the-art performance on two circle-packing tasks and outperforms its base-model counterpart on the Erdős minimum-overlap problem.

• Key finding: Scaling training tasks from 15 to 355 improves held-out performance by 14.1% on average, demonstrating that EFT provides a scalable training signal.

---

Introduction and Theoretical Foundation

Background

The paper addresses optimization tasks—problems where a candidate solution can be scored against an objective, but the optimal solution is not directly computable. Examples include:

• Open mathematical conjectures (e.g., the Erdős minimum-overlap problem)
• Design of high-performance GPU kernels
• Discovery of new scientific laws from data

Limitations of Prior Work

The authors identify three fundamental limitations in existing LLM-driven evolutionary search approaches:

1. Test-time search methods (e.g., AlphaEvolve, OpenEvolve) rely on proprietary frontier-scale LLMs as mutation operators, because scaffolds demand consistent, high-quality proposals. Open-source models smaller than 9B parameters fail to follow evolutionary trajectories within such scaffolds.

2. Test-time learning methods (e.g., ThetaEvolve, TTT-Discover) allow smaller LLMs to adapt weights based on search experience, but these updates are tailored to a single search loop and a single task—strategies are not consolidated into reusable capability.

3. Fundamental issue: In neither branch does the model itself acquire the evolving capability (knowing which part to mutate, what to keep, when to backtrack). The capability resides in the scaffold or is discarded after task completion.

Theoretical Foundation

The optimization setup is formalized as:

$$x^{\star} = \underset{x \in \mathcal{X}}{\arg \operatorname{opt}} \mathcal{E}(x), \qquad \text{opt} \in \{\max, \min\}$$

where $\mathcal{E}$ is the evaluator, $\mathcal{X}$ is the candidate set, and the optimization direction depends on the task (max for accuracy, min for bounds).

A discovery is defined as a solution $x^{\star}$ that improves upon the previous best-known solution $x_{\mathrm{sota}}$ within budget $T$: $\mathcal{E}(x^{\star}) > \mathcal{E}(x_{\mathrm{sota}})$ for maximization tasks.

---

Methodology

3.1 ℱinch Collection Construction

The construction pipeline consists of three steps:

Step 1: Seed Optimization Task Collection

• Sources tasks from existing optimization benchmarks with executable, externally validated objectives
• Selection criteria: (i) requires nontrivial search, (ii) does not reduce to matching a known ground-truth answer, (iii) provides a deterministic evaluator
• Collects 371 seed tasks from 10 benchmarks including AlphaEvolve's mathematical discovery problems, FrontierCS, ALE-Bench, AlgoTune, GPU Mode, LLM-SRBench, Function Minimization, K-Module tasks, scRNA-seq denoising, and variants of Erdős problems

Step 2: Evolutionary Trajectory Collection

• Runs OpenEvolve scaffold with Qwen3.5-397B-A17B as teacher mutation operator
• Collects trajectories under two mutation strategies:
  • Diff-based edit: emphasizes exploitation (50.3% of trajectories)
  • Full rewrite: encourages broader exploration (49.7%)
• Each trajectory stores: $(I, x_{t-1}, \mathcal{H}_{t-1}, x_t, \mathcal{E}(x_t), \mathcal{F}_t)$ where $I$ is task instruction, $x_{t-1}$ is parent solution, $\mathcal{H}_{t-1}$ is search history, $x_t$ is generated candidate, and $\mathcal{F}_t$ includes execution logs and feedback
• Total: 172,997 raw trajectories

Step 3: Trajectory Filtering

• Removes systematic errors (3.7%): missing parent scores, timeout errors, syntax-check failures, import failures, evaluator crashes
• Removes unrecoverable cases (both parent and child erroneous; 0.2%) and breakage cases (error-free parent → erroneous child; 0.8%)
• Discards excessively long inputs (>16,384 token responses or >32,768 token serialized I/O; 5.0%)
• Final: 156,731 trajectories (90.6% retention)

Trajectory Classification

Trajectories are classified by improvement outcome $\Delta = \mathcal{E}(x_t) - \mathcal{E}(x_{t-1})$:

3.2 Dataset Analysis

Key statistics:

• Languages: 68.5% Python, 31.5% C++
• Input length: avg. 8,902 tokens; Output length: avg. 6,865 tokens (1.3× ratio)
• NC trajectories have 2.9× longer median output length, suggesting more extensive reasoning without improvement
• Reg trajectories have comparable output length to Imp, implying regressions arise from misguided exploration rather than insufficient reasoning
• Most frequent packages: numpy, bits/stdc++

3.3 ℱinch Model Training

• Base models: Qwen3.5 series (2B, 4B, 9B) and Qwen3-8B
• Training data: Only Imp trajectories (30,445 total), one trajectory per task to mitigate imbalance
• Training setup: Full SFT via LLaMA-Factory, one epoch, global batch size 128, learning rate 1e-5, eight NVIDIA H200 140GB GPUs
• Additional training: KTO (preference learning) on Imp and Reg jointly to teach self-judgment of solution quality

---

Empirical Validation / Results

4.2 Main Results

Cross-task discovery generalization (Table 3): ℱinch achieves substantial relative gains across 22 tasks:

Notable improvements:

• ahc058: +290.59% (ℱinch-9B)
• Transaction: +74.30% (ℱinch-4B)
• CP (n=26): +65.09% (ℱinch-9B)

Competitive Programming (Table 4): ℱinch-9B achieves 46.01 average score vs. 32.46 for base Qwen3.5-9B, with P263 improving from 55.09 to 86.10.

Offline RL (KTO) (Table 5): Further improves performance; ℱinch-8B + KTO surpasses best human scores on AC1 (1.5089 vs. 1.5097) and AC2 (0.9146 vs. 0.9015).

Test-time RL synergy (Table 6): ℱinch-8B with nanodiscover achieves state-of-the-art on circle packing (n=26: 2.635983; n=32: 2.939573) and improves Erdős performance.

Key Ablation: Effect of Improvement Type

Training on only Imp trajectories is crucial:

Scaling Analysis

Increasing training tasks from 15 to 355 improves held-out performance by 14.1% on average across AC2, CP (n=26), and PRISM, demonstrating scalable training signal.

Case Study: Cross-Domain Transfer

On the Convolve2D task, ℱinch-9B transfers knowledge by switching from scipy to jax for computational efficiency—a pattern learned from uncertainty inequalities and matrix multiplication tasks in the collection.

---

Theoretical and Practical Implications

Theoretical Implications

1. Discovery as learnable capability: The paper demonstrates that the ability to iteratively evolve solutions (knowing what to mutate, when to backtrack) can be internalized into model weights rather than residing solely in search scaffolds—fundamentally shifting the paradigm from "search with a model" to "model that searches."

2. Cross-task generalization: EFT confers the ability to compose strategies from diverse domains when tackling new problems, suggesting that discovery skills are transferable across task boundaries.

3. Scaling laws for discovery: The positive scaling trend with training tasks suggests discovery capability follows similar scaling properties to other learned capabilities.

Practical Implications

1. Democratization of discovery: EFT enables open-source models (2B–9B) to match or exceed the performance of proprietary frontier models in search scaffolds, reducing dependence on expensive closed-source LLMs.

2. Mid-training paradigm: EFT serves as a "practice phase" for discovery agents, complementing existing test-time RL approaches and providing a foundation for further adaptation.

3. Efficiency gains: Smaller EFT models can match or exceed performance of non-EFT models twice their size (e.g., ℱinch-4B achieves 0.386460 on Erdős vs. Qwen3-8B's 0.403585).

---

Conclusion

Summary

This work introduces Evolution Fine-Tuning (EFT), a mid-training paradigm that teaches LLMs to evolve solutions across optimization tasks by converting evolutionary search trajectories into supervision. The resulting ℱinch Collection (156K trajectories, 371 tasks, 10 domains) and ℱinch model family demonstrate:

• Cross-task discovery generalization across 22 held-out tasks (10.22% average improvement)
• Synergistic effects when combined with test-time RL
• Scalable training signal with increasing task diversity
• Emergent cross-domain strategy transfer behaviors

Future Directions

The authors identify several limitations and extensions:

1. Mixed test-time search scaffolds: Training on OpenEvolve-style trajectories may not generalize to stronger scaffolds (e.g., EvoX); collecting trajectories from mixed scaffolds is needed for cross-scaffold generalization.

2. Extending test-time RL experiments: Verification of positive synergy with test-time RL on practical tasks (e.g., kernel engineering) beyond mathematical tasks.

3. Multi-modal discovery: Extending EFT to Vision-Language Models for scientific discovery involving visual observations, leveraging recent VL distillation techniques.

4. Multi-turn interactions: Extending from single-turn generation to multi-turn reasoning over previously explored lineages, enabling continual learning of promising evolution directions.